Game developers' approaches to comunicating climate change

Educational games are potential tools for communicating climate science to the public and thus improving public understanding of climate change. In this article we explore the use of co-design methodologies, a participatory open design process, to communicate climate change to a wider audience. To this end, we hosted Climate Jam 2018, a game jam with the objective of creating games to communicate climate change science and to gain insight into how developers approach educational game design. The inclusive event attracted professional game developers and hobbyists from four continents. Participants received a science pack with scientiﬁc information about climate change and completed a pre-and post-game-jam survey containing questions relating to climate change, motivations, and game design principles. We present a description of select games that highlight different approaches to communicating climate change to a general audience. Additional results from the surveys showed that few game developers engaged with the science pack and other resources in depth, that communicating climate science was of medium interest to game developers, and that the games’ potential learning effects relate mostly to memorizing and recalling the information communicated in the games. The results are discussed with respect to improving communication between scientists and game developers in the co-creation process.


INTRODUCTION
Climate change and the increase in extreme weather patterns represents one of the biggest challenges facing humanity.The global and regional impacts of climate change are increasingly driving global politics, food and water security, socioeconomics, and population migration (United Nations, 2018).Hassol (2008, p. 106) argues that the "need for scientists to communicate more effectively about climate change is urgent" (cf.also Moser, 2010;Wu and Lee, 2015).However, due to the complexity of the science, geopolitics and the global nature of the problem, the general public in many countries has a poor understanding of key issues surrounding climate change and the impact individual behavioral changes could have on the future of the climate system.For example, Howe et al. (2015) found that even though 70% of American citizens believed climate change was real, only 53% believed it was caused by humans and, worryingly, only 43% believed that scientists were in agreement about climate change.In addition, long-term records indicate that in the US there has been little change in public opinion and understanding of climate change in recent years (cf.Hassol, 2008;Leiserowitz et al., 2017) and that understanding remains superficial overall (Nisbet and Myers, 2007;Leiserowitz et al., 2013).Such evidence suggests that current methods for communicating and teaching about climate change are ineffective.
Various reasons for the poor communication of climate science have been proposed in the literature.Hassol (2008) points out that scientists are generally not effective communicators, partially because their scientific training hinders them from communicating effectively with a non-scientist audience (cf.Moser, 2010).Furthermore, scientists frequently do not communicate directly with a lay audience.Rather, the communication is often filtered through media outlets, for example, when scientists are being interviewed or when published scientific findings are reported on in the media.While scientists may aim for their communication to combat science misinformation and educate the public (Dudo and Besley, 2016), media outlets may not necessarily share these aims.For example, Ereaut and Segnit (2006) found that alarmist language, dramatic imagery and small actions were dominant features across all communications in the 600 newspaper articles, TV and radio clips, press ads and websites that they analyzed.Even though such representations were memorable and captured people's initial attention, their use led to feelings of helplessness and failed to promote engagement or a sense of personal responsibility (O'Neill and Nicholson-Cole, 2009).Similarly, Witte and Allen's (2000) meta analysis suggests that appeals to fear can lead to denial and the adoption of avoidance strategies.As a result, it is recommended that fear-increasing messages are accompanied by credible remedial actions (cf.Moser, 2010;Nerlich et al., 2010).The focus on individuals' small actions has also been criticized because a small number of corporations, not individuals, are largely responsible for global emissions (Griffin and Heede, 2017;Byskov, 2019).This, however, does not mean that communicating climate change science to the wider public is of little use, as the public play a vital role in shaping issues (Dryzek, 2002) and critical mass in terms of awareness, and the need for action, is needed to force corporations and governments to change.In fact, without significant improvement in public understanding of the complexities of climate change, it is difficult to envisage any change on an individual or at a societal level.Consequently, it is vital that effective methods of communicating the complexities of climate change are developed.
A growing body of research is therefore exploring how to effectively communicate climate change science to a general audience (see Moser, 2010, for a historical overview of climate change communication).This research shows a trend away from a "public understanding of science" or "deficit model" (Wynne, 1993(Wynne, , 2007;;Irwin and Wynne, 2003) that takes a top-down approach and assumes that the general public has a knowledge deficit that scientists need to remedy.The deficit model is monologic in the sense that scientists (i.e., those who know) convey messages to the general audience (i.e., those who don't know) so that the general audience decreases their knowledge deficit and converges with experts in terms of knowledge and attitudes.In contrast, communication is typically dialogic, reflexive, and requires contextual understanding (Moser, 2010;Nerlich et al., 2010).As a result, more recent approaches have focused on engaging the general public on an affective and emotional level through the use of more egalitarian or bottomup methods (Nerlich et al., 2010).Thus, part of the challenge is to make the issue of climate change "appealing, interesting, and meaningful to the individual" (Nerlich et al., 2010, p. 100).This involves engaging audiences not only at the level of understanding, but also at the level of emotions and behavior.
Research has begun to explore how to best engage general audiences on these different levels.In terms of understanding, Van der Linden et al. (2014) suggest that climate change is most effectively communicated in short, simple, easy to understand, and easy to remember messages and with the help of imagery (cf.also Sheppard, 2005;Moser, 2010).Dahlstrom (2014) proposes that narratives and storytelling are effective means to reach audiences at the level of emotions and engagement.Narratives and storytelling are effective because a lay audience has an "inherent understanding of what it means to tell a story" (Dahlstrom, 2014, p. 13614).While scientists value data, laypeople are more easily engaged through anecdotes.Specifically, lay audiences find narratives easier to understand and more engaging than scientific communication (Green, 2006;Bruner, 2009).In fact, laypeople get most of their scientific information through mass media, which is already biased toward a narrative format (Bruner, 2009).
There has also been a growing interest in using entertainment media to communicate scientific information (cf.Brodie et al., 2001).Entertainment media have the potential to reach a wide audience because they represent a majority of the media content consumed (Dahlstrom, 2014).They also typically use narratives and have the potential to engage audiences not just at the level of understanding, but also emotionally.Here, we focus on one entertainment medium-video games-to communicate climate change to a lay audience.There is, in fact, a growing interest in the potential use and effectiveness of video games to communicate science broadly, and climate change in particular (Connolly et al., 2012;Boyle et al., 2016;Clark et al., 2016;Harker-Schuch and Grant, 2017;Benton et al., 2018;Bianco, 2018b;Farrell and Homatash, 2018;Melcer and Isbister, 2018;Zhang et al., 2018).Video games are inherently engaging because they provide "designed experiences" (Squire, 2006) where players learn first-hand by doing and being.This kind of learning not only increases understanding, but also engages the audience emotionally (Mendler de Suarez et al., 2012).Importantly, video games can also simulate complex situations and thus allow audiences to experience these complexities.Given this, using this kind of simulation has the potential to help lay people gain a better understanding of the complexities of climate change (Wu and Lee, 2015).In line with this, Reckien and Eisenack (2013) argued that climate change games could contribute to learning, mitigation and adaptation in response to climate change, and Meya and Eisenack (2018) report evidence that simulation games can facilitate experiential learning about the difficulties of international climate politics.In addition, from their review of games attempting to teach players about climate change, Wu and Lee (2015) concluded that games are uniquely suited to communicating climate change science to young people, who are less set in their worldview and more open to new information and change (Stevenson et al., 2014;Harker-Schuch and Grant, 2017).However, they also noted that games varied greatly in format, technical sophistication, scientific accuracy, and effectiveness, highlighting the need for further experimentation with new game types and mechanics to communicate the complexities of climate change.
Game jams (Kultima, 2015) represent one way to explore how games can communicate climate change.Game jams are events where participants from diverse backgrounds, such as art, programming and design, create games under the constraints of a time limit and theme (Chatham et al., 2013;Cook, 2015).They have been identified as excellent opportunities for research in a wide variety of fields (Fowler et al., 2013a,b;Zook and Riedl, 2013;Ho et al., 2014), where time constraints encourage rapid and exploratory game design and development (Hrehovcsik et al., 2016).In addition, participants in game jams gain valuable skills in prototyping, collaboration (Preston et al., 2012;Fowler et al., 2013a,b), and collaborative learning (Shin et al., 2012;Arya et al., 2013).In a study exploring participants' motivations for participating in game jams, Zook and Riedl (2013) found that the majority set goals for personal benefit.The games which participants create are often rated by other participants and the public, providing a feedback mechanism that places an emphasis on making games which are enjoyed by players (Fowler et al., 2013a,b).
The overarching aim of this study was to gain an understanding of how game developers from diverse backgrounds approach the task of designing an educational game in a co-creation process and how they communicate aspects of climate science to a general audience.To explore these topics, we hosted Climate Jam 2018, a game jam for educational games which communicate the complexities of climate change.We chose a game jam to explore developers' approaches to communicating climate science because educational game design is a relatively new discipline (Ke, 2016;Lameras et al., 2017) and can be challenging due to the various stakeholders involved (Zhang et al., 2018).This often leads to the use of co-design frameworks (De Jans et al., 2017) that encourage a participatory open design process, such as game jams.Furthermore, Mauser et al. (2013, p. 420) identify co-design methodologies as part of "a paradigm for research toward global sustainability that will be both designed and conducted in partnership between science and society." A focus on game developers with respect to communicating climate change through games is particularly relevant because game developers play a large role in what is communicated in a game and how it is communicated.We therefore asked the following research questions: • RQ1: What motivates game developers when designing an educational game on climate change?
• RQ2: How do game developers engage with climate science when designing an educational game on climate change?To encourage participation, we also offered prizes and assistance to game developers to apply for science communication funding following the game jam in order to expand their prototype.

Rules
Since our aim was to encourage experimentation, the rules of Climate Jam 2018 were minimal.The only requirement was that each game needed to communicate one or more of the following concepts related to climate change, which are commonly misunderstood, miscommunicated, or not sufficiently appreciated: • Weather is not climate.
• Global warming doesn't mean it's getting warmer everywhere.
• I can make a difference to climate change.
Since there is substantial misunderstanding of climate science amongst lay audiences (Howe et al., 2015), a science pack was produced and provided to game jam participants via a link on the game jam homepage.The science pack was put together in collaboration between DR and science communicators at the National Museum of Wales, Cardiff, UK.The pack was aimed at any non-expert audience, including the game jam participants and the audience of gamers for whom they develop their games.It contained a concise suite of information regarding the state-of-the-art predictions of likely future climate change.In addition, links were provided to web pages that presented reliable information designed for the lay reader.Finally, the science pack contained examples of key feedback mechanisms in the climate system.Participants also had the opportunity to ask climate science experts in our team for advice via on-line forums and chat rooms throughout the duration of the jam.

Judging and Rating
Itch.io provides tools for the judging and rating of entries to the game jam.Three criteria were selected for judging: 1. Gameplay/Fun 2. Educational Value 3. Scientific Accuracy Gameplay/Fun and Educational Value were set as public criteria, meaning that anybody could submit a rating out of 5 stars, with 5 being the best possible rating.In addition, anybody could leave comments on each game.The project team also rated games sand provided comments.Scientific Accuracy was set as a manually ranked criterion determined by the game jam organizers, and a team of ten climate scientists were recruited to judge this particular criterion.This team of climate scientists comprised university professors, leading research scientists (based at Cardiff University, Exeter University and the UK Met Office), and science communicators (based at the National Museum of Wales, UK).In addition to rating Scientific Accuracy, climate scientists and the project team were also able to write official judges' comments.

Participants
Different numbers of game developers participated in the pre-jam survey, submitted games to Climate Jam 2018, and participated in the post-jam survey.
Participants also provided information about their prior exposure to, and beliefs about, climate change science.Four (13.8%) participants reported having received basic training in climate change science and/or a related discipline, three (10.3%)advanced training, and 22 (75.9%)no formal training.To measure trend and attribution skepticism (Rahmstorf, 2004), we also asked participants whether climate change is happening and what is causing climate change.The majority of participants believed that the world's climate is changing (N = 27; 93.1%).Only one (3.4%)participant reported that the world's climate is not changing, and only one (3.4%)did not know.The vast majority of participants thought that climate change was at least "partly" or "mostly" attributable to human activity (N = 27; 93.2%).One participant (3.4%) thought it was caused entirely by natural processes, one (3.4%)did not know.

Game Submissions
A total of 15 games were entered into Climate Jam 2018, some of which are free to download and play via the game jam website (https://itch.io/jam/climate-jam-2018). Of the 15 games entered, 6 were removed from our analysis because the relationship to climate change was unclear, or they were in an unfinished state so could not be played.This resulted in a total of N = 9 games.

Post-jam Survey
A total of 15 participants accessed the post-jam survey, but data for 5 participants was subsequently removed for the following reasons: (a) did not fully consent to participate (N = 2); (b) consented to take part but did not finish the survey (N = 1); (c) were under 18 years of age (N = 2).The final sample consisted of 10 participants, of whom 6 (60%) were male, 3 (30%) female, and 1 (10%) preferred not to say.Mean age was 25.0 years of age (SD = 3.94; range = 19-31 years).

Materials
A science pack (see information above) and two surveys for administration pre-and post-game jam were developed.

Pre-jam Survey
In addition to questions capturing the information about game developers' demographics, experience with game development, and exposure to climate change science described above, the survey also contained questions related to: • Engagement with science pack: Participants were asked whether they had read the science pack prior to the start of the game jam and, if so, to indicate their level of engagement.
• Opinions about climate change: Twenty-two questions designed to assess beliefs, opinions and skepticism about climate change (Steentjes et al., 2017) which participants rated on five-point Likert-type scales (e.g., strongly disagreestrongly agree; definitely not-definitely yes; not at all worried-extremely worried).
• Important aspects of game design and development: Nine questions capturing different aspects of game design/development were included, with respondents asked to rate the relative importance of each item when designing/developing a game on a 5-point scale (extremely important-not at all important).A "not applicable" option was also included.The self-assessed potential communicative value of the games developed was assessed via two questions which asked participants whether they believed their planned game could influence others' views on climate change and why (yes/no/maybe; open text response).

Post-jam Survey
In addition to basic demographic information and the information mentioned in section 2.3.1, the survey also contained questions related to: • Game submission: Participants reported whether they submitted a game, and if not, at what stage they withdrew.They were also asked whether they worked on their game alone or as part of a team.
• Engagement with science pack: Participants were asked to specify how much they had engaged with the science pack during game development, and to explain why.They were also asked how informative they found the science pack on a 5-point Likert-type scale (definitely not-definitely yes) and asked whether they used other resources to obtain information about climate change (e.g., books, newspaper articles).
• Opinions about climate change: As described in the pre-jam survey.
• Impact of participation on knowledge and behavior: Participants were asked to respond to a series of statements regarding change in knowledge and behavior with respect to climate change following participation in the jam.
• Impact of game: On a 7-point Likert-type scale ranging from extremely unlikely to extremely likely, participants were asked to rate their game's potential to influence others via a variety of statements.They were also asked to explain their answers in an open-ended format.
• Game design: Participants were asked to rank their priorities when designing and developing their game entry, and if working as part of a team, to report how much their team discussed each of the priorities.They were also asked to report whether they expected participating in this jam to influence their future game design.

Procedure
Ethical approval was obtained from the Cardiff University School of Psychology ethics committee1 .Participants were recruited through itch.io,Twitter and Facebook and invited to take part in an on-line survey pre-and post-game-jam.Participants who completed both parts of the survey were informed that they would be entered into a free prize draw to win one of three prizes (e.g., $420 to buy Unity Plus for 12 months).Participants were presented with an electronic information page outlining the purpose of the research, what the study would involve, study data protection and confidentiality arrangements, and details about who was carrying out the research.Participants were then asked to provide informed consent and to generate a unique personal identifier (initials, month and year of birth; CW0212) to allow their pre-and post-jam data to be subsequently linked.Following completion of the pre-jam survey, which took approximately 15-20 min to complete, participants were thanked for their time, debriefed, and asked whether they would like to be invited to take part in the post-jam survey upon completion.
Participants who consented to be contacted to take part in the post-jam survey were emailed details about how to take part after the game jam had ended.Details about the post-jam survey were also posted on itch.io at the end of the game jam.Participants were asked to read an electronic information page and to provide informed consent for a second time.They were then asked to record their unique personal identifier from the pre-jam survey before proceeding to complete the survey questions as described previously.The post-jam survey took approximately 15-20 min to complete and participants were debriefed and thanked for their time upon completion.

General Game Design Motivations (RQ1)
To assess what motivates game developers in general, participants were asked to respond on a 5 point scale from "not a all important" (1) to "extremely important" (5) to the question "When you have designed and/or developed games, how important have each of the following objectives been to you?"Of direct relevance for the communication of climate science through games were the objectives "simulation of reality, " "to educate or inform, " and "game narrative and/or role play."These three objectives only received low to medium importance ratings with average ratings of 2.65 (SD = 1.06;NAs = 3), 2.7 (SD = 1.1;NAs = 2), and 3.41 (SD = 1.12;NAs = 2), respectively.In contrast, the objectives "playability" and "mechanics, dynamics, and systems" were rated as most important with average ratings of 4.5 (SD = 0.69; NAs = 1) and 4.07 (SD = 1.05;NAs = 1), respectively.Other objectives with low and medium importance ratings were "to make the most of up-to-date technology and audio-visual effects" (mean = 2.12; SD = 1.42;NAs = 4), "distraction from daily routines" (mean = 3.07; SD = 1.15;NAs = 1), "game difficulty and adaptivity" (mean = 3.32; SD = 0.98; NAs = 1), and "level design" (mean = 3.7; SD = 1.06;NAs = 2).Overall, participants considered objectives related to the quality of the games to be more important than objectives related to communicating climate science.

Engagement With Climate Science Before the Game Jam (RQ2)
The vast majority of participants reported engaging with the science pack prior to completing the pre-jam survey and thus also prior to developing a game, though only a minority reported engaging with it thoroughly.Five (17.2%) engaged several times/thoroughly, thirteen (44.8%) once or twice but not thoroughly, nine (31.0%) briefly skimmed it and two (6.9%) did not read it at all.Despite the relatively shallow engagement with the science pack, participants expressed medium levels of confidence that they could influence views on climate change with their games.Participants were asked to respond to the question "Thinking about Climate Jam 2018, do you think you can influence views on climate change with your game?" on a 5 point scale from "probably not" (1) to "definitely yes" (5).The mean response was 3.62 (SD = 0.75; NAs = 3), suggesting medium confidence in influencing views on climate change.Participants were then asked to explain their responses in an open-ended question, and whilst a full thematic analysis is beyond the scope of this paper, we have selected some typical responses to report.One confident participant stated "[Other media] cannot give the reader/viewer the chance to see that system move and work and to poke at it.Only games can do that."Many participants discussed the idea of fun, for example "Games give people a fun way to experience something they might otherwise find boring."A less confident participant expressed an opinion that games are not effective educational tools "due to gameplay which must balance between 'textbook' about climate change and game component."Finally, one participant explained that they were not confident because "If someone possesses heavy connotative bias about climate change, there is no guarantee any medium would change their opinion."

Communicating Climate Science in Games (RQ3)
To explore how climate science was communicated in the games, we first describe three entries in detail to provide information about the range of the games submitted and how they sought to communicate climate change.A general analysis of all entries (N = 9) follows.

On Thin Ice
On Thin Ice (Bianco, 2018a) was created by Karn Bianco.Bianco describes it as a short game about the impacts of climate change and what can be done to reduce them.As the game starts players are introduced to Nanuq, a time traveling bear from the future, shown in Figure 1.Nanuq acts as a narrator throughout the game, teaching players how to play and explaining how the game play mechanics relate to climate science.
As the player, you control a block of ice which, as explained by Nanuq, reflects rays of sunlight into space.If you miss the rays, which fly down from the top of the screen, the global temperature increases.Once the global temperature, indicated as a bar on the left-hand side of the screen (see Figure 2) reaches a certain value, the player fails the level.The goal is to survive a given amount of time, which is made challenging by the addition of carbon dioxide particles.Rays bounce off the carbon dioxide particles FIGURE 1 | On Thin Ice's narrative element (Bianco, 2018a).
back down toward Earth.Each level in the game represents a different year in Earth's history.At the start, the player travels back in time when there were fewer carbon dioxide particles in the atmosphere, which reduces the games difficulty.Then the player travels to present day where the challenge is increased by more carbon dioxide particles, a smaller ice block caused by ice cap melting, and a higher initial starting temperature.Finally, the player travels to the future where the difficulty level increases further.The learning objectives of On Thin Ice are embedded within the game play mechanics, and the addition of the narrator Nanuq makes this link explicit as he communicates climate science concepts in the context of the game play mechanics.

Isorropia
Isorropia is a city management game created by Return Null (2018).The player is tasked with generating enough power for a city by building structures to generate power.If the player has managed to generate enough power for the city, it will automatically grow.The goal is to grow the city as much as possible.To make things challenging, the player also has to manage the climate, in the form of a danger meter which ends the game if it fills up.This meter represents the amount of carbon dioxide in the atmosphere, which is affected by the emission and absorption of structures the player builds.For example, the player can build a Coal Plant which generates 20 Mega Watts of power and emits 6,000 tons of carbon dioxide, or a tree which generates no power but absorbs 50 tons of carbon dioxide.All four structures which the player can build are shown in Figure 3.The player has a number of resources to balance and consider, all structures cost money, more money is generated as the city grows, and the city can only grow if there is enough power.This forces difficult decisions about building coal plants which are the most cost effective way of generating power, yet emit the most carbon dioxide.However, as the location of the buildings seems to have no impact on the simulation, this makes the game feel more like an economics management game than city planning.There is no narration or tutorial, with the climate change information entirely communicated through the game play mechanics and systems.

Climate Cards
Climate Cards is a deck building card game by Hopeless Productions (2018), in which the player is tasked with "exploiting" a small ecosystem while avoiding climate catastrophe.The player takes on the role of a planetary manager employed by the fictional company LifeCorp, whose job is to maximize profits.There are three resources to manage in Climate Cards; (1) gold, (2) energy and (3) carbon dioxide levels  in the atmosphere.Cards are then played to spend and gain these resources.For example, playing the generator card provides two additional energy and two additional carbon dioxide.Playing the crops card generates one gold if you have zero energy, but two gold if you spend one energy.Figure 4 shows a player's hand of six cards randomly drawn from the draw pile.On each turn, the player can play as many or as few cards from their hand as they wish; then all cards are placed on a discard pile.At the end of each turn a number of events happen: 1.The player must pay a small sum of gold, starting at two but increasing as the player performs better, else they will lose a life and eventually the game.2. The sun heats up the planet.Normally the heat radiates away, but if the player has too much carbon dioxide in the atmosphere, it does not.As the temperature increases, turn by turn, ice on the planet melts and flood cards are added to the player's deck.These are useless cards that reduce the options available to the player if they appear in their hand.If enough ice melts, the player loses the game.3. Energy is reset to zero. 4. The player may pick one of three random cards to add to their deck.These cards open up more options when they randomly appear in the players' hand.
In Climate Cards, climate science concepts are communicated through the effects of the cards.As the player performs better, they are required to produce more gold each turn forcing tough decisions about generating gold at the expense of generating large amounts of carbon dioxide.

Analysis of Games
Ke (2016), Aleven et al. (2010) and Lameras et al. (2017) provide general frameworks for analyzing educational games that allow us to look at how climate change is communicated through the games.A combination of concepts from these frameworks were used to formalize the analysis.SPW and CW classified and labeled all nine entries independently, reaching 100% inter-rater agreement.

Communicating Climate Science in Games (RQ3)
Ke ( 2016) identified three modes in which learning objectives (cf.Aleven et al., 2010) are integrated into a game: representation, simulation and contextualization.Conceptual representation maps abstract concepts to illustrative game objects.For example, in On Thin Ice (Bianco, 2018a), increasing global temperature is represented in the game by the player's ice block decreasing in size.Thus, the connection between global temperature and melting ice caps is communicated through the size of the block of ice.Simulation integrates learning objectives by simulating a complex system and giving players agency over it.Simulation is therefore especially well suited for conveying the complexities of climate change to a lay audience.One such example is Isorropia (Return Null, 2018), where players build power plants for a simulated city.Here climate change information is communicated as players explore what happens in response to the climate-relevant decisions that they make.Finally, learning objectives can be mapped into games via contextualization.
Here the learning objectives are expressed by the theme, game characters or game world without any relationship with the game play mechanics.Contextualization tends to be context-irrelevant, and therefore, the same core game could be transformed from a game teaching about climate change to a game teaching mathematics without touching the game mechanics.An example of an entry which used contextualization to communicate climate science is Life of Fish by Vanadium Games (2018).In this game, the player controls a fish swimming in the ocean and must avoid predators to survive.As time progresses, fictional news stories appear which give predictions of the future of climate change.The player's actions have no effect on the climate, but the climate slowly kills the fish as acidity in the ocean rises.We classified this as contextualization since climate change simply happens around the player and the act of playing does not seem to relate to any of the learning objectives.We classified 11% of the entries as representation, 33% as simulation and 56% as contextualization.
Ke ( 2016) also identified that educational games create environments for learning through game mechanics, the game world, or a combination of the two.56% of the entries only communicated climate science through the game world or narrative; these were the same group of entries which used contextualization to map learning objectives.The remaining 44% of entries additionally communicated climate change information through the game mechanics.Games which embedded the learning goals within the game play mechanics mostly required the player to manage the level of carbon dioxide in the atmosphere.Players would often be tasked with making political and economic decisions, and then watch as the effects of these decisions are simulated.These games all took a global view point, where players would be managing the decisions of city planners or world leaders.Conversely, in the narrative games the focus was often on the individual experiencing climate change passively.Lameras et al. (2017) presents a taxonomy for linking learning outcomes to game attributes in educational games.Specifically, they mapped game attributes to Bloom's (1956) taxonomy of learning outcomes.Bloom considered three domains of learning: cognitive, affective and psychomotor.Lameras et al. (2017) argue that the most relevant domain to serious games is the cognitive domain.The cognitive domain is defined as a student's intellectual level, what the student knows and how they organize ideas, opinions and thoughts.Bloom defined six classifications of learning outcomes related to the cognitive domain: 1. Remembering: Learner can memorize and recall information.2. Understanding: Learner can comprehend, explain and predict.3. Applying: Learner can use information and solve problems.4. Analysis: Learner can analyze data patterns or concepts and relate these findings to prior knowledge.

Evaluating:
Learner can compare and make justifiable judgments about the value of ideas, methodologies or products.6. Creating: Learner can design, build, invent, plan or produce original knowledge and transfer it to new contexts.
These outcomes are listed in order of skill level, i.e., Creating is considered a higher level skill than Remembering.Higher levels lead to deeper learning, such that games that communicate climate change by applying higher-level learning outcomes should lead to more engagement and deeper learning.All entries were classified using the above learning outcomes.For example, in Life of Fish (Vanadium Games, 2018) contextualization is used to map learning objectives onto the game play; it is tagged with the Remembering classification only since the game does not provide opportunity for higher level outcomes.Games which do not embed the learning goals within the game play mechanics do not offer players the opportunity to apply their knowledge.This applied to 56% of the entries (5 out of 9), which are only classified as Remembering.In On Thin Ice (Bianco, 2018a) the learning objectives are embedded in the game play mechanics using a conceptual representation, but the player has no agency related to those learning objectives.For example, as the amount of carbon dioxide in the atmosphere increases in the game, the challenge increases, since more carbon dioxide particles block light rays.This helps the player understand the concept, but does not give the opportunity to apply that knowledge.Only 44% of games (4 out of 9) reached this level and were labeled as Understanding in addition to Remembering.Contrastingly, in Isorropia (Return Null, 2018) the player has agency over the learning objectives which are simulated.They are given knowledge about trees absorbing carbon dioxide, apply that knowledge by planting trees to reduce the carbon dioxide level, analyze the amount of carbon dioxide they absorb vs. the cost of planting trees, and continually evaluate their decisions as the game offers feedback through the growth of the city.Only 33% of games (3 out of 9) reached this level and additionally received the labels Applying, Analysis, and Evaluating.None of the entries offered the opportunity for the player to create knowledge.Thus, the majority of games only reached the lowest level of Remembering.These games may communicate information about climate change, but are unlikely to help people gain a better understanding of the complexities of climate change or promote personal engagement.
These results are broadly in line with participants' thoughts on how the game mechanics and dynamics (i.e., the run-time behavior of the mechanics when the player is interacting with them; cf.Hunicke et al., 2004) relate to communicating climate change and players' emotional responses.One participant did not consider the relationship between mechanics and dynamics of the game and engagement with climate change, expressing that "As long as the game I make is fun and engaging it can be very valuable in a school."Another participant described how they aimed to design a set of mechanics that allows players to grasp part of the complexities of climate change: "My intent is to make a simulation that shows how changing means of production in an industrial setting can not only be beneficial economically, but also as a means to reduce climate change."A further participant commented on the challenge of controlling players' emotional responses to the game through designing the mechanics: "A welldesigned game can express an author's opinion about that system, but it's never, ever clear at the beginning of the development process how that opinion will land with the audience.The interplay of game mechanics and story, especially in computer games, is far too complicated to tell ahead of time." 3.4.Game Developers' Approach (Post-jam Survey)

Game Submission
Participants who completed the post-jam survey were asked whether they submitted a game to Climate Jam 2018: six (60%) participants submitted a game, one (10%) worked on a game but as it was unfinished, they did not submit it; two (20%) reported having an idea/potential game but did not go on to develop/produce it, and one (10%) participant signed up to Climate Jam 2018 but subsequently withdrew.

Motivations While Developing the Climate Change Game (RQ1)
We asked participants about their motivations while developing their climate change game by inquiring about their design priorities.Participants were asked to rank ten design priorities by inserting a number from 1 to 10 against each option, where 1 was the least important priority and 10 was the most important priority.Mean rank scores for all priorities are shown in Table 1.
The most important concern for developers was the "accuracy of information in their game related to climate change science, " followed closely by "educational value" and then "playability." The least important considerations were "other, " "visual effects, " "technical issues/considerations, " and "narration."This suggests that game developers were concerned about the accuracy of the information they were conveying.Compared to game developers' overall motivations presented above, developers placed more of a focus on educational value when designing their Climate Jam 2018 game.

Engagement With Climate Science During the Game Jam (RQ2)
One (10%) participant reported engaging with the science pack several times when developing their game, seven (70%) engaged with it once or twice, and two (20%) reported that they had only engaged with it when originally signing up.In a free text question one participant noted that "the Science Pack did not contain all of the information I required" and another "I engaged with it when coming up with the idea I did not follow through on".
We also asked participants where else they obtained information about climate change.80% of participants reported obtaining information from the internet, 20% from newspaper articles, 20% from television programs, 20% from family/friends, 10% from books, 30% from journals/science articles, and 20% from survey research.No participants reported obtaining information from the radio programs, podcasts or magazines.

Educational Value and Gameplay/Fun
Each game was rated in terms of educational value and gameplay/fun by between eight and thirteen people, comprising the project team and those members of the general public who had engaged with a game and opted to leave a rating on the game's website.The average educational value was 2.39 (SD = 0.83) out of a maximum rating of 5, and the average gameplay/fun rating was 2.58 (SD = 0.58) out of a maximum rating of 5, suggesting that the games overall were considered to be of medium educational value and fun.Interestingly, the games that were rated higher in terms of gameplay/fun were generally also considered to be more educationally valuable, suggesting that educational value does not need to occur at the expense of fun.

Scientific Accuracy
Climate scientists ranked each game in terms of its scientific accuracy and both climate scientists and the project team provided written feedback on the game.Here, we focus on the four games with the highest scientific accuracy ratings to see how communication of climate science can further be improved in those games that already show promise.These four games coincide with those that reached the level of Understanding or above in Bloom's (1956) classification.Overall, climate scientists valued climate change being communicated through the game mechanics, allowing players to see the consequences of their actions: "It's nice to see the science embedded in the game mechanics rather than tacked on between levels"; "The effects of the cards help educate about climate change"; "Climate change concepts are taught through the effects of the cards and choices, and content maps across to the number of core themes: climate protection, economic efficiency, power supply and resource efficiency"; "[M]ost of what you learn is achieved from playing the game and seeing what happens when you take a different course of action."One comment also suggests that this approach can be quite effective: "It is quite shocking to see how many trees you need to plant to balance a coal power plant, and how many wind turbines you need."Some of the comments highlight the difficulties of communicating a complex topic such as climate change in the context of a game, both in terms of playability and accuracy: "I did struggle a little working out the cause and effect of my actions while playing"; "[T]his builds in both science and geopolitics which is exactly what we had in mind.The game play is complicated though"; "A basic simulation of the Earth and making gamers make the same sort of tough decisions that have to be made in the real world.[...] Some refining of the science might be needed though." The comments also reflect that even among the most highly rated games, there is room for improvement: "One thing I wondered was whether educational value could be improved by suggesting changes that we could make as individuals, rather than the big global changes (i.e., stop using fossil fuels).This way it may help players feel like they have more control and can make small changes to make a difference"; "The one thing that would be nce [sic.] to change would be the size of the ice patch on the 3d globe between each of the time periods to stitch together the idea that the ice caps are melting through time.";"You do learn about climate change through playing a game, but some more explicit narration would be great (i.e., if you plant this, this will happen; links to our lives now)."

DISCUSSION
Our motivation for running Climate Jam 2018 was ultimately to increase the public's awareness of climate change science and to learn about the process game developers take in designing educational games and communicating the climate science that we asked them to engage with.A number of key themes were identified, which will be discussed below followed by an evaluation of our overall approach.

Engagement With the Subject Matter
The results of the pre-game jam survey indicate that the majority of participants had no formal training in climate change science, yet the pre-and post-jam surveys indicate that most participants only engaged with the science pack once or twice.Further, even though climate science experts were available to answer questions throughout the game jam, only one question was asked relating to the subject matter.This low engagement with climate science is also reflected in the games: Six of the original 15 entries were not related to climate change, and none of the analyzed 9 entries clearly communicated the specific concepts which were included in the brief and science pack and of which developers were asked to incorporate one into their games.This is despite most of the participants having taken part in game jams in the past.
This highlights the importance of continued communication and engagement between game developers and climate change experts in the creation of educational games aimed at communicating climate change science accurately.There are a number of possible reasons why game developers in the current study did not engage with the science pack as in depth as desired and why game developers did not engage with the science experts during the duration of the game jam.Even though the science pack was developed with a lay audience in mind, game developers may still not have found it accessible enough to engage with in depth.The science pack included five pages and two additional pages as an appendix, which may have been too long a document for a lay audience.That said, none of the game developers mentioned the length or perceived complexity as a reason for non-engagement.
Each page of the science pack contained several graphs to help readers visualize the information.However, the text also contained terminology that Hassol (2008) previously suggested should be avoided when conveying climate change information to a lay audience.For example, the science pack contained some words that mean something different to climate scientists and lay people, potentially leading to misinterpretation.For example, enhance, which means improve to a lay audience and increase to the climate scientist, or positive feedback, which sounds positive to a lay audience and for which Hassol (2008) recommends self-reinforcing cycle instead.A shorter science pack avoiding the above-mentioned terminology may have improved engagement with the science pack.One way to drastically reduce the length of the science information would have been to give game developers no choice in terms of the climate concepts that they should communicate.We asked game developers to communicate one or more of three concepts related to climate change.Focusing on just one of these concepts would have reduced the information presented in the science pack and may also have sent a stronger message to game developers about what they should communicate through their game.
There are also several reasons why game developers may not have engaged with the climate experts during the game jam.Game developers may have been reluctant to contact experts whom they did not know and had not met.There are several ways to improve this.For virtual game jams, experts can be introduced on the game jam web page along with a message encouraging developers to contact them.However, to encourage in-depth engagement with climate experts, a physical game jam may have been more suitable.Climate experts could have been present at a physical game jam and could have approached the game developers and offered their support rather than waiting to be approached by the developers.This approach would have been more aligned with the process of co-creation.We opted for a virtual game jam to be as inclusive as possible and to give game developers maximum flexibility.However, a physical game jam would likely have improved collaboration between game developers and climate scientists and would have facilitated co-creation more readily.

Game Developers' Motivations
In both the pre-and post-game-jam surveys we asked participants about their design objectives.In the pre-jam survey the most important objectives to our participants were those related to the core game design.The objective "to educate or inform" was only considered very or extremely important by 17.2% of those who answered, making it the third lowest rated objective in terms of importance.This perhaps partially explains the lack of engagement with the subject matter discussed above.However, the post-jam survey paints a different picture, "accuracy of information related to climate change science" and "educational value" were among the highest ranked objectives on average.This difference could be interpreted in two ways, either that participants who generally prioritized educational value were more likely to work on and submit an entry, or that the participants who answered the second survey were trying to meet the objectives of the game jam.However, the former seems unlikely as even though our sample size is limited, "educational value" was rated similarly by developers who did and did not go on to submit a game.The key message from our data as a whole is that game developers, perhaps obviously, are motivated by making compelling core game play experiences.It is therefore important for climate experts to both respect this motivation, remind developers of the educational focus when designing educational games, and support developers in engaging with the concepts to be communicated in the games.Again, this could potentially be done more effectively through a physical game jam.Even though climate change is a global issue, a local physical game jam would have other advantages.For example, Sheppard (2012) suggests that the best way to communicate climate change is to make the communication local, visual and connected.This could have more easily been achieved through a local game jam where developers were asked to communicate a local climate change issue.Game developers participating in a local climate change game jam may also feel like they could make a difference to their local environment and their own communities.

Game Developers' Approaches to Communicating Climate Science
When analyzing entries, we found that 56% of the games communicated learning objectives through contextualization in the game world or narrative.This can be linked to our prejam survey, which shows that our participants prioritized core game design above everything else.Communicating through contextualization allows the developer to design any game and simply put it in the context of climate change.Many of the participants expressed a belief that as long as their game is fun, it would effectively communicate climate change.However, when analyzing the learning outcome classifications we found that those games which relied on contextualization to communicate climate change issues would only be able to achieve the lowest level outcome; Remembering.This is an important finding in the context of educational game design.It highlights the need to discuss communication methods and learning outcomes at the start of the design process.Without doing this there is a risk that game developers will follow their own motivations to create effective core games which tackle the learning objectives through contextualization, limiting the potential learning outcomes of the game.
As mentioned above, simulation is especially well suited for communicating the complexities of climate change because it allows for the simulation of a complex system and lets players interact with this system.In the current study, the games that were rated most highly in terms of scientific accuracy used simulations.Moser (2010) mentions various challenges for communicating climate change that simulations can tackle: climate change is often invisible and not immediate.For example, greenhouse gases are not visible and have no immediate health effects (Kirkman, 2007).In fact, many of the negative health effects from climate change are cumulative.Simulations could both visualize invisible aspects of climate change and model cumulative effects on health.Since modern humans spend much of their time indoors, simulations could also help us notice and understand "creeping" environmental changes (Glantz, 1999).Finally, climate change is also not immediate in the sense that we are trying to prevent situations that are predicted to happen in the future.This means that we are not yet experiencing, or have only just begun to experience, any of the predicted scenarios, making it difficult to come to grips with the myriad of ways that climate change can affect our lives.Again, simulations can help us visualize these future scenarios and explore how our current behaviors are likely to affect the future climate.All of these are important aspects of climate change to communicate, and all of them can benefit from games that use simulation to communicate climate change.This highlights the importance of supporting game developers in going beyond contextualization when creating games to communicate the complexities of climate change.

Limitations of Our Approach and Future Directions
Even though we attracted a diverse pool of game developers, in terms of demographics and experience, few participants answered the second survey.This is not unexpected and is often the case with longitudinal follow-up studies.A larger data set would have allowed us to more comprehensively explore differences between groups of developers in terms of motivation.It might also be possible to gather more data by hosting a physical game jam over a weekend and observing the game developers.This could also lead to more effective interaction between game developers and climate scientists.However, the trade off would be a less diverse pool of participants, likely limited to local students.Furthermore, the nature and relatively short duration of the game jam may have influenced the range of game genres found in the current study.Many of the games were modeled on familiar existing video games.The relatively short duration and the topical constraints that we introduced may have limited game developers' ability to create more complex or highly innovative games.Finally, it needs to be mentioned that our study focused on game developers and how they communicated climate science through their games, not on how the games affect the audiences that engage with them.Thus, even though we analyzed the games in terms of their educational value, we do not know how the games affect understanding of and attitudes toward climate change in those who engage with the games.Future studies focusing on the games' audiences are needed to explore this issue.

SUMMARY
In this paper we share insights gained during Climate Jam 2018 on how game developers approach educational game design when tasked with the communication of climate science through games.We highlight the importance of continued engagement between game developers and climate experts to ensure that appropriate learning outcomes are addressed in the final game and appropriate and engaging messages are communicated.We also identify an interesting bridge between the needs of climate experts and the motivation of game developers.Specifically, the nature of climate change and the difficulties in communicating it require games that go beyond contextualization and are specifically developed with the complexities and idiosyncrasies of climate change science in mind.Developers are clearly motivated by creating compelling game play experiences.If game developers are directed to simulate the learning objectives in the game mechanics, this would both better link the education and communication aspect to their motivations and enable higher level learning outcomes from the final product.

FIGURE 3 |
FIGURE 3 | Screen shot from Isorropia showing the available structures for the player to buy (Return Null, 2018).

FIGURE 4 |
FIGURE 4 | The main game play screen of Climate Card showing the player their hand of cards Hopeless Productions (2018).

TABLE 1 |
Priorities developers considered when designing/developing their game.