Making the Case for a Null Effects Framework in Environmental Education and K-12 Academic Outcomes: When “Just as Good” Is a Great Thing

Introduction

Although environmental education (EE) is a lifelong endeavor, there is a logical nexus with K-12 education. Roth (1969) and Stapp (1969) both called for education across the lifespan, and this sentiment has been echoed in the Tblisi Delcaration (UNESCO, 1977) and in more contemporary statements on EE (Monroe and Krasny, 2016). Though many initiatives exist for older or multi-generation audiences, children represent a significant audience of EE programming (Ardoin et al., 2017). As early intervention can set individuals on a trajectory of lifelong environmental engagement (Chawla, 1999; Wells and Lekies, 2006), many environmental educators see younger audiences as an opportunity for greatest impact. Further, working with K-12 schools represents an opportunity to reach a wide and diverse audience, as education is compulsory in many countries across the globe.

As K-12 school programming is a major focus in EE efforts, it is not surprising that there has been a recent call for more evaluation in EE, paralleling trends in K-12 education. In the United States, A Nation at Risk (US Department of Education, 1983) marked the beginning of a decades-long emphasis on accountability and testing. Responding to the perception that students were being left behind by public education and that better measurement would help mitigate these trends, subsequent iterations of the Elementary and Secondary Education Act (Every Student Succeeds: 114th Congress, 2015; e.g., No Child Left Behind: 107th Congress, 2002) have required standardized tests at every grade level (Darling-Hammond et al., 2016). Similarly, granting agencies have increasingly emphasized the need for evaluation (Boris and Kopczynski Winkler, 2013). In parallel, and perhaps in response to, these trends, EE scholars and practitioners have called for more rigorous evaluation to improve efficacy of programs (Carleton-Hug and Hug, 2010; Heimlich, 2010) and attend to the culture within schools and requirements of funders (Ardoin et al., 2017). As formal education is a key audience for EE, these calls have included the need for more data supporting links between EE and academic outcomes (e.g., grades, standardized test scores) (Jordan and Chawla, in press).

Increased frequency and rigor of EE evaluation, however, is a double-edged sword, particularly as it relates to academic outcomes. On one side, rigorous evaluation linking EE to academic outcomes has the potential to accelerate adoption of EE in schools. Links to academic achievement is required to justify participation in EE programs (Ernst, 2012; Stevenson et al., 2014) and funders of EE programs demand quantitative proof of efficacy, often linked to academic outcomes (Boris and Kopczynski Winkler, 2013). Some research supports a link between EE and academic outcomes, including in the context of test scores (State Education and Environment Roundtable [SEER], 2000) and science knowledge gains (Ballantyne et al., 2001; Barnett et al., 2006). By and large, this research supports positive associations between EE and test scores when EE is fully integrated into the curriculum (State Education and Environment Roundtable [SEER], 2000). Little research has been published on shorter-term EE programs (e.g., field trips, EE lessons). Of 119 EE articles systematically reviewed between 1994 and 2013, only seven included academic outcomes (Ardoin et al., 2017). When academic outcomes are engaged, as in the case of the statewide Oregon Outdoor School program (Braun, 2019), academic outcomes may be indirectly measured through teacher perceptions of student learning and student self-report measures. The relatively few studies directly measuring academic outcomes may represent a disinterest among EE researchers in academic outcomes in favor of others (e.g., connection to nature), rejection of academic outcomes a measure of meaningful learning, or a failure to report null or negative results (Stern et al., 2013). This latter explanation highlights a challenge posed by increased evaluation of EE in academic contexts—generating research that shows that EE is no better than non-EE strategies at boosting academic outcomes when best teaching practices are used in both contexts. If the EE community is to respond to calls for increased evaluation, a framework may be needed to present those null results in ways that can build the case for EE, rather than undercut it, and contribute to scientific understanding of the contributions of EE to student learning.

In this paper, we describe how a null effects framework for EE research and evaluation in in K-12 contexts works toward these goals. Such a framework shifts the emphasis from reporting positive results (e.g., EE strategies being better than non-EE strategies) to celebrating null effects (i.e., no difference between EE and non-EE strategies). Many readers likely adopt the view that academic measures (e.g., standardized test scores) are poor indicators of student learning that ignore critical factors such as motivation or affective gains (Aydeniz and Southerland, 2012; Tienken et al., 2017). We agree with this view but argue that academic outcomes are clearly a metric of interest to policy makers and funders and are essential to advocate for EE in schools. We begin by describing how to detect null effects and, then, suggesting how they may be useful in the context of EE research and evaluation while avoiding unrealistic expectations. We argue a null effects framework fits the objectives of EE and addresses the context of standardized testing in K-12 education without being bound by it.

What Does it Mean to use a Null Effects Framework for EE and Academic Outcomes?

Although it is possible that null effects around EE and academic outcomes are going unreported, EE researchers may not directly study academic outcomes because doing so creates logistical challenges. For example, many EE programs serve K-12 students but are not affiliated directly with schools. Accordingly, evaluation or research efforts focusing on academic outcomes require seeking permission from schools for access to those measures, a procedure that can be prohibitive. Student-level data (rather than aggregate) is typically difficult to obtain from schools and districts, due to concerns about student anonymity and privacy (Family Educational Rights and Privacy Act (FERPA), 1974). Other options may include working with third party data repositories (e.g., North Carolina Education Research Data Center), but these sources are often associated with high access fees to support the infrastructure needed to maintain the databases. These types of barriers make studying academic outcomes nearly impossible for individual EE programs, but universities and associated research grants could overcome them in some instances by providing funds to access the data and assurance that the work meets ethical standards for human research to secure the necessary permissions.

Focusing on null effects for EE on academic outcomes will require minor statistical changes to some studies and a major shift in perspective. Regarding the former, most well-designed studies include statistical elements required to evaluate the likelihood of null effects, and EE scholars have already called for inclusion of these elements in future research. Specifically, rigorous research designs called for in EE studies require large samples and precise variables (i.e., low variance) to ensure high power of analysis, treatment-control experimental designs, measures of effect size in addition to p-values, and consistent metrics to evaluate across programs (Creswell, 2008; Carleton-Hug and Hug, 2010; Stern et al., 2013). These suggestions were made in the context of identifying attributes of EE that are linked to positive outcomes in the realm of environmental literacy (e.g., environmental sensitivity), but they apply equally to evaluating null effects. Statistically, it is not possible to demonstrate an effect size of zero, but studies with large sample sizes and variables measured with high precision can demonstrate effect sizes so close to zero that differences are socially meaningless (Vaske, 2008). For instance, test scores for treatment (EE) and control (non-EE) groups may have confidence intervals which overlap almost completely, potentially yielding a p-value of 0.95 or higher. Though there are multiple statistical techniques for rigorously evaluating the degree of overlap in confidence intervals and testing for no difference, the statistical testing is a relatively small challenge compared to adopting alternatives to research focused on rejecting null hypotheses (Anderson et al., 2000; Trafimow, 2013). The intense pressure in many disciplines to conduct and publish research that obtains statistically significant effects, interpretable as disconfirming alternative theories can lead to inappropriate conclusions (Trafimow, 2013), stifle creativity (Guthery et al., 2001), and encourage evaluators and stakeholders to cherry-pick data to highlight positive associations with treatments and outcomes of interest (Munafò et al., 2017). This might include a tendency to interpret pre-post-increases in academic measures as attributable to a program when no control group is included or to rely on p-hacking, wherein researchers sift through variables, report only the outcomes and drivers with significant relationships (Head et al., 2015), and, ultimately, underreport results which show no difference between non-EE and EE-based methods in terms of academic growth. A null effects framework would instead encourage more transparent reporting of research without threatening adoption of EE in schools.

Stepping outside a hypothesis testing paradigm requires practitioners, researchers, reviewers, journal editorial boards, and program funders to treat hypothesis testing as one of many ways to evaluate evidence rather than the sole way. Doing so would open the door to research focusing on null effects, rather than disconfirming alternative theories, and allow use of unusual but informative models of portraying data. Fortunately, the EE community already values other ways of evaluation beyond typical hypothesis testing. For instance, EE has a rich history in qualitative work (Palmer, 2002) and has recently begun to use methods such as photo elicitation, art, and story-telling (Flowers et al., 2015; Chanse et al., 2017; Piersol and Timmerman, 2017). A null effects framework for EE and academic outcomes provides opportunities for creative use of diverse metrics of evidence including utilizing qualitative analyses, providing basic frequencies and graphing, using equivalence tests to report null effects, and employing information theoretic approaches linked to Bayesian models (Guthery et al., 2001). One first step toward making this change could involve special journal issues focused on exploring the dynamics of EE treatments on academic outcomes, where null effects are welcome, if not required. Further, outreach efforts through organizations such as the North American Association for Environmental Education (NAAEE) or the Children & Nature Network (C&NN) could work to equip practitioners with the tools they need to clearly communicate how null effects fit into the broader narrative of how EE contributes to student growth and learning.

Why Adopt a Null Effects Framework for EE Evaluation and Research?

A primary reason to focus on null effects of EE treatments on academic outcomes is that null effects are arguably what EE practitioners are trying to achieve. Although EE practitioners and organizations argue for EE to be integrated into the fabric of K-12 teaching, the majority of EE in schools is structured as a supplement to classroom teaching (e.g., field trips, isolated activities: Ardoin et al., 2017). In our experience, EE practitioners in these contexts are not trying to beat K-12 teachers at their own game. Rather, they are working to develop mutual gains where academic or cognitive outcomes are supported while building environmental sensitivity, connection to nature, stewardship, or other outcomes related to environmental literacy. Outcomes including connection to nature are more directly related to EE programs' mission statements and theories of change, but EE practitioners recognize the need to report academic outcomes to appeal to their K-12 stakeholders. For instance, we conducted an informal poll of environmental educators in North Carolina and found that nearly all explicitly linked their programs to academic standards, but few listed academic outcomes as a primary goal of their program, and none expected to generate better classroom outcomes on concepts teachers were already teaching. Instead, practitioners commonly listed connection to nature, natural history knowledge, and environmental behaviors as program goals but identified linking to academic standards as important to communicate the worth of their program to K-12 stakeholders. These responses suggest that a null effects framework for EE and academic outcomes is congruent with practitioner conceptualizations of success.

Not only does a null effect framework better fit EE objectives, it may prevent those objectives from being warped by mission creep linked to an emphasis on standardized testing. The troubling impacts of the increased emphasis on standardized testing on teaching and learning is well documented. These include classes being dominated by test preparation (Jacob, 2002), non-tested subjects like art and social studies being deemphasized or dropped completely (Jorgenson and Vanosdall, 2002; Jones et al., 2003), teacher attrition (Jalongo and Heider, 2006), and higher student stress (Amrein and Berliner, 2002). Further, the emphasis on accountability associated with standardized tests imbeds a level of competition and one-upmanship into attending to these outcomes (e.g., schools being graded on standardized test outcomes) (Jones et al., 2003). This type of competition is arguably antithetical to EE culture (Krasny et al., 2016), and it also represents a trap in which EE programs seeking to help achieve higher academic outcomes are forced to participate in a culture that can sacrifice student learning to improve test scores. A null effects framework for EE and academic outcomes would help EE avoid slipping into this paradigm.

By communicating null effects with respect to academic outcomes, EE evaluation and research can present EE as a way to maintain academic achievement while providing a myriad of co-benefits, particularly when paired with time outdoors. Many teachers and administrators fear that EE may detract from student learning (Ernst, 2009, 2012), and to some degree, this view may even be shared by students (Carrier et al., 2014). A null effects framework for EE and academic outcomes assuages these fears to allow the focus to shift to other benefits. Some of these are congruent with outcomes of interest within formal schooling. For instance, EE has been linked to improved cognitive skills (Stevenson et al., 2013), motivation (Legault and Pelletier, 2000), self-efficacy (Barnett et al., 2006), and social-emotional learning (Carter, 2016), all of which have been highlighted as priority areas in national education legislation and standards in the United States and internationally (Breiting and Wickenberg, 2010; NRC, 2012; NGSS Lead States., 2013). Further, when EE incorporates outdoor instruction, students may realize benefits such as improved attention in and out of the classroom (Kuo et al., 2018; Szczytko et al., 2018), lower stress levels (Wells and Evans, 2003), and improved cognitive and social function (Chawla, 2015). Teachers engaging in EE may find new avenues for connecting with their students (Carrier, 2009b; Ernst, 2009), helping curb teacher attrition plaguing many schools. While not related to academic achievement, all of these benefits are likely of interest to formal educators, and when paired with an acknowledgment of test scores, environmental educators may have more of opportunity to gain access to schools. Other environmental literacy-related outcomes such as improved connection to nature (Cheng and Monroe, 2010), environmental sensitivity (Chawla, 2010), environmental behavior (Ardoin et al., 2015) may be of less direct interest to formal classroom educators but are central to the EE community.

Another reason to consider a null effects framework in academic-related EE research is that it avoids setting unrealistic expectations. Education scholars and practitioners have been perfecting classroom instruction as a tool for conveying knowledge for centuries, and an evaluation model bent on “beating” those methods with new EE curricula simply is not realistic. Further, as mentioned above, EE aims to do much more than convey knowledge. In general, EE will likely not outperform classroom teaching in improving academic outcomes as measured using traditional assessments except in special contexts (see below). EE curriculum (e.g., Project Learning Tree, Project WILD) and approaches (e.g., the NAAEE Guidelines for Excellence: NAAEE, 2010) are rooted in educational theory and research, employing constructivist approaches to teach interdisciplinary concepts (e.g., natural sciences, civic engagement) in authentic settings (e.g., the natural world) (Carrier and Stevenson, 2017). However, classroom approaches and teacher training programs have also benefitted from this same theory and research, in an arguably more focused, long-term, and larger scale manner. Specifically, many environmental educators are natural scientists trained during short-term staff training programs in theory- and research-based pedagogical techniques. These programs are often excellent but cannot compare to the pedagogical training typically required of classroom teachers, which includes a 4-year college degree focused on teaching and months of mentored student teaching. This training ensures classroom teachers are equipped to use the same techniques that make EE high-quality education (e.g., project based learning, experiential learning) (Carrier and Stevenson, 2017). Although EE programming, whether implemented through schools and teachers (e.g., Project Learning Tree activities) or through external programing (e.g., field trips), may be a great complement to what teachers are already doing, teachers following best practices of education should be expected to produce student academic outcomes as well as, or better than, EE programming. Because classroom teachers often have more training than environmental educators in pedagogical techniques, null results can be seen as a measure of a program's success in fostering student learning.

Finally, framing null results as supporting EE in K-12 settings sets up few cases of positive effects as the spectacular successes they are, rather than a mere “better than nothing” finding. Although large treatment effects across the general population are likely unrealistic, in some instances, we may see treatment effects and establishing a norm of null effects helps highlight them. For instance, Muddy Sneakers, a North Carolina EE program, organizes their program around the eight content themes as identified in state standards (e.g., energy, forces and motion, ecosystems). While students participating in the program improved science grades over a control group, overall treatment effects were small with respect to academic outcomes. However, academic gains were higher among students with cognitive, emotional, or behavioral disabilities (Szczytko et al., 2018). Similarly, a pilot evaluation project of Oregon's outdoor schools documented gains in self-assessed measures of academic achievement, but those gains were not as high as gains in non-academic areas (e.g., social-emotional learning, environmental learning) (Braun, 2019). However, the academic gains that were reported were particularly pronounced among girls, students identifying as non-binary or trans-gender, and students with substantial or special needs, particularly among measures of academic self-efficacy (Braun, 2019). Other studies have found socially meaningful differences in academic outcomes in specific curricular contexts (State Education and Environment Roundtable [SEER], 2000) or differential academic benefits for girls (Carrier, 2009a) and African American students (Lieberman and Hoody, 1998). These types of results, paired with overall null results for academic outcomes, could make a strong case that EE is not taking away from instructional time, and making progress on factors that may give a boost to those who need it most (Stevenson et al., 2017).

Conclusion

Linking EE to academic outcomes may be necessary to ensure EE is relevant in K-12 settings, which crucially ensures that EE audiences are broad and diverse. Adoption of a null effects framework for EE and academic outcomes will allow for a narrative framework that can communicate why EE is beneficial to key K-12 stakeholders, including administrators, teachers, and parents. This is particularly true in a policy setting in which funding is dedicated to interventions, such as widespread technology adoption, which have little evidence for academic learning (Kimmons, 2015) but rapidly mounting evidence for coupled downsides (e.g., higher stress: Twenge, 2017), a frame in which the EE community can advocate for adoption of EE programming in schools is warranted. We invite researchers to join in this work by including academic outcomes in their studies, reporting the null results, and beginning to shape the conversation. Similarly, we call on journals to encourage and support this work by promoting it through special issues and calls for papers. Standardized tests and grades may be poor measures of learning (Finn et al., 2014), and accordingly, uninteresting or even unappealing to EE researchers. However, they produce arguably the single most powerful data in the context of allocating resources for K-12 schooling, and the EE community would be well-served to utilize them in ways that support integration of EE into schools.

Author Contributions

KS, MP, and SC contributed to the conceptualization and writing of this manuscript. RLS, RO, and RES contributed to the conceptualization and critical editing of the writing.

Funding

This work was supported in part by Muddy Sneakers and NC Sea Grant (2016-R/16-ELWD-1).

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We would like to thank the numerous students and teachers who have participated in the research that helped shape our ideas around this topic as well as Steve Braun for his insight into the Oregon outdoor schools project.

References

107th Congress (2002). No Child Left Behind (NCLB) Act of 2001. United States.

PubMed Abstract

114th Congress (2015). Every Student Succeeds Act (ESSA). Available online at: https://www.govtrack.us/congress/bills/114/s1177

Amrein, A. L., and Berliner, D. C. (2002). High-stakes testing, uncertainty, and student learning. Educ. Policy Anal. Arch. 10, 1–74. doi: 10.14507/epaa.v10n18.2002

CrossRef Full Text

Anderson, D. R., Burnham, K. P., and Thomson, W. L. (2000). Null hypothesis testing: problems, prevalence, and an alternative. J. Wildl. Manage. 64, 912–923. doi: 10.2307/3803199

CrossRef Full Text | Google Scholar

Ardoin, N. M., Bowers, A. W., Roth, N. W., and Holthuis, N. (2017). Environmental education and K-12 student outcomes: a review and analysis of research. J. Environ. Educ. 49, 1–17. doi: 10.1080/00958964.2017.1366155

CrossRef Full Text | Google Scholar

Ardoin, N. M., Wheaton, M., Bowers, A. W., Hunt, C. A., Durham, H., Ardoin, N. M., et al. (2015). Nature-based tourism's impact on environmental knowledge, attitudes, and behavior : a review and analysis of the literature and potential future research. J. Sustain. Tour. 23, 838–858. doi: 10.1080/09669582.2015.1024258

CrossRef Full Text | Google Scholar

Aydeniz, M., and Southerland, S. A. (2012). A national survey of middle and high school science teachers' responses to standardized testing: is science being devalued in schools? J. Sci. Teacher Educ. 23, 233–257. doi: 10.1007/s10972-012-9266-3

CrossRef Full Text | Google Scholar

Ballantyne, R., Fien, J., and Packer, J. (2001). Program effectiveness in facilitating lntergenerational influence in environmental education: lessons from the field. J. Environ. Educ. 32, 8–15. doi: 10.1080/00958960109598657

CrossRef Full Text | Google Scholar

Barnett, M., Lord, C., Strauss, E., Rosca, C., Langford, H., Chavez, D., et al. (2006). Using the urban environment to engage youths in urban ecology field studies. J. Environ. Educ. 37, 3–11. doi: 10.3200/JOEE.37.2.3-11

CrossRef Full Text | Google Scholar

Boris, E. T., and Kopczynski Winkler, M. (2013). “The emergence of performance measure- ment as a complement to evaluation among U.S. foundations,” in Performance Management and Evaluation. New Directions for Evaluation, eds S. B. Nielsen and D. E. K. Hunter, 69–80.

Google Scholar

Braun, S. M. (2019). Outdoor School for All: Diverse Programming and Outcomes in Oregon: 2017 Pilot Study Evaluation (PNW XXX). Corvallis, OR: Oregon State University.

Breiting, S., and Wickenberg, P. (2010). The progressive development of environmental education in Sweden and Denmark. Environ. Educ. Res. 16, 9–37. doi: 10.1080/13504620903533221

CrossRef Full Text | Google Scholar

Carleton-Hug, A., and Hug, J. W. (2010). Challenges and opportunities for evaluating environmental education programs. Eval. Program Plann. 33, 159–164. doi: 10.1016/j.evalprogplan.2009.07.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Carrier, S. J. (2009a). Environmental education in the schoolyard: learning styles and gender. J. Environ. Educ. 40, 2–12. doi: 10.3200/JOEE.40.3.2-12

CrossRef Full Text | Google Scholar

Carrier, S. J. (2009b). The effects of outdoor science lessons with elementary school students on preservice teachers' Self-Efficacy. J. Elem. Sci. Educ. 21, 35–48. doi: 10.1007/BF03173683

CrossRef Full Text | Google Scholar

Carrier, S. J., and Stevenson, K. (2017). “Children experiencing the outdoors,” in Reforming Teaching and Teacher Education, eds E. Kimonen and R. Nevalainen (Rotterdam: SensePublishers), 67–88.

Google Scholar

Carrier, S. J., Thomson, M. M., Tugurian, L. P., and Stevenson, K. T. (2014). Elementary science education in classrooms and outdoors: stakeholder views, gender, ethnicity, and testing. Int. J. Sci. Educ. 36, 2195–2220. doi: 10.1080/09500693.2014.917342

CrossRef Full Text | Google Scholar

Carter, D. (2016). A nature-based social-emotional approach to supporting young children's holistic development in classrooms with and without walls: the Social-Emotional and Environmental Education Development (SEED) Framework. Int. J. Early Child. Environ. Educ. Int. J. Early Child. Environ. Educ. 4, 9–24.

Google Scholar

Chanse, V., Mohamed, A., Wilson, S., Dalemarre, L., Leisnham, P. T., Rockler, A., et al. (2017). New approaches to facilitate learning from youth: exploring the use of Photovoice in identifying local watershed issues. J. Environ. Educ. 48, 109–120. doi: 10.1080/00958964.2016.1256260

CrossRef Full Text | Google Scholar

Chawla, L. (1999). Life paths into effective environmental action. J. Environ. Educ. 31, 15–26. doi: 10.1080/00958969909598628

CrossRef Full Text | Google Scholar

Chawla, L. (2010). Significant life experiences revisited: a review of research on sources of environmental sensitivity. J. Environ. Educ. 29, 11–21. doi: 10.1080/00958969809599114

CrossRef Full Text | Google Scholar

Chawla, L. (2015). Benefits of nature contact for children. J. Plan. Lit. 30, 433–452. doi: 10.1177/0885412215595441

CrossRef Full Text | Google Scholar

Cheng, J. C.-H., and Monroe, M. C. (2010). Connection to nature: children's affective attitude toward nature. Environ. Behav. 44, 31–49. doi: 10.1177/0013916510385082

CrossRef Full Text | Google Scholar

Creswell, J. W. (2008). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. Thousand Oaks, CA: Sage Publications, Inc.

Darling-Hammond, L., Bae, S., Cook-Harvey, C. M., Lam, L., Mercer, C., Podolsky, A., et al. (2016). Pathways to New Accountability Through the Every Student Succeeds Act. Palo Alto, CA. Available online at: http://learningpolicyinstitute.org/our-work/publications-resources/ pathways-new-accountability-every-student-succeeds-act.%0AThis

Ernst, J. (2009). Influences on US middle school teachers' use of environment-based education. Environ. Educ. Res. 15, 71–92. doi: 10.1080/13504620802710599

CrossRef Full Text | Google Scholar

Ernst, J. (2012). Influences on and obstacles to K-12 administrators' support for environment-based education. J. Environ. Educ. 43, 73–92. doi: 10.1080/00958964.2011.602759

CrossRef Full Text | Google Scholar

Family Educational Rights and Privacy Act (FERPA) (1974). 20 U.S.C. § 1232g; 34 CFR Part 99.

Finn, A. S., Kraft, M. A., West, M. R., Leonard, J. A., Bish, C. E., Martin, R. E., et al. (2014). Cognitive skills, student achievement tests, and schools. Psychol. Sci. 25, 736–744. doi: 10.1177/0956797613516008

PubMed Abstract | CrossRef Full Text | Google Scholar

Flowers, A. A., Carroll, J. P., Green, G. T., and Larson, L. R. (2015). Using art to assess environmental education outcomes. Environ. Educ. Res. 21, 846–864. doi: 10.1080/13504622.2014.959473

CrossRef Full Text | Google Scholar

Guthery, F. S., Lusk, J. J., and Peterson, M. J. (2001). The fall of the null hypothesis: liabilities and opportunities. J. Wildl. Manage. 65, 379–384. doi: 10.2307/3803089

CrossRef Full Text | Google Scholar

Head, M. L., Holman, L., Lanfear, R., Kahn, A. T., and Jennions, M. D. (2015). The extent and consequences of p-hacking in science. PLoS Biol. 13:e1002106. doi: 10.1371/journal.pbio.1002106

PubMed Abstract | CrossRef Full Text | Google Scholar

Heimlich, J. E. (2010). Environmental education evaluation: reinterpreting education as a strategy for meeting mission. Eval. Program Plann. 33, 180–185. doi: 10.1016/j.evalprogplan.2009.07.009

PubMed Abstract | CrossRef Full Text | Google Scholar

Jacob, B. A. (2002). Accountability, Incentives and Behavior: The impact of high-stakes testing in the Chicago Public Schools. Cambridge, MA. Available online at: http://www.nber.org/papers/w8968%0ANATIONAL

Jalongo, M. R., and Heider, K. (2006). Editorial teacher attrition: an issue of national concern. Early Child. Educ. J. 33, 379–380. doi: 10.1007/s10643-006-0122-y

CrossRef Full Text | Google Scholar

Jones, M. G., Jones, B. D., and Hargrove, T. (2003). The Unintended Consequences of High Stakes Testing. Rowman and Littlefield Publishers, Inc.

Google Scholar

Jordan, C., and Chawla, L. (in press). A coordinated agenda for nature-based learning. Front. Psychol.

Jorgenson, O., and Vanosdall, R. (2002). The death of science? Phi Delta Kappan 83, 601–605. doi: 10.1177/003172170208300808

CrossRef Full Text | Google Scholar

Kimmons, R. (2015). Online system adoption and K-12 academic outcomes. J. Comput. Assist. Learn. 31, 378–391. doi: 10.1111/jcal.12101

CrossRef Full Text | Google Scholar

Krasny, M. E., Mukute, M., Aguilar, O., Masilela, M. P., and Olvitt, L. (2016). “Community Environmental Education,” in Essays in Urban Environmental Education, eds M. E. Krasny and A. Kudryavtsev (Ithica, NY: Cornell University Press), 18–26.

Kuo, M., Browning, M. H. E. M., and Penner, M. L. (2018). Do lessons in nature boost subsequent classroom engagement? Refueling students in flight. Front. Psychol. 8:2253. doi: 10.3389/fpsyg.2017.02253

PubMed Abstract | CrossRef Full Text | Google Scholar

Legault, L., and Pelletier, L. G. (2000). Impact of environmental education program on students' and parents' attitudes, motivations and behaviours. Can. J. Behav. Sci. 32, 243–250. doi: 10.1037/h0087121

CrossRef Full Text | Google Scholar

Lieberman, G. A., and Hoody, L. L. (1998). Closing the Achievement Gap: Using the Environment as an Integrating Context for Learning. Results of a Nationwide Study. San Diego, CA.

Google Scholar

Monroe, M. C., and Krasny, M. E. (eds). (2016). Across the Spectrum: Recources for Environmental Educators, 3rd Edn. North American Association for Environmental Education

Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie Du Sert, N., et al. (2017). A manifesto for reproducible science. Nat. Hum. Behav. 1, 1–9. doi: 10.1038/s41562-016-0021

CrossRef Full Text

NAAEE (2010). Excellence in Environmental Education: Guidelines for Learning (K.-12). Washington, DC: North American Association of Environmental Education.

NGSS Lead States. (2013). Next Generation Science Standards: For States, By States. Washington, DC: The National Academies Press.

PubMed Abstract

NRC (2012). A Framework for K-12 Science Education: Practices, Crosscutting Concepts, an Core Ideas. Washington, DC: National Academies Press.

Palmer, J. A. (2002). Environmental Education in the 21st Century: Theory, Practice, Progress and Promise. New York, NY: Routledge.

Google Scholar

Piersol, L., and Timmerman, N. (2017). Reimagining environmental education within academia: storytelling and dialogue as lived ecofeminist politics. J. Environ. Educ. 48, 10–17. doi: 10.1080/00958964.2016.1249329

CrossRef Full Text | Google Scholar

Roth, R. E. (1969). Fundamental concepts for environmental management education (K-16). Environ. Educ. 1, 65–74. doi: 10.1080/00139254.1969.10801495

CrossRef Full Text | Google Scholar

Stapp, W. B. (1969). The concept of environmental education. J. Environ. Educ. 1, 30–31. doi: 10.1080/00139254.1969.10801479

CrossRef Full Text | Google Scholar

State Education Environment Roundtable [SEER] (2000). The Effects of Environment-Based Education on Student Achievement. San Diego, CA. Available online at: http://www.seer.org/pages/research/educeff.pdf

Stern, M. J., Powell, R. B., and Hill, D. (2013). Environmental education program evaluation in the new millennium: what do we measure and what have we learned? Environ. Educ. Res. 20, 581–611. doi: 10.1080/13504622.2013.838749

CrossRef Full Text | Google Scholar

Stevenson, K. T., Carrier, S. J., and Peterson, M. N. (2014). Evaluating strategies for inclusion of environmental literacy in the elementary school classroom. Electron. J. Sci. Educ. 18.

Google Scholar

Stevenson, K. T., Peterson, M. N., Bondell, H. D., Mertig, A. G., and Moore, S. E. (2013). Environmental, institutional, and demographic predictors of environmental literacy among middle school children. PLoS ONE 8:e59519. doi: 10.1371/journal.pone.0059519

PubMed Abstract | CrossRef Full Text | Google Scholar

Stevenson, K. T., Peterson, M. N., and Dunn, R. R. (2017). Leveraging natural capital to solve the shared education and conservation crisis. Conserv. Biol. 32, 490–492. doi: 10.1111/cobi.13039

PubMed Abstract | CrossRef Full Text | Google Scholar

Szczytko, R., Carrier, S. J., and Stevenson, K. T. (2018). Impacts of outdoor environmental education on teacher reports of attention, behavior, and learning outcomes for students with emotional, cognitive, and behavioral disabilities. Front. Educ. 3:46. doi: 10.3389/feduc.2018.00046

CrossRef Full Text | Google Scholar

Tienken, C. H., Colella, A., Angelillo, C., Fox, M., McCahill, K. R., and Wolfe, A. (2017). Predicting middle level state standardized test results using family and community demographic data. RMLE Online 40, 1–13. doi: 10.1080/19404476.2016.1252304

CrossRef Full Text | Google Scholar

Trafimow, D. (2013). To disconfirm or not to disconfirm: a null prediction vs. no prediction. Front. Psychol. 4:733. doi: 10.3389/fpsyg.2013.00733

PubMed Abstract | CrossRef Full Text | Google Scholar

Twenge, J. M. (2017). iGen: Why Today's Super-Connected kids are Growing up Less Rebellious, More Tolerant, Less Happy: and Completely Unprepared for Adulthood. New York, NY: ATRIA books.

Google Scholar

UNESCO (1977). “The Tbilisi declaration,”in Intergovernmental Conference on Environmental Education (Tbilisi: USSR), 1–8.

US Department of Education (1983). A Nation at Risk: The Imperative for Educational Reform.

Vaske, J. J. (2008). Survey Research and Analysis: Applications in Parks, Recreation and Human Dimensions. State College, PA: Venture Publishing.

Wells, N. M., and Evans, G. W. (2003). Nearby nature: a buffer of life stress among rural children. Environ. Behav. 35, 311–330. doi: 10.1177/0013916503035003001

CrossRef Full Text | Google Scholar

Wells, N. M., and Lekies, K. S. (2006). Nature and the life course: pathways from childhood nature experiences. Child. Youth Environ. 16, 1–25. Available online at: http://www.jstor.org/stable/10.7721/chilyoutenvi.16.1.0001

Google Scholar