Bobby Habig
Franny Geller3,4
Preeti Gupta
Jennifer D. Adams
Mandë Holford- 1Department of Biology, Mercy University, Bronx, NY, United States
- 2American Museum of Natural History, New York, NY, United States
- 3Research Foundation of The City University of New York, New York, NY, United States
- 4The Graduate Center, City University of New York, New York, NY, United States
- 5University of Calgary, Calgary, AB, Canada
- 6Department of Organismic and Evolutionary Biology, Harvard University, New York, NY, United States
A significant challenge is to identify strategies that motivate youth to consider a STEM career pathway. For many young people, the first barrier to entry is lack of awareness of STEM majors and careers. Promisingly, informal STEM programs are potential vehicles for addressing issues of underrepresentation and for bolstering youths’ knowledge and understanding of STEM career pathways. The goal of this study was to conduct a meta-synthesis of research on the outcomes of participants of informal STEM programs. We used a mixed methods approach to address three research questions: (1) How and to what extent do informal STEM learning experiences impact K-12 participants’ knowledge and understanding of STEM career pathways and careers? (2) What are program design principles, effective practices, and technological innovations of rigorously designed studies that exhibit strong or exemplary evidence of impact? (3) What strategies work most effectively in underserved populations? Across the informal STEM education landscape, we found that many studies exhibited weak or adequate research design. However, among rigorously designed studies, we found a significant positive association between participation in informal STEM programs and increased STEM career awareness (Cohen’s d = 0.584, 95% CI: 0.295–0.872, p < 0.001). Our meta-syntheses revealed that informal STEM programs that target high school students, girls, or members of historically underrepresented groups, and that focus on career exploration and experiential learning were most effective at fostering STEM career awareness. Our results suggest that informal STEM institutions play an unappreciated role in ensuring that youth are introduced to potential future careers. Critically, our findings can be used by practitioners to inform program design and confront issues of underrepresentation.
Introduction
Early awareness of a range of specific STEM fields is an important factor influencing STEM career engagement (van Tuijl and Molen, 2016). As articulated by Dorsen et al. (2006): “Young people cannot choose a specific STEM career or field of study if they do not know of its existence.” Indeed, lack of awareness of STEM majors and careers is considered the first barrier to entry for certain STEM disciplines (Dorsen et al., 2006; van Tuijl and Molen, 2016; Tillinghast and Mansouri, 2020). To address this issue, informal STEM institutions, including museums, science centers, universities, aquaria, botanical gardens, and zoological parks, can play an important role in increasing young people’s awareness of a multitude of STEM career opportunities (Blanchard et al., 2020).
There are several reasons why informal STEM education programs are especially well-suited for expanding youths’ realm of career possibilities. First, informal STEM education programs are voluntary, and are often characterized by inquiry-based learning experiences that emphasize choice learning (National Research Council, 2009; Allen and Peterman, 2019); these settings have the potential to provide experiences that allow for exploration in an environment free from the performance demands of school (Braund and Reiss, 2006; Adams et al., 2012; Habig and Gupta, 2021). While many formal education institutions are effective at exposing students to traditional STEM careers (e.g., health science and medicine), most public and private schools are limited by bureaucratic restrictions, including a rigid curriculum and preparation for standardized examinations (Adams et al., 2012). Second, many informal STEM institutions have unique infrastructures and research programs that make them especially amenable to promoting STEM major and STEM career awareness beyond the scope of a formal education. For example, Project TRUE, facilitated by Fordham University and the Wildlife Conservation Society, provides research experiences that expose high school students to careers in ecology and conservation (Aloisio et al., 2018). The High School Aerospace Scholars Program, facilitated by the NASA Johnson Space Center, affords students opportunities to meet NASA scientists and to gain knowledge of space science careers, including NASA astronaut, engineer, and mission controller (Demirci, 2018). The Lang Science Program, a multi-year program for middle school and high school students facilitated by the American Museum of Natural History, introduces students to a variety of STEM careers, including but not limited to astrophysics, entomology, geology, ornithology, paleontology, and physical anthropology (Habig et al., 2020). Finally, informal STEM learning programs are thought to be important vehicles for confronting issues of underrepresentation (Falk and Dierking, 2010; National Research Council, 2015; Habig et al., 2021). With an estimated 2,500 STEM institutions nationwide, museums, science centers, zoos, and other cultural and educational institutions present a unique opportunity to reach out to individuals who have felt alienated by the sciences (Phillips et al., 2007; National Research Council, 2015). To confront this challenge, many informal STEM programs have made concerted efforts to offer experiences that specifically target girls and members of historically marginalized racial and ethnic groups (e.g., Chi et al., 2010; McCreedy and Dierking, 2013). Hence, the youth activities offered by informal STEM institutions exhibit program design principles, pedagogical practices, and unique infrastructures that make them especially well-suited for promoting STEM major and STEM career knowledge outside the bounds of a traditional education.
Although many informal STEM institutions have developed programming specifically designed to foster STEM major and STEM career awareness (e.g., McCreedy and Dierking, 2013; Phelan et al., 2017; Soto-Lara et al., 2022), it is often quite challenging to gauge the effectiveness of these initiatives (Institute for Learning Innovation, 2007; Allen et al., 2019; Habig, 2020). This is largely because many studies and evaluations of informal STEM programs vary considerably in research design quality (Institute for Learning Innovation, 2007; Fu et al., 2016). To address this gap in knowledge, several validated rubrics have been developed specifically for informal STEM programs to assess research design quality and evidence of impact (Habig, 2020). With the help of these tools, it is now possible to quantify how and to what extent informal STEM programs impact participants’ knowledge of STEM majors and STEM careers.
While it is crucial to assess whether informal STEM programs are effective at fostering STEM major and STEM career awareness, it is also important to understand what programmatic features and pedagogical practices are most effective in facilitating these outcomes (Habig, 2020) so that STEM educators can adopt practices proven effective. Furthermore, in alignment with National Research Council (2013), it is critical to determine outcomes centered on ethnicity, gender, language status, race, and socioeconomic status in order to forefront the lived experiences, voices, and standpoints of historically underrepresented groups. In the context of this study, we hypothesize that not all practices work universally and that specific strategies are needed to foster STEM major and STEM career awareness for individuals who are members of groups historically marginalized from STEM fields. For example, data suggest that boys’ interest in pursuing a STEM pathway tends to persist while girls’ interest tends to decline throughout adolescence and beyond (Sadler et al., 2012; Shapiro et al., 2015; Wigfield et al., 2015; Holmes et al., 2018). Additionally, girls from low socioeconomic backgrounds express more limited STEM career aspirations than girls from high socioeconomic backgrounds (Aschbacher et al., 2014; Mau and Li, 2018). Furthermore, African American girls report feeling less welcome in STEM subjects than White girls (Hanson, 2007; Dortch and Patel, 2017), and some Latina girls report feeling pressure to exit STEM trajectories to pursue more traditional roles (Aschbacher et al., 2010; Gonzalez, 2020). Consequently, gender gaps continue to persist in certain STEM fields, notably in computer science, engineering, and physics (Cimpian et al., 2020; Perez-Felkner et al., 2024). Given these findings, a synthesis of the literature is essential for identifying factors that are most effective for maximizing impact and cultivating STEM major and STEM career awareness across different demographic groups.
The goal of this study was to perform a quantitative assessment and qualitative synthesis of research on the outcomes of K-12 participants in informal STEM programs. Specifically, we addressed the following research questions: (1) How and to what extent do informal STEM learning experiences impact K-12 participants’ knowledge and understanding of STEM career pathways and careers (STEM major awareness and STEM career awareness)? (2) What are program design principles, target audiences, effective practices, and technological innovations of rigorously designed studies of programs that exhibit strong or exemplary evidence of impact? (3) What are program design principles, effective practices, and technological innovations proven to work effectively in underserved and underrepresented populations? To answer our research questions, we conducted a landscape analysis and meta-synthesis of peer-reviewed K-12 studies, conference proceedings, and evaluations of informal STEM learning programs. From our systematic review, we extracted studies that measured the impact of informal STEM programs on participants’ STEM major awareness and STEM career awareness, and from these extracted studies, we used validated rubrics to quantify research design quality and evidence of impact (Habig, 2020). Finally, from a subset of rigorous studies that exhibited strong or exemplary evidence of impact, we extracted program design principles, effective practices, and technological innovations that can be adopted by STEM practitioners to maximize impact and broaden participation of girls and members of historically marginalized racial and ethnic groups. Because the pursuit of STEM pathways can lead to many positive outcomes, including the opportunity to earn a sustainable living and contribute to society (e.g., Carnevale et al., 2011; Venville et al., 2013), knowing what factors strengthen participants’ knowledge and understanding of STEM career pathways and careers is an important step to ensure that youth have access to high quality STEM education and opportunities.
Theoretical frameworks
Several theoretical frameworks have been developed to address how individuals navigate their career trajectories. Three theoretical frameworks applicable to this review are: (1) Possible Selves Theory (Markus and Nurius, 1986); (2) Social Cognitive Career Theory (Lent et al., 1994); and (3) Expectancy Value Theory (Wigfield and Eccles, 2000). Many of the studies we discuss in this synthesis were grounded in one or more of these theories. We briefly summarize each of the three theories below.
Possible selves theory
Possible selves are conceptions of what youth imagine themselves in the future (Markus and Nurius, 1986). As youth examine future-oriented identities, some “possible selves” are regarded as more plausible than others (Oyserman et al., 1995). Markus and Nurius (1986) note that a young person might explore a range of future-oriented identities, but “the pool of possible selves derives from the categories made salient by the individual’s particular sociocultural and historical context” (p. 954). Notably, “the pool of possible selves” is largely influenced by a young person’s social experiences. In an informal STEM learning context, this means that exposure to different STEM careers is one way for young people to “try on” possible selves and make decisions on whether or not to pursue a particular STEM pathway (Dorsen et al., 2006). These experiences are thought to be valuable for facilitating knowledge of STEM majors and STEM careers and sustained engagement in STEM pathways (Kelly et al., 2013; Habig et al., 2020).
Social cognitive career theory
Many studies of informal STEM programs are grounded in Social Cognitive Career Theory (Lent et al., 1994). Social Cognitive Career Theory, which originates from Social Cognitive Theory (Bandura, 1991), is a conceptual framework that explains how young people develop and enact academic and career choices (Lent et al., 1994). Social Cognitive Career Theory is comprised of three core concepts: (1) self-efficacy (Bandura, 1977) (self-assurance an individual has in their capacity to succeed); (2) outcome expectations (an individual’s belief about the end results of their efforts); and (3) career goals (aspirations to achieve a future outcome in a specific professional discipline). The theory is grounded in the idea that early engagement in subjects or disciplines that increase students’ self-efficacy are predictive of future career interest. For example, young people who engage in stimulating informal STEM learning experiences, such as participation in science research experiences or engineering design work to solve community problems, are more likely to develop interest in STEM majors and STEM careers (Bicer and Lee, 2019). Likewise, a student exposed to computer science at a young age is more likely to seek out technology-related activities, including afterschool clubs and robotics camps, or advanced placement classes (Bicer et al., 2018). The probability that a young person will persist in a specific career trajectory increases when these informal learning experiences increase self-efficacy and a sense of competence (Lent et al., 1994; Diegelman and Subich, 2001; Fox and Cater, 2015).
Expectancy-value theory
Expectancy-Value Theory was first advanced in the field of education by Eccles (1983, 1994) and further refined over the past several decades (e.g., Wigfield and Eccles, 2000; Simpkins et al., 2006; Eccles, 2009; Wigfield et al., 2009). Expectancy-Value Theory is a framework for helping educators understand what motivates youth, or more specifically, what is important to them, what they value, and what they want to do with their lives (Eccles, 2009). According to Expectancy-Value Theory, three key constructs, (1) ability beliefs, (2) expectancies for success, and (3) subjective values, directly impact young people’s educational and career pathways (Wigfield and Eccles, 2000). In this framework, “ability beliefs” refer to an individual’s perception of their competence. “Expectancies” describe an individual’s confidence in their ability to succeed. “Subjective values” refer to the motivation orientations of an individual. Wigfield and Eccles (2000) describe four subjective values predicted to influence academic and career choices: (1) attainment value (perceived importance to the individual), (2) intrinsic value (perceived interest or enjoyment), (3) utility value (perceived relevance or usefulness), and (4) cost (perceived effort demands and psychological tolls). Research centered on Expectancy Value Theory has been instrumental as a guiding framework for studies of STEM major and STEM career awareness and for addressing issues of gender disparity in certain STEM disciplines (e.g., Fadigan and Hammrich, 2004; Simpkins et al., 2006; Wang and Degol, 2013). Eccles (1994) found that girls need both an expectancy of success and a sense that the field of study has value when considering whether to pursue a specific STEM career pathway. Notably, informal STEM programs that adopt an expectancy-value framework, that is, those that enact practices that reduce stereotypes, increase competence, and communicate the value of a STEM discipline, can influence STEM career awareness and sustained engagement (Dorsen et al., 2006; van Tuijl and Molen, 2016).
Methods
Systematic literature review
We conducted a systematic literature review based on a search and inclusion process for meta-analyses (Belland et al., 2017; Figure 1). We searched peer-reviewed journal articles, publicly available program evaluations, conference proceedings, and published books to identify studies that assessed the impact of informal STEM programs on STEM major and STEM career awareness. Following Crane et al. (1994), we defined an informal STEM program as one that involves voluntary participation during out-of-school time hours. This definition encompasses many settings, including but not limited to museums, zoos, universities, industries, non-profit organizations, and remote learning. The two outcomes of interest were defined as follows: (1) STEM major awareness: knowledge and understanding of STEM career pathways; (2) STEM career awareness: knowledge and understanding of STEM professions (Habig, 2020).
Figure 1. Search and inclusion process for meta-analysis; n refers to number of studies. The meta-analysis included 18 studies with 36 associated outcomes.
We conducted a primary search using three databases: (1) Academic Search Premier; (2) Google Scholar; and (3) Web of Science. From these databases, we used all possible pairwise combinations of two search terms, one each from either: (1) informal education, museum education, out-of-school program; AND (2) major, courses, career, attitude, awareness, interest, STEM, science, technology, engineering, mathematics. Each of our search parameter combinations, the number of abstracts reviewed based on each combination of search terms, and the number of candidate papers extracted based on each combination of these terms are summarized in Supplementary Table S1. To standardize our search, in cases in which a search parameter combination exceeded 1,000 records, we stopped reviewing papers if there were 50 consecutive papers that did not meet the inclusion criteria. Furthermore, we searched for literature directly from organizations that curate papers on informal STEM education including the (1) American Society of Engineering Education; (2) Frontiers in Education; (3) Advances in Engineering Education; (4) Association for Computing Machinery; and (5) online repositories (e.g., InformalScience.org; Harvard Family Research Project).
To be included in the systematic review, studies had to be published between January 1, 1980, and December 31, 2023. We accepted all K-12 studies that assessed two outcomes of youth participants in informal STEM education programs: (1) STEM major awareness and (2) STEM career awareness. Studies that assessed adults were excluded from the review with one exception: We included retrospective studies of K-12 informal STEM programs that assessed STEM outcomes of alumni. We also excluded studies of summer bridge programs for graduating high school seniors. To reduce heterogeneity, we only included studies of informal STEM programs that took place in the United States. During the review process, we completed a supplementary search of the reference pages of all accepted papers and reviewed the citations of relevant books, review articles, and papers to identify additional studies that may not have been available in the electronic database search.
During the literature review, we extracted the following information: (1) citation information; (2) name of program; (3) organization; (4) type of informal STEM education program (e.g., aquarium, botanical garden, museum, university, zoo, etc.); (5) program time frame (e.g., summer, after school, longitudinal [i.e., greater than 1 year in duration]); (6) contact hours; (7) type of paper (e.g., peer-reviewed journal article, conference proceeding, evaluation) (8) general description of each study; (9) theoretical framework; (10) summary of theoretical framework; (11) type of analysis (qualitative; quantitative; mixed methods); (12) grade level of program participants (e.g., elementary school [K-5], middle school [6–8], high school [9–12]); (12) demographic characteristics of program participants (e.g., gender; ethnic/racial background); (13) outcome of interest (STEM major or STEM career awareness); (14) sample size; and (15) summary of results. We included conference proceeding articles in our systematic review because most research of informal STEM programs is published exclusively in the proceedings of their respective societies. For example, the American Society of Engineering Education and the Association for Computing Machinery conference publications are the primary venues for publishing refereed research results on engineering education and computer science education (Chowdhury et al., 2019; Lunn et al., 2021). We also included program evaluations, dissertations, and white papers in our systematic review because all studies in this review were assessed for research design quality (see next section). If there was a study or studies that were not rigorously designed, they were excluded in subsequent rounds of analysis.
Quantitative assessment of research design quality
We conducted a quantitative analysis of research design quality by using a STEM Research Design Rubric specifically developed for assessing research design quality of quantitative, qualitative, and mixed methods studies of informal STEM education programs (Habig, 2020). This research design rubric is informed by the Theory of Impact Analysis (Mohr, 1995) and proposes a hierarchy of study designs ranging from (1) experimental designs (randomized controlled trials), (2) quasi-experimental designs (well-matched comparison groups), and (3) other designs (e.g., pre/post studies, comparison groups without careful matching) (U.S. Department of Education, 2007). According to this framework, a well-designed randomized controlled trial is the preferred research design; quasi-experimental designs are preferred when experimental designs are not feasible, and other designs are considered when the first two designs are not feasible. The STEM Research Design Rubric allowed us to evaluate quantitative, qualitative, and mixed methods studies on a 4-point scale: 1 = weak research design; 2 = adequate research design; 3 = strong research design; and 4 = exemplary research design. We used these tools to provide a quantitative rating (i.e., mean rubric score) of research design quality across the K-12 informal STEM education landscape for studies of STEM major awareness and STEM career awareness. Briefly, a quantitative study was coded as exhibiting exemplary research design if it included an experimental or quasi-experimental research design, was grounded in a theoretical framework, and included a minimum sample size of 50 subjects per group (Habig, 2020). Likewise, a qualitative study was coded as exhibiting exemplary research design if it included a random or quasi-experimental design (i.e., a qualitative comparison group), was grounded in a theoretical framework, and consisted of a minimum of 20 participants per group (Habig, 2020). The inclusion of a qualitative comparison group in a qualitative study enhances research rigor as it enables investigators to assess similarities and differences between participants and non-participants (Lindsay, 2019). For example, Cheung et al. (2019) conducted a qualitative study of resilience among rising college students using a quasi-experimental design, which included interviews with both a treatment group (students from foster care) and a comparison group. Criteria for strong research design, adequate research design, and weak research design are described in detail in Habig (2020). Notably, this rigidly structured process allowed us to exclude weak and adequately designed studies from our next round of analysis.
Evidence of impact
We used a STEM Impact Rubric to assess evidence of STEM outcomes (i.e., STEM major awareness and STEM career awareness) (Habig, 2020). The STEM Impact Rubric, which is informed by the Theory of Impact Analysis (Mohr, 1995) and the Theory of Triangulation (Denzin, 1970; Ammenwerth et al., 2003), allowed us to consider multiple sources of data, both quantitative and qualitative, when assessing evidence of impact. We evaluated evidence of outcome (i.e., STEM major awareness and STEM career awareness) on a 4-point scale: 1 = little or no evidence of impact; 2 = moderate evidence of impact; 3 = strong evidence of impact; and 4 = exemplary evidence of impact. Briefly, a study was coded as exhibiting exemplary evidence of impact if there was a statistically significant difference between the treatment (program participants) and comparison groups (Habig, 2020). Additionally, a qualitative study was coded as exhibiting exemplary evidence of impact if the outcome of interest was identified as an emerging theme in the treatment group (Habig, 2020). Criteria for strong evidence of impact, moderate evidence of impact, and little or no evidence of impact are described in detail in Habig (2020).
Quantitative meta-analyses
For a subset of rigorously designed quantitative studies, we conducted meta-analyses to compare (1) STEM major and STEM career outcomes (STEM major awareness and STEM career awareness) between informal STEM program participants and comparison students (experimental designs; quasi-experimental designs) and (2) STEM major and STEM career outcomes among program participants before and after participation in an informal STEM program (pre-test, post-test design). The inclusion criteria for the quantitative meta-analyses were: (1) quantitative K-12 studies of informal STEM programs published between January 1, 1980 to December 31, 2023; (2) studies that assessed STEM major and STEM career awareness; (3) studies exhibiting exemplary research design (a quantitative comparison between program participants and comparison students) or strong research design (pre-test, post-test design); and (4) studies that provided enough information to calculate an effect size. Studies were excluded if they did not meet criteria for meta-analyses (e.g., insufficient information for calculation of effect sizes; did not meet criteria for strong or exemplary research design; see Table 1).
Table 1. Best supported model of predictors of STEM career awareness based on averaging of parameter estimates (conditional average).†
Meta-analyses were conducted using the “rma.mv” function in the metafor package (Viechtbauer, 2010) in R (R Core Team, 2023). We used both the esc package (Lüdecke and Lüdecke, 2022) and an effect size calculator (Wilson, 2023) to calculate effect sizes into a standardized mean difference, Cohen’s d (Cohen, 1988). We included study ID as a random effect to control for non-independence among data points. Following the recommendations of Morris and Deshon (2002), we calculated effect sizes for two types of study design: (1) independent design and (2) single group pre-post design. For independent study designs (i.e., quasi-experimental), we determined standardized mean differences between comparison and treatment groups. For single-group study designs (i.e., pre-test, post-test), we determined Cohen’s d by calculating the mean difference between the pre-test and post-test groups, and then by dividing this result by the pooled standard deviation (Roberts et al., 2017). To determine significance, we used random effects models and calculated the 95% confidence intervals surrounding d (Thompson, 2002). We conducted heterogeneity analyses using the I2 statistic, which allowed us to assess whether the effect sizes varied more than what is expected from sampling error (Higgins and Thompson, 2002). We assessed for publication bias visually using funnel plots (Sterne and Harbord, 2004; Lin, 2019) and quantitatively by conducting Egger’s tests (Egger et al., 1997). We also used the trim and fill method to assess the significance of publication bias and to determine whether our analyses required bias-adjusted results (Duval and Tweedie, 2000a,b; Shi and Lin, 2019).
Multiple linear regression
Using the same inclusion criteria as meta-analyses, multiple linear regressions were conducted using the “lmer” function in the lmerTest package (Kuznetsova et al., 2017) in R (R Core Team, 2023). We used a mixed modeling approach to test for predictors of STEM career awareness (Supplementary Table S2). We included the standardized mean difference of each study as a response variable. We included the following moderators in our analyses: (1) study design (independent design or single group pre-post); (2) type of informal STEM program (university or informal STEM institution); (3) program time frame (short term [<1 year] or longitudinal [>1 year]); (4) grade level of program participants (middle school [6–8] or high school [9–12]); (5) gender (female-only; all genders); and (6) underrepresented (focus on historically underrepresented groups based on ethnicity, socioeconomic status, or race; no focus on historically underrepresented groups). For grade level, if a study included a combination of middle and high school participants, we coded the study as “middle school.” The six moderators described above were included as predictor variables, and we modeled study ID as a random effect to control for nonindependence among data points (equation: Y ~ study design + informal program type + program time frame + grade level + gender + underrepresented + 1 | study ID where Y is the standardized mean difference of each study of STEM career awareness). Before performing multiple regression analyses, we used the car package to calculate generalized variance inflation factors (GVIFs), which allowed for testing of multicollinearity (Fox and Weisberg, 2018). We found no evidence of problematic multicollinearity as all VIFs were <2.5, which was well below the cutoff of five (Sheather, 2009). For model selection, we used the muMin package (Bartoń, 2020), which allowed for testing of all possible parameter combinations. We used the “model.avg” function to average all models with an AICc difference <2 using the summed weight method (Burnham and Anderson, 2004), and we calculated model coefficients using conditional R2 (Nakagawa and Schielzeth, 2013).
Qualitative analyses
We conducted qualitative analyses of studies that exhibited strong or exemplary evidence of research design quality and evidence of impact. From these studies, we extracted the following information: (1) program design principles; (2) effective practices; (3) target audiences; and (4) technological innovations. We defined program design principles as specific curricular or programmatic features of an informal STEM education program applied to achieve programmatic goals (Laursen et al., 2007; Baldwin et al., 2015). Some examples of program design principles might include the development of research apprenticeships, STEM competitions, and college readiness workshops. We defined effective practices as specific approaches that are effective in improving the cognitive and social–emotional outcomes of informal STEM participants (Ellis et al., 2018). Examples of effective practices might include inquiry-based learning, engaging in discourse and practices that embody the research process, and incorporating real-world applications in the learning process. We defined target audiences as specific groups of participants based on identifiable demographic characteristics (Struminger et al., 2018). Examples of target audiences might include programs that specifically focus on middle school students, programs that focus on girls, and/or programs that focus on historically underrepresented groups. Finally, we defined technology-based innovations as cutting-edge technologies not previously (or rarely) used in STEM educational settings and the innovative use of existing technology. Examples might include innovative artificial intelligence projects, immersive extended reality experiences, such as virtual reality or augmented reality, environmental DNA (eDNA) ecology projects, and bioinformatics. During our qualitative content analysis of the informal STEM literature, we independently reviewed each manuscript to identify specific program design principles, effective practices, target audiences, and technological innovations linked to STEM major and STEM career awareness. Through a collaborative process, we identified common themes centered around our four categories of interest.
Validity
As recommended by Creswell and Poth (2018), we applied triangulation methods to ensure the validation of our findings by making use of multiple and different: (1) sources, (2) methods, (3) investigators, and (4) theories to corroborate our findings. Multiple sources included the systematic review and extraction of studies from different scholarly publications including journal articles, evaluations, and conference proceedings. In terms of multiple methods, we applied a mixed methods approach incorporating both quantitative analyses and qualitative syntheses to validate our findings. We also made use of multiple investigators to independently evaluate and synthesize sources of quantitative and qualitative data. Finally, our study is grounded in multiple theories that were uncovered from the systematic review of the informal STEM literature.
Reliability
To assess consistency across coders, we tested for reliability for both quantitative and qualitative analyses using two methodologies (Supplementary Table S3): (1) percent agreement (Lombard et al., 2002) and (2) Cohen’s kappa (k) (Cohen, 1968). For quantitative analyses (research design quality; evidence of impact), we tested for reliability between two primary coders [BH, FG] and for qualitative analyses (program design principles, effective practices, target audiences, and technological innovations), we tested for reliability between four coders [BH, FG, and two undergraduate research assistants]. For quantitative analyses, percent agreement for research design quality ranged from 97.0 to 100% and Cohen’s kappa ranged from 0.98 (near perfect agreement) to 1 (perfect agreement) (Landis and Koch, 1977). For evidence of impact, percent agreement ranged from 86.8 to 88.9% and Cohen’s kappa ranged from 0.88 to 0.90 (both indicative of near perfect agreement) (Landis and Koch, 1977). For qualitative analyses, percent agreement ranged from 85.1 to 100% and Cohen’s kappa ranged from 0.63 (substantial agreement) to 1 (perfect agreement) (Landis and Koch, 1977).
Results and Discussion
Studies of STEM major awareness across the informal STEM education landscape
We identified 27 studies spanning from 1997 to 2021 that assessed the impact of informal STEM programs on STEM major awareness. Of the 27 studies of STEM major awareness, 14 (51.9%) were peer reviewed journal articles, 10 (37.0%) were conference proceeding articles, two (7.4%) were dissertations, and one (3.7%) was a program evaluation. Six (22.2%) of the studies of STEM major awareness were qualitative, 15 (55.6%) were quantitative, and six (22.2%) applied a mixed methods approach.
Studies of STEM career awareness across the informal STEM education landscape
We identified 266 studies spanning from 1980 to 2021 that assessed the impact of informal STEM programs on STEM career awareness (Supplementary Table S4). Of the 266 studies of STEM career awareness, 134 (50.4%) were conference proceeding articles, 78 (29.3%) were peer reviewed journal articles, 43 (16.2%) were evaluations, and eight (3.0%) were dissertation or thesis chapters. The remaining three studies were extracted from a book chapter (n = 1; 0.4%), a conference poster (n = 1; 0.4%), and a white paper (n = 1; 0.4%). Twenty-two (8.3%) studies of STEM career awareness were qualitative, 147 (55.3%) were quantitative, and 97 (36.5%) employed a mixed methods approach.
Quantitative analyses
Research design quality of studies of STEM major and STEM career awareness varies across the informal STEM education landscape
Studies of STEM major awareness and STEM career awareness varied in research design quality, with most studies exhibiting adequate or weak research design. The average research design rubric score for studies of STEM major awareness was 1.85 ± 0.60 (n = 28) and the average research design rubric score for studies of STEM career awareness was 1.95 ± 0.69 (n = 266), both indicative of adequate research design. Research design quality for STEM major awareness was slightly higher for studies published in peer reviewed journals (mean = 2.00 ± 0.55, n = 14) than studies published in conference proceedings (mean = 1.60 ± 0.52, n = 10). Likewise, research design quality for STEM career awareness was slightly higher for studies published in peer reviewed journals (mean = 2.15 ± 0.71, n = 78) than studies published in conference proceedings (1.78 ± 0.63, n = 134).
Strikingly, only three of 28 studies of STEM major awareness and only 29 of 266 studies of STEM career awareness exhibited strong or exemplary research design. Of the 28 studies of STEM major awareness, three studies (two peer-reviewed journal articles and one dissertation) exhibited strong research design and zero studies exhibited exemplary research design. Among the 266 studies of STEM career awareness, 27 studies (14 peer-reviewed journal articles, seven conference proceedings, three evaluations, and three dissertations) exhibited strong research design and two studies (one peer-reviewed journal article, one evaluation) exhibited exemplary research design. The mean research design rubric score for these 29 studies was 3.07 ± 0.26.
Evidence of impact of studies of STEM major and STEM career awareness varies across the informal STEM education landscape
Evidence of impact varied among studies of informal STEM programs that assessed STEM major awareness and STEM career awareness. On average, we found that studies of informal STEM programs provided adequate evidence of impact. Specifically, the mean evidence of impact rubric score for studies of STEM major awareness was 2.39 ± 0.80 (n = 28) and for studies of STEM career awareness, the mean rubric score was 2.45 ± 0.99 (n = 266). Evidence of impact for STEM major awareness was slightly higher for studies published in peer reviewed journals (mean = 2.67 ± 0.61, n = 14) than studies published in conference proceedings (mean = 2.22 ± 0.61, n = 10). Conversely, evidence of impact for STEM career awareness was slightly lower for studies published in peer reviewed journals (mean = 2.41 ± 0.78, n = 78) than studies published in conference proceedings (2.55 ± 0.76, n = 134). All three of the studies of STEM major awareness that exhibited strong research design also exhibited strong evidence of impact (rubric score = 3). For the 29 studies of STEM career awareness that exhibited strong or exemplary research design, the mean rubric score for evidence of impact was 3.03 ± 0.33, which was indicative of strong evidence of impact.
Recommendations for improving research design quality and evidence of impact
Our findings indicate that improvements in research design quality across the informal STEM landscape are necessary so that researchers and educators can identify whether informal programs are indeed facilitating youths’ knowledge of and persistence in STEM career pathways and careers. Most studies that were evaluated did not include a comparison group, lacked theoretical frameworks, and in many cases, lacked adequate sample sizes. We recommend that STEM researchers make use of the validated rubrics employed in this meta-synthesis to assess research design quality and evidence of impact of studies of informal STEM programs (Habig, 2020). Where feasible, we also recommend that researchers compare program participants to a comparison group using either an experimental or quasi-experimental design (Mohr, 1995; U.S. Department of Education, 2007). Moreover, we suggest that in addition to program time frames (i.e., summer, after school), that researchers also report the precise number of participants’ contact hours (Wilkerson and Haden, 2014), as this information was missing from most of the studies. In this way, informal STEM educators can use rigorously designed data-driven research to maximize participant impact. For example, Cooper et al. (2000) found that a minimum of 60, but fewer than 120 contact hours, were necessary to achieve academic outcomes in a summer school program. By using data-driven results, informal STEM practitioners can make a strong case to funding agencies, policy makers, and the public to invest in informal STEM education, especially given the considerable economic and human resources dedicated to supporting informal STEM initiatives (Wilkerson and Haden, 2014; Habig, 2020).
Participation in informal STEM programs is associated with increased STEM career awareness
From a subset of studies that met the inclusion criteria for meta-analyses (18 studies, 36 analyses), we found evidence that participation in informal STEM programs was predictive of STEM career awareness. Specifically, there was a significant positive association between participation in informal STEM programs and increased STEM career awareness (d = 0.584, 95% CI: 0.295–0.872, p < 0.001; Figure 2). However, we also found high levels of heterogeneity (Q (35) = 412.7, p < 0.001; I2 = 95.02%), which suggests that certain moderator variables might account for variation in effect sizes across studies. Therefore, we used a mixed modeling approach to test additional parameters that might explain additional variables associated with increased STEM career awareness (reported in the next section). For studies of STEM major awareness, we did not conduct meta-analyses as there were no studies that met the inclusion criteria.
Figure 2. A forest plot showing the effect sizes of studies that tested the association between participation in an informal STEM program and STEM career awareness. Positive values indicate higher STEM career awareness of participants relative to a comparison group, or higher STEM career awareness in a post-assessment versus a pre-assessment. The error bars represent the 95% confidence intervals (CIs). The length of the bar is indicative of the precision of the estimate; bars crossing the vertical lines indicate non-significant findings. The diamond represents the effect size for all studies of STEM career awareness included in the analysis.
We found no evidence of publication bias based on qualitative (funnel plot: Supplementary Figure S1) and quantitative (Egger’s test; trim and fill analysis) analyses of studies of STEM career awareness. An Egger’s test revealed no significant publication bias across studies (z = −0.896, p = 0.371). Moreover, a trim and fill analysis estimated that there were zero studies missing from the meta-analysis. A fail-safe N analysis, which tests for the stability of the meta-analysis results, indicated that 5,468 null results would need to be added to the observed outcomes to yield a non-significant (p > 0.05) mean effect size.
Informal STEM programs that target high school students and girls are associated with STEM career awareness
Multiple linear regression analyses revealed that two model parameters best predicted STEM career awareness: (1) participation in informal STEM high school programs and (2) participation in female-only informal STEM programs (Table 1). Informal STEM programs that focused specifically on high school students exhibited significantly higher effect sizes for STEM career awareness than informal STEM programs that focused mainly on middle school students (β = 1.192, SE = 0.303, z = 3.811, p < 0.001). Likewise, female-only informal STEM programs exhibited significantly higher effect sizes for STEM career awareness than informal STEM programs that served all genders (β = 0.698, SE = 0.268, z = 2.501, p < 0.012). For studies of STEM major awareness, we did not conduct multiple linear regressions, as there were no studies that met the inclusion criteria.
Implications of quantitative analyses
Our meta-analysis provided quantitative evidence that informal STEM programs are significantly associated with increased STEM career awareness. Moreover, the results of our multiple linear regression analyses suggest that informal STEM programs are especially effective at fostering STEM career awareness for two target audiences: (1) high school students and (2) girls. These results are encouraging, given that STEM interest tends to decline during adolescence, especially for girls (Sadler et al., 2012; Shapiro et al., 2015; Wigfield et al., 2015; Holmes et al., 2018). Therefore, our findings suggest that from a programmatic perspective, there are two strategies that might be most effective at fostering STEM career awareness: (1) offering female-only informal STEM programs that expose girls to an array of STEM career opportunities; and (2) offering informal STEM programming that exposes high school students to an array of STEM career opportunities. Hence, our quantitative analyses suggest that not only do informal STEM institutions play a pivotal role in fostering knowledge and understanding of STEM career pathways, but that specific types of programs—i.e., high school programs and female-only programs—might be most effective at expanding youths’ realm of possibilities.
Qualitative analyses
Program design principles associated with increased STEM major and STEM career awareness
Eight program design principles were extracted from rigorously designed studies that exhibited strong or exemplary evidence of impact (Table 2). The four most common program design principles associated with STEM major and STEM career awareness were: (1) mentoring (26 of 32 studies; 81.3%), (2) same sex/same race/same ethnicity role models (20 of 32 studies; 62.5%), (3) authentic research experiences (12 of 32 studies; 37.5%), and (4) the apprenticeship model (10 of 32 studies; 31.2%).
Table 2. Program design principles, effective practices, target audiences, and technological innovations extracted from rigorously designed studies of STEM major awareness and STEM career awareness that exhibited strong or exemplary evidence of impact.
Effective practices associated with increased STEM major and STEM career awareness
Nine effective practices were extracted from rigorously designed studies that exhibited strong or exemplary evidence of impact (Table 2). The four most common effective practices associated with STEM major and STEM career awareness were: (1) hands-on learning experiences (29 of 32 studies, 90.6%), (2) community of practice/collaborative learning (25 of 32 studies, 78.1%), (3) exposure to careers typically not learned in formal education (24 of 32 studies, 75.0%), and (4) specific workshops that focus on career exploration (19 of 32 studies, 59.4%).
Target audiences associated with increased STEM major and STEM career awareness
Two target audiences were identified from rigorously designed studies that exhibited strong or exemplary evidence of impact (Table 2). The two target audiences that exhibited increased STEM major awareness and STEM career awareness as a result of participation in informal STEM programs were: (1) girls (15 of 32 studies; 46.9%), and (2) members of historically underrepresented groups based on socioeconomic status, ethnicity, or race (14 of 32 studies; 43.8%).
Technological innovations associated with increased STEM major and STEM career awareness
Eleven technological innovations were extracted from rigorously designed studies that exhibited strong or exemplary evidence of impact (Table 2). The four most common technological innovations associated with STEM major awareness or STEM career awareness were computer programming (e.g., Linux, Python) (9 of 32 studies, 28.1%), online workshops and modules (7 of 32 studies, 21.9%), robotics (6 of 32 studies, 18.8%), and Lego Mindstorm (4 of 32, 12.5%).
Program design principles, effective practices, and technological innovations that foster STEM major and STEM career awareness for participants of female-only programs
There were 15 studies of STEM career awareness that exhibited strong or exemplary evidence of research design quality and evidence of impact that specifically focused on girls. From these 15 studies, two program design principles were associated with increased STEM career awareness: (1) same sex role models (14 of 15 studies, 93.3%) and (2) mentoring (12 of 15 studies, 80.0%). Moreover, four effective practices were associated with increased STEM career awareness: (1) hands-on learning experiences (15 of 15 studies, 100%), (2) community of practice/collaborative learning (13 of 15 studies, 86.7%), (3) exposure to careers typically not learned in formal education (12 of 15 studies, 80.0%), and (4) specific workshops that focus on career exploration (11 of 15 studies, 73.3%). In terms of technological innovations, four of 15 studies (26.7%) provided female participants with training in computer programming.
Program design principles, effective practices, and technological innovations that foster STEM major and STEM career awareness for participants of programs that specifically focus on historically underrepresented groups
There were two studies of STEM major awareness and 12 studies of STEM career awareness that exhibited strong or exemplary evidence of research design quality and evidence of impact that specifically focused on historically underrepresented groups based on socioeconomic status, ethnicity, or race. From these 14 studies, there were three program design principles that were associated with increased STEM major and STEM career awareness: (1) mentoring (13 of 14 studies, 92.9%), (2) authentic research experiences (8 of 14 studies, 57.1%), and (3) same sex/same race/same ethnicity role models (8 of 14 studies, 57.1%). Three effective practices were associated with increased STEM major and STEM career awareness: (1) hands-on learning experiences (12 of 14 studies, 85.7%), (2) community of practice/collaborative learning (11 of 14 studies, 78.6%), and (3) exposure to careers typically not learned in formal education (11 of 14 studies, 78.6%).
Implications of qualitative analyses
Our qualitative analyses suggest that there are commonalities in program design principles and effective practices among informal STEM programs that exhibit strong or exemplary evidence of increased STEM major and STEM career awareness. For example, two program design principles associated with increased knowledge of STEM career pathways and careers were mentoring (81.3% of studies) and research apprenticeships (37.5% of studies). Moreover, two effective practices associated with increased STEM major and STEM career awareness were hands-on learning experiences (90.6% of studies) and communities of practice (78.1% of studies). A representative example of an informal STEM program that adopted these aforementioned program design principles and effective practices is Project TRUE. In this program, undergraduate students, trained by Wildlife Conservation Society and Fordham University researchers, mentored a cohort of high school students to help them formulate urban ecology research questions, carry out hands-on research in local parks, and communicate their findings in a research symposium. A post-assessment of Project TRUE revealed that high school participants gained an understanding and awareness of science and conservation majors and careers (Aloisio et al., 2018). This example illustrates how a combination of program design principles and effective practices can be applied to facilitate knowledge of STEM career pathways and careers. The program design principles, effective practices, and technological innovations identified during our qualitative analyses, in synthesis with our quantitative findings, can be adopted by STEM practitioners to expand youths’ realm of possibilities in terms of STEM career pathways.
Emerging themes
Based on a synthesis of both qualitative and quantitative analyses, we identified three emerging themes that summarize features of informal STEM programs that foster STEM major and STEM career awareness: (1) programs that target girls or historically underrepresented groups; (2) programs that focus on career exploration; and (3) experiential learning.
Theme 1: Informal STEM programs that target girls or historically underrepresented groups
We found that informal STEM programs that specifically targeted girls or historically underrepresented groups based on socioeconomic status, ethnicity, or race were effective at fostering STEM major and STEM career awareness. Moreover, among these programs, two program design principles that were positively associated with increased knowledge of STEM career pathways and STEM careers were mentoring and same sex/same race/same ethnicity role models.
Examples of informal STEM programs that were effective at fostering STEM major and STEM career awareness for historically underrepresented groups included the Summer Experience in Earth and Mineral Science (SEEMS) Program, the Broadening Access to Science Education (BASE) camp, and the Cybersecurity Summer Program. The SEEMS Program, facilitated by Pennsylvania State University, specifically focuses on low-income high school students. During this six-week program, participants were given the opportunity to work alongside faculty from the College of Earth and Mineral Science, conducting research in the geosciences, and communicating their findings to the university community. A study of SEEMS participants reported a significant increase in knowledge about careers in the geosciences field (Baber et al., 2010). BASE, a free two-week residential camp for high school girls facilitated by Fairfield University, provides participants from low-income communities with authentic research experiences in university laboratories. The program focuses on same-sex role models and provides high school girls with both near-peer mentoring (undergraduate students) and mentoring by college professors. As a result of their experience, BASE participants’ awareness of STEM careers increased significantly from pre- to post-participation (Phelan et al., 2017). Finally, the Cybersecurity Summer Program facilitated by Columbia University offers high school girls an introduction to careers in cybersecurity. During this two-year program, youth participants are assigned female mentors and work collaboratively to solve cyber forensic problems. A qualitative analysis, based on participant interviews, revealed that exposure to cybersecurity careers was an emergent theme (Jethwani et al., 2017).
Theme 2: Informal STEM programs that focus on career exploration
Informal STEM programs that intentionally exposed participants to careers typically not learned in formal education or that provided workshops that specifically focused on career exploration exhibited strong or exemplary evidence of increased STEM major and STEM career awareness. These practices were particularly salient among longitudinal programs and in programs that focused on high school students. For example, Techbridge, a longitudinal (up to 6 years) afterschool and summer program facilitated by the Chabot Space and Science Center in Oakland, CA, provides middle and high school girls exposure to careers not typically learned in formal education, including electrical engineering and computer science. A study of Techbridge participants reported a significant increase in knowledge of engineering and technology careers, but no significant difference in a comparison group that did not participate in the program (Mosatche et al., 2013).
One technological innovation related to career exploration identified in nine high impact studies from our meta-synthesis was computer programming. This is notable because many public schools are not able to offer computer science education as part of their curriculum, and students therefore often have limited exposure to computer science careers (Yadav et al., 2016). One example of an informal STEM program from our meta-synthesis that concentrated on computer programming was the Summer Math and Science Honors Academy (SMASH) in Oakland, CA, a longitudinal (37.5 h per summer for a total of 112.5 h over 3 summers) residential computer science camp for high school students, designed to increase knowledge of and broaden participation in computer science (Scott et al., 2016). SMASH offers high school participants computer science programming workshops situated in a culturally relevant computing framework, i.e., a pedagogical approach that connects computing concepts to students’ personal identities and lived experiences (Eglash et al., 2013; Scott et al., 2016). Based on a quantitative assessment of the program, there was a significant increase in participants’ understanding of the computer science field and computing careers following participation (Scott et al., 2016).
Theme 3: Informal STEM programs that focus on experiential learning
Informal STEM programs that focused on experiential learning were associated with increased STEM major and STEM career awareness. Two program design principles related to the theme of experiential learning were authentic research experiences and the apprenticeship model. Authentic research experiences are programmatic components that afford participants the opportunity to engage as practitioners of science and to conduct research that contributes to a specific STEM discipline (Braund and Reiss, 2006; Habig and Gupta, 2021). The apprenticeship model is a program design principle in which participants are provided the opportunity to advance through a “ladder of achievement” while cultivating their professional workplace skills and practices (Barros-Smith et al., 2012). In addition to program design principles, studies that exhibited strong or exemplary evidence of impact included informal STEM programs that incorporated the following effective practices related to the theme of experiential learning: (1) project-based learning (Schneider et al., 2002), a practice in which students collaborate to solve real world problems and to develop products or solutions to address these problems (e.g., Techbridge: Mosatche et al., 2013; BUILDERS program: Escobar and Qazi, 2020); (2) communities of practice/collaborative learning (Lave and Wenger, 1991), i.e., a network of scientists, educators, and students working collaboratively to achieve shared goals (e.g., Project Exploration: Chi et al., 2010; AMNH Lang Science Program: Habig et al., 2020); (3) inquiry-based learning (Keselman, 2003; Lazonder and Harmsen, 2016), a practice in which students construct their own learning experience by asking questions, planning and carrying out investigations, analyzing and interpreting data, and communicating results (e.g., Girls Inc.: McCreedy and Dierking, 2013; NASA: Ortiz et al., 2018); and (4) engineering design process (Haik et al., 2015), a practice in which students follow the steps used by engineers to solve problems in a systematic manner (e.g., Park and Ride Program: Dell et al., 2011; BUILDERS program: Escobar and Qazi, 2020). A representative example of an informal STEM program that provided experiential learning opportunities was the BUILDERS program, a three-week summer program for underserved high school students facilitated by Tuskegee University. In this program, participants engaged in project-based learning where they worked in teams to design, construct, and test prototypes of technology-based solutions to community problems. A post-assessment of the BUILDERS program revealed that participants increased their knowledge of careers in STEM (Escobar and Qazi, 2020).
Implications based on meta-synthesis of quantitative and qualitative findings
Based on a synthesis of our quantitative and qualitative analyses, our findings suggest that there are program design principles and effective practices that can be applied universally to help increase youths’ knowledge of STEM career pathways and careers. Specifically, informal STEM programs that systematically exposed youth to an array of STEM careers and that provided opportunities for experiential learning were found to be effective at fostering STEM major and STEM career awareness across multiple programs affording students opportunities to try on “possible selves” (Dorsen et al., 2006) and to build self-efficacy (Bandura, 1977). These experiences map to existing theoretical frameworks, including Social Cognitive Career Theory (Bandura, 1977), Possible Selves Theory (Markus and Nurius, 1986), and Expectancy-Value Theory (Eccles, 2009). Moreover, in support of our hypothesis that not all practices work universally and that specific strategies are needed to foster STEM major and STEM career awareness for individuals who are members of groups historically marginalized from STEM fields, we found that informal STEM programs that specifically focus on girls and historically underrepresented groups, and that provide mentoring and same sex/same race/same ethnicity role models, were particularly effective at facilitating knowledge of STEM careers. Our findings suggest the possibility that girls and members of historically underrepresented groups are not being encouraged to explore certain STEM disciplines because of the persistence of historical stereotypes (van Tuijl and Molen, 2016) and that informal STEM programs might play an underappreciated role in introducing these audiences to a range of STEM fields. As we emphasized at the onset of this study: “Young people cannot choose a specific STEM career or field of study if they do not know of its existence” (Dorsen et al., 2006). Informal STEM programs might play an important role in cultivating awareness of STEM majors and STEM careers and countering deep-rooted negative stereotypes that serve as barriers to entry.
Limitations
We acknowledge that there are limitations to our study. First, many studies were excluded from our meta-analyses, multiple linear regressions, and qualitative syntheses because they exhibited poor or adequate research design. However, we think our decision to exclude studies that were not designed rigorously was the correct choice as we would not be able to reliably infer evidence of outcome. In data science, the term “garbage-in, garbage-out” is used to illustrate circumstances in which the quality of input is tightly linked to the quality of output (Kilkenny and Robinson, 2018). This is not to say that these studies were not impactful, only that we cannot confidently infer evidence of impact. The United States Department of Education (2007) recommends that researchers and educators use non-rigorous research designs to refine hypotheses and to better inform more rigorous studies, but not to assess evidence of impact. A second limitation was the limited number of studies that assessed STEM major awareness. Because there were so few studies of STEM major awareness (n = 28) and even fewer that exhibited strong evidence of research design (n = 3), many of our analyses concentrated on STEM career awareness. Finally, a third limitation is the potential for bias in estimates of treatment effects in our meta-analyses due to high levels of heterogeneity. There were several factors that may have contributed to statistical heterogeneity, including selection effects, measurement effects, and time effects (Morris and DeShon, 2002). Indeed, consistent with previous research, informal STEM programs tend to vary based on student characteristics and programmatic features (Young et al., 2017). Moreover, some of the statistical heterogeneity in the current study might be explained by qualitative differences in the rigor of peer review (e.g., refereed manuscripts published in conference proceedings versus refereed manuscripts published in journals) and measurement heterogeneity (e.g., differences in the instruments used to assess STEM career awareness). To account for these potential biases, we used a mixed modeling approach and included moderator variables in our analyses to identify sources of heterogeneity (Viechtbauer, 2007; Kuznetsova et al., 2017).
Future directions
There are three main steps to long-term engagement in a STEM discipline. The first step in a STEM career pathway is knowledge and understanding of possible STEM career pathways and careers (STEM major awareness and STEM career awareness). The second step is increased curiosity, motivation, and attention to a STEM discipline as a focus of study at a higher education institution and as a profession (STEM major interest and STEM career interest). The third step is a formal commitment to a STEM discipline in a higher education institution and in the workforce (STEM major engagement and STEM career engagement). In this study, we focused on the first of these three steps: STEM major awareness and STEM career awareness. We showed that informal STEM programs can play a pivotal role in fostering knowledge and understanding of possible STEM career pathways and careers. The next logical step, in terms of future directions, is to assess how and to what extent informal STEM learning experiences impact youth participants’ STEM major interest and STEM career interest. The third logical step in this sequence is to focus on alumni of informal STEM programs, and to assess how and to what extent informal STEM learning experiences impact participants’ STEM major engagement and STEM career engagement. Similar to the present study, future studies should also extract program design principles, effective practices, target audiences, and technological innovations from rigorously designed studies that can be adopted by STEM practitioners. We also recommend that future studies continue to center on strategies that can be applied to maximize impact and broaden participation of girls and members of historically marginalized racial and ethnic groups.
Conclusion
Awareness of STEM majors and STEM careers is the first step to a long-term trajectory in a STEM discipline. In this landscape analysis and meta-synthesis of the informal STEM education literature, we found support for our thesis that informal STEM education programs are pivotal for fostering STEM major and STEM career awareness. Across the informal STEM education landscape, we found that many studies of STEM major and STEM career awareness exhibited weak or adequate research design. However, among the rigorously designed studies, we found a significant positive association between participation in informal STEM programs and increased STEM career awareness. Informal STEM education programs that specifically focused on high school students and girls were particularly effective at facilitating knowledge of STEM careers. A thematic analysis revealed that three features of informal STEM education programs were associated with increased STEM major and STEM career awareness: (1) programs that focus on girls or historically underrepresented groups; (2) programs that focus on career exploration; and (3) experiential learning. Among programs that focused on historically marginalized populations, two programmatic features—same sex/same race/same ethnicity role models and mentoring—were particularly salient for facilitating knowledge of STEM careers. There are important economic and ethical justifications for augmenting young people’s knowledge and understanding of STEM career pathways and careers. Our results suggest that informal STEM institutions play an important role in ensuring that young people are introduced to potential future careers.
Author contributions
BH: Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing. FG: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Visualization, Writing – original draft, Writing – review & editing. PG: Writing – original draft, Writing – review & editing. JA: Writing – original draft, Writing – review & editing. MH: Project administration, Supervision, Writing – original draft, Writing – review & editing, Funding acquisition.
Funding
The author(s) declared that financial support was received for this work and/or its publication. This project was funded by the National Science Foundation Innovative Technology Experiences for Students and Teachers (ITEST) Grant (Award #2048544) (Principal Invesigator: MH, Key Personnel: BH). This project was also partially supported by the Texaco Research Foundation Endowment.
Acknowledgments
We thank Jackie Faherty, Stephanie London-Wortel, and Jamaal Young for their advice during project planning and their constructive feedback during project implementation. We thank Niouma Gassama and Jayda Smith for their assistance during coding.
Conflict of interest
The author(s) declared that this work was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Generative AI statement
The author(s) declared that Generative AI was not used in the creation of this manuscript.
Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary material
The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/feduc.2025.1717945/full#supplementary-material
References
Adams, J. D., Gupta, P., and DeFelice, A. (2012). Schools and informal science settings: collaborate, co-exist, or assimilate? Cult. Stud. Sci. Educ. 7, 409–416. doi: 10.1007/s11422-012-9399-x
Allen, P. J., Chang, R., Gorrall, B. K., Waggenspack, L., Fukuda, E., Little, T. D., et al. (2019). From quality to outcomes: a national study of afterschool STEM programming. Int. J. STEM Educ. 6, 1–21. doi: 10.1186/s40594-019-0191-2
Allen, S., and Peterman, K. (2019). Evaluating informal STEM education: issues and challenges in context. New Dir. Eval. 2019, 17–33. doi: 10.1002/ev20354
Aloisio, J. M., Johnson, B., Lewis, J. D., Alan Clark, J., Munshi-South, J., Roberts, S. J., et al. (2018). Pre-college urban ecology research mentoring: promoting broader participation in the field of ecology for an urban future. J. Urban Ecol. 4, 1–9. doi: 10.1093/jue/juy023
Ammenwerth, E., Iller, C., and Mansmann, U. (2003). Can evaluation studies benefit from triangulation? A case study. Int. J. Med. Inform. 70, 237–248. doi: 10.1016/S1386-5056(03)00059-5,
Aritajati, C., Rosson, M. B., Pena, J., Cinque, D., and Segura, A. (2015). “A socio-cognitive analysis of summer camp outcomes and experiences” in Proceedings of the 46th ACM Technical Symposium on Computer Science Education, Kansas City, MO: SIGCSE 2015, 581–586. doi: 10.1145/2676723.2677292
Aschbacher, P. R., Ing, M., and Tsai, S. M. (2014). Is science me? Exploring middle school students' STE-M career aspirations. J. Sci. Educ. Technol. 23, 735–743. doi: 10.1007/s10956-014-9504-x
Aschbacher, P. R., Li, E., and Roth, E. J. (2010). Is science me? High school students' identities, participation and aspirations in science, engineering, and medicine. J. Res. Sci. Teach. 47, 564–582. doi: 10.1002/tea.20353
Baber, L. D., Pifer, M. J., Colbeck, C., and Furman, T. (2010). Increasing diversity in the geosciences: recruitment programs and student self-efficacy. J. Geosci. Educ. 58, 32–42. doi: 10.5408/1.3544292
Baldwin, C. K., Stromwall, K., and Wilder, Q. (2015). Afterschool youth program design and structural quality: implications for quality improvement. Child Youth Serv. 36, 226–247. doi: 10.1080/0145935X.2015.1046592
Bandura, A. (1977). Self-efficacy: toward a unifying theory of behavioral change. Psychol. Rev. 84, 191–215. doi: 10.1037/0033-295X.84.2.191,
Bandura, A. (1991). Social cognitive theory of self-regulation. Organ. Behav. Hum. Decis. Process. 50, 248–287. doi: 10.1016/0749-5978(91)90022-L
Barros-Smith, R., Porro, I., and Norland, E. (2012). “Developing teenage youth's science identity through an astronomy apprenticeship: summative evaluation results” in Connecting people to science: a national conference on science education and public outreach, vol. 457, 209–212.
Belland, B. R., Walker, A. E., Kim, N. J., and Lefler, M. (2017). Synthesizing results from empirical research on computer-based scaffolding in STEM education: a meta-analysis. Rev. Educ. Res. 87, 309–344. doi: 10.3102/0034654316670999,
Bicer, A., and Lee, Y. (2019). Effect of STEM PBL embedded informal learning on student interest in STEM majors and careers. J. Math. Educ. 12, 57–73.
Bicer, A., Lee, Y., Capraro, R. M., Capraro, M. M., Barroso, L. R., Bevan, D., et al. (2018). “Cracking the code: the effects of using microcontrollers to code on students’ interest in computer and electrical engineering” in 2018 IEEE Frontiers in Education Conference (FIE) (San Jose, CA: IEEE), 1–7. doi: 10.1109/FIE.2018.8658617
Blanchard, M., Gutierrez, K., Habig, B., Gupta, P., and Adams, J. (2020). “Informal STEM program learning” in Handbook of research on STEM education (Routledge), 138–151.
Braund, M., and Reiss, M. (2006). Towards a more authentic science curriculum: the contribution of out-of-school learning. Int. J. Sci. Educ. 28, 1373–1388. doi: 10.1080/09500690500498419
Burnham, K. P., and Anderson, D. R. (2004). Multimodel inference: understanding AIC and BIC in model selection. Sociol. Methods Res. 33, 261–304. doi: 10.1177/0049124104268644
Cano, R., Berliner-Heyman, S., Koppel, N., Gibbons, S., and Kimmel, H. (2004). “Evaluation of summer enrichment programs for women students” in 2004 ASEE Annual Conference & Exposition, Salt Lake City, Utah. 9–582. doi: 10.18260/1-2--12910
Carnevale, A. P., Smith, N., and Melton, M. (2011) STEM: science technology engineering mathematics Georgetown University Center on Education and the Workforce
Cheung, J. R., Lietz, C. A., Carpenter, B. M., Sitz, E., and Lietz, B. C. (2019). Cultivating resilience in college students with a foster care background. J. Public Child Welf. 15, 182–202. doi: 10.1080/15548732.2019.1679694
Chi, B., Snow, J. Z., Goldstein, D., Lee, S., and Chung, J. (2010). Project Exploration: 10-year retrospective program evaluation summative report. Berkeley, CA: Center for Research, Evaluation, & Assessment, 1–82.
Chowdhury, T. M., Taylor, A. R., Murzi, H., Özkan, D. S., and Strom, H. C. (2019). “Where do we meet? Understanding conference participation in a department of engineering education” in ASEE Annual Conference & Exposition. Tampa, Florida. doi: 10.18260/1-2--33241
Cimpian, J. R., Kim, T. H., and McDermott, Z. T. (2020). Understanding persistent gender gaps in STEM. Science 368, 1317–1319. doi: 10.1126/science.aba7377,
Cohen, J. (1968). Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit. Psychol. Bull. 70, 213–220. doi: 10.1037/h0026256,
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. New York, NY: Lawrence Erlbaum Associates.
Cooper, H., Charlton, K., Valentine, J. C., Muhlenbruck, L., and Borman, G. D. (2000). Making the most of summer school: a meta-analytic and narrative review. Monogr. Soc. Res. Child Dev., i–127.
Crane, V., Nicholson, T., and Chen, M. (1994). Informal science learning: what research says about television, science museums, and community-based projects. Philadelphia, PA: Science Press.
Creswell, J. W., and Poth, C. N. (2018). Qualitative inquiry and research design: choosing among five approaches. 4th Edn. Thousand Oaks, CA: Sage Publications.
Dell, E., Bailey, M. B., O'Hurley, S., Lillis, R. P., Khol, B., Garrick, R. D., et al. (2011). “WE-IMPACT: women in engineering-improving program assessment tools for outreach and retention programs” in ASEE Annual Conference & Exposition. Vancouver, BC. doi: 10.18260/1-2--18770
Demetry, C., Hubelbank, J., Blaisdell, S. L., Sontgerath, S., Nicholson, M. E., Rosenthal, L., et al. (2009). Supporting young women to enter engineering: long-term effects of a middle school engineering outreach program for girls. J. Women Minor. Sci. Eng. 15, 119–142. doi: 10.1615/JWomenMinorScienEng.v15.i2.20
Demetry, C., and Sontgerath, S. (2013). “Does a middle school intervention for girls have long-lasting differential effects on their perceptions of and participation in engineering? (research to practice)” in ASEE Annual Conference & Exposition. Atlanta, Georgia. 23–441. doi: 10.18260/1-2--19455
Demirci, E. (2018). The influence of NASA'S High School Aerospace Scholars (HAS) program on student's STEM attitude and 21st century skills (Publication no. 10844761) ProQuest Dissertations & Theses Global: (Doctoral dissertation, Oklahoma State University; ProQuest Dissertations & Theses Global).
Diegelman, N. M., and Subich, L. M. (2001). Academic and vocational interests as a function of outcome expectancies in social cognitive career theory. J. Vocat. Behav. 59, 394–405. doi: 10.1006/jvbe.2001.1802
Dixon, C. S. (2015). The effects of “Girls in Science Day” on middle school girls’ attitudes and interests in science. (Publication No. 3729407) ProQuest Dissertations & Theses Global: (Doctoral dissertation, Ohio University.
Dorsen, J., Carlson, B., and Goodyear, L. (2006). Connecting informal STEM experiences to career choices: identifying the pathway. Waltham, MA: ITEST Learning Resource Center, 17.
Dortch, D., and Patel, C. (2017). Black undergraduate women and their sense of belonging in STEM at predominantly White institutions. NASPA J. About Women Higher Educ. 10, 202–215. doi: 10.1080/19407882.2017.1331854
Duran, M., Höft, M., Lawson, D. B., Medjahed, B., and Orady, E. A. (2014). Urban high school students’ IT/STEM learning: findings from a collaborative inquiry-and design-based afterschool program. J. Sci. Educ. Technol. 23, 116–137. doi: 10.1007/s10956-013-9457-5
Duval, S., and Tweedie, R. (2000a). A nonparametric “trim and fill” method of accounting for publication bias in meta-analysis. J. Am. Stat. Assoc. 95, 89–98. doi: 10.1080/01621459.2000.10473905,
Duval, S., and Tweedie, R. (2000b). Trim and fill: a simple funnel-plot–based method of testing and adjusting for publication bias in meta-analysis. Biometrics 56, 455–463. doi: 10.1111/j.0006-341X.2000.00455.x,
Eccles, J. (1983). “Expectancies, values and academic behaviors” in Achievement and achievement motives: Psychological and sociological approaches. ed. J. T. Spence (San Francisco, CA: Freeman), 75–146.
Eccles, J. S. (1994). Understanding women's educational and occupational choices: applying the Eccles et al. model of achievement-related choices. Psychol. Women Q. 18, 585–609. doi: 10.1111/j.1471-6402.1994.tb01049.x
Eccles, J. (2009). Who am I and what am I going to do with my life? Personal and collective identities as motivators of action. Educ. Psychol. 44, 78–89. doi: 10.1080/00461520902832368
Egger, M., Smith, G. D., Schneider, M., and Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. BMJ 315, 629–634. doi: 10.1136/bmj.315.7109.629,
Eglash, R., Gilbert, J. E., Taylor, V., and Geier, S. R. (2013). Culturally responsive computing in urban, after-school contexts: two approaches. Urban Educ. 48, 629–656. doi: 10.1177/0042085913499211
Ellis, G. D., Lacanienta, A., and Freeman, P. (2018). “Reducing attrition from youth programs through structuring deep, valued, and impactful programs for youth” in Youth development principles and practices in out-of-school time settings. eds. P. Witt and L. Caldwell (Urbana, IL: Sagamore-Venture), 351–387.
Escobar, M., and Qazi, M. (2020). Builders: a project-based learning experience to foster STEM interest in students from underserved high schools. J. STEM Educ. Innov. Res. 21, 35–43.
Fadigan, K. A., and Hammrich, P. L. (2004). A longitudinal study of the educational and career trajectories of female participants of an urban informal science education program. J. Res. Sci. Teach. 41, 835–860. doi: 10.1002/tea.20026
Falk, J. H., and Dierking, L. D. (2010). The 95 percent solution. Am. Sci. 98, 486–493. doi: 10.1511/2010.87.486
Farland-Smith, D. (2009). Exploring middle school girls’ science identities: examining attitudes and perceptions of scientists when working “side-by-side” with scientists. Sch. Sci. Math. 109, 415–427. doi: 10.1111/j.1949-8594.2009.tb17872.x
Foutz, S., and Luke, J. (2010). A youth-directed science café: impacts on teen participants. Institute for Learning Innovation. Available online at: https://informalscience.org/evaluation/youth-directed-science-cafe-impacts-teen-participants/ (Accessed March 10, 2024).
Fox, J., and Cater, M. (2015). An analysis of adolescents' science interest and competence in programs with and without a competitive component. J. Agric. Educ. 56, 90–106. doi: 10.5032/jae.2015.04090
Fox, J., and Weisberg, S. (2018). An R companion to applied regression. 3rd Edn. Thousand Oaks, CA: Sage.
Fu, A. C., Kannan, A., Shavelson, R. J., Peterson, L., and Kurpius, A. (2016). Room for rigor: designs and methods in informal science education evaluation. Vis. Stud. 19, 12–38. doi: 10.1080/10645578.2016.1144025
Gonzalez, S. I. (2020). Latinas in higher education: How receiving a higher education influences a feminist identity, career aspirations, and shapes familial relationships (Master’s thesis, California State University, Sacramento). Available online at: https://hdl.handle.net/10211.3/218680 (Accessed March 10, 2024).
Habig, B. (2020). Practical rubrics for informal science education studies: (1) a STEM research design rubric for assessing study design and a (2) STEM impact rubric for measuring evidence of design. Front. Educ. 5:554806. doi: 10.3389/feduc.2020.554806
Habig, B., and Gupta, P. (2021). Authentic STEM research, practices of science, and interest development in an informal science education program. Int. J. STEM Educ. 8, 1–18. doi: 10.1186/s40594-021-00314-y
Habig, B., Gupta, P., and Adams, J. D. (2021). Disrupting deficit narratives in informal science education: applying community cultural wealth theory to youth learning and engagement. Cult. Stud. Sci. Educ. 16, 509–548. doi: 10.1007/s11422-020-10014-8
Habig, B., Gupta, P., Levine, B., and Adams, J. (2020). An informal science education program’s impact on STEM major and STEM career outcomes. Res. Sci. Educ. 50, 1051–1074. doi: 10.1007/s11165-018-9722-y
Haik, Y., Sivaloganathan, S., and Shahin, T. M. (2015). Engineering design process. Boston, MA: Cengage Learning.
Hanson, S. L. (2007). Success in science among young African American women. J. Fam. Issues 28, 3–33. doi: 10.1177/0192513X06292694
Higgins, J. P., and Thompson, S. G. (2002). Quantifying heterogeneity in a meta-analysis. Stat. Med. 21, 1539–1558. doi: 10.1002/sim.1186,
Hirsch, L. S., Kimmel, H., Rockland, R., and Bloom, J. (2006). “A study of the impact of enrichment programs on girls’ attitudes towards engineering” in Proceedings of the 2006 International Conference on Engineering Education. San Juan, PR.
Holmes, K., Gore, J., Smith, M., and Lloyd, A. (2018). An integrated analysis of school students’ aspirations for STEM careers: which student and school factors are most predictive? Int. J. Sci. Math. Educ. 16, 655–675. doi: 10.1007/s10763-016-9793-z
Hubelbank, J., Demetry, C., Nicholson, S. E., Blaisdell, S., Quinn, P., Rosenthal, E., et al. (2007). Long term effects of a middle school engineering outreach program for girls: A controlled study. In 2007 ASEE Annual Conference & Exposition, Honolulu, Hawaii. doi: 10.18260/1-2--2098
Institute for Learning Innovation (2007). Evaluation of learning in informal learning environments, in paper prepared for the Committee on Science Education for Learning Science in Informal Environments. Available online at: https://www.informalscience.org/evaluation-learning-informal-learning-environments (Accessed March 10, 2024).
Jethwani, M. M., Memon, N., Seo, W., and Richer, A. (2017). “I can actually be a super sleuth”: promising practices for engaging adolescent girls in cybersecurity education. J. Educ. Comput. Res. 55, 3–25. doi: 10.1177/073563311665197
Kelly, K., Dampier, D. A., and Carr, K. (2013). Willing, able, and unwanted: high school girls' potential selves in computing. J. Women Minor. Sci. Eng. 19, 67–85. doi: 10.1615/JWomenMinorScienEng.2013004471
Keselman, A. (2003). Supporting inquiry learning by promoting normative understanding of multivariable causality. J. Res. Sci. Teach. 40, 898–921. doi: 10.1002/tea.10115
Kilkenny, M. F., and Robinson, K. M. (2018). Data quality: “garbage in–garbage out”. Health Inf. Manag. J. 47, 103–105. doi: 10.1177/1833358318774357,
Kuznetsova, A., Brockhoff, P. B., and Christensen, R. H. (2017). lmerTest package: tests in linear mixed effects models. J. Stat. Softw. 82, 1–26. doi: 10.18637/jss.v082.i13
Landis, J. R., and Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics 33, 159–174. doi: 10.2307/2529310,
Laursen, S., Liston, C., Thiry, H., and Graf, J. (2007). What good is a scientist in the classroom? Participant outcomes and program design features for a short-duration science outreach intervention in K–12 classrooms. CBE Life Sci. Educ. 6, 49–64. doi: 10.1187/cbe.06-05-0165,
Lave, J., and Wenger, E. (1991). Situated learning. Legitimate peripheral participation : Cambridge University Press.
Lazonder, A. W., and Harmsen, R. (2016). Meta-analysis of inquiry-based learning: effects of guidance. Rev. Educ. Res. 86, 681–718. doi: 10.3102/0034654315627366
Lent, R. W., Brown, S. D., and Hackett, G. (1994). Toward a unifying social cognitive theory of career and academic interest, choice, and performance. J. Vocat. Behav. 45, 79–122. doi: 10.1006/jvbe.1994.1027
Lin, L. (2019). Graphical augmentations to sample-size-based funnel plot in meta-analysis. Res. Synth. Methods 10, 376–388. doi: 10.1002/jrsm.1340,
Lindsay, S. (2019). Five approaches to qualitative comparison groups in health research: a scoping review. Qual. Health Res. 29, 455–468. doi: 10.1177/1049732318807208,
Lombard, M., Snyder-Duch, J., and Bracken, C. C. (2002). Content analysis in mass communication: assessment and reporting of intercoder reliability. Hum. Commun. Res. 28, 587–604. doi: 10.1111/j.1468-2958.2002.tb00826.x
Lüdecke, D., and Lüdecke, M. D. (2022). Package ‘esc’. R Package Version 0.5.1, 2019. Package ‘esc’. R Package Version 0.5.1, 2022. Available online at: https://strengejacke.github.io/esc
Lunn, S., Marques Samary, M., and Peterfreund, A. (2021). “Where is computer science education research happening?” in Proceedings of the 52nd ACM Technical Symposium in Computer Science Education, Virtual Event, USA. 288–294. doi: 10.1145/3408877.3432375
Markus, H., and Nurius, P. (1986). Possible selves. Am. Psychol. 41, 954–969. doi: 10.1037/0003-066X.41.9.954
Mau, W. C. J., and Li, J. (2018). Factors influencing STEM career aspirations of underrepresented high school students. Career Dev. Q. 66, 246–258. doi: 10.1002/cdq.12146
McCreedy, D., and Dierking, L. D. (2013). Cascading influences: long-term impacts of informal STEM experiences for girls. Presented at 27th Annual Visitor Studies Association Conference. Available online at: https://www.informalscience.org/cascading-influences-long-term-impacts-stem-informal-experiences-girls (Accessed March 10, 2024).
McKay, M. M., Lowes, S., Tirthali, D., and Camins, A. H. (2015). “Student learning of STEM concepts using a challenge-based robotics curriculum” in 2015 ASEE Annual Conference & Exposition, Seattle, Washington. 26–1419. doi: 10.18260/p.24756
Morris, S. B., and DeShon, R. P. (2002). Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. Psychol. Methods 7, 105–125. doi: 10.1037/1082-989X.7.1.105,
Mosatche, H. S., Matloff-Nieves, S., Kekelis, L., and Lawner, E. K. (2013). Effective STEM programs for adolescent girls: three approaches and many lessons learned. Afterschool Matters 17, 17–25.
Nakagawa, S., and Schielzeth, H. (2013). A general and simple method for obtaining R2 from generalized linear mixed-effects models. Methods Ecol. Evol. 4, 133–142. doi: 10.1111/j.2041-210x.2012.00261.x
National Research Council (2009). Learning science in informal environments: people, places, and pursuits. Washington, DC: The National Academies Press.
National Research Council (2013). Monitoring progress toward successful K12 STEM education: a nation advancing? Washington, DC: The National Academies Press.
National Research Council (2015). Identifying and supporting productive STEM programs in out-of-school settings. Washington, DC: The National Academies Press.
Ortiz, A. M., Amaya, L. R., Warshauer, H. K., Torres, S. G., Scanlon, E., and Pruett, M. (2018). “They choose to attend academic summer camps? A mixed methods study exploring motivation for, and the impact of, an academic summer pre-engineering camp upon middle school students in a Latino community” in 2017 ASEE Annual Conference & Exposition. Columbus, Ohio. doi: 10.18260/1-2--29025
Oyserman, D., Gant, L., and Ager, J. (1995). A socially contextualized model of African American identity: possible selves and school persistence. J. Pers. Soc. Psychol. 69, 1216–1232. doi: 10.1037/0022-3514.69.6.1216
Perez-Felkner, L., Erichsen, K., Li, Y., Chen, J., Hu, S., Ramirez Surmeier, L., et al. (2024). Computing education interventions to increase gender equity from 2000 to 2020: a systematic literature review. Rev. Educ. Res. 95, 536–580. doi: 10.3102/00346543241241536,
Phelan, S. A., Harding, S. M., and Harper-Leatherman, A. S. (2017). BASE (Broadening Access to Science Education): a research and mentoring focused summer STEM camp serving underrepresented high school girls. J. STEM Educ. Innov. Res. 18, 65–72.
Phillips, M., Finkelstein, D., and Wever-Frerichs, S. (2007). School site to museum floor: how informal science institutions work with schools. Int. J. Sci. Educ. 29, 1489–1507. doi: 10.1080/09500690701494084
R Core Team (2023). R: a language and environment for statistical computing. Vienna: R Foundation for Statistical Computing.
Roberts, B. W., Luo, J., Briley, D. A., Chow, P. I., Su, R., and Hill, P. L. (2017). A systematic review of personality trait change through intervention. Psychol. Bull. 143, 117–141. doi: 10.1037/bul0000088,
Sadler, P. M., Sonnert, G., Hazari, Z., and Tai, R. (2012). Stability and volatility of STEM career interest in high school: a gender study. Sci. Educ. 96, 411–427. doi: 10.1002/sce.21007
Schneider, R. M., Krajcik, J., Marx, R. W., and Soloway, E. (2002). Performance of students in project-based science classrooms on a national measure of science achievement. J. Res. Sci. Teach. 39, 410–422. doi: 10.1002/tea.10029
Scott, A., Martin, A., McAlear, F., and Madkins, T. C. (2016). Broadening participation in computer science: existing out-of-school initiatives and a case study. ACM Inroads 7, 84–90. doi: 10.1145/2994153
Shapiro, M., Grossman, D., Carter, S., Martin, K., Deyton, P., and Hammer, D. (2015). Middle school girls and the “leaky pipeline” to leadership: an examination of how socialized gendered roles influences the college and career aspirations of girls is shared as well as the role of middle level professionals in disrupting the influence of social gendered messages and stigmas. Middle Sch. J. 46, 3–13. doi: 10.1080/00940771.2015.11461919
Shi, L., and Lin, L. (2019). The trim-and-fill method for publication bias: practical guidelines and recommendations based on a large database of meta-analyses. Medicine 98:e15987. doi: 10.1097/MD.0000000000015987,
Simpkins, S. D., Davis-Kean, P. E., and Eccles, J. S. (2006). Math and science motivation: a longitudinal examination of the links between choices and beliefs. Dev. Psychol. 42, 70–83. doi: 10.1037/0012-1649.42.1.70,
Soto-Lara, S., Yu, M. V. B., Pantano, A., and Simpkins, S. D. (2022). How youth-staff relationships and program activities promote Latinx adolescent outcomes in a university-community afterschool math enrichment activity. Appl. Dev. Sci. 26, 619–637. doi: 10.1080/10888691.2021.1945454
Sterne, J. A., and Harbord, R. M. (2004). Funnel plots in meta-analysis. Stata J. Promot. Commun. Stat. Stata 4, 127–141. doi: 10.1177/1536867X0400400204
Struminger, R., Zarestky, J., Short, R. A., and Lawing, A. M. (2018). A framework for informal STEM education outreach at field stations. Bioscience 68, 969–978. doi: 10.1093/biosci/biy108
Thompson, B. (2002). What future quantitative social science research could look like: confidence intervals for effect sizes. Educ. Res. 31, 25–32. doi: 10.3102/0013189X031003025
Tillinghast, R. C., and Mansouri, M. (2020). Identifying key development stages of the STEM career pipeline. IEEE Trans. Technol. Soc. 3, 58–66. doi: 10.1109/TTS.2020.3046424,
U.S. Department of Education (2007). Report of the academic competitiveness council. Washington, DC.
van Tuijl, C., and van der Molen, J. H. W. (2016). Study choice and career development in STEM fields: an overview and integration of the research. Int. J. Technol. Des. Educ. 26, 159–183. doi: 10.1007/s10798-015-9308-1
Venville, G., Rennie, L., Hanbury, C., and Longnecker, N. (2013). Scientists reflect on why they chose to study science. Res. Sci. Educ. 43, 2207–2233. doi: 10.1007/s11165-013-9352-3
Viechtbauer, W. (2007). Accounting for heterogeneity via random-effects models and moderator analyses in meta-analysis. J. Psychol. 215, 104–121. doi: 10.1027/0044-3409.215.2.104,
Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. J. Stat. Softw. 36, 1–48. doi: 10.18637/jss.v036.i03
Wang, M. T., and Degol, J. (2013). Motivational pathways to STEM career choices: using expectancy–value perspective to understand individual and gender differences in STEM fields. Dev. Rev. 33, 304–340. doi: 10.1016/j.dr.2013.08.001,
Wigfield, A., and Eccles, J. S. (2000). Expectancy–value theory of achievement motivation. Contemp. Educ. Psychol. 25, 68–81. doi: 10.1006/ceps.1999.1015,
Wigfield, A., Eccles, J. S., Fredricks, J. A., Simpkins, S., Roeser, R. W., and Schiefele, U. (2015). “Development of achievement motivation and engagement” in Handbook of child psychology and developmental science. eds. M. E. Lamb, R. M. Lerner, M. E. Lamb, and R. M. Lerner, vol. 3. 7th ed (Hoboken, NJ: Wiley), 657–700.
Wigfield, A., Tonks, S., and Klauda, S. L. (2009). “Expectancy-value theory” in Handbook of motivation at school (Routledge), 69–90.
Wilkerson, S. B., and Haden, C. M. (2014). Effective practices for evaluating STEM out-of-school time programs. Afterschool Matters 19, 10–19.
Wilson, D. B. (2023). Effect size calculator. Available online at: https://campbellcollaboration.org/escalc/html/EffectSizeCalculator-SMD-main.php (Accessed July 24, 2023).
Yadav, A., Gretter, S., Hambrusch, S., and Sands, P. (2016). Expanding computer science education in schools: understanding teacher experiences and challenges. Comput. Sci. Educ. 26, 235–254. doi: 10.1080/08993408.2016.1257418
Keywords: informal STEM education, museum education, out-of-school time programs, program design principles, STEM career awareness, STEM major awareness
Citation: Habig B, Geller F, Gupta P, Adams JD and Holford M (2026) Expanding the realm of possibilities: the role of informal STEM programs in promoting STEM major and STEM career awareness. Front. Educ. 10:1717945. doi: 10.3389/feduc.2025.1717945
Edited by:
Rany Sam, National University of Battambang, CambodiaReviewed by:
Daniel H. Solis, Instituto Politécnico Nacional, MexicoVireak Keo, University of Battambang, Cambodia
No Sinath, University of Battambang, Cambodia
Copyright © 2026 Habig, Geller, Gupta, Adams and Holford. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Bobby Habig, YmhhYmlnQGFtbmgub3Jn