The STEM Enhancement in Earth Science “Mosquito Mappers” Virtual Internship: Outcomes of Place-Based Engagement with Citizen Science

In this paper, we describe a virtual high school earth science enhancement program that embeds a citizen science experience within a scientist-mentored research internship. We demonstrate the success of the program as measured by knowledge of pertinent science concepts and processes, and changes in future career orientation, and explore the role of place-based citizen science in interns’ experience of their research projects. The STEM Enhancement in Earth Science (SEES) Mosquito Mappers Virtual Internship connected high school interns with research opportunities applying NASA Earth Observations to the global health threat of mosquito-vector borne diseases. The interns engaged in 120–150 h of research working closely with NASA subject matter expert mentors, and making use of the GLOBE Observer citizen science infrastructure to collect and analyse data. In the virtual format, interns were able to participate in activities synchronously or asynchronously through an online learning platform, web conferencing software and social media. Students developed and completed either an independent or small team research project leveraging data collected using the place-based citizen science mobile application, GLOBE Observer. We found that participants were motivated by conducting fieldwork in and around their communities using the GLOBE Observer app. Local fieldwork enabled the student researchers to apply their prior knowledge of their natural and built landscapes, while also contributing to the reduction of mosquito-borne disease risk in their communities. This finding highlights the benefit of coupling citizen science to a virtual internship. Interns shared that their interest in research, and research in the earth sciences specifically, increased as a result of participating in the internship program. By leveraging citizen science data collection and reporting tools, existing data, and analysis tools, the program represented a robust, cost effective research experience conducted in a virtual environment, lowering barriers to participation and broadening access to STEM enhancement opportunities for all.


INTRODUCTION
Engaging more youth in citizen science is an essential goal for citizen science practitioners (Mueller et al., 2011;Newman et al., 2012;Harris et al., 2020). Today's youth are "digital natives", an admittedly controversial term that has been used to describe those who experienced their formative years with easy access to sophisticated computers, tablets, and smartphones for entertainment, communication, and learning (Akçayır et al., 2016). We posit that connecting digitally savvy youth to citizen science programs that rely on devices like smartphones and tablets engages young people in STEM learning, and expands the population of capable volunteers who contribute time and data to their programs.
At the same time, children and teens interested in science stand to benefit greatly from participation in the kinds of authentic STEM experiences that citizen science programs offer (National Academies of Sciences, Engineering, and Medicine, 2018). Youth who participate in citizen science programs increase their scientific knowledge, science process skills, and self-efficacy . As STEM disciplines increasingly seek to uphold democratic values like diversity, access and inclusion, citizen science can provide a valuable tool with which to authentically engage a broader coalition of young people in authentic STEM experiences.
Many citizen science programs that engage youth do so through in-school programs led by classroom teachers who incorporate citizen science into their curricula (National Academies of Sciences, Engineering, and Medicine, 2018). Authentically engaging young people outside of the classroom is a greater challenge (Bonney et al., 2015). One potential strategy for engaging youth in citizen science outside of formal schooling is to bring citizen science to young people through structured extracurricular programs. Embedding citizen science within existing STEM programs creates scaffolding that could help young volunteers contextualize what they learn in citizen science experiences.
In this paper, we describe the outcomes and evaluate the impacts of Mosquito Mappers, a virtual internship offered in summer 2020 to high school students. Mosquito Mappers leveraged citizen science data collection tools and an existing data archiving and analysis infrastructure to coordinate participation of 113 students distributed across the country in collaborative research. Interns engaged both synchronously and asynchronously with scientist mentors and NASA subject matter experts through discussion boards, online webinars, virtual coworking events, social media and a learning management platform. Because Mosquito Mappers interns could participate in the virtual internship from their own homes, this opportunity was more accessible to lower-income participants, interns with disabilities, or those with family or work responsibilities (Mayo and Shethji 2010).
Mosquito Mappers was one of several scientific investigation teams sponsored by the STEM Education and Enhancement in the Earth Sciences (SEES), a nationally competitive STEM Summer Intern program targeting diverse and underserved students in STEM. Funded by NASA, the SEES program is directed by the University of Texas Austin Center for Space Research and hosted by the Texas Space Grant. Mosquito Mappers was one of two teams in 2020 that was designed to be conducted completely virtually (All teams designed for inperson internships shifted to a remote format in response to the COVID-19 pandemic.) The Mosquito Mapper internship provided an 8-weeks, hands-on experience in multiple aspects of the scientific research process: question or hypothesis identification, data collection and analysis, result interpretation and communication of outcomes. To coordinate the virtual engagement of the interns in fieldwork and data analysis, we employed the GLOBE Observer citizen science mobile data collection app and the data infrastructure of the GLOBE Program (Global Learning and Observations to Benefit the Environment).
GLOBE is an international science and education program engaging students, teachers, community members and scientists in environmental monitoring and research. Created 25 years ago to support inquiry-based learning in K-12 classrooms, GLOBE recently expanded its mission to support citizen scientist data collection and investigation by launching the GLOBE Observer mobile app, a place-based citizen science app for volunteers of all ages (Amos et al., 2020).
The GLOBE Observer app includes four different protocols, each of which allows the user to collect observations of their immediate environment using their smartphone camera: land cover, clouds, trees, and mosquito habitats. Users have the option of simply taking geo-tagged photographs of a location, or to go further by identifying key features of the observations they take. The data collected by thousands of GLOBE observers is then used by scientists in conjunction with remote sensing data to better understand Earth science phenomena. In the case of the Mosquito Habitat Mapper protocol, reported observations of productive larval habitats can be of immediate use in local surveillance and mitigation efforts. The data can also support the refinement of the secondary land cover products derived from remotely-sensed satellite data, and can be employed by scientists in the development of vector borne disease risk models. In addition, several species of invasive mosquitoes are expanding their range poleward, partially in response to climate change. Volunteer-collected data can support public health departments in tracking the locations of potential disease vectors, and thus help in estimating their geographic spread (Palmer et al., 2017).
For the Mosquito Mapper internship, two GLOBE Observer tools, Mosquito Habitat Mapper and Land Cover, were employed as data collection tools in the intern research experience. Interns contributed to a group data collection and analysis project that required systematic sampling and analysis of a series of 3 km 2 Areas of Interest (AOI). They used the Mosquito Habitat Mapper and Land Cover tools to take coincident observations of the local environments within the AOI located near their home. These data were then uploaded to the GLOBE database and subsequently accessed and uploaded into web-based data analysis platforms to inform analysis of corresponding satellite imagery. Land cover classification and analysis was conducted on Collect Earth Online, an open-source, high resolution satellite image viewing and interpretation system (Saah et al., 2019). Once the shared project dataset was prepared, each student applied the data in a research project of their own choosing. The research projects provided the students with individualized contexts within which to conduct their investigation and answer their research questions.
Our evaluation of the Mosquito Mappers internship was an opportunity to assess not only the success of a new, virtual internship opportunity, but to also investigate how place-based citizen science contextualized in a project of students' choosing may influence young people in STEM. Research in STEM learning has shown that place-based learning is an effective way to help students understand the real-world connections between scientific research and human lives (Avery 2013;Cruz et al., 2019;Kirsch et al., 2019). By coupling place-based citizen science to a virtual internship, Mosquito Mappers interns were able to connect to their communities while building the science process skills that can contribute to successful secondary and post-secondary STEM coursework and careers.

METHODS
We employed a pre-post knowledge quiz to evaluate content knowledge acquisition in conjunction with a post-only online survey to evaluate intern attitudes and outcomes related to the Mosquito Mappers internship.

Participant Consent
Interns were made aware that the program was being evaluated by external evaluators, and were given the opportunity to consent to the evaluation process. A consent document explaining the evaluation process was posted on the Blackboard website, and interns were asked to read the document carefully. Minor students (under age 18) were asked to read the document with their parents. Students were then asked to complete a quiz that included options for both minors and adults to consent to the research, decline to participate, or request further information.

Data Collection and Survey Instrument
Students twice completed a science content knowledge quiz, administered at the beginning and end of the internship via the Blackboard learning management system that was used for the internship. The quiz included 12 multiple choice questions of varying difficulty, capturing basic concepts in Earth science, mosquito biology, and scientific processes that would be covered during the webinar sessions with SMEs (Supplementary Material). Content questions were developed based on Next Generation Science Standards performance expectations.
A post-internship survey was distributed as the last assignment of the internship program, administered via Qualtrics (Supplementary Material). The survey included both closed and open response questions designed to assess students' experiences with the SEES virtual internship. The survey was open for 2 weeks after distribution before being closed to new responses. Closed response question responses were based on a 5-point Likert scale.
Key quantitative measures assessed using the survey included: perceived changes in career and subject matter interest, and respondent attitudes towards 1) program content and execution, 2) collaboration within the program, and 3) connections to interns' local communities. Demographic data on racial and gender identities were also collected.
In addition to the quantitative measures, an open-response question explored how students felt about conducting citizen science research in their local communities: "We encouraged you to perform your research in your local community. Please describe how, if at all, working in your local community impacted your motivations for doing research."

Data Analysis
Quantitative data were analyzed using SPSS Statistics (IBM) software. In addition to descriptive statistics, Wilcoxon signedrank tests were used to assess the change in scores on the Earth science content knowledge quiz, as well as the change in career and subject matter interest. Effect sizes were calculated based on the Z value, using the equation r (effect size) z/square root of N, and evaluated based on Cohen's criteria (Cohen 1988). Qualitative data were compiled and evaluated using a thematic analysis approach to identify common themes across responses; these data were used to illustrate quantitative findings related to interns' community connections.

SEES Demographics
In Summer 2020, 113 interns enrolled in the SEES Mosquito Mapper Virtual Internship. Interns were all entering either 11th or 12th grade; approximately between 15 and 18 years of age. Although most interns were located in the southern United States (49%), reflecting the program's roots in Texas, interns were distributed across all regions of the United States, representing 20 different states, including Puerto Rico. By the end of the project, more than 8,300 observations were reported using the GLOBE Observer tool, and 1,800 classified 100 m 2 land cover observations were submitted within 49 AOI. 60 interns completed the full program and consented to participate in the program evaluation. Intern demographic characteristics (race and gender) can be seen in Table 1. All 60 consenting interns completed the post-internship survey (100% response rate), while 57 of those interns completed both the pre and post post-internship knowledge quizzes.

Program Appeal and Success
Intern research projects were reasonably successful in terms of project completion and follow-up. Of the 113 interns who enrolled in the program, 73 completed the full program, including final research projects employing data obtained using the Mosquito Habitat Mapper and Land Cover tools on the GLOBE Observer app. Nine SEES students submitted their projects to the Columbia Junior Science Journal and 32 submitted posters to the International Virtual Science Symposium. Video examples of intern research presentations can be viewed in Supplementary Material. The scientific output of these intern contributions is available in Low et al. (2020).  "By doing the 36 point land cover grid, I explored many areas in my community that I did not know existed. It was very eye opening to be able to explore and look at my area through a new, scientific lens. I also was able to find out that majority of land owners spray their property with mosquito treatment pesticides." Neutral "It was nice since I knew the area and how to get to places for the 36 grid. Other than that, there wasn't much else." "I wasn't able to do a lot of research due COVID-19, but the time that I was able to use for research was very helpful because I was able to learn the different ways to retrieve data." Motivation unchanged "Working in my local community did not improve my motivations much." "I can't really say much about this, since I've never encountered a mosquito outside yet nor have a caught a mosquito larvae." Frontiers in Environmental Science | www.frontiersin.org August 2021 | Volume 9 | Article 682669 The internship program achieved its goal of engaging young people; over 80% of survey respondents (n 60) strongly agreed with the statements "I enjoyed being a part of this program" and "I felt the program was interesting". Mean Likert-scale values (with 1 Strongly disagree and 5 Strongly agree) were 4.80 and 4.83, respectively.

Community Learning Outcomes
The SEES virtual internship made use of a place-based citizen science program to situate the interns' research projects in their local communities. In order to assess how interns responded to this, they were asked to rate the statement "It was motivating to do research in my local community" (5-point Likert), as well respond to the open response question "We encouraged you to perform your research in your local community. Please describe how, if at all, working in your local community impacted your motivations for doing research." A majority of post survey respondents agreed (20.0%) or strongly agreed (63.3%) to the statement "It was motivating to do research in my local community". The mean value for this item was 4.35. Of the qualitative responses to the open-response question, 39 of the 60 responses in the post-survey described a strong, positive influence of community-oriented citizen science research on their motivation to conduct research. Themes that emerged from the positive responses included 1) being able to do research that impacted themselves and their loved ones, 2) being able to do research in a place they loved, 3) doing research in a convenient location, and 4) doing research helped change the way they saw their community. Three of the remaining 21 responses stated that their motivations for doing research were not impacted, and the remaining responses were neutral, or not directly pertinent to the respondents' personal motivations. Table 2 illustrates these response types and themes with a selection of representative responses.

Program Learning and Attitudinal Outcomes
SEES interns increased their knowledge of relevant science concepts during the internship. On a 12-point knowledge quiz including questions about Earth science, mosquito biology, and scientific processes, administered at the beginning and end of the internship program, mean scores increased by 3.6% at the end of the internship (n 57), from a mean score of 10.92-11.35. Although the value of the change in knowledge scores was small, a Wilcoxon Signed Rank Test revealed a statistically significant increase in content knowledge (z −3.282; p-value <0.001), with a medium-large effect size (r 0.434). The knowledge test revealed existing strong content knowledge among the interns in the pre-test; as a result, the change in content knowledge may even have been more robust than was measured using our instrument.
Interns were also asked about their interest level in various career outcomes, prior to and after the internship (Figure 1). The career outcome (career orientation) was operationalized through six items: "majoring in an Earth science field in college", "a career in Earth science", "a career as a researcher", "using remote sensing in my research", "researching mosquitoes", and "researching land cover". Interest in all of these outcomes increased from pre to post, indicating that the program had at least a short-term impact on career orientation in participating youth. Wilcoxon Signed Rank Tests revealed statistically significant increases in all six of these items (p-values <0.001), with moderate effect sizes. Z-values and effect sizes are as follows: "majoring in an Earth science field in college" (z −4.112; r 0.545), "a career in Earth science" (z −4.536; r 0.601), "a career as a researcher" (z −4.287; r 0.568), "using remote sensing in my research" (z −5.729), "researching mosquitoes" (z −5.183; r 0.759), and "researching land cover" (z −5.593; r 0.741).

DISCUSSION
The SEES Mosquito Mappers team engaged interns in citizen science and Earth science research by incorporating the GLOBE Observer app in a research experience accessible from their own homes and communities. Our evaluation of this program validates the feasibility and success of a virtual research internship approach, particularly by using a place-based, community-oriented approach that leveraged the fact that participants remained at home for their internship. By leveraging the citizen science data collection and reporting tools, the GLOBE Observer citizen science data archive, and online data access and analysis tools, it was possible to create a robust, cost-effective research experience conducted in a virtual setting based on a sense-of-place pedagogy while lowering barriers to participation; broadening access to STEM enhancement opportunities for all.
We found that interns experienced a significant increase in their Earth science content knowledge, despite the fact that most already scored high on the knowledge pre-test. Students had multiple opportunities to develop their understanding of Earth science during the internship; through live webinars with NASA Subject Matter Experts (SMEs), direct mentorship by research scientists, through the collection and analysis of citizen science data, and through the completion and presentation of a final research project. Although participation in citizen science is just one of several ways in which interns engaged with Earth science concepts, research on citizen science programs have shown that an increase in content knowledge can be a significant outcome for volunteers (Brossard et al., 2011;Bonney et al., 2016;National Academies of Sciences, Engineering, and Medicine, 2018).
Students also demonstrated changes in their interest in various future career paths, being significantly more interested in Earth science research and topics after the internship than they were previously. Students were more interested in majoring in Earth science, careers in Earth science, careers in research generally, and in doing research on the specific topics that the interns explored using the GLOBE Observer platform, remote sensing, land cover, and mosquitoes. Notably, the effect was greatest for the measures of change in interest in those specific topics-remote sensing, land cover, and mosquitos-suggesting that interns' experiences in the virtual internship helped them make meaning out of topics that may have previously been obscure or uninteresting to them.
Our evaluation also found that the interns were motivated by a perceived connection to their own communities-the people they care about within the place that they live. Place-based education situates learning in a local context, connecting concepts and skills directly to places that hold meaning for the learners (Semken et al., 2017). Mosquito Mappers leveraged such place-connections for interns by asking them to develop their own research studies and analyze data that they collected within their own communities. Interns expressed positive responses regarding research in their communities and their motivation to work on their research projects ( Table 2). The personal bond that forms between volunteer and community in place-based STEM programs is one that has been demonstrated to particularly enhance scientific knowledge and literacy (Brewer 2002;Danielsen et al., 2005;Sullivan et al., 2009;Jordan et al., 2011), "scientific thinking", and science process skills (Kountoupes and Oberhauser 2008;Braschler et al., 2010). These outcomes would serve interns in their future careers, and improve the quality of the data that they provide to GLOBE Observer in the future.
Studies in STEM education have demonstrated the myriad benefits of place-based learning for students (Ernst and Monroe. 2004;Semken et al., 2017;Johnson et al., 2020). This is particularly so in the geosciences: many Earth science phenomena, such as climate change, are abstract and take place at a spatial scale that is hard for learners to fully grasp (Sarkar and Frazier 2008;DeFelice et al., 2014). Place-based learning contextualizes these challenging topics and makes them locally relevant to students in formal education settings (Semken et al., 2017). Citizen science programs bring place-based education into informal learning (National Research Council, 2009). "Sense of place" has been described as the emotional, generative interaction between a person and a place, a sense which is emphasized and reified by experiences with place-based citizen science (Haywood 2014;Newman et al., 2017;Haywood et al., 2021). Evaluation of the Mosquito Mappers research internship demonstrated that the local citizen science fieldwork successfully used a sense-of-place pedagogy to motivate students. Because citizen scientists are necessarily interacting closely with their local environment when they take observations, the GLOBE Observer helps promote a connection between the volunteer and their community. By allowing interns to craft research projects centered within their own communities, for the benefit of community members (including family, neighbors and friends), the program created personal significance and community relevance.
The SEES Mosquito Mappers virtual internship represents a viable pathway towards the development of a more dynamic and diverse population of STEM engaged citizen science volunteers. A well-known challenge that citizen science programs face is in recruiting and retaining a diverse cohort of volunteers (Pandya 2012;Fischer et al., 2021). The common profile of a citizen science volunteer is older, white, and well-educated (Blake et al., 2020;Fischer et al., 2021). It can be a struggle to attract younger and demographically diverse volunteers to citizen science programs. For young people who are focused on school, social activities, and college readiness, citizen science may not seem like an appealing hobby. By incorporating citizen science into a structured virtual internship, and-as described above-adding a layer of significance and relevance to the experience, younger generations of STEM learners can be drawn into citizen science in ways that are aligned with their interests and career goals. The virtual nature of this particular internship also makes research experience, and citizen science, accessible to a broader, more diverse group of young people (Medeiros et al., 2015).
Internship programs like the Mosquito Mappers internship, but likely many other informal STEM learning programs for youth may stand to benefit from the inclusion of citizen science assets in the program experience. The digital infrastructure provided by citizen science programs enable robust and intensive, and also authentic science research experiences to be conducted remotely. They can readily substitute for some of the critical services once previously only found in brick and mortar research facilities. Not only can these investments in data collection tools, mobile apps, data archives, and analysis tools be deployed at little to no cost, it also frees internship opportunities from accessing university assets that were previously a necessity. The COVID-19 pandemic has underlined the importance of developing quality virtual educational opportunities for future STEM leaders, as residential internships were not possible during the summer of 2020. The SEES Mosquito Mappers research internship was already equipped with a virtual pedagogic model, distributed data collection sampling strategy and collaborative data analysis platform that ended up being critical for engaging students in a robust research experience during pandemic conditions.
The young people who completed the SEES virtual internship represent a generation of STEM learners that can bridge the gap between citizen scientists and scientists. Unlike the typical citizen science volunteer, the SEES interns are just beginning their journeys in STEM. By participating in an internship opportunity that introduces them to place-based citizen science as both volunteers and researchers in their own communities, interns experience first-hand the democratic values of inclusion, access, and community engagement. These are values that they will bring with them as they pursue careers in STEM. It also provides authentic science experiences to students who are unable to travel, because of either work or family responsibilities, or mobility issues. Future evaluative and research work should explore the longer-term impacts and outcomes of programs like this that introduce young people to place-based citizen science and aims at fostering career orientation and possible selves.
In summary, internship opportunities such as the SEES program are a viable way to connect young people with citizen science initiatives, in a way that benefits both the citizen science program as well as the internship itself. The infrastructure and tools of the GLOBE Observer citizen science program provided a ready to use framework that supported a freestanding science research internship experience accessible to students within their own communities. The project described here engaged interns' personal knowledge of place and had a potential impact on the well-being of their own community, two factors that appear to contribute to the successful engagement of this student cohort.

DATA AVAILABILITY STATEMENT
The raw data supporting the conclusion of this article will be made available by the authors, without undue reservation.

ETHICS STATEMENT
Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. Written informed consent from the participants' legal guardian/next of kin was not required to participate in this study in accordance with the national legislation and the institutional requirements.

AUTHOR CONTRIBUTIONS
HC and HF collected evaluation data. HC performed the data analysis and was the primary author of the article. HF and MS reviewed and supported the preparation of the article. RL contributed to sections of the article describing the GLOBE Observer citizen science app and the SEES Mosquito Mappers Summer High School Research Internship. The science and education team at the Institute for Global Environmental Strategies, led by RL, conceived of the diversity-driven, virtual outreach model for the SEES Mosquito Mappers. RL designed and directed the place-based summer internship program, and co-mentored the interns with NASA SME Peder Nelson.