Emergency remote instruction in 2020: Differential impacts on science, technology, engineering, and mathematics students’ confidence and belonging, by gender, race/ethnicity, and socioeconomic status

Introduction

The COVID-19 pandemic severely impacted higher education in many ways, such as the emergency shutdown of campuses around the world in spring 2020. These shutdowns made the standard, in-person class experience impossible for many months. Even throughout academic year 2020–2021 face-to-face instruction was the exception, not the rule (see Garrett et al., 2021). We consider most non-face-to-face instruction throughout the first year of the COVID-19 pandemic to be emergency remote instruction, which is defined as teaching in response to crisis and disaster rather than a well-planned, permanent, and intentionally online course (Hodges et al., 2020). This distinction is crucial for many reasons such as the fact that instructors of well-planned, permanent, intentionally online courses typically have more experience, training, and support, and their students typically have a choice regarding this modality, something not always available during emergency remote instruction. Documenting how emergency remote instruction impacted students is important for several reasons. First, it may inform interventions meant to help current students recover from this shock to higher education. Second, such knowledge may help faculty and university administrators better prepare for future crises, whether natural, political, or economic. Finally, we may glean surprising insights for the simple reason that we have no precedent for this instructional context. Thus, the overarching aim of this work was to document and explore the impact of emergency remote instruction on undergraduate STEM students in Spring and Fall 2020.

The present work is part of a parent project to study impacts of COVID-19 on undergraduate students in the United States majoring in science, technology, engineering, and mathematics (STEM). Students can experience the same classroom in different ways depending on the relationships of their identities (e.g., gender, race, socioeconomic status) to systems of power, privilege, and oppression (Harper and Hurtado, 2007). These impacts can have complex and sometimes compounding influences on STEM students’ experiences (Seymour and Hunter, 2019), thus we utilize a framework inspired by theories of intersectionality from the outset (Crenshaw, 2018). STEM fields suffer from a persistent lack of diversity (Fry et al., 2021), hence understanding experiences associated with multiple student identities is particularly salient.

Our focus is on students’ confidence and sense of belonging. Confidence in an academic setting is the belief in one’s ability as a student (Naird, 2005). Some courses such as calculus can have a substantial impact on confidence (Bressoud, 2015) which can, in turn, influence a student’s choice of major (Moakler and Kim, 2014; Ellis et al., 2016). Sense of belonging refers to a students’ feeling of connectedness to and acceptance in an academic community (Strayhorn, 2019). It is a critical factor in the overall success of students, particularly underrepresented students who may otherwise feel out of place or suffer from a “self-fulfilling prophecy” of stereotypes (Strayhorn, 2019, p. 90), and students’ perceptions of climate and connections at an institution can impact belonging in positive and negative ways (see Johnson, 202). This study focuses on confidence and belonging for two main reasons: both are associated with student persistence and success (e.g., degree attainment and GPA) and both are malleable (National Academies of Sciences Engineering and Medicine, 2017), meaning that new learning environments may impact them. Since emergency remote instruction drastically altered learning environments, it is reasonable to expect impacts on both variables. However, emergency remote instruction was unprecedented, so we cannot assume previous research on confidence and belonging, whether in the face-to-face or online-learning setting (Peacock et al., 2020), generalizes to this context. Thus, the specific aim of this work was to explore impacts of emergency remote instruction on undergraduate STEM students’ confidence and sense of belonging by gender, race/ethnicity, and socioeconomic status (SES).

Methods

In December 2020 and January 2021, we recruited students attending US colleges and universities to complete an online survey about their experiences and perceptions during Fall 2020. The present work analyzes a subset of data from this parent study. Participants received a $30 Amazon gift card as compensation for completing the survey. Penn State University’s and University of Connecticut’s Institutional Review Boards approved this study.

Students for the parent study were recruited via social media platforms (e.g., Twitter, Facebook), university-affiliated listservs, STEM student organizations (e.g., National Society for Black Engineers), and emails to faculty in the Math Alliance (a national organization of faculty in mathematics). Eligible students were 18 years or older, declared or intended STEM majors, enrolled full-time at a college or university in the United States, and met the criteria for inclusion in one of 18 groups stratified by race/ethnicity, gender, and SES as described below. Enrollment in each stratum was capped at 63 students due to financial constraints. Once the cap for a particular stratum was reached, subsequent respondents meeting eligibility criteria for that stratum were not enrolled. One stratum contained 64 participants (the 64th was completing the survey when the enrollment cap for their stratum was met, so the data collection platform kept them in the study) while the other 17 strata contained 63 participants, for a total of 1,135 participants enrolled in the parent study.

STEM majors included students in the natural sciences (e.g., biology, physics), engineering, mathematical sciences, and technology (e.g., computer science) but excluded students in the social sciences (e.g., psychology). Participants were asked to provide their race, gender, ethnicity, age, family income, and parental education level. Participants reported their gender as woman, man, or “other” with a write-in option. Students who did not identify as women or men were excluded due to our inability to recruit enough non-binary participants in each of the racial/ethnic and socioeconomic categories under consideration. Participants reported whether their ethnic identity is Hispanic/Latinx and reported how they describe their race(s), including Native American (e.g., American Indian, Alaskan Native, Indigenous), Native Hawaiian/Pacific Islander, Black/African American, White/Caucasian, Middle Eastern/Arab American, South East/East Asian (e.g., Malaysian, Vietnamese, Chinese, Japanese), or South Asian (e.g., Indian, Sri Lankan), or another race. Based on responses, we categorized participants as Hispanic/Latinx (any race or combination of races), non-Hispanic Asian (South East/East or South Asian), non-Hispanic Black, non-Hispanic White, or “other,” which included multiracial participants and those whose racial identity is other than Asian, Black, or White. We categorized participants as lower SES if they met one or more of the following criteria: (1) were eligible for work-study or Pell grants (forms of financial aid in the United States), (2) reported that it was very hard for their family to pay for basics (e.g., rent, heat, food), or (3) parental education of at most a high school diploma or GED combined with a household income < $75,000. We categorized them as higher SES if they met all of the following criteria: (1) were not eligible for work-study, (2) were not eligible for a Pell Grant, (3) reported that it was not hard at all or somewhat hard to pay for basics, and (4) parental education was a bachelor’s degree or higher. Students whose responses did not place them in either SES group were excluded from the parent study. With this information, participants were stratified into one of 18 categories. Sixteen enrollment strata were defined by gender (men vs women), race/ethnicity (Hispanic, non-Hispanic Asian, non-Hispanic Black, non-Hispanic white), and SES (lower vs higher). Two additional strata enrolled men and women who met criteria for lower or higher SES and identified as multiracial or another race.

For the current study, we collapsed race/ethnicity according to their representation in STEM fields. Non-Hispanic whites and Asians were collapsed into “non-underrepresented minorities (non-URM),” since their percentages in STEM fields exceeds their percentages in the US population, while all other races/ethnicities were collapsed into “underrepresented minorities (URM).” Thus, participants for this study were stratified by representation in STEM (non-URM vs. URM), gender (men vs. women), and SES (lower vs. higher). Our categorization of students identifying as Asian is consistent with the National Institute of Health: “A person having origins in any of the original peoples of the Far East, Southeast Asia, or the Indian subcontinent including, for example, Cambodia, China, India, Japan, Korea, Malaysia, Pakistan, the Philippine Islands, Thailand, and Vietnam” (U.S. Department of Health and Human Services, n.d.). However, we acknowledge that this masks variation in STEM representation among some southeast Asian demographics, such as Hmong, who remain underrepresented (Her, 2019). We also acknowledge that overrepresentation does not eliminate harmful stereotypes and discrimination.

Measures for this study were derived from the following questions in the parent survey.

In Spring and Fall 2020, many college courses were conducted online. To what extent has learning online impacted the following:

• Your confidence that you can succeed in a STEM major?

• Your sense that you belong in a STEM major?

Responses were captured with a continuous visual analog scale ranging from –100 to 100, with anchors at –100 (much less confident/belonging); 0 (no impact); and 100 (much more confident/belonging). The default position of the slider was 0, so participants could move in the negative/positive direction to indicate impact. If a participant did not click on a particular slider, it was recorded as “no response” by REDCap (Harris et al., 2019), our data-collection platform. In previous studies conducted by members of our research team, participants commented that they didn’t click on the slider if it was already where they would place it. Therefore, for participants who clicked a slider for at least one item within the block of questions, we imputed missing responses as zeros (i.e., “no impact”). Two participants did not click any of the sliders, did not receive imputed values, and were excluded from our analytic sample (N = 1133). We categorized impact ratings as “decreased” if they were below –10; “minimal impact” if they were between –10 and 10; and “increased” if they were greater than 10. We used chi-squared tests to examine whether perceived impact of remote learning (decreased, minimal impact, increased) differed by demographic group. Sensitivity analyses were performed and revealed consistent results with and without imputation (data not shown).

Results

Table 1 shows the contingency table for how students reported the impact emergency remote learning had on their confidence by demographic strata. Table 2 shows a similar contingency table for how students reported the impact on their sense of belonging in STEM.

TABLE 1

Table 1. Impact of emergency remote instruction during 2020 on students’ confidence (N = 1133), column% (N).

TABLE 2

Table 2. Impact of emergency remote instruction during 2020 on students’ sense of belonging in STEM (N = 1133), column% (N).

Confidence

Overall, 48.2% of students reported that emergency remote instruction decreased their confidence that they can succeed in a STEM major, 26.1% reported minimal impact, and 25.7% reported increased confidence. Impact on confidence statistically differed across the demographic groups (χ²(14) = 39.86, p < 0.0001). While the effect size is modest (Cramer’s V = 0.13’ Cohen, 1988), these are impactful differences. For example, nearly two-thirds (63.3%) of URM, low SES women reported a decreased sense of confidence, compared to 38.9% of non-URM, high SES men. Non-URM, high SES women had the highest percentage reporting minimal impact to their confidence (31.7%). Over one third of URM, low SES men (34.6%) reported an increase in confidence levels, compared to 18.4% of URM, low SES women.

Belonging

Overall, 36.5% of students reported that emergency remote instruction decreased their sense of belonging in STEM, 38.3% reported minimal impact, and 25.2% reported an increased sense of belonging. The impact of emergency remote instruction on students’ sense of belonging differed among the demographic groups (χ²(14) = 39.86, p < 0.0001; Cramer’s V = 0.16; Table 2). More than half of URM, low SES women (54.4%) reported a decrease in belonging, compared to 24.6% of non-URM high SES men. Half of non-URM, high SES students reported minimal impact to their belonging (50.0% for men and 49.2% for women), compared to about a quarter of low SES URM students (28.8% for men and 26.6% for women). About 1/3 of URM men from both low SES (35.3%) and high SES (33.3%) reported an increase in sense of belonging, which was higher than other groups.

Discussion

Emergency remote instruction disrupted traditional forms of instruction, communication, and socialization on campus. The altered campus climate and learning environment affected students’ confidence and sense of belonging. Across all demographic groups, a large percentage of undergraduate STEM students reported that emergency remote instruction in 2020 decreased their confidence to succeed and sense of belonging in STEM. However, the impact was not uniform across demographic groups: for both confidence and sense of belonging, URM, low SES women had the highest percentage reporting decreases while non-URM, high SES men had the lowest. These findings are consistent with previous studies of confidence and sense of belonging of women of color in STEM (see MacPhee et al., 2013; Rainey et al., 2018), and suggest a compounding effect of gender, race/ethnicity, and SES. They also illustrate the potential of non-intersectional research methods to obfuscate the experience of URM students, since data from non-URM students will drive results when pooled together with URM students. The variation we observed by gender, race/ethnicity, and SES reaffirms the need for an intersectional lens in STEM-education research, and more broadly.

The increases in confidence and sense of belonging reported by URM, low SES men were unexpected and suggest a direction for future work. Indeed, research suggests that traditional face-to-face STEM classrooms can be aversive environments for students of color (Leyva et al., 2021, 2022). For instance, “problems with climate and belonging often manifest from status competitions that are instigated by some students to gain advantage, and assert their own superior belonging, by stigmatizing and excluding others” [emphasis in original] (Seymour and Hunter, 2019, p. 303). It is possible that in the traditional classroom URM, low SES men’s racial/ethnic identities and socioeconomic status are used against them in these status competitions. However, removing the physical presence through remote instruction may allow their identities as men to take precedent and therefore reduce the effects of race and SES. This could also explain why a similar increase in confidence and sense of belonging is not reported in URM, low SES women, because their gender identities alone do not signal the same belonging in STEM as men. However, research into URM men’s experience is needed to further understand the impact of classroom climate and behaviors on their confidence and sense of belonging.

This work is not without its limitations. First, due to length constraints, the parent survey study from which our data is derived used researcher generated questions about confidence and belonging, as opposed to validated measures, which potentially limits the ability of the data to capture these constructs. Second, it may be difficult to disentangle students’ experience of emergency remote instruction from other important contextual issues such as mental or physical health, basic needs insecurities, living arrangements, or familial support. The parent study asked about these issues and more, but analyses attempting disentanglement are beyond the scope of the current work. Finally, because our sample was comprised of students from various years in college this may have resulted in some uncertainty about what online learning was compared to. For example, participants in their second year may be comparing their experiences to their first year of college while first year students are comparing online learning to high school. We were interested in the broad impact of emergency remote instruction on confidence and belonging and future work should examine the potential implications of various comparisons.

Consistent with prior research contexts (Rainey et al., 2018; Seymour and Hunter, 2019), our results suggest differential impacts of emergency remote instruction during the COVID-19 pandemic by gender, race/ethnicity, and SES. This is also consistent with research during the pandemic that demonstrates differential impacts by gender on the sense of belonging of students in computer science (Mooney and Becker, 2021), though insufficient sample sizes may have prevented that study from observing more nuanced differences by race/ethnicity and their intersections with gender. This leads us to several implications. First, future research in distance education should attend to student’s multiple identities to address educational equity even when it is seemingly unseen because of modality. Further, interventions meant to help students recover from the COVID-19 pandemic, or future students facing similar crises, must take an intersectional approach to students’ identities, lest they risk further exacerbating disparities in STEM education. Institutions should explore how students’ confidence and sense of belonging changed during the pandemic, and how to retain diverse students in STEM in the coming years. Our work highlights the importance of an intersectional perspective in research related to the impact of the pandemic on STEM higher education. For example, research has found that STEM students had both health (e.g., mental health, socialization) as well as educational concerns (e.g., hands-on experiences; Palmer et al., 2021) but this research did not investigate these concerns with an intersectional perspective. The current work suggests that an intersectional framework may provide a nuanced understanding to student concerns and challenges during the pandemic. Additionally, the lasting impact on academic success because of the pandemic is unknown and may challenge what is previously known for how students engage with and feel about their academic major. As higher education researchers continue to map out this landscape, it is imperative to take an intersectional approach, attending to students’ multiple identities and developing targeted interventions to prevent losses of URM students in the STEM workforce. Finally, as higher education administrators and faculty continue to welcome students back to campus, the impact of emergency remote learning on confidence and sense of belonging cannot be overlooked. Reviews and adaptations to student programming, strategic plans, policies, and enrollment targets with retention efforts are warranted.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving human participants were reviewed and approved by Institutional Review Board (IRB) at The Pennsylvania State University and IRB at University of Connecticut. The patients/participants provided their written informed consent to participate in this study.

Author contributions

All authors listed have made a substantial, direct, and intellectual contribution to the work, and approved it for publication.

Funding

This research was supported by grants from the National Science Foundation (PI: Brown, NSF #2028344; PI: Pagoto, co-PI: Waring, NSF #2028341), with additional support for SP provided by National Institute of Health (NIH) grant K24HL124366.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Bressoud, D. (2015). Insights from the MAA national study of college calculus. Math. Teach. 109, 179–185. doi: 10.5951/mathteacher.109.3.0178

CrossRef Full Text | Google Scholar

Cohen, J. (1988). Statistical power analysis for the behavioral sciences, 2nd Edn. Hillsdale, NJ: L. Erlbaum Associates.

Google Scholar

Crenshaw, K. (2018). “Demarginalizing the intersection of race and sex: A Black feminist critique of antidiscrimination doctrine, feminist theory, and antiracist politics [1989],” in Feminist LEGAL theory, eds K. Bartlett and R. Kennedy (New York, NY: Routledge), 57–80. doi: 10.4324/9780429500480-5

CrossRef Full Text | Google Scholar

Ellis, J., Fosdick, B. K., and Rasmussen, C. (2016). Women 1.5 times more likely to leave STEM pipeline after calculus compared to men: Lack of mathematical confidence a potential culprit. PLoS One 11:e0157447. doi: 10.1371/journal.pone.0157447

PubMed Abstract | CrossRef Full Text | Google Scholar

Fry, R., Kennedy, B., and Funk, C. (2021). STEM jobs see uneven progress in increasing gender, racial and ethnic diversity. Pew Research Center Science & Society. Available online at: https://www.pewresearch.org/science/2021/04/01/stem-jobs-see-uneven-progress-in-increasing-gender-racial-and-ethnic-diversity/

Google Scholar

Garrett, R., Simunich, B., Legon, R., and Fredericksen, E. E. (2021). CHLOE 6: Online learning leaders adapt for a post-pandemic world, the changing landscape of online education, 2021. Available online at: qualitymatters.org/qa-resources/resource-center/articles-resources/CHLOE-project

Google Scholar

Harper, S., and Hurtado, S. (2007). Nine themes in campus racial climates and implications for institutional transformation. Racial Ethnic Divers. High. Educ. 120, 7–24. doi: 10.1002/ss.254

CrossRef Full Text | Google Scholar

Harris, P. A., Taylor, R., Minor, B. L., Elliott, V., Fernandez, M., O’Neal, L., et al. (2019). REDCap Consortium, The REDCap consortium: Building an international community of software partners. J. Biomed. Inform. 95:103208. doi: 10.1016/j.jbi.2019.103208

PubMed Abstract | CrossRef Full Text | Google Scholar

Her, T. (2019). The underrepresentation of Hmong American college students in STEM majors. Ph.D. dissertation. Fresno, CA: California State University.

Google Scholar

Hodges, C. B., Moore, S., Lockee, B. B., Trust, T., and Bond, M. A. (2020). The difference between emergency remote teaching and online learning. EDUCAUSE Review. Available online at: https://er.educause.edu/articles/2020/3/the-difference-between-emergency-remote-teaching-and-online-learning

Google Scholar

Johnson, R. (202). Campus racial climate perceptions and overall sense of belonging among racially diverse women in STEM majors. J. Coll. Stud. Dev. 53, 336–346. doi: 10.1353/csd.2012.0028

PubMed Abstract | CrossRef Full Text | Google Scholar

Leyva, L. A., Amman, K., Wolf, McMichael, E. A., Igbinosun, J., and Khan, N. (2022). Support for all? Confronting racism and patriarchy to promote equitable learning opportunities through undergraduate calculus instruction. Int. J. Res. Undergrad. Math. Educ. 8, 339–364. doi: 10.1007/s40753-022-00177-w

CrossRef Full Text | Google Scholar

Leyva, L. A., McNeill, R. T., Marshall, B. L., and Guzmán, O. A. (2021). “It seems like they purposefully try to make as many kids drop”: An analysis of logics and mechanisms of racial-gendered inequality in introductory mathematics instruction. J. High. Educ. 92, 784–814. doi: 10.1080/00221546.2021.1879586

CrossRef Full Text | Google Scholar

MacPhee, D., Farro, S., and Canetto, S. (2013). Academic self-efficacy and performance of underrepresented STEM majors: Gender, ethnic, and social class patterns. Anal. Soc. Issues Public Policy 13, 347–369. doi: 10.1111/asap.12033

CrossRef Full Text | Google Scholar

Moakler, M., and Kim, M. (2014). College major choice in STEM: Revisiting confidence and demographic factors. Career Dev. Q. 62, 128–142. doi: 10.1002/j.2161-0045.2014.00075.x

CrossRef Full Text | Google Scholar

Mooney, C., and Becker, B. A. (2021). Investigating the impact of the COVID-19 pandemic on computing students’ sense of belonging. ACM Inroads 12, 38–45. doi: 10.1145/3463408

CrossRef Full Text | Google Scholar

Naird, T. (2005). College students’ experiences with diversity and their effects on academic self-confidence, social agency, and disposition toward critical thinking. Res. High. Educ. 46, 365–387. doi: 10.1007/s11162-005-2966-1

CrossRef Full Text | Google Scholar

National Academies of Sciences Engineering and Medicine (2017). Supporting students’ college success: The role of assessment of intrapersonal and interpersonal competencies. Washington, DC: National Academies Press.

Google Scholar

Palmer, L. E., Pagoto, S. L., Workman, D., Lewis, K. A., Rudin, L., de Luna, N., et al. (2021). Health and education concerns about returning to campus and online learning during the COVID-19 pandemic among US undergraduate STEM majors. J. Am. Coll. Health. [Epub ahead of print]. doi: 10.1080/07448481.2021.1979009

PubMed Abstract | CrossRef Full Text | Google Scholar

Peacock, S., Cowan, J., Irvine, L., and Williams, J. (2020). An exploration into the importance of a sense of belonging for online learners. Int. Rev. Res. Open Distrib. Learn. 21, 18–35. doi: 10.19173/irrodl.v20i5.4539

CrossRef Full Text | Google Scholar

Rainey, K., Dancy, M., Mickelson, R., Stearns, E., and Moller, S. (2018). Race and gender differences in how sense of belonging influences decisions to major in STEM. Int. J. STEM Educ. 5:10. doi: 10.1186/s40594-018-0115-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Seymour, E., and Hunter, A. B. (2019). Talking about leaving revisited: Persistence, relocation, and loss in undergraduate STEM education. Cham: Springer. doi: 10.1007/978-3-030-25304-2

CrossRef Full Text | Google Scholar

Strayhorn, T. (2019). College students’ sense of belonging: A key to educational success for all students, 2nd Edn. New York, NY: Routledge. doi: 10.4324/9781315297293

CrossRef Full Text | Google Scholar

U.S. Department of Health and Human Services (n.d.). Not-OD-15-089: Racial and ethnic categories and definitions for NIH Diversity Programs and for other reporting purposes. National Institutes of Health. Available online at: https://grants.nih.gov/grants/guide/notice-files/not-od-15-089.html#:∼:text=The%20revised%20standards%20contain%20five,%22Not%20Hispanic%20or%20Latino.%22 (accessed August 26, 2022).

Google Scholar