What Kind of Students Attend Cyber Schools? Pandemic Enrollment as Evidence of Negative Selection

Background

Full-time, online schools, known as “cyber schools,” have rapidly expanded in the last 20 years, making research a priority (Molnar et al., 2019). Student enrollment in these schools increased by nearly 30,000 students between 2017–2018 and 2019-2020 alone (Molnar et al., 2019). Early research by Clark (2000) first named these schools as “cyber schools,” categorizing them among six other kinds of online learning programs. Since that point, this terminology has been used in the “Virtual Schools in the U.S.” reports that are regularly published by the National Educational Policy Center (e.g., Molnar et al., 2019).

Cyber schools have consistently underperformed academically compared to traditional public schools (for summaries by charter school supporters see Finn et al., 2016; see Saultz and Fusarelli, 2017 for a summary by critics of cyber charter schools; also see Molnar et al., 2019 for a more balanced summary). The Center for Research on Educational Outcomes (CREDO) produced oft-cited descriptive research that indicates that these negative effects are statistically and practically significant. Still, some research hints at disproportionate negative selection into virtual charter schools (Beck et al., 2014; Bueno, 2020; Paul and Wolf, 2020). Recent research concludes that even controlling for prior achievement does not sufficiently account for this negative selection (Paul and Greene, 2022), a finding that supports the theory “that parents choose to enroll their children in [virtual schools] because of problems or ‘shocks’ experienced in their previous school that might be connected to drops in student performance.” (Lueken et al., 2015, p. 328).

The influx of students into virtual schools during the COVID-19 pandemic provides a unique opportunity to shed further light on what type of students typically select virtual charters. In theory, students enrolling due to COVID-19 should closely resemble the general population of brick and mortar students on observable and unobservable characteristics because they are enrolling due to extrinsic (i.e., a pandemic) rather than intrinsic forces (e.g., social emotional challenges). To that end, academic differences between the “COVID cohort” and other virtual charter students might offer clues regarding the differences between virtual charter students and public school students generally. In this study, we use data from a large cyber charter network (“Countrywide Cyber”) to assess whether students who enrolled in full time virtual schools due to COVID-related concerns performed better on entry diagnostic assessments.

COVID-19 Pandemic and Cyber Schools

The worst pandemic since the 1918-1920 flu outbreak, COVID-19 has killed over one million Americans and thoroughly disrupted all facets of life, including schooling. There is already significant literature addressing how the pandemic has affected in person public schools, the majority of which shifted to temporary remote learning. Yet there is scant literature regarding COVID-19’s impacts on preexisting online schooling. Molnar et al. (2019) report that as of the 2017-2018 school year 501 full-time virtual schools enrolled approximately 297,000 full time students. Charter schools accounted for 79.1% of enrollment.

The rapidly growing research literature on the impact of COVID-19 on K-12 schooling has a noted gap: no prior work focuses on how COVID-19 affected cyber charter schools. Prior empirical work finds that cyber charter schools have lower academic value added than both charter and traditional public in person schools (e.g., Woodworth et al., 2015), and that artificial testing conditions only play a marginal role in explaining this gap (Beck et al., 2018; Kingsbury et al., 2020). Some work suggests that lower cyber charter performance may in part reflect student composition, with students whose needs are not met in in-person settings disproportionately choosing cyber options, but also relatively more in need of the in-person support which virtual schools have difficulty providing (Ahn and McEachin, 2017; Paul and Greene, 2022).

The COVID-19 pandemic provides an opportunity to better understand whether the observed performance of cyber charter schools is explained by school performance or negative selection of students into cyber schools. Rather than enrolling due to negative shocks or other circumstances that negatively predict achievement, the “COVID cohort” believed that dedicated virtual programs (district or charter) could deliver a better virtual education than brick and mortar schools that switched to emergency remote learning (Flanders, 2021). Recent research by Maranto et al. (2021) compared parent survey responses from newly enrolled students entering a large national cyber charter school network in Spring 2020, during the pandemic, with parents of students entering in 2019 and 2018, before the pandemic, and found that the COVID cohort resembled prior groups demographically but reported a substantially lower incidence of bullying, mental health issues, and physical health issues as reasons for enrollment. However, the only performance metrics that the study compared were grade point average and curriculum-based assessments completed. While both metrics hint at stronger performance from the COVID cohort, neither metric is standardized, and the study only reports averages, omitting analysis of where changes occurred along the distribution of outcomes. Our analysis fills a gap by using a standardized performance metric and assessing where changes occurred along the distribution of achievement outcomes. Our analysis can inform the degree to which cyber charter observed performance reflects school quality rather than student composition.

Methods

Data was provided by a large education management organization (EMO) that manages tuition-free virtual charter schools across the United States. Student records contained standard demographic information as well as information about how students performed on their beginning of year 2020-21 STAR and NWEA tests, third-party computer-adaptive assessments used to diagnose learning levels. Both assessments are widely used in American public schools and have been deemed reliable and valid diagnostic tests (Bulut and Cormier, 2018; Institute for Education Sciences, n.d.). Student-level de-identified scores were provided as national proficiency ranks, a percentile rank for performance on each subject test compared to the universe of American students who participated in the same grade-level test. Data was provided for students who enrolled between March 14, 2020 and September 9, 2020. March 14 is one day after President Trump declared a national emergency in response to COVID-19, whereas the latter date represents the beginning of the 2020-21 school year for all schools served by the management organization. Overall, 95.2% of eligible students participated in 2020-2021 beginning of year NWEA and STAR assessments. English language arts (ELA) tests are dispensed to students in kindergarten through 12th grade (i.e., all primary years in American public education, from about age 5–18) whereas math tests are dispensed to students from 3rd through 12th grade. Though it is not clear why 4.8% of the eligible student body did not participate, missing data is unlikely to pose a threat to the validity of our analysis, as it would only bias our estimates if it was missing in a way that correlated with both whether enrollment was tied to COVID-19 and observed achievement, an unlikely scenario.

Critically, the EMO polled families in Fall 2021 asking them to assess the importance of several factors in their decision to enroll. Specifically, the survey prompts respondents that “Below are some reasons that parents have said they chose [School Name] for their child. For each, please indicate how well it describes why you chose [School Name] for [Student Name].” Respondents respond on a Likert scale with ranges from 1 (very unimportant or strongly disagree) to 5 (very important or strongly agree). Among the 16 potential reasons provided is “concerns related to COVID-19.” Adjoined to student-level assessment data, these responses allow us to gauge the extent to which the “COVID cohort” (i.e., those who reported that COVID-19 was an important enrollment factor) profiled differently from the traditional population served by cyber charter schools. Overall, responses were collected for 9,091 students, representing 34.3% of students who enrolled during that time. Among those, math scores were available for 7,243 students and ELA scores for 8,981.

Results

We begin by observing the degree to which the “COVID cohort” profiles differently from other cyber charter students by conducting two-sided t-tests on demographic variables, as seen in Table 1.

TABLE 1

Table 1. Demographics of those who enrolled in cyber charters depending on COVID as enrollment factor.

Overall, parents of students who responded that concerns about COVID-19 were “important” or “very important” regarding their decision to enroll profile very similarly in their demographic composition to parents who responded that COVID-19 was unimportant or very unimportant. Indeed, none of the t-tests indicated statistically significant differences.

Next, we employ the regression model below to gauge differences in the two groups of students according to academic preparedness.

${\text{Y}}_{\text{i}}=\mathrm{\beta }0+\mathrm{\beta }1\mathrm{C}\mathrm{O}\mathrm{V}\mathrm{I}{\mathrm{D}}_{\text{i}}+\mathrm{\beta }2{\mathrm{X}}_{\text{i}}+\mathrm{\beta }3{\mathrm{D}}_{\text{i}}+\mathrm{\beta }4{\mathrm{Q}}_{\text{i}}+e_{\text{i}}$

Y_i represents student percentile ranks on ELA or math test scores on their 2021 beginning of year NWEA or STAR assessment. Coefficient β1 denotes responses to the Likert scale question about the importance of COVID as an enrollment factor (1 = very unimportant, 5 = very important). X_i is a vector of indicator variables for each school managed by the EMO, D_i is a vector of demographic characteristics,¹ and Q_i is a vector of responses to other survey items that were asked alongside the question about COVID.²

Overall, results (Table 2) indicate that students whose enrollment was motivated by COVID were academically stronger, and that this difference was practically and statistically significant. Results were especially pronounced regarding ELA scores. For example, in the unadjusted model, a one-point increase on the Likert scale COVID question is associated with a 1.19 point increase in national percentile rank. Notably, the difference in the two groups of students is driven by differences on both tails of the distribution, as seen in Figure 1. That is, students enrolling due to COVID were more likely to be at the high-achieving end of the distribution and less likely to be at the low-achieving end. Illustratively, in averaging math and ELA percentile ranks, 14.6% of students not concerned about COVID scored between the 1st and 20th percentile compared to 10.6% of students concerned about COVID. Meanwhile, 12.8% of students not concerned about COVID were drawn from the top quintile compared to 15.7% of students concerned about COVID.

TABLE 2

Table 2. Concern about COVID as predictor of beginning of year percentile rank on NWEA/STAR assessments.

FIGURE 1

Figure 1. Distribution of average math/ELA scores on NWEA/STAR assessments depending on concern about COVID.

Discussion

Our findings indicate that students who enrolled in cyber charters due to COVID-19 were better academically prepared than students who enrolled for other reasons. The pandemic and associated switches to emergency remote learning plausibly invited a representative cross-section of students to enroll in cyber charters, including families who anticipated higher-quality virtual learning in devoted cyber schools and others who had concerns about COVID-19 or associated mitigation measures in schools with in-person learning. To that end, the results support evidence that students are often negatively selected into cyber charter schools (Paul and Greene, 2022).

Still, our findings do not put to rest discussion surrounding the degree to which cyber charter performance reflects school composition rather than quality. To be sure, we cannot be certain of the true representativeness of the “COVID cohort.” While our analysis of observed differences in learning levels between students who enrolled due to COVID versus other students hints at authentic differences, we do not have enough information about the representativeness of the COVID cohort to conclude that our estimates represent the discrete differences between students in the cyber charter sector versus students in brick and mortar schools.

Another notable limitation is that we did not have the data to assess differences in academic growth according to whether students enrolled due to COVID-19, as much of the criticism of the performance of virtual charter schools is directed not only at low test scores, but low observed year-over-year student growth. However, even with longitudinal assessment data, it’s not clear that the COVID-19 enrollment surge provides an instructive counterfactual to assess student growth in virtual charters. To the degree that the disruption caused by COVID-19 is a shock that likely impacts the academic performance of the COVID cohort downstream, it is dubious whether the growth of the COVID cohort provides an instructive counterfactual for academic growth within virtual charters.

Conclusion: Normalizing Cyber Schools?

Prior research indicated that many parents who chose cyber schools did so out of serious dissatisfaction at their in-person schools, often due to social factors like bullying, but also due to concerns that the child’s academic needs were unmet at their in-person school. This was particularly true for students with special education needs (Beck et al., 2014). These student-related factors may be among the drivers of the relatively weak cyber charter measured academic performance found in many studies (Lueken et al., 2015; Ahn and McEachin, 2017; Paul and Greene, 2022). The National Cyber data indicates that historical struggles characterized the COVID cohort to a lesser degree than prior cohorts.

In short, in the COVID era a greater percentage of new entrants to cyber schools may be those who were thriving rather than struggling at in person schools, but who were dissatisfied by their traditional public schools’ adjustments to COVID. Survey research indicates that nationally, such parental dissatisfaction may reflect relatively ineffective implementation of hastily prepared online learning options in traditional schools (Henderson et al., 2021; Kingsbury, 2021).

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Author Contributions

IK spearheaded data collection and analysis, and authored the results section. DB wrote the literature review. MB-D reviewed the analysis and revised the first draft of the manuscript. All authors contributed to the article and approved the submitted version.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

1. ^ The demographic control variables are the same variables that appear in Table 1.
2. ^ The EMO that provided data has requested that we not reproduce the full list of survey items, though generally speaking these 16 survey items cover a rich variety of factors and all responses are expressed on a 1–5 Likert scale reflecting responses from strongly disagree to strongly agree or very unimportant to very unimportant.

References

Ahn, J., and McEachin, A. (2017). Student enrollment patterns and achievement in Ohio’s online charter schools. Educ. Res. 46, 44–57. doi: 10.3102/0013189X17692999

CrossRef Full Text | Google Scholar

Beck, D., Egalite, A., and Maranto, R. (2014). Why they choose and how it goes: comparing special education and general education cyber student perceptions. Comput. Educ. 76, 70–79. doi: 10.1016/j.compedu.2014.03.011

CrossRef Full Text | Google Scholar

Beck, D., Watson, A., and Maranto, R. (2018). Do testing conditions explain cyber charter schools’ failing grades? Am. J. Distance Educ. 33, 46–58. doi: 10.1080/08923647.2019.1554989

CrossRef Full Text | Google Scholar

Bueno, C. (2020). Bricks and Mortar vs. Computers and Modems: The Impacts of Enrollment in K-12 Virtual Schools. Available Online at: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3642969 (accessed March 1, 2022).

Google Scholar

Bulut, O., and Cormier, D. (2018). Validity evidence for progress monitoring with Star reading: slope estimates, administration frequency, and number of data points. Front. Educ. 3:68. doi: 10.3389/feduc.2018.00068

CrossRef Full Text | Google Scholar

Clark, T. (2000). Virtual High Schools: State of the States - A Study of Virtual High School Planning and Preparation in the United States: Center for the Application of Information Technologies. Macomb, IL: Western Illinois University.

Google Scholar

Finn, C., Manno, B., and Wright, B. (2016). Charter Schools at the Crossroads. Cambridge, MA: Harvard Education Press.

Google Scholar

Flanders, W. (2021). Opting out: enrollment trends in response to continued public school shutdowns. J. Sch. Choice 15, 331–343. doi: 10.1080/15582159.2021.1917750

CrossRef Full Text | Google Scholar

Henderson, M., Houston, D., Peterson, P., and West, M. (2021). What American Families Experienced When COVID-19 Closed Their Schools. Education Next. Available online at: https://www.educationnext.org/wp-content/uploads/2022/01/ednext_XXI_1_poll_pandemic_henderson_et_al.pdf

Google Scholar

Institute for Education Sciences (n.d.). The Impact of the Measures of Academic Progress on Differentiated Instruction and Student Achievement. Available Online at: https://ies.ed.gov/ncee/edlabs/projects/rct_245.asp?section=ALL

Google Scholar

Kingsbury, I. (2021). Online learning: how do brick and mortar schools stack up to virtual schools? Educ. Inf. Technol. 26, 6567–6588. doi: 10.1007/s10639-021-10450-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Kingsbury, I., Maranto, R., and Beck, D. (2020). Road weary? Testing whether long commutes to testing sites explain deficient cyber charter school academic performance. J. Sch. Choice 14, 471–481. doi: 10.1080/15582159.2020.1845526

CrossRef Full Text | Google Scholar

Lueken, M., Ritter, G., and Beck, D. (2015). Value-added in a virtual learning environment: an evaluation of a virtual charter school. J. Online Learn. Res. 1, 305–335.

Google Scholar

Maranto, R., Beck, D., Clark, T., Tran, B., and Liu, F. (2021). Choosing cyber during COVID. Phi Delta Kappan 103, 30–33. doi: 10.1177/00317217211043622

CrossRef Full Text | Google Scholar

Molnar, A., Miron, G., Elgeberi, N., Barbour, M. K., Huerta, L., Shafer, S. R., et al. (2019). Virtual Schools in the U.S. 2019. Boulder, CO: National Education Policy Center.

Google Scholar

Paul, J., and Greene, J. (2022). Investigating the Relationship between Negative Selection into Online Schooling and Achievement Growth. Education Reform Faculty and Graduate Students Publications. Available Online at: https://scholarworks.uark.edu/edrepub/133 (accessed February 27, 2022).

Google Scholar

Paul, J., and Wolf, P. (2020). Moving on up? A Virtual School, Student Mobility, and Achievement. Annenberg Ed Working Paper No. 20-309. Available Online at: https://www.edworkingpapers.com/sites/default/files/ai20-309.pdf (accessed February 21, 2022).

Google Scholar

Saultz, A., and Fusarelli, L. (2017). Online schooling: a cautionary tale. J. Sch. Choice 11, 29–41. doi: 10.1002/hpja.203

PubMed Abstract | CrossRef Full Text | Google Scholar

Woodworth, J., Raymond, M., Chribas, K., Gonzalez, M., Negassi, Y., Snow, W., et al. (2015). Online Charter School Study. Stanford, CA: Center for Research on Education Outcomes.

Google Scholar