Self-Regulation Versus Self-Discipline in Predicting Achievement: A Replication Study With Secondary Data

Self-regulation is one of multiple noncognitive measures that have been found to relate to academic achievement. Zimmerman and Kitsantas (2014) have distinguished self-regulation from the construct of self-discipline and provided evidence that only the former is predictive of high-school GPA, while controlling for the latter. In this brief report we present a preregistered replication study of their finding using secondary data obtained from a published article by Jung et al. (2017). Despite minor differences in the sample, the measures and the analysis approach, the replication supported the original claim that self-regulation was predictive of academic achievement for undergraduate students, while self-discipline was unrelated to the outcome. The positive association for the self-regulation variable with academic achievement was smaller, but in the same direction as in the original study.


INTRODUCTION
Educational achievement depends on many factors (Hattie, 2009). Contextual and institutional variables, as well as individual difference variables appear to predict educational achievement. At the individual student level, cognitive abilities are positively associated with achievement (e.g., Deary et al., 2007). Self-regulation, self-efficacy, and other personality traits and noncognitive characteristics are also considered as conducive to positive school outcomes (e.g., Laidra, 2007;Lee and Shute, 2010;Lee and Stankov, 2018;Duckworth et al., 2019).
Broadly defined, "self-regulation is a multidimensional construct that includes the regulation of emotion, cognition, and behavior" (McClelland et al., 2010, p.510). Within the academic domain, self-regulation encompasses purposeful actions, monitored and sustained by the learner, to achieve learning goals. Consequently, research on self-regulation has developed from the study of metacognitive and cognitive strategies, to include motivational beliefs, such as goal orientations, self-efficacy, interest, and causal attributions (Zimmerman, 2011). Interventions focusing on the promotion of these motivational beliefs have been developed to support planning, adapting, and guiding their behavior, for attaining personal goals such as high academic achievement (e.g., Schunk and Gunn, 1986;Wibrowski et al., 2017). The theoretical framework of self-regulated learning, a broader umbrella term for variables influencing learning, has flourished with multiple models appearing in the literature (Panadero, 2017) and underscores the importance of this area of research in the teaching and learning process. Zimmerman and Kitsantas (2014) distinguished between selfregulation, as defined above, from self-discipline, which emphasizes the conscious control of "adverse personal performance" (p.146) to succeed in a task. Self-discipline is oriented towards successful performance outcomes, by overcoming impediments. Examples of self-discipline in academic tasks include "reading test instructions before proceeding to the questions, paying attention to a teacher rather than daydreaming [. . .] choosing homework over TV, and persisting on long-term assignments despite boredom and frustration (Duckworth and Seligman, 2006, p.199). In another study, Duckworth and Seligman (2005) refer to their multisource, multimethod instrumentation to measure "trait self-discipline" (p.942). This conceptualization diverges from the gradual skill development in processes such as planning, self-monitoring, and sustained effort emphasized in the self-regulation paradigm. In the five-factor model of personality, self-discipline is often referred to as a facet of conscientiousness, which has been found to correlate strongly with self-regulation (de la Fuente et al., 2020). Self-regulation and self-discipline independently have been found to empirically predict academic outcomes (e.g., Duckworth and Seligman, 2005;Zimmerman and Schunk, 2011). In a direct test of their relative contribution in predicting academic achievement, Zimmerman and Kitsantas (2014) found strong evidence in favor of self-regulation, but not for self-discipline. The present research attempts to replicate this claim with secondary data published in the literature.

METHODS
A secondary data analysis was conducted using data from the paper by Jung et al. (2017), a study that was conceptually related to Zimmerman and Kitsantas (2014). Their data were collected online from 366 undergraduate psychology students from a large mid-western public university in the United States (71.3% female, 76.5% white, M age 20.05).
We used the summary statistics and correlation coefficients from Table 2 in Jung et al. (2017) to construct a covariance matrix that served as input in a path analysis using R and the lavaan package (Rosseel, 2012). As an indicator of self-regulation, we used the "academic self-discipline" variable, a five-item scale score developed by Jung and others; items from the Big Five Inventory were adjusted to the school context "to measure contextualized self-regulation of effort" (p.36). Importantly, it included items like "I do a thorough job of studying for exams," and "When studying, I make a plan and follow through with it" which describe purposeful actions that the student monitors and sustains to achieve learning goals. As an indicator of selfdiscipline, we used the "self-discipline facet of conscientiousness" variable from Jung et al., a ten-item subscale score from the International Personality Item Pool (Goldberg et al., 2006); "get chores done right away" and "postpone decisions" are two indicative items from this subscale which highlight trait-like characteristics conducive to optimal task performance. Academic self-efficacy, a six-item scale score obtained from the Academic Self-Efficacy Scale (Roeser et al., 1996) was used as an additional control variable for exploratory purposes; an example item states "I can master the skills taught in school this year." Reported Cronbach's alphas for the three scales were 0.82, 0.90, and 0.84, respectively. The outcome variable was the semester Grade Point Average (GPA) obtained by the institution where the study was conducted. GPA was initially regressed on self-regulation; then, self-discipline was added as a predictor, as a test of the Zimmerman and Kitsantas (2014) original finding. Finally, an exploratory model was tested with academic self-efficacy specified as a third predictor.
This replication study was preregistered (https://osf.io/hj8bd/? view_only cdf8f39add9c4f2b8d850abc75d3d4cd) and was part of the effort by the Center for Open Science (COS) under the Systematizing Confidence in Open Research and Evidence (SCORE; cos.io/score) project, which aimed at evaluating the reproducibility of published findings in the social and behavior sciences. The sample size was, of course, out of our control, but power analyses conducted by COS suggested that the 366 observations exceeded the minimum threshold of 78 observations defined by power analysis needed to achieve 50% power to detect the full original effect, and was close to the 379 observations that would be needed to attain 90% power to detect 75% of the original effect. The power analysis, materials, data, and R code we used are provided on the Open Science Framework: https://osf.io/a4swd/?view_ only b871e151cbee4d60b59d260002cab1da.

RESULTS
In a baseline model, self-regulation of effort significantly predicted students' GPA (b 0.243, β 0.300, SE 0.040, p < 0.001; R 2 9.0%). For the primary path model, students' GPA was simultaneously regressed on the self-regulation of effort variable and the self-discipline facet of conscientiousness. Self-regulation of effort was positively and statistically significantly associated with students' GPA at a moderate degree (b 0.272, β 0.336, SE 0.056, p < 0.001), while controlling for students' levels on the self-discipline facet of conscientiousness, which was not associated with GPA (b −0.040, β −0.052, SE 0.053, p 0.454). The two predictors were strongly correlated, r 0.690, p < 0.001. Together they explained 9.1% of the variance in students' GPA.
Following the main replication result, academic self-efficacy was added as a predictor of GPA along with the two primary predictors in an exploratory mode, as described in the preregistration report. It had a weak, positive and statistically significant effect on GPA (b 0.141, β 0.138, SE 0.055, p 0.010). Controlling for academic self-efficacy, the unique association between self-regulation of effort and GPA was slightly smaller compared to the previous models, but still statistically significant (b 0.238, β 0.293, SE 0.057, p < 0.001), and the association between the self-discipline facet of conscientiousness and GPA remained nonsignificant (b −0.052, β −0.068, SE 0.053, p 0.324). Academic self-Frontiers in Education | www.frontiersin.org November 2021 | Volume 6 | Article 724711 efficacy was moderately correlated with the two predictors, r 0.390 and 0.330 respectively (ps < 0.001). This final model explained 10.7% of the variance in the outcome.

CONCLUSION
The current brief investigation aimed at replicating a finding by Zimmerman and Kitsantas (2014) on the predictive role of selfregulation on academic achievement for high school students, while controlling for self-discipline. Employing published summary data by Jung et al. (2017) from a tertiary education sample, we found support for that claim: self-regulation of effort was moderately predictive of semester GPA at the university. Selfdiscipline was not associated with GPA over and above selfregulation effort. Even though the two primary constructs were strongly correlated, only one was uniquely predictive of academic performance. This is consistent with findings suggesting that students emphasizing process versus product goals on learning tasks fare better in achievement outcomes (Schunk and Swartz, 1993), and with the distinction between mastery and performance in goal orientation theory (Ames and Archer, 1988). The replicated finding can be considered robust, since the relative contributions of self-regulation and self-discipline did not change when the additional noncognitive predictor of academic selfefficacy collected by Jung et al. (2017) was included as a control variable. The positive association for the focal self-regulation variable with the outcome was in the same direction as in the original study, albeit substantially smaller in magnitude (β 0.336 vis-àvis 0.96); higher self-regulation predicted higher achievement, but to a lesser degree compared to the strong original effect. While not invalidating the replication result, these differences may be attributed to deviations in the samples and the measures between the study by Jung et al. (2017) and Zimmerman and Kitsantas (2014) original study: in the replication, the sample consisted of undergraduate students, with larger proportions of females and whites, who completed an online survey, compared to the original study with high-school students who completed questionnaires in school. GPA was the outcome variable in both studies, however a semester GPA at the university, which may rely on more idiosyncratic selection of courses, may be a less representative indicator of academic performance compared to a high-school GPA based on a wide range of curriculum subjects. The selfregulation and self-discipline predictors in the original study were each measured with multiple scales, from both student and teacher reports, and modeled as latent variables; hence they were of broader construct coverage and higher measurement reliability, compared to the two manifest variables based on two brief scales in the replication. Finally, the smaller effect could be partially driven by truncated range in the replication sample that results from the non-random admittance to college, which is itself partially dependent on both high school GPA and self-regulation.
Taken together, the original finding and the current replication suggest that self-regulation plays a unique and specific role in academic achievement that cannot be simply boiled down to broader personality trait differences. Self-regulation competencies can be modeled, trained, and developed by educators and mentors to promote successful engagement of the students in their learning (Osher et al., 2016). Further investigation into the specific mechanisms that underlie the association between self-regulation and achievement could facilitate interventions that boost academic achievement.
Noncognitive characteristics are useful predictors of academic outcomes (Duckworth et al., 2019), but as the replication finding of the current study shows not all are equally important in predicting higher achievement. A related line of research with international large-scale assessments points to differences in the relative contribution of self-competence and self-efficacy in an academic domain over motivational constructs such as interest, enjoyment, and valuing a specific school subject (Lee and Stankov, 2018;Michaelides et al., 2019). Identification of the exact noncognitive variables that are robust and replicable predictors of academic achievement will enable the development of effective educational programs.

DATA AVAILABILITY STATEMENT
Publicly available data were analyzed in this study. The data can be found here: https://osf.io/a4swd/?view_ only b871e151cbee4d60b59d260002cab1da.

ETHICS STATEMENT
Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

AUTHOR CONTRIBUTIONS
MM designed the replication study, prepared the data for the replication, and wrote the introduction and discussion of the manuscript. PD wrote the code for the analysis, drafted the methods and results section and provided comments on the manuscript.

FUNDING
The authors would like to acknowledge funding for this replication study by the Center for Open Science.