Introduction
Job et al. (2010; hereafter JDW) proposed that one's susceptibility to the ego-depletion effect—a performance decrement on a self-control task after performing on another self-control task—depends on one's implicit beliefs or mindset about whether willpower is limited or not (hereafter willpower mindset). Specifically, JDW demonstrated that only individuals holding a limited-willpower mindset exhibit a significant ego-depletion effect, a finding often regarded as a compelling refutation of a dominant resource-based account of the ego-depletion effect, the strength model (Baumeister et al., 2007). Berkman (2020) even called JDW's finding “the biggest bombshell” against the strength model.
Although we agree that the strength model has various theoretical problems (Lurquin and Miyake, 2017), JDW's willpower-mindset account also has major problems. Here, we first describe statistical problems that cast doubts on the robustness of JDW's original finding. We then present a review of existing ego-depletion studies we conducted to assess its replicability. We conclude by discussing additional major challenges surrounding JDW's account.
Statistical problems
The statistical evidence for JDW's original finding (Study 1) is on shaky ground. JDW used a letter-cancelation (e-crossing) task as the depletion task and a Stroop task as the outcome task (N = 60). They analyzed the data using logistic hierarchical linear models (HLMs), with binary-coded accuracy on each Stroop incongruent trial (24 trials) as the dependent variable, and reported a significant interaction between the experimental manipulation (depletion vs. control) and willpower mindset: “t(1433) = 3.88, p < 0.01” (p. 1688).
The reported degrees of freedom (1433), however, suggest that the model was likely misspecified, failing to properly nest trials under participants and thereby treating all 1440 trials—24 trials × 60 subjects—as independent observations (Schimmack, 2016). Their model also included three theoretically unmotivated covariates [trial order, reaction time (RT), and age], which may have increased the odds for false-positive results (Simmons et al., 2011).
JDW's subsequent studies analyzed the data differently, even when Stroop was the outcome task. Job et al. (2013) Experiment 1 (N = 87) used standard regressions, with Stroop interference in RTs (incongruent-congruent differences) as the dependent measure and age and lab rooms as covariates, and demonstrated depletion-reducing benefits of glucose consumption only among individuals holding a limited-willpower mindset. Savani and Job (2017) Study 3 used HLMs (properly nesting trials under subjects), with log-transformed RTs as the dependent measure and age and SES as covariates, and demonstrated a reverse ego-depletion effect (better Stroop performance under depletion) only among participants from India (n = 56) who collectively believed that exerting self-control is energizing (p = 0.045).
Such substantial data-analytic differences across studies raise doubts on the robustness of JDW's original finding, a concern exacerbated by our own failure to replicate the moderation effect in a preregistered study (N = 187; Carruth et al., 2023). To further assess the replicability of JDW's finding, we reviewed the existing literature.
Review of existing literature
We conducted a literature search on Google Scholar in January 2023. We first identified all articles that cited the JDW study and then excluded (a) review/theoretical papers without any data and (b) research papers that did not report any ego-depletion studies. We examined the remaining papers to identify studies that administered JDW's willpower-mindset measure and tested its moderating influence on ego depletion. We included only studies treating willpower mindset as an individual-differences variable; those that experimentally manipulated willpower mindset were excluded (e.g., JDW's Study 2; Vohs et al., 2012, Experiment 1).
Thirteen independent studies (from 11 papers) met our inclusion criteria. We did not conduct a formal meta-analysis because different methods used to analyze the data in these studies (e.g., HLM, standard regressions)—some including covariates—made it difficult to cleanly derive a common effect-size metric for the hypothesized interaction effect without having access to the raw data.
We instead classified the 13 studies into two categories—those that observed moderating influences (Category A) and those that did not (Category B)—and examined whether they differed systematically. For this purpose, we coded the following variables: (a) final sample size; (b) data-analytic method; (c) timing of mindset measurement; (d) the study's primary focus; (e) significance of the overall ego-depletion effect; (f) significance of the moderation effect; (g) preregistration; and (h) data availability. The results are summarized in Table 1.
Table 1
| References | (a) Final sample size (# of participating labs) | (b) Analysis method used to test moderation | (c) The timing of mindset measure | (d) Is primary focus the moderation effect? | (e) Overall ego depletion significant? | (f) Moderation effect significant? | (g) Preregistered? | (h) Data shared publicly? | Notes |
|---|---|---|---|---|---|---|---|---|---|
| Category A: Studies that reported a moderating influence of willpower mindset on the ego-depletion effect | |||||||||
| Chow et al. (2015) Experiment 3 | N = 126 | Regression | Beginning | Yes | Yes | Yes | No | No | |
| Dang et al. (2021) | N = 1,775 (k = 12) | Meta-analysis across labs | Beginning | No | Yes | Yes? (but only for a secondary DV) | Yes | Yes | The moderation was significant for a secondary RT measure, but not for the primary accuracy measure; it also disappeared when chance-level performance was excluded. |
| Job et al. (2010) Study 1 | N = 60 | Logistic HLM | Beginning | Yes | Yes? | Yes? | No | No | As noted in the main text, their HLMs appear to be misspecified and included post-hoc, theoretically unmotivated covariates. |
| Singh and Göritz (2019) Study 1 | N = 115 | Regression | Beginning | Yes | Yes? | Yes? (but in the opposite direction) | No | No | Willpower mindset moderated the ego-depletion effect, but the effect was stronger for those holding a nonlimited-willpower mindset. Because the journal bans the reporting of p-values, it is not clear whether this moderation effect was significant. |
| Singh and Göritz (2019) Study 2 | N = 633 | Regression | Beginning | Yes | Yes? | Yes? (but in the opposite direction) | No | No | In the initial analysis, there was no moderation effect, but, when mood (a hypothesized moderator) was added to the regression, the moderation effect emerged. Again, it was the opposite of what JDW found (i.e., more depletion for unlimited-willpower mindset). No p-values are reported due to the journal's policy. |
| Category B: Studies that Reported No Significant Moderation Effect | |||||||||
| Calvillo et al. (2022) | N = 192 | Repeated-measure ANCOVA | End | No | Yes? | No | Yes | Yes | Depletion was manipulated within-subjects in this study. The overall ego-depletion effect was significant in an exploratory analysis that removed subjects who performed below-chance on the outcome task. |
| Carruth et al. (2023) | N = 187 | Repeated-measure ANOVA and HLM | Beginning | Yes | No | No | Yes | Yes | A replication of JDW's Study 1. Tried different ways to analyze data to address statistical issues we noted in the main text. |
| Dang et al. (2017) | N = 176 | Regression | Beginning | No | Yes | No | Yes | Yes | This study was the basis for the Dang et al. (2021) preregistered multilab study. |
| Garrison et al. (2019) Experiment 1 | N = 657 | HLM | End | No | Yes | No | Yes | Yes | |
| Garrison et al. (2019) Experiment 2 | N = 359 | HLM | End | No | Yes | No | Yes | Yes | |
| Singh and Göritz (2018) | N = 1,385 | Regression | Beginning | No | No | No | No | Yes | |
| Vohs et al. (2021) | N = 3,531 (k = 36) | HLM | End | No | Yes? | No | Yes | Yes | The overall ego-depletion effect was significant only in an exploratory analysis based on the full sample, but not in the preregistered analysis. |
| Wenzel et al. (2019) Study 1 | N = 187 | HLM | Between Sessions | No | Yes | No | No | No | Depletion was manipulated within-subjects in this study. |
Summary of the existing studies testing the moderating influence of willpower mindset on the ego-depletion effect.
This table includes only studies that administered Job et al. (2010) willpower mindset measure and tested its moderating influence on the ego-depletion effect using behavioral measures and comparing the deletion and control (no-depletion) conditions, just like JDW's Study 1. Studies that assessed the ego-depletion effect using self-reports without including depletion and control conditions are excluded. A small number of studies that manipulated subjects' willpower mindset experimentally (as opposed to treating it as an individual differences variable) are also not included here (e.g., JDW, Studies 2 and 3). DV, dependent variable; RT, reaction time; ANOVA, analysis of variance; ANCOVA, analysis of covariance; HLM, hierarchical linear modeling.
Five studies reported the moderating influence of willpower mindset on ego depletion (Category A). As explained in the table, however, the evidence in 4 of the 5 studies was ambiguous (including JDW's original study). Particularly noteworthy are two studies by Singh and Göritz (2019), both reporting the opposite moderation effect (greater depletion for those holding an unlimited-willpower mindset). Only one study (Chow et al., 2015) seems to have demonstrated the hypothesized moderation effect unambiguously.
Eight studies reported no significant moderation effect (Category B), even though the Category B studies collectively had greater sample sizes (mean N = 834.3; median N = 275.5) than the Category A studies (mean N = 464.5; median N = 96.5). Additionally, the hypothesized moderation effect did not seem to depend on the statistical significance of the overall ego-depletion effect: Although all 5 Category A studies reported some evidence for it, so did the majority of the Category B studies (6 out of 8).
The two categories of studies differed in several intriguing ways, however. First, all 5 studies that used HLMs and properly nested trials under participants failed to observe the moderation effect. Thus, it would be informative to know whether JDW's original moderation effect would still be significant if their data were analyzed with proper nesting.
Second, all Category A studies administered the willpower-mindset measure before the depletion manipulation. Administering willpower-mindset and other self-control-related measures early has the advantage of avoiding questionnaire responses being influenced by the depletion manipulation, but it runs the risk of the questionnaire items influencing participants' subsequent behaviors. Interestingly, all 5 studies that administered the mindset questionnaire at the end and thereby minimized such potential priming influences failed to observe the moderation effect.
Third, testing the moderating influence of willpower mindset on ego depletion was the primary focus in 4 of the 5 Category A studies, but in only 1 of the 8 Category B studies (the other studies focused primarily on the replicability of the ego-depletion effect itself; willpower mindset was included as one of the few potential moderators and did not receive much discussion). One way to interpret this discrepancy is in terms of publication bias: If the study's main purpose is to test the moderation effect, publishing a nonsignificant result may be more difficult, hence resulting in few such studies in Category B.
Finally, almost all Category B studies were preregistered (minimizing the likelihood of p-hacking) and have publicly shared the data (enabling other researchers to verify the results and reanalyze the data), but not for the Category A studies, with one notable exception of the Dang et al. (2021) study. The evidence reported in that preregistered multilab study was ambiguous, however: The moderation effect was significant (p = 0.039) only for their secondary dependent measure (RTs for the antisaccade task), and the effect became nonsignificant when the analyses excluded subjects who likely failed to understand the task requirements and performed the outcome task at chance level.
This brief review is limited in two important ways. First, we did not conduct a formal meta-analysis and hence cannot provide an aggregate effect-size estimate for the target interaction term or quantitative tests of the moderating variables we coded for each study. Second, this review was restricted to published studies. Accordingly, it is important to keep in mind that a meta-analysis that includes unpublished studies may yield different results. It would therefore be worthwhile to conduct such a meta-analysis, especially based on raw data so that effect sizes for the target interaction term can be estimated more equivalently across studies.
Despite these limitations, our review showed that the existing evidence for JDW's original finding is weak at best: No large-sample preregistered study currently exists that yielded robust evidence for the hypothesized moderation effect for the primary dependent measure. The finding that only one of the 13 studies yielded an unambiguous moderation effect in the predicted direction casts seriously questions the replicability of JDW's original finding.
Additional challenges
JDW's willpower-mindset account of ego depletion also faces other challenges. First, their account assumes that the ego-depletion effect is “real” at least among some individuals. However, both meta-analyses (Carter and McCullough, 2014; Carter et al., 2015) and large-scale multilab studies (Hagger et al., 2016; Vohs et al., 2021) have seriously challenged the replicability of the ego-depletion effect itself (Friese et al., 2019). Moreover, self-reports from researchers who have conducted ego-depletion studies revealed an alarming rate of prior engagement in questionable research practices, especially selective reporting of outcomes (Wolff et al., 2018). Thus, it is far from clear whether the ego-depletion effect is real.
Even if this controversial effect is real, its overall effect size would likely be small. Three preregistered multilab studies of ego depletion, for example, yielded effect-size estimates (Cohen's d) of only 0.04 (Hagger et al., 2016), 0.08 (Vohs et al., 2021), and 0.10 (Dang et al., 2021), respectively. Detecting a difference between two independent groups with a 0.80 power requires >3,000 participants when the effect size is 0.10. Given that detecting a statistical moderation effect involving individual-differences variables has been known to be difficult (McClelland and Judd, 1993), replicating JDW's original finding (N = 60) with sufficient power would present a major challenge in future research, requiring much larger sample sizes than used before in ego-depletion studies.
JDW's account is also lacking theoretically (Bertrams, 2020). Although not explicitly stated in the original article, Job's preferred explanation is motivational: Performing a demanding self-control task reduces one's motivation level, especially for those holding a limited-willpower mindset, thus making them more willing to “slack off” on a subsequent task (see Carruth et al., 2023, for further elaboration). Though plausible, this motivational account of ego depletion is not precise enough to allow other researchers to unambiguously derive testable (falsifiable) predictions, a point we also raised against the strength model (Lurquin and Miyake, 2017). Without better specifying under what circumstances individuals with a limited-willpower mindset decide to “slack off,” it is difficult to unambiguously predict whether willpower mindset has a moderating influence on ego depletion in a given study.
Equally challenging is the observation that the existing studies showed three different patterns: (a) stronger depletion for limited-willpower mindset; (b) stronger depletion for unlimited-willpower mindset; and (c) no effect of willpower mindset. Given that JDW's account is unidirectional (limited-willpower mindset → greater depletion effects), it cannot explain the opposite pattern reported by Singh and Göritz (2019). Considering that Clarkson et al. (2016) and Vohs et al. (2012) (Experiment 1) demonstrated that an experimentally induced unlimited-willpower mindset can lead to worse self-control performance under some circumstances, one cannot simply dismiss the Singh and Göritz (2019) finding as a fluke. If (a) and (b) are both real and reflect two different ways in which willpower mindset impacts participants' motivation for self-control, JDW's account clearly needs a further theoretical specification.
One possible way forward is to specify potential underlying moderators and mediators that can help reconcile patterns (a) and (b). In fact, Clarkson et al. (2016) proposed (dis)fluency (subjective ease/difficulty of retrieving or processing information) as the moderator (fluency promotes JDW's original pattern, whereas disfluency reverses it) and perceived mental fatigue as the mediator. It is an intriguing proposal, but, because Clarkson et al. (2016) did not test their account using an ego-depletion paradigm, it is unclear whether this account is readily applicable to the ego-depletion results reviewed here.
We believe that willpower mindset will likely remain a useful concept in self-control research (see Francis and Job, 2018, for a review). However, its application to ego depletion—the very phenomenon whose existence has been questioned—is not as compelling as often portrayed in the literature: The moderating influence of willpower mindset on ego depletion is weak at best, starting with JDW's original study. Until more compelling evidence is provided and a more precise model is developed, JDW's original finding must be treated with great caution.
Statements
Author contributions
AM and NC jointly developed the ideas and the plan for the literature review presented in this article. NC conducted the literature search and produced an initial draft of the summary table for the literature review. AM checked the entries, edited the table, and produced an initial draft of the main text. Both contributed to the editing of the text.
Funding
Publication of this article was funded in part by the University of Colorado Boulder Libraries.
Acknowledgments
We thank Michael J. Kane for providing thoughtful feedback on this manuscript.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
1
Baumeister R. F. Vohs K. D. Tice D. M. (2007). The strength model of self-control. Curr. Dir. Psychol. Sci. 16, 351–355. 10.1111/j.1467-8721.2007.00534.x
2
Berkman E. T. (2020). The last thing you need to know about ego depletion. Psychology Today. Available online at: https://www.psychologytoday.com/us/blog/the-motivated-brain/202012/the-last-thing-you-need-know-about-ego-depletion
3
Bertrams A. (2020). A schema-activation approach to failure and success in self-control. Front. Psychol.11, 2256. 10.3389/fpsyg.2020.02256
4
Calvillo D. P. Rodriguez K. Nguyen T. N. (2022). A within-subject test of ego depletion and potential moderators. J. Gen. Psychol.149, 509–523. 10.1080/00221309.2021.1922341
5
Carruth N. P. Ramos J. A. Miyake A. (2023). Does willpower mindset really moderate the ego-depletion effect?: A preregistered replication of Job, Dweck, and Walton (2010). PsyArXiv. 10.31234/osf.io/8cqpk
6
Carter E. C. Kofler L. M. Forster D. E. McCullough M. E. (2015). A series of meta-analytic tests of the depletion effect: Self-control does not seem to rely on a limited resource. J. Exp. Psychol.144, 796–815. 10.1037/xge0000083
7
Carter E. C. McCullough M. E. (2014). Publication bias and the limited strength model of self-control: has the evidence for ego depletion been overestimated?Front. Psychol.5, 823. 10.3389/fpsyg.2014.00823
8
Chow J. T. Hui C. M. Lau S. (2015). A depleted mind feels inefficacious: Ego-depletion reduces self-efficacy to exert further self-control. Eur. J. Soc. Psychol.45, 754–768. 10.1002/ejsp.2120
9
Clarkson J. J. Otto A. S. Hirt E. R. Egan P. M. (2016). The malleable efficacy of willpower theories. Pers. Soc. Psychol. Bull.42, 1490–1504. 10.1177/0146167216664059
10
Dang J. Barker P. Baumert A. Bentvelzen M. Berkman E. Buchholz N. et al . (2021). A multilab replication of the ego depletion effect. Soc. Psychol. Personal. Sci.12, 14–24. 10.1177/1948550619887702
11
Dang J. Liu Y. Liu X. Mao L. (2017). The ego could be depleted, providing initial exertion is depleting: A preregistered experiment of the ego-depletion effect. Soc. Psychol.48, 242–245. 10.1027/1864-9335/a000308
12
Francis Z. Job V. (2018). Lay theories of willpower. Soc. Personal. Psychol. Compass. 12, e12381. 10.1111/spc3.12381
13
Friese M. Loschelder D. D. Gieseler K. Frankenbach J. Inzlicht M. (2019). Is ego depletion real? an analysis of arguments. Pers. Soc. Psychol. Review. 23, 107–131. 10.1177/1088868318762183
14
Garrison K. E. Finley A. J. Schmeichel B. J. (2019). Ego depletion reduces attention control: Evidence from two high-powered preregistered experiments. Pers. Soc. Psychol. Bull. 10.1177/0146167218796473
15
Hagger M. S. Chatzisarantis N. L. D. Alberts H. Anggono C. O. Batailler C. Birt A. R. et al . (2016). A multilab preregistered replication of the ego-depletion effect. Perspect. Psychol. Sci.11, 546–573. 10.1177/1745691616652873
16
Job V. Dweck C. S. Walton G. M. (2010). Ego depletion—Is it all in your head? Implicit theories about willpower affect self-regulation. Psychol. Sci. 21, 1686–1693. 10.1177/0956797610384745
17
Job V. Walton G. M. Bernecker K. Dweck C. S. (2013). Beliefs about willpower determine the impact of glucose on self-control. Proc. Natl. Acad. Sci. U.S.A.110, 14837–14842. 10.1073/pnas.1313475110
18
Lurquin J. H. Miyake A. (2017). Challenges to ego-depletion research go beyond the replication crisis: a need for tackling the conceptual crisis. Front. Psychol.8, 568. 10.3389/fpsyg.2017.00568
19
McClelland G. H. Judd C. M. (1993). Statistical difficulties of detecting interactions and moderator effects. Psychol. Bull.114, 376–390. 10.1037/0033-2909.114.2.376
20
Savani K. Job V. (2017). Reverse ego-depletion: acts of self-control can improve subsequent performance in Indian cultural contexts. J. Pers. Soc. Psychol.113, 589–607. 10.1037/pspi0000099
21
Schimmack U. (2016). Replicability report No. 1: Is ego-depletion a replicable effect? [Blog post]. Available online at: https://replicationindex.wordpress.com/2016/04/18/is-replicability-report-ego-depletionreplicability-report-of-165-ego-depletion-articles/
22
Simmons J. P. Nelson L. D. Simonsohn U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol. Sci.22, 1359–1366. 10.1177/0956797611417632
23
Singh R. K. Göritz A. S. (2018). Ego depletion does not interfere with working memory performance. Front. Psychol.9, 538. 10.3389/fpsyg.2018.00538
24
Singh R. K. Göritz A. S. (2019). Revisiting ego depletion: moderators and measurement. Basic Appl. Soc. Psych.41, 1–19. 10.1080/01973533.2018.1530671
25
Vohs K. D. Schmeichel B. J. Lohmann S. Gronau Q. F. Finley A. J. Ainsworth S. E. et al . (2021). A multisite preregistered paradigmatic test of the ego-depletion effect. Psychol. Sci.32, 1566–1581. 10.1177/0956797621989733
26
Vohs K. D. Baumeister R. F. Schmeichel B. J. (2012). Motivation, personal beliefs, and limited resources all contribute to self-control. J. Exp. Soc. Psychol.48, 943–947. 10.1016/j.jesp.2012.03.002
27
Wenzel M. Lind M. Rowland Z. Zahn D. Kubiak T. (2019). The limits of ego depletion: a crossover study on the robustness of performance deterioration in consecutive tasks. Soc. Psychol.50, 292–304. 10.1027/1864-9335/a000365
28
Wolff W. Baumann L. Englert C. (2018). Self-reports from behind the scenes: Questionable research practices and rates of replication in ego depletion research. PLoS ONE.13, e0199554. 10.1371/journal.pone.0199554
Summary
Keywords
ego depletion, implicit theories of willpower, willpower beliefs, self-control, replicability
Citation
Miyake A and Carruth NP (2023) On the robustness and replicability of the moderating influence of willpower mindset on the ego-depletion effect: existing evidence is weak at best. Front. Psychol. 14:1208299. doi: 10.3389/fpsyg.2023.1208299
Received
18 April 2023
Accepted
24 May 2023
Published
15 June 2023
Volume
14 - 2023
Edited by
June J. Pilcher, Clemson University, United States
Reviewed by
Oulmann Zerhouni, Université de Rouen, France
Updates
Copyright
© 2023 Miyake and Carruth.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Akira Miyake akira.miyake@colorado.edu
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.