Commentary: “Misguided Effort with Elusive Implications” and “A Multi-Lab Pre-Registered Replication of the Ego Depletion Effect”

A recent Registered Replication Report (RRR; Hagger et al., 2016) was conducted to examine the ego depletion effect first reported by Baumeister et al. (1998) by replicating Sripada et al.'s (2014) study in 23 international laboratories. Participants completed either an easy or difficult version of a letter “e” task (i.e., Depletion Condition) followed by a multi-source interference task (MSIT) in which participants' reaction time (RT) and reaction time variability (RTV) were recorded as they made congruent and incongruent responses (see https://osf.io/v79xp/ for complete protocol). Ego depletion theory predicts that a sufficiently depleting letter “e” task should result in poorer self-regulation on incongruent MSIT trials (i.e., greater RTs and RTVs). Hagger et al. (2016) found that the ego depletion effect was trivial in effect size and not significantly different from zero. This has called some to question the verisimilitude of ego depletion theory (e.g., Engber, 2016). Yet, several commentaries have also argued that Hagger et al's procedures may be an invalid test of ego depletion (e.g., Baumeister and Vohs, 2016; Sripada et al., 2016; c.f., Hagger and Chatzisarantis, 2016 for an eloquent reply).

One intriguing criticism from Baumeister and Vohs (2016) is that the depleting letter “e” task used in the RRR procedures did not require the exertion necessary to evoke ego-depletion effects on the MSIT. Although the RRR's depleting letter “e” task was rated as a more effortful, difficult, and frustrating than the easy version (Hagger et al., 2016), the task may not have sufficiently taxed self-regulation. One method to examine if this criticism is plausible is to test the underlying process of ego-depletion theory using the existing RRR data.

A recent reanalysis of the Hagger et al.'s (2016) data attempted to address some of Baumeister and Vohs's (2016) concerns by employing a moderation analysis to investigate whether ego depletion was evident in participants who reported exerting more effort on the depleting letter “e” task (Dang, 2016). Although intriguing, Dang's (2016) reanalysis is problematic on three fronts. First, Baumeister et al.'s (1998) ego depletion model theorizes a process by which more challenging, habit-breaking tasks cause depleted self-regulation, and that depletion, in turn, causes ego depletion effects. In other words, effort should be a mediating variable. A moderation model is therefore problematic: It doesn't consider depleted self-regulation as the process through which ego depletion occurs, instead specifying self-regulation (indexed via effort) as an additional independent factor. Second, Baumeister and Vohs (2016) suggest that fatigue is the closest index of depleted self-regulation used in the RRR¹—not effort as examined by Dang (2016). Finally, the reanalysis used an Effort × Depletion Condition interaction term to predict changes in dependent measures. However, participants' effort ratings significantly differed by depletion condition in the RRR. There is significant collinearity between effort and depletion condition, violating the little-to-no collinearity assumption for regression analyses (e.g., Tabachnick et al., 2007). In addition to being logically problematic, statistically the collinearity between the IV and the moderator results in less stable estimates of the moderator's effect. In light of these issues, we undertook a process-focussed reanalysis of Hagger et al.'s (2016) data to investigate whether the RRR's seeming lack of ego depletion effects may be attributed to an insufficiently depleting letter “e” task, with a particular interest in the indirect effect of the manipulation through self-reported fatigue.

We conducted a series of mediation analyses on Hagger et al's RRR data using the PROCESS macro (Hayes, 2013; model 4; Figure S1) to test whether there was a significant indirect effect of depletion condition on the dependent measures of RT and RTV through participants' self-reported fatigue, effort, difficulty, or frustration on the letter “e” task. A bootstrapped mediation analysis is advantageous because it can detect an indirect effect of depletion condition on the dependent measures through a mediator even in the absence of a direct or total effect. This analysis can detect whether the depletion manipulation produced an insuffiecient change in self-regulation for ego depletion effects to occur.

The raw data for each of the 23 participating laboratories was downloaded from the OSF website (https://osf.io/jymhe/), allowing a complete reanalysis of the dependent measures and self-report measures. For all but one laboratory (Schlinkert et al.), we were able to source self-report ratings of fatigue, effort, difficulty, and frustration on the letter “e” task (2,059 participants in total). We did not detect any ratings beyond the 1–7 range of the scales and used the same exclusion criteria for the analysis as were applied in the original RRR.

We undertook bootstrapped mediation analyses (50,000 bootstraps²) to give the greatest chance of detecting an indirect effect of the depletion condition on the dependent measures (RT and RTV) via participants' self reported fatigue, effort, difficulty, and frustration scores. As in the original RRR, there was no significant direct or total effect of depletion condition on RT or RTV, ps > 0.46. Table 1 presents results of the mediation analyses. Small, positive, significant indirect effects were observed on RT and RTV for all mediators except effort and task difficulty, which only showed a significant indirect effect for RTV. The fatigue mediator shows the smallest indirect effect, and this is the construct that Baumeister and Vohs (2016) suggest is the best indicator of the theorized self-regulation mediator.

We find mixed support for the assertion that Hagger et al.'s (2016) task was insufficient to produce the ego depletion effect. Although we find a significant ego depletion effect operating through the mediator of fatigue on both RT and RTV, the effect sizes were trivial (Cohen, 1988). Indirect effects through effort, difficulty, and frustration were larger. However, Baumeister and Vohs (2016) suggest that effort, difficulty, and frustration are not necessarily indicative of changes in self-regulation, so these indirect effects may suggest that the RRR manipulation affected something other than self-regulation. We concur with suggestions (e.g., Baumeister and Vohs, 2016; Hagger et al., 2016) that further replication attempts will be critical for understanding and documenting the boundary conditions and processes that lead to ego depletion effects in laboratory settings.

Statements

Summary

Keywords

ego depletion, self control, registered replication report, reanalysis, social psychology

Citation

Drummond A and Philipp MC (2017) Commentary: “Misguided Effort with Elusive Implications” and “A Multi-Lab Pre-Registered Replication of the Ego Depletion Effect”. Front. Psychol. 8:273. doi: 10.3389/fpsyg.2017.00273

Received

24 October 2016

Accepted

13 February 2017

Published

28 February 2017

Volume

8 - 2017

Edited by

Thomas L. Webb, University of Sheffield, UK

Reviewed by

Martin S. Hagger, Curtin University, Australia; Nikos Chatzisarantis, Curtin University, Australia; Roy Frederick Baumeister, Florida State University, USA

Updates

Check for updates

Copyright

© 2017 Drummond and Philipp.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Aaron Drummond a.drummond@massey.ac.nz

This article was submitted to Personality and Social Psychology, a section of the journal Frontiers in Psychology

Disclaimer

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.