1 Introduction: replicability crisis
In this Editorial Grand Challenge, I will first summarize some of the multiple (intrinsic and extrinsic) reasons why cognitive psychology, along with many other scientific fields, has experienced a replicability crisis. I will then illustrate some of the proposed remedies that are leading to a credibility revolution. I will highlight that the proposed best practices for complex epistemological problems, while necessary, also need to be critically evaluated to ward off potentially new concerns. Finally, I will advocate for a new balance to ensure methodologically sound and credible science while also safeguarding scientific sustainability, creativity and enthusiasm.
In the history of psychology, periods of vibrant discovery and exciting research have occasionally been perturbed by troubling waves of replicability crisis, the most recent one of which is still ongoing (Lakens, 2023; Open Science Collaboration, 2015). While cognitive psychology appears to be slightly less affected by the replicability crisis than social psychology (Open Science Collaboration, 2015), it has certainly not been immune to similar concerns.
First, publications with small sample sizes and inadequate statistical power often led to findings that could not always be reliably reproduced (Button et al., 2013). Second, the pressure to publish novel and positive results, coupled with distorting incentives, has fostered questionable research practices (or even fraud, in the worst cases), such as for instance selective reporting and p-hacking, in which some researchers may, more or less deliberately, manipulate or select data to achieve statistically significant outcomes (e.g., Lilienfeld, 2017; Nosek et al., 2012). Additionally, complex experimental designs, researchers' degrees of freedom and variability in methodological practices have contributed to inconsistencies across studies. The prevalence of a confirmation bias, that is, the tendency of researchers to favor data that support their hypotheses, has further exacerbated this issue. Moreover, a lack of transparency in data sharing and reporting of methodological details has hindered efforts to accurately replicate findings.
These challenges are currently prompting a massive cultural paradigm shift that would imply entirely rethinking the research process. It is not surprising that a growing movement, largely and meritoriously driven by the new generation of researchers, is advocating for more rigorous and transparent scientific practices. This shift is a virtuous reaction to the replicability crisis (e.g., Lindsay, 2020). These practices include for instance, (i) registering reports or at least pre-registering studies to minimize questionable practices such as Harking (Kerr, 1998), cherry-picking and confirmation bias (e.g., Clark et al., 2022a), (ii) implementing open data initiatives to increase transparency and maximizing collective scientific efforts (e.g., Mayernik, 2017; Rasti et al., 2025), and (iii) conducting replication efforts through coordinated multi-laboratory projects to achieve the large sample sizes necessary for sufficient statistical power. Therefore, awareness of the replicability crisis is currently driving positive and praiseworthy changes in the methods, procedures and, importantly, policies that will eventually and ideally benefit the credibility of the scientific community as a whole (e.g., Korbmacher et al., 2023). However, as I will argue throughout this contribution, one would be naïve to think that the proposed solutions to these complex epistemological problems are always straightforward and without their own drawbacks. Any new remedy needs to be critically evaluated and a new balance sought.
2 Multiple contributions to the replicability crisis
As a first consideration, it is undeniable that the replicability crisis can be, at least partially, attributed to a series of questionable research practices that have been commonly used, tolerated, or even favored by standard practices in the field and by ill-conceived academic incentives. However, we should not overlook other potential factors that may also have contributed to the replicability crisis in our specific field. Indeed, replicability issues may also stem, in part, from deeper epistemological and methodological problems.
The social sciences, and cognitive psychology is no exception, are concerned with objects of investigation that cannot be observed directly but are captured primarily through latent variables and constructs. These constructs require careful, and often challenging, operationalization in order to be scientifically attacked through some measurable proxies. Because cognitive processes are inherently “fuzzier” and more ambiguous compared to more basic biological processes, there is typically no single best operationalization of a given construct to test, and ontological definitions require additional effort. To take an example from cognitive psychology used by Poldrack and Yarkoni (2016), performance on N-back tasks and OSPAN tasks shows weak, and often non-significant, correlations (Kane et al., 2007; Oberauer, 2005; Roberts and Gibson, 2002). This is unsettling as both tasks are designed to assess the same cognitive construct, that is, working memory. Such discrepancies suggest issues such as ambiguous operational definitions and task impurity (Kane et al., 2007), which contribute to the entropy in the literature. Some promising remedies have been proposed. These include the Cognitive Atlas Project (Poldrack and Yarkoni, 2016), a collaborative project to develop a knowledge base that represents the current state of cognitive science, or the Cognitive Paradigm Ontology (Turner and Laird, 2012), a systematic collection of cognitive paradigms for functional neuroimaging applications. While these tools are commendable, there is a risk that only global inferences can be made. This would happen if the tasks included in these databases were too general and lacked subtask-specific manipulations. Indeed, despite their methodological idiosyncrasies, subtle experimental manipulations are often more theoretically meaningful than the common features across tasks.
Additionally, the findings obtained need to be interpreted, with more than one account often being equally plausible at the same time, thus lowering internal validity. This produces uncertainty about which interpretation of the findings might be the most accurate, increasing the likelihood that biased conclusions will be preferred instead. A related consequent issue is the so-called motivated research, which occurs when factors unrelated to accuracy influence how scientists engage with existing data, generate new hypotheses, collect, analyze and interpret data, and communicate research outcomes to their peers and the general audience (e.g., Clark et al., 2022b; Kunda, 1990). These factors include confirmation biases and the need for social and moral approval of specific research topics and conclusions.
Furthermore, delving deeper into the core of the issue, the inherent variability of the object of investigation is another major challenge in the social sciences, which also include cognitive psychology. The manifestation of cognitive processes can indeed vary considerably, not only between individuals but also within individuals (e.g., Dhawale et al., 2017; Fiske and Rice, 1955; Kahana et al., 2018; MacPherson et al., 2019), with variability being a key aspect of cognition that cannot be dismissed (or at least not exclusively) as a measurement error (Judd et al., 2024). Indeed, variability is not only spuriously caused by ill-defined operationalizations of cognitive constructs, imprecise measurements of their proxies (e.g., behavior), violations of statistical assumptions, power issues, and more generally, poor experimental control and other methodological flaws (e.g., Clark and Watson, 2019; Podsakoff et al., 2012). There are also other fundamental sources of variability linked to the very nature of mental processes.
Natural fluctuations characterize the manifestations of mental functions such as perception, emotion, memory, executive functions, and decision-making, to name a few. This variability arises from numerous factors, which include psychobiological variables, genetic differences, developmental stages, but also situational contexts and environmental influences (e.g., epigenetics).
Specifically, cognitive performance can be affected by an individual's mood (Chepenik et al., 2007), motivation (Braver et al., 2014), stress (Steinhauser et al., 2007), sleep deprivation (Killgore, 2010), and prior experience (Pastukhov and Braun, 2011). Moreover, the brain's complex neural architecture and the dynamic nature of its networks contribute to this variability, as its connectome is constantly shaped and reshaped by new experiences, providing the neural foundation for an ever-changing cognitive life (e.g., Dosenbach et al., 2010; Kolb and Gibb, 2011; Seung, 2012; Tost et al., 2015). Acknowledging, and ideally accounting for, this intrinsic variability and lability of the object of investigation in cognitive psychology (i.e., mental processes) is essential for developing robust, or at least plausible, theories and applications in cognitive psychology and neuroscience.
Additionally, many findings cannot be easily generalized across different contexts (e.g., cultural, spatial, temporal) or experimental conditions (Stroebe and Strack, 2014). Indeed, the environmental or experimental context can affect both how and to what extent a cognitive process is activated and expressed. An example is provided in the work by Hommel et al. (2012). Their study showed that cognitive control can be, somewhat counterintuitively, more effective when exercised in the context of the acoustic noise typical of fMRI sequences than in standard settings. This specific example raises more general considerations about the potential applicability of Heisenberg's uncertainty principle in cognitive neuroscience, where the object of investigation (the mind) is being transformed by the very tools used to study it. A more classic example of this issue is the Hawthorne effect, in which individuals alter their behavior because they are aware of being observed and measured (e.g., Adair, 1984). Although, ironically enough, this famous phenomenon is probably not fully reliable, or at least not in its initial reports (Letrud and Hernes, 2019; Levitt and List, 2011), it represents just one example of the broader (but often neglected) modulatory role of participants' motivation and demand characteristics in task performance (e.g., Orne, 1962; Weber and Cook, 1972).
To summarize this section, while variability can be seen as an obstacle to scientific progress, as it can lead to average null effects, uncovering its intrinsic (but also exogenous) sources can also have a leveraging effect on theory development. Indeed, variability could be exploited to better understand inter-individual differences in cognition and to advance cognitive theories (e.g., Langerock et al., 2025; Miller and Schwarz, 2017; Wang et al., 2012; cf., Rowe and Healy, 2014).
3 Remedies for the replicability crisis and new concerns of the credibility revolution
One potential concern with the proposed remedies for the replicability crisis is that, while highly desirable in principle, some of them conceal other risks that should at least be acknowledged, if not actively countered. As the Latin writer Horace once wrote: “Est modus in rebus”, which reminds us to pursue balance in life by avoiding deleterious extremes. As will become clear from the following, the coveted solutions proposed to overcome the replicability crisis are also subject to this caveat. Therefore, we need to avoid applying them uncritically.
3.1 Data-driven approaches in cognitive psychology: potential risks and antidotes
While the widespread availability of large open datasets promotes reproducibility and collective efficiency, it also inevitably encourages data mining and exploratory approaches. Far be it from me to want to unleash or incite a witch hunt, but such a data-driven mindset (which could bring to a so-called fishing expedition attitude) can sometimes temptingly lead to the post hoc generation of hypotheses (HARKing) and, even worse, to a lack of theoretical depth. A data-driven attitude is also encouraged by the unprecedented, exponential expansion of Machine Learning and other Artificial Intelligence (AI) algorithms. AI can undoubtedly enhance the efficient exploration and analysis of complex scientific data in a number of ways, such as identifying patterns and trends, allowing predictive modeling, integrating multidimensional data and so on, accelerating data-driven discoveries including in the field of cognitive psychology. However, would powerful but opaque AI algorithms always help enhance our theoretical understanding of underlying cognitive mechanisms? And, in case this is achievable, could this AI-based understanding then generalize to new datasets? Although the debate in the field is currently heated with these and other open questions, it would definitely benefit from a more extensive and deeper epistemological reflection, hopefully with contributions also coming from cognitive psychology.
One proposed solution to the risks of a lenient data-driven attitude is the pre-registration of studies, which forces researchers to truly specify their a priori hypotheses in advance (and to clearly label as exploratory all the other hypotheses that are generated a posteriori with respect to the pre-registration). However, these and other credibility-enhancing practices are still adopted on a voluntary basis and are not yet routinely applied as gold standards. Additionally, pre-registration alone (as a mere bureaucratic formality) is not sufficient. It would be equally important to address the deeper issue of the theoretical crisis: the hypotheses that are pre-registered could still be purely empirical in nature or, even worse, poorly formulated, unjustified and without strong theoretical foundations. Thus, pre-registering hypotheses helps to reassure the scientific community that our thinking remains unbiased but it does not in itself substitute theoretical and logic-based reasoning. Simply pre-registering a hypothesis, as a sort of fig leaf, without substantiating it with a solid theoretical rationale serves little purpose and does not in itself enhance the credibility of the hypothesis (Oberauer and Lewandowsky, 2019). Moreover, it would be fine to candidly admit that a given study was exploratory, especially if at the beginning of a new line of research, as long as the data obtained could inspire theory development, new hypotheses, and further research to test those hypotheses and refine the theory.
An excessive focus on trying to empirically reproduce or replicate published effects or phenomena, while extremely valuable to reshape the trust and the foundation of what we already know about human cognition, if an end in itself, can divert attention away from developing robust theories (Oberauer and Lewandowsky, 2019). Proposing strong theories would allow scholars to formulate credible and specific hypotheses, which would benefit scientific progress (Clark et al., 2022b). Developing accurate and comprehensive theories or at least models of cognitive processes is inherently challenging due to their fuzzy nature and complexity. Thus, theories and models need to consider a broad spectrum of variables and interactions, which necessitates advanced computational approaches, biological plausibility and extensive experimental testing (e.g., Busemeyer and Diederich, 2010; Turner et al., 2017). This process could benefit from collective, multidisciplinary scientific coordination toward common, challenging goals (Brignol et al., 2024; Rasti et al., 2025).
One special type of scientific coordination that has been proposed as a remedy for the lack of a theory-oriented mindset is represented by adversarial collaborations (sometimes referred to as “coopetition”; Clark et al., 2022a; Ellemers et al., 2020). Although challenging, engaging with those who propose alternative theories can be scientifically rewarding in the long run (see Cowan et al., 2020; Mellers et al., 2001, for some examples). These collaborations require clear, testable hypotheses, mutual understanding of differing viewpoints, and methods that can distinguish and potentially falsify both hypotheses. This process limits researchers' biases and beliefs, enhances the integrity of methodological approaches, and effectively advances debates. By committing to methods before testing, even better when coupled with best practices such as pre-registering, open data and replication, scholars reduce the risk of post-hoc criticism or hiding unwanted results. This would also make the findings more credible to the scientific community. More plausible hypotheses will prevail, and incorrect ones will be falsified sooner, preventing wasted time and resources and increasing the reliability and credibility of scientific findings.
Relying on increasingly sophisticated methods to analyze rich and complex data poses similar risks. The field of neuroimaging can be used as an example. While advances in neuroimaging data analysis have provided valuable insights into brain activity, understanding how these findings are related to cognitive processes is still difficult. Explaining how complex mental functions arise from brain regions and neural networks has traditionally been a major challenge (e.g., Coltheart, 2006; Hommel and Colzato, 2017; Niv, 2021). More specifically, while recent advances in the field of connectomics in cognitive neuroscience (e.g., graph theory) have significantly enhanced our ability to explain the complexity of brain organization, at least in principle, clear relationships between topological indices of network integration and segregation and cognitive processes have yet to be fully established (Litwińczuk et al., 2023). These relationships would be crucial for assessing the biological plausibility and behavioral relevance of these highly derived, network-based measures.
Complicating matters, it is acknowledged in the neuroimaging literature that behavioral data are not always characterized with sufficient fidelity to support robust and reliable brain-behavior correlations (Tiego and Fornito, 2022). Additionally, some common interpretations of cognitive neuroscience findings are unjustified because they rely on flawed reasoning or other unproven assumptions. A well-known example of flawed reasoning is reverse inference, that is, deducing the involvement of particular mental processes solely based on observed patterns of brain activation, without the constraints of elegant experimental manipulations that would enhance internal validity (Kriegeskorte et al., 2009; Poldrack, 2006). As another example, many compensatory cognitive interpretations of brain patterns in older adults are not fully justified unless data are analyzed longitudinally (e.g., Pudas et al., 2013).
Another potential drawback of widely available open data is the inherent risk of salami slicing, in which data from a research study that should form a single publication are divided into several smaller publishable units. Salami slicing, which has long been considered a questionable and unethical research practice in various fields (Karlsson and Beaufils, 2013; Siegel and Baveye, 2010; Spielmans et al., 2010) remains a significant concern, and even more so in the era of open science. This practice can lead to various issues, such as consuming excessive resources during the editorial and review process and overwhelming science readers. It can also lead to an inflated number of articles cluttering the literature, increasing the risk of publication bias, and distorting effect sizes in meta-analyses. To overcome this issue, the same or overlapping data samples should always be properly acknowledged as such in different publications (e.g., Hilgard et al., 2019; Urbanowicz and Reinke, 2018).
Furthermore, it is also worth mentioning that the risk of wasting resources and producing flawed science is amplified if the tasks used to collect behavioral data in large open datasets were not carefully designed and suffer from some error.
3.2 Potential side effects of the credibility revolution and excessive methodological control
A well-known trade-off exists between experimental control and generalizability. On the one hand, more attention to well-designed experimental paradigms that account for confounding factors when trying to fully characterize a psychological effect is desirable to increase internal validity (e.g., Verbruggen et al., 2019; Viviani et al., 2024), and hopefully also to enhance scientific credibility and replicability. On the other hand, excessive methodological control in research poses a significant risk, as it can lead to a decrease in generalizability, that is, external and ecological validity (Shadish et al., 2002). In other words, the more tightly controlled experiments become, the less accurately their findings reflect real-world conditions, making it difficult to apply the results to broader contexts or diverse populations. This undermines the relevance and applicability of the research, limiting its potential impact outside the highly controlled environment in which it was conducted.
Another potential risk of placing too much emphasis on the remedies for the replicability crisis is a reduction in the enthusiasm for exploration, creativity and discovery, which are also essential aspects of the scientific process that complement methodological rigor. Focusing too much on strict standards, while being undeniably vital for overcoming the pressing credibility crisis, may also lead to excessive caution, hindering divergent thinking and the serendipitous attitude that drives new discoveries (Kaufman and Glǎveanu, 2018; Ness, 2015). In principle, the majority of researchers would support recommendations, best practices, and, more generally, to virtuous organized skepticism, in the hope that the scientific community would universally adopt them (Anderson et al., 2007). However, when these begin to become imposed norms, a looming risk is that they may lead to additional paperwork in fulfilling the already hypertrophic and self-referential accountability and evaluation processes. The latter require additional time and mental energy, limited resources that researchers would prefer to devote, with much more motivation and enthusiasm, to their actual research activities. If new practices become excessively demanding due to an increased bureaucratic burden, they risk reducing the time and resources available for creative activities (Kaufman and Glǎveanu, 2018). In other words, with excessive constraints, creativity could also be suppressed (Medeiros et al., 2014). This risk within the scientific field should be considered in addition to the already observed decline in creativity in the general population (Kim, 2011).
In other words, while best practices in science are indispensable and non-negotiable, the burdensome bureaucracy and other possible downsides that might accompany their implementation should be kept strictly in check, and new incentives should be envisioned that could promote slower but more accurate science and allow for more quality time. Scientists are therefore compelled to closely monitor the tendency toward hyper-bureaucratization of the scientific process and to continually seek a new balance between the apparently conflicting, yet equally essential needs of methodological rigor and scientific creativity (Ness, 2015; Scheffer, 2014; Simonton, 2004), while moving away from purely quantity-based institutional evaluation processes and incentives.
4 Concluding remarks
The awareness of a massive replicability crisis has been a slap in the face for the scientific community. It is undeniable that we can no longer continue with business as usual and that remedies to address this crisis must be implemented. However, as I have tried to argue here, this cannot be done naively or uncritically, as any new practice could also bring its own drawbacks, including changes in how we value scientific research. It has been shown, for instance, that awareness of replication failures and criticism of questionable research practices can diminish trust in past research (Anvari and Lakens, 2018). Worse, being informed about potential solutions, such as increased transparency, may be, somewhat paradoxically, associated with reduced trust in future research (Anvari and Lakens, 2018). This highlights a significant issue with the trustworthiness of research, which is exacerbated by a lack of enthusiasm.
In conclusion, as discussed in this contribution, the proposed remedies aimed to foster the credibility revolution are highly desirable and non-negotiable. However, these remedies (along with others yet to be envisaged) cannot, sic et simpliciter, be seen as a panacea for the replicability crisis without first being critically evaluated for their own hidden risks or costs. To avoid throwing the baby out with the bathwater, the scientific community desperately needs to find a new balance in the scientific process between imperative methodological rigor and best practices on the one hand and enthusiastic theory-advancing creativity and sustainability on the other. Both types of needs require more quality time for researchers, free from ill-conceived (and only apparently objective) institutional academic incentives and excessive bureaucratic burdens. We need to fine-tune best practices that ensure methodological thoroughness, without renouncing greasing the wheels of scientific creativity, while minimizing the overproduction of non-replicable results. Although I am certainly not the first to propose this sort of call to action, I hope that our journal, Frontiers in Psychology: Cognition, can serve (through specific Research Topics or other formats) as an inspiring receptacle for new ways of conceiving the entire scientific process in the pursuit of this new balance, free of any chauvinistic preconceptions.
Statements
Author contributions
AV: Conceptualization, Writing – original draft, Writing – review & editing.
Funding
The author(s) declare that financial support was received for the research and/or publication of this article. AV was partially supported by the Italian Ministry of Health (project code: GR-2018-12367927, FINAGE), by the Italian Ministry of University and Research (MUR), and by the European Union in the framework of PNRR NextGenerationEU with projects: MNESYS (CUP #B83C22004960002; PE0000006) and ORCA (CUP #J93C24000190007; PE0000013).
Acknowledgments
Part of this work was inspired by fruitful discussions with AV and other esteemed colleagues.
Conflict of interest
The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
The author(s) declared that they were an editorial board member of Frontiers, at the time of submission. This had no impact on the peer review process and the final decision.
Generative AI statement
The author(s) declare that Gen AI was used in the creation of this manuscript. ChatGPT (OpenAI, 2024) was occasionally used for language editing purposes.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
1
Adair J. G. (1984). The Hawthorne effect: a reconsideration of the methodological artifact. J. Appl. Psychol.69, 334–345. 10.1037//0021-9010.69.2.334
2
Anderson M. S. Martinson B. C. De Vries R. (2007). Normative dissonance in science: results from a national survey of U.S. Scientists. J. Empir. Res. Human Res. Ethics2, 3–14. 10.1525/jer.2007.2.4.3
3
Anvari F. Lakens D. (2018). The replicability crisis and public trust in psychological science. Compreh. Results Soc. Psychol.3, 266–286. 10.1080/23743603.2019.1684822
4
Braver T. S. Krug M. K. Chiew K. S. Kool W. Westbrook J. A. Clement N. J. et al . (2014). Mechanisms of motivation–cognition interaction: challenges and opportunities. Cognit. Affect. Behav. Neurosci.14, 443–472. 10.3758/s13415-014-0300-0
5
Brignol A. Paas A. Sotelo-Castro L. St-Onge D. Beltrame G. Coffey E. B. J. (2024). Overcoming boundaries: interdisciplinary challenges and opportunities in cognitive neuroscience. Neuropsychologia200, 108903. 10.1016/j.neuropsychologia.2024.108903
6
Busemeyer J. R. Diederich A. (2010). Cognitive Modeling. Thousand Oaks, CA: SAGE.
7
Button K. S. Ioannidis J. P. A. Mokrysz C. Nosek B. A. Flint J. Robinson E. S. J. et al . (2013). Power failure: why small sample size undermines the reliability of neuroscience. Nat. Rev. Neurosci.14, 365–376. 10.1038/nrn3475
8
Chepenik L. G. Cornew L. A. Farah M. J. (2007). The influence of sad mood on cognition. Emotion7, 802–811. 10.1037/1528-3542.7.4.802
9
Clark C. J. Costello T. Mitchell G. Tetlock P. E. (2022a). Keep your enemies close: adversarial collaborations will improve behavioral science. J. Appl. Res. Mem. Cogn.11, 1–18. 10.1037/mac0000004
10
Clark C. J. Honeycutt N. Jussim L. (2022b). “Replicability and the psychology of science,” in Avoiding Questionable Research Practices in Applied Psychology (Cham: Springer Nature Switzerland AG), 45–71
11
Clark L. A. Watson D. (2019). Constructing validity: New developments in creating objective measuring instruments. Psychol. Assess.31, 1412–1427. 10.1037/pas0000626
12
Coltheart M. (2006). What has functional neuroimaging told us about the mind (so far)? Cortex42, 323–331. 10.1016/S0010-9452(08)70358-7
13
Cowan N. Belletier C. Doherty J. M. Jaroslawska A. J. Rhodes S. Forsberg A. et al . (2020). How do scientific views change? Notes from an extended adversarial collaboration. Perspect. Psychol. Sci.15, 1011–1025. 10.1177/1745691620906415
14
Dhawale A. K. Smith M. A. Ölveczky B. P. (2017). The role of variability in motor learning. Annual Rev. Neurosci.40, 479–498. 10.1146/annurev-neuro-072116-031548
15
Dosenbach N. U. F. Nardos B. Cohen A. L. Fair D. A. Power J. D. Church J. A. et al . (2010). Prediction of individual brain maturity using fMRI. Science329, 1358–1361. 10.1126/science.1194144
16
Ellemers N. Fiske S. T. Abele A. E. Koch A. Yzerbyt V. (2020). Adversarial alignment enables competing models to engage in cooperative theory building toward cumulative science. Proc. Nat. Acad. Sci.117, 7561–7567. 10.1073/pnas.1906720117
17
Fiske D. W. Rice L. (1955). Intra-individual response variability. Psychol. Bull.52, 217–250. 10.1037/h0045276
18
Hilgard J. Sala G. Boot W. R. Simons D. J. (2019). Overestimation of action-game training effects: publication bias and salami slicing. Collabra: Psychol.5:1. 10.1525/collabra.231
19
Hommel B. Colzato L. S. (2017). The grand challenge: integrating nomothetic and ideographic approaches to human cognition. Front. Psychol.8:100. 10.3389/fpsyg.2017.00100
20
Hommel B. Fischer R. Colzato L. S. van den Wildenberg W. P. M. Cellini C. (2012). The effect of fMRI (noise) on cognitive control. J. Exp. Psychol. Hum. Percept. Perform.38, 290–301. 10.1037/a0026353
21
Judd N. Aristodemou M. Klingberg T. Kievit R. (2024). Interindividual differences in cognitive variability are ubiquitous and distinct from mean performance in a battery of eleven tasks. J. Cognit.7:371. 10.5334/joc.371
22
Kahana M. J. Aggarwal E. V. Phan T. D. (2018). The variability puzzle in human memory. J. Exp. Psychol. Learn. Mem. Cogn.44, 1857–1863. 10.1037/xlm0000553
23
Kane M. J. Conway A. R. A. Miura T. K. Colflesh G. J. H. (2007). Working memory, attention control, and the N-back task: a question of construct validity. J. Exp. Psychol. Learn. Mem. Cogn.33, 615–622. 10.1037/0278-7393.33.3.615
24
Karlsson J. Beaufils P. (2013). Legitimate division of large data sets, salami slicing and dual publication, where does a fraud begin?Knee Surg Sports Traumatol. Arthrosc.21, 751–752. 10.1007/s00167-013-2413-3
25
Kaufman J. C. Glǎveanu V. P. (2018). The road to uncreative science is paved with good intentions: ideas, implementations, and uneasy balances. Perspect. Psychol. Sci.13, 457–465. 10.1177/1745691617753947
26
Kerr N. L. (1998). HARKing: hypothesizing after the results are known. Personal. Soc. Psychol. Rev.2, 196–217. 10.1207/s15327957pspr0203_4
27
Killgore W. D. S. (2010). “Effects of sleep deprivation on cognition,” in Progress in Brain Research, eds. G. A. Kerkhof & H. P. A. van Dongen (London: Elsevier), 105–129.
28
Kim K. H. (2011). The creativity crisis: the decrease in creative thinking scores on the torrance tests of creative thinking. Creat. Res. J.23, 285–295. 10.1080/10400419.2011.627805
29
Kolb B. Gibb R. (2011). Brain plasticity and behaviour in the developing brain. J. Canadian Acad. Child Adolescent Psychiat.20, 265–276.
30
Korbmacher M. Azevedo F. Pennington C. R. Hartmann H. Pownall M. Schmidt K. et al . (2023). The replication crisis has led to positive structural, procedural, and community changes. Commun. Psychol.1, 1–13. 10.1038/s44271-023-00003-2
31
Kriegeskorte N. Simmons W. K. Bellgowan P. S. Baker C. I. (2009). Circular analysis in systems neuroscience – the dangers of double dipping. Nat. Neurosci.12, 535–540. 10.1038/nn.2303
32
Kunda Z. (1990). The case for motivated reasoning. Psychol. Bull.108, 480–498. 10.1037//0033-2909.108.3.480
33
Lakens D. (2023). Concerns about Replicability, Theorizing, Applicability, Generalizability, and Methodology across Two Crises in Social Psychology. Dover, PA: OSF.
34
Langerock N. Oberauer K. Throm E. Vergauwe E. (2025). The cognitive load effect in working memory: refreshing the empirical landscape, removing outdated explanations. J. Mem. Lang.140:104558. 10.1016/j.jml.2024.104558
35
Letrud K. Hernes S. (2019). Affirmative citation bias in scientific myth debunking: a three-in-one case study. PLoS ONE14:e0222213. 10.1371/journal.pone.0222213
36
Levitt S. D. List J. A. (2011). Was there really a hawthorne effect at the hawthorne plant? An analysis of the original illumination experiments. Am. Econ. J.3, 224–238. 10.1257/app.3.1.224
37
Lilienfeld S. O. (2017). Psychology's replication crisis and the grant culture: righting the ship. Persp. Psychol. Sci.12, 660–664. 10.1177/1745691616687745
38
Lindsay D. S. (2020). Seven steps toward transparency and replicability in psychological science. Can. Psychol.61, 310–317. 10.1037/cap0000222
39
Litwińczuk M. C. Muhlert N. Trujillo-Barreto N. Woollams A. (2023). Using graph theory as a common language to combine neural structure and function in models of healthy cognitive performance. Hum. Brain Mapp.44, 3007–3022. 10.1002/hbm.26258
40
MacPherson S. E. Gillebert C. R. Robinson G. A. Vallesi A. (2019). Editorial: Intra- and inter-individual variability of executive functions: determinant and modulating factors in healthy and pathological conditions. Front. Psychol.10:432. 10.3389/fpsyg.2019.00432
41
Mayernik M. S. (2017). Open data: accountability and transparency. Big Data Soc.4:2. 10.1177/2053951717718853
42
Medeiros K. E. Partlow P. J. Mumford M. D. (2014). Not too much, not too little: The influence of constraints on creative problem solving. Psychol. Aesthet. Creat. Arts8, 198–210. 10.1037/a0036210
43
Mellers B. Hertwig R. Kahneman D. (2001). Do frequency representations eliminate conjunction effects? An exercise in adversarial collaboration. Psychol. Sci.12, 269–275. 10.1111/1467-9280.00350
44
Miller J. Schwarz W. (2017). Implications of individual differences in on-average null effects. J. Exp. Psychol. General147, 377–397. 10.1037/xge0000367
45
Ness R. B. (2015). The Creativity Crisis: Reinventing Science to Unleash Possibility. Oxford: Oxford University Press.
46
Niv Y. (2021). The primacy of behavioral research for understanding the brain. Behav. Neurosci.135, 601–609. 10.1037/bne0000471
47
Nosek B. A. Spies J. R. Motyl M. (2012). Scientific Utopia: II. restructuring incentives and practices to promote truth over publishability. Perspect. Psychol. Sci.7, 615–631. 10.1177/1745691612459058
48
Oberauer K. (2005). Binding and inhibition in working memory: individual and age differences in short-term recognition. J. Exp. Psychol.134, 368–387. 10.1037/0096-3445.134.3.368
49
Oberauer K. Lewandowsky S. (2019). Addressing the theory crisis in psychology. Psychon. Bull. Rev.26, 1596–1618. 10.3758/s13423-019-01645-2
50
Open Science Collaboration . (2015). Estimating the reproducibility of psychological science. Science349:aac4716. 10.1126/science.aac4716
51
Orne M. T. (1962). On the social psychology of the psychological experiment: With particular reference to demand characteristics and their implications. Am. Psychol.17, 776–783. 10.1037/h0043424
52
Pastukhov A. Braun J. (2011). Cumulative history quantifies the role of neural adaptation in multistable perception. J. Vis.11, 12. 10.1167/11.10.12
53
Podsakoff P. M. MacKenzie S. B. Podsakoff N. P. (2012). Sources of method bias in social science research and recommendations on how to control it. Annu. Rev. Psychol.63, 539–569. 10.1146/annurev-psych-120710-100452
54
Poldrack R. A. (2006). Can cognitive processes be inferred from neuroimaging data?Trends Cogn. Sci.10, 59–63. 10.1016/j.tics.2005.12.004
55
Poldrack R. A. Yarkoni T. (2016). From brain maps to cognitive ontologies: informatics and the search for mental structure. Annu. Rev. Psychol.67, 587–612. 10.1146/annurev-psych-122414-033729
56
Pudas S. Persson J. Josefsson M. de Luna X. Nilsson L.-G. Nyberg L. (2013). Brain characteristics of individuals resisting age-related cognitive decline over two decades. J. Neurosci.33, 8668–8677. 10.1523/JNEUROSCI.2900-12.2013
57
Rasti S. Vaesen K. Lakens D. (2025). A Framework for Describing the Levels of Scientific Coordination. Dover, PA: OSF.
58
Roberts R. Gibson E. (2002). Individual differences in sentence memory. J. Psycholinguist. Res.31, 573–598. 10.1023/A:1021213004302
59
Rowe C. Healy S. D. (2014). Measuring variation in cognition. Behav. Ecol.25, 1287–1292. 10.1093/beheco/aru090
60
Scheffer M. (2014). The forgotten half of scientific thinking. Proc. Nat. Acad. Sci.111, 6119–6119. 10.1073/pnas.1404649111
61
Seung S. (2012). Connectome: How the Brain's Wiring Makes Us who We are. Boston, MA: Houghton Mifflin Harcourt.
62
Shadish W. R. Cook T. D. Campbell D. T. (2002). Experimental and Quasi-experimental Designs for Generalized Causal Inference. Boston, MA: Houghton Mifflin.
63
Siegel D. Baveye P. (2010). Battling the paper glut. Science329:1466. 10.1126/science.329.5998.1466-a
64
Simonton D. K. (2004). Creativity in Science: Chance, Logic, Genius, and Zeitgeist. Cambridge: Cambridge University Press.
65
Spielmans G. I. Biehn T. L. Sawrey D. L. (2010). A case study of salami slicing: pooled analyses of duloxetine for depression. Psychother. Psychosom.79, 97–106. 10.1159/000270917
66
Steinhauser M. Maier M. Hübner R. (2007). Cognitive control under stress: how stress affects strategies of task-set reconfiguration. Psychol. Sci.18, 540–545. 10.1111/j.1467-9280.2007.01935.x
67
Stroebe W. Strack F. (2014). The alleged crisis and the illusion of exact replication. Persp. Psychol. Sci.9, 59–71. 10.1177/1745691613514450
68
Tiego J. Fornito A. (2022). “Putting behaviour back into brain–behaviour correlation analyses,” in Aperture Neuro, BWAS Editorials (Roseville, MN: Organization for Human Brain Mapping), 1–4.
69
Tost H. Champagne F. A. Meyer-Lindenberg A. (2015). Environmental influence in the brain, human welfare and mental health. Nat. Neurosci.18, 1421–1431. 10.1038/nn.4108
70
Turner B. M. Forstmann B. U. Love B. C. Palmeri T. J. Van Maanen L. (2017). Approaches to analysis in model-based cognitive neuroscience. J. Math. Psychol.76, 65–79. 10.1016/j.jmp.2016.01.001
71
Turner J. A. Laird A. R. (2012). The cognitive paradigm ontology: design and application. Neuroinformatics10, 57–66. 10.1007/s12021-011-9126-x
72
Urbanowicz C. Reinke B. A. (2018). Publication overlap: building an academic house with salami shingles. Bullet. Ecol. Soc. Am.99:e01425. 10.1002/bes2.1425
73
Verbruggen F. Aron A. R. Band G. P. Beste C. Bissett P. G. Brockett A. T. et al . (2019). A consensus guide to capturing the ability to inhibit actions and impulsive behaviors in the stop-signal task. Elife8:e46323. 10.7554/eLife.46323
74
Viviani G. Visalli A. Montefinese M. Vallesi A. Ambrosini E. (2024). The Stroop legacy: A cautionary tale on methodological issues and a proposed spatial solution. Behav. Res. Methods56, 4758–4785. 10.3758/s13428-023-02215-0
75
Wang L. Hamaker E. Bergeman C. S. (2012). Investigating inter-individual differences in short-term intra-individual variability. Psychol. Methods17, 567–581. 10.1037/a0029317
76
Weber S. J. Cook T. D. (1972). Subject effects in laboratory research: an examination of subject roles, demand characteristics, and valid inference. Psychol. Bull.77, 273–295. 10.1037/h0032351
Summary
Keywords
replicability crisis, credibility revolution, scientific discovery, creativity, psychological theories, open science
Citation
Vallesi A (2025) Credibility revolution: pursuing a balanced and sustainable approach, without dogmas, without magic elixirs. Front. Psychol. 16:1581160. doi: 10.3389/fpsyg.2025.1581160
Received
21 February 2025
Accepted
28 March 2025
Published
16 April 2025
Volume
16 - 2025
Edited and reviewed by
Axel Cleeremans, Université libre de Bruxelles, Belgium
Updates
Copyright
© 2025 Vallesi.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Antonino Vallesi antonino.vallesi@unipd.it
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.