Hypothesis-testing demands trustworthy data—a simulation approach to inferential statistics advocating the research program strategy
- 1University of Hamburg, Germany
- 2Lund University, Sweden
- 3Universität Konstanz, Germany
- 4Slovak Academy of Sciences (SAS), Slovakia
- 5Sun Yat-sen University, China
- 6Université de Genève, Switzerland
In psychology as elsewhere, the main statistical inference strategy to establish empirical effects is null-hypothesis significance testing (NHST). The recent failure to replicate allegedly well-established NHST-results, however, implies that such results lack sufficient statistical power, and thus feature unacceptably high error-rates. Using data-simulation to estimate the error-rates of NHST-results, we advocate the research program strategy (RPS) as a superior methodology. RPS integrates Frequentist with Bayesian inference elements, and leads from a preliminary discovery against a (random) H0-hypothesis to a statistical H1-verification. Not only do RPS-results feature significantly lower error-rates than NHST-results, RPS also addresses key-deficits of a “pure” Frequentist and a standard Bayesian approach. In particular, RPS aggregates underpowered results safely. RPS therefore provides a tool to regain the trust the discipline had lost along the ongoing replicability-crisis.
Keywords: inferential statistics, t-test, Wald-criterion, likelihood, Bayes Theorem, Research Program Strategy
Received: 18 Oct 2017;
Accepted: 19 Mar 2018.
Edited by:Holmes Finch, Ball State University, United States
Reviewed by:Paul T. Barrett, Advanced Projects R&D Ltd., Australia
Aristides (Aris) Moustakas, Universiti Brunei Darussalam, Brunei
Copyright: © 2018 Witte, Zenker and Krefeld-Schwalb. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
* Correspondence: Dr. Antonia Krefeld-Schwalb, Université de Genève, Geneva, Switzerland, firstname.lastname@example.org