Psychology’s continued crises in replicability, validity, generalisability—and thus confidence in its findings—are currently attributed primarily to questionable research practices (QRPs; e.g., p-hacking, HARKing, cherry-picking). Popular pertinent debates therefore centre on proposals for remedying inappropriate method applications (e.g., larger samples, robust statistics, pre-registration, replication). Replicable non-random effects of psychological interventions and differentiations between individuals are essential for applied purposes. But current approaches to replicability, validity and psychometric modelling are merely pragmatic, providing evidence of just utility to discriminate well and consistently between cases and in ways considered important (e.g., social relevance, relations to future outcomes). Yet without understanding the study phenomena in themselves (e.g., individuals’ experience, beliefs, thought processes) and without elaborating how relevant features of them can be made amenable to quantitative investigation, and if at all (the quantity objection), the actual causes of replicable quantitative findings cannot be explored.
These causes, however, may be completely unrelated to the actual study phenomena (e.g., methodomorphism). This concerns in particular the ‘attributes’ and ‘latent constructs’ studied in psychology given that these are theoretical and statistical concepts that are attributed and construed as means of investigation and that only refer to but do not constitute the actual study phenomena in themselves. We believe the currently debated deficiencies of quantitative practices in psychology are just surface symptoms whereas fundamental problems (e.g., the inappropriateness of statistical assumptions for psychical phenomena) and the actual root causes underlying psychology’s crises and disputes around its scientific status are still hardly addressed.
With this Research Topic, we want to give new impetus to critical debates on the epistemological and methodological foundations of quantitative methods and measurement in psychology, including problems long known but still hardly considered. Examples are:
- The behavioural and psychical phenomena’s non-ergodicity, rendering sample-level statistics uninformative about individual-level phenomena;
- Psychometricians’ alignment of results to statistical theories rather than to features of the study phenomena, biasing inferences on the latter;
- Ideas that laypeople’s intuitive judgements could reveal quantitative structures in complex, largely unobservable phenomena;
- The still unsupported claim that psychical phenomena could have quantitative properties at all and could thus be measured;
- The fact that psychometric structures shown for verbal item judgements can also be produced by semantic and Artificial Intelligence (AI) algorithms (e.g., ChatGPT) without collecting any empirical data.
Still, most quantitative psychologists seem fairly unconcerned about the strong reliance on everyday language (e.g., in ratings ‘scales’) and the imprecise concepts of psychological ‘measurement’—perhaps because their consequences do not become as straightforwardly apparent as in other disciplines, such as in physics, chemistry and medicine where imprecise measurement can entail the collapse of buildings, chemical explosions or drug overdoses. But where does the loose jargon around ‘measurement’, ‘variables’ and ‘psychometrics’ go seriously wrong in psychology? These and further contradictions, problems and challenges underlying established quantitative practices in psychology must be addressed to tackle the discipline’s continued crises and to develop new approaches that are needed to advance psychology as a science.
We invite from all fields contributions that critically explore and open up new perspectives on the foundations of quantification and measurement in psychology, including perspectives still hardly considered (e.g., qualitative mathematics, complex dynamic systems, fuzzy systems modelling, machine learning). We are also interested in methods for generating quantitative data either executed by persons (e.g., time-based observations, computer-based coding) or by automated technology (e.g., ecological momentary monitoring, life logging, reality mining).
However, other than commonly done, we are explicitly not seeking proposals for improving specific techniques of data analysis (e.g., statistical or latent variable models). Instead, our focus is on making explicit and scrutinising the (implicitly) underlying rationales and fundamental principles—thus, the theory, philosophy and methodology on which quantitative research in psychology currently is or should be built in the future. This also includes critical views on whether or not quantitative investigations are meaningful at all for a science exploring behaviour, mind and society, and thus, also contributions to the psychology, sociology and philosophy of science in this field.
Psychology’s continued crises in replicability, validity, generalisability—and thus confidence in its findings—are currently attributed primarily to questionable research practices (QRPs; e.g., p-hacking, HARKing, cherry-picking). Popular pertinent debates therefore centre on proposals for remedying inappropriate method applications (e.g., larger samples, robust statistics, pre-registration, replication). Replicable non-random effects of psychological interventions and differentiations between individuals are essential for applied purposes. But current approaches to replicability, validity and psychometric modelling are merely pragmatic, providing evidence of just utility to discriminate well and consistently between cases and in ways considered important (e.g., social relevance, relations to future outcomes). Yet without understanding the study phenomena in themselves (e.g., individuals’ experience, beliefs, thought processes) and without elaborating how relevant features of them can be made amenable to quantitative investigation, and if at all (the quantity objection), the actual causes of replicable quantitative findings cannot be explored.
These causes, however, may be completely unrelated to the actual study phenomena (e.g., methodomorphism). This concerns in particular the ‘attributes’ and ‘latent constructs’ studied in psychology given that these are theoretical and statistical concepts that are attributed and construed as means of investigation and that only refer to but do not constitute the actual study phenomena in themselves. We believe the currently debated deficiencies of quantitative practices in psychology are just surface symptoms whereas fundamental problems (e.g., the inappropriateness of statistical assumptions for psychical phenomena) and the actual root causes underlying psychology’s crises and disputes around its scientific status are still hardly addressed.
With this Research Topic, we want to give new impetus to critical debates on the epistemological and methodological foundations of quantitative methods and measurement in psychology, including problems long known but still hardly considered. Examples are:
- The behavioural and psychical phenomena’s non-ergodicity, rendering sample-level statistics uninformative about individual-level phenomena;
- Psychometricians’ alignment of results to statistical theories rather than to features of the study phenomena, biasing inferences on the latter;
- Ideas that laypeople’s intuitive judgements could reveal quantitative structures in complex, largely unobservable phenomena;
- The still unsupported claim that psychical phenomena could have quantitative properties at all and could thus be measured;
- The fact that psychometric structures shown for verbal item judgements can also be produced by semantic and Artificial Intelligence (AI) algorithms (e.g., ChatGPT) without collecting any empirical data.
Still, most quantitative psychologists seem fairly unconcerned about the strong reliance on everyday language (e.g., in ratings ‘scales’) and the imprecise concepts of psychological ‘measurement’—perhaps because their consequences do not become as straightforwardly apparent as in other disciplines, such as in physics, chemistry and medicine where imprecise measurement can entail the collapse of buildings, chemical explosions or drug overdoses. But where does the loose jargon around ‘measurement’, ‘variables’ and ‘psychometrics’ go seriously wrong in psychology? These and further contradictions, problems and challenges underlying established quantitative practices in psychology must be addressed to tackle the discipline’s continued crises and to develop new approaches that are needed to advance psychology as a science.
We invite from all fields contributions that critically explore and open up new perspectives on the foundations of quantification and measurement in psychology, including perspectives still hardly considered (e.g., qualitative mathematics, complex dynamic systems, fuzzy systems modelling, machine learning). We are also interested in methods for generating quantitative data either executed by persons (e.g., time-based observations, computer-based coding) or by automated technology (e.g., ecological momentary monitoring, life logging, reality mining).
However, other than commonly done, we are explicitly not seeking proposals for improving specific techniques of data analysis (e.g., statistical or latent variable models). Instead, our focus is on making explicit and scrutinising the (implicitly) underlying rationales and fundamental principles—thus, the theory, philosophy and methodology on which quantitative research in psychology currently is or should be built in the future. This also includes critical views on whether or not quantitative investigations are meaningful at all for a science exploring behaviour, mind and society, and thus, also contributions to the psychology, sociology and philosophy of science in this field.