Measurement problems in psychology are noteworthy because of measurement’s crucial role in fostering scientific progress. Reviews of scientific development indicate that the history of science is associated with the history of measurement and that scientific advancement rests on valid measurement. Even a cursory review of the history of science indicates that new measurement techniques drive scientific development. Continuous innovation in measurement theory and techniques seems to be important for progress in any science. Scientific progress is enabled by measurement's capacity to provide new data for ideas, extend human senses into new domains, and correct for limitations of the senses. Despite decades of conversation about the importance of measurement, scholars continue to warn us that measurement problems are pervasive. Both basic and applied researchers must remain vigilant about best measurement practices to protect the replicability of findings in our discipline.
We encourage authors to submit papers that describe current measurement issues in psychology that significantly hinder research progress in their particular domains. As an example, researchers in both experimental and correlational studies often default to a single operation for obtaining data, particularly if the operation is an economical self-report. In their review of the occupational stress literature, for example, researchers found that the self-report Maslach Burnout Inventory (MBI) was employed in approximately 80% of studies. Researchers in these areas appear to sidestep thorny issues related to construct explication by adopting a single measure that becomes accepted as a standard, particularly for publication purposes.
We further encourage authors to describe innovative solutions to address measurement problems. The emergence of open data practices and open science tools, such as the Open Science Framework (osf.io) for conducting research, for example, offers new possibilities for tackling measurement issues. Researchers now have can more easily communicate across international borders to share ideas, data, and code. These collaborations should enhance the field’s ability to identify and address both unique and common measurement problems.
During recent sessions at the 2021 Psychological Science Accelerator Conference and the 2021 Society for the Improvement of Psychological Science Conference, attendees from 14 countries discussed the prevalence of measurement problems in their respective areas of research and the contributions that poor measurement practices make to the ongoing replication crisis in social and behavior sciences. Attendees noted that researchers have been stymied by such long-standing problems as how to translate psychological tests into other languages (including whether constructs in one culture have equivalent meanings in other cultures), dealing with self-reports of sensitive issues such as sexual aggression (which has low base rates, ambiguous definition of terms, and potential legal jeopardy when disclosing illegal acts), and the correspondence (or lack thereof) between self-reports and behaviors.
Many researchers at the conferences noted that they did not begin their work with a particular interest in measurement, but “stumbled into measurement” as an inevitable consequence of thinking deeply about problems in their studies.
Measurement problems in psychology are noteworthy because of measurement’s crucial role in fostering scientific progress. Reviews of scientific development indicate that the history of science is associated with the history of measurement and that scientific advancement rests on valid measurement. Even a cursory review of the history of science indicates that new measurement techniques drive scientific development. Continuous innovation in measurement theory and techniques seems to be important for progress in any science. Scientific progress is enabled by measurement's capacity to provide new data for ideas, extend human senses into new domains, and correct for limitations of the senses. Despite decades of conversation about the importance of measurement, scholars continue to warn us that measurement problems are pervasive. Both basic and applied researchers must remain vigilant about best measurement practices to protect the replicability of findings in our discipline.
We encourage authors to submit papers that describe current measurement issues in psychology that significantly hinder research progress in their particular domains. As an example, researchers in both experimental and correlational studies often default to a single operation for obtaining data, particularly if the operation is an economical self-report. In their review of the occupational stress literature, for example, researchers found that the self-report Maslach Burnout Inventory (MBI) was employed in approximately 80% of studies. Researchers in these areas appear to sidestep thorny issues related to construct explication by adopting a single measure that becomes accepted as a standard, particularly for publication purposes.
We further encourage authors to describe innovative solutions to address measurement problems. The emergence of open data practices and open science tools, such as the Open Science Framework (osf.io) for conducting research, for example, offers new possibilities for tackling measurement issues. Researchers now have can more easily communicate across international borders to share ideas, data, and code. These collaborations should enhance the field’s ability to identify and address both unique and common measurement problems.
During recent sessions at the 2021 Psychological Science Accelerator Conference and the 2021 Society for the Improvement of Psychological Science Conference, attendees from 14 countries discussed the prevalence of measurement problems in their respective areas of research and the contributions that poor measurement practices make to the ongoing replication crisis in social and behavior sciences. Attendees noted that researchers have been stymied by such long-standing problems as how to translate psychological tests into other languages (including whether constructs in one culture have equivalent meanings in other cultures), dealing with self-reports of sensitive issues such as sexual aggression (which has low base rates, ambiguous definition of terms, and potential legal jeopardy when disclosing illegal acts), and the correspondence (or lack thereof) between self-reports and behaviors.
Many researchers at the conferences noted that they did not begin their work with a particular interest in measurement, but “stumbled into measurement” as an inevitable consequence of thinking deeply about problems in their studies.
