Impact Factor 2.089

The world's most-cited Multidisciplinary Psychology journal

Methods ARTICLE Provisionally accepted The full-text will be published soon. Notify me

Front. Psychol. | doi: 10.3389/fpsyg.2018.02298

Item-Score Reliability as a Selection Tool in Test Construction

 Eva A. Zijlmans1*, Jesper Tijmstra1, L. Andries Van der Ark2 and Klaas Sijtsma1
  • 1Methodology and Statistics, Tilburg University, Netherlands
  • 2University of Amsterdam, Netherlands

This study investigates the usefulness of item-score reliability as a criterion for item selection in test construction. Methods MS, lambda-6, and CA were investigated as item-assessment methods in item selection and compared to the corrected item-total correlation, which was used as a benchmark. An ideal ordering to add items to the test (bottom-up procedure) or omit items from the test (top-down procedure) was defined based on the population test-score reliability. The orderings the four item-assessment methods produced in samples were compared to the ideal ordering, and the degree of resemblance was expressed by means of Kendall's tau. To investigate the concordance of the orderings across 1000 replicated samples, Kendall's W was computed for each item-assessment method. The results showed that for both the bottom-up and the top-down procedure, item-assessment method CA and the corrected item-total correlation most closely resembled the ideal ordering. Generally, all item assessment methods resembled the ideal ordering better, and concordance of the orderings was greater, for larger sample sizes and greater variance of the item discrimination parameters.

Keywords: Correction for attenuation, corrected item-total correlation, item-score reliability, item selection in test construction, method CA, method lambda 6, method MS

Received: 16 Aug 2018; Accepted: 05 Nov 2018.

Edited by:

Holmes Finch, Ball State University, United States

Reviewed by:

Fernando Marmolejo-Ramos, University of Adelaide, Australia
Elena Zaitseva, University of Žilina, Slovakia  

Copyright: © 2018 Zijlmans, Tijmstra, Van der Ark and Sijtsma. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

* Correspondence: Ms. Eva A. Zijlmans, Tilburg University, Methodology and Statistics, Tilburg, 5000 LE, Netherlands,