Modelling and Assessing Multiplicative Operation Sense – Validation of a Test Instrument for 5th Grade

Abstract

A considerable number of students enter secondary school with fundamental gaps in basic mathematical skills, particularly in their understanding of operations related to multiplication and division. The conceptual understanding of operations — operation sense — is the ability to relate situations (e.g., word problems) to mathematical-symbolic notations (e.g., calculations, equations) and vice versa and it is crucial for further learning in mathematics. In order to provide adaptive support for operation sense, the understanding or the lack thereof must be systematically assessed. Therefore, a sensitive assessment instrument specifically tailored to the operations of multiplication and division is needed, which can be used for a focused diagnosis and for evaluation of specific interventions in this domain. Thus, we developed an instrument for assessing multiplicative operation sense, which represents theoretically grounded levels of understanding operations in multiplicative situations. In the pilot study (N = 66), general linear mixed models showed that – as expected – the estimated solution rate decreases with increasing level and that 86 % of the variance in item difficulty can be explained by the four theoretically grounded levels of multiplicative operation sense. After revising three items and eliminating other shortcomings of the pilot study, the main study (N = 464) showed 94% explained variance and significant mean differences between all levels. These findings support the validity of the instrument and its usefulness for both research and practice.