How do we collect all this data? A performative account of International Large-Scale Assessment data collection in times of systemic diversity

International Large-Scale Assessments (ILSAs) serve to monitor and benchmark educational achievements, providing accurate, valid, and reliable data for participating countries to govern the development of their education systems. While standardized operating procedures and practices in administering these studies are well documented, and achievement reports are produced both nationally and internationally, there is less emphasis on how the intention of standardized processes manifests in each participating country during data collection. The feasibility of data collection in nationally representative samples, the populations being sampled, the strata used, exclusions at the system and school level, and the number of languages involved are critical questions that need to be addressed to understand the accuracy of PIRLS data. The data collection process reveals important insights about a country's education system, reflecting not just deficits but also the specific culture, changes over time, and the embedding of education into its society. This research topic aims to bring the process of data collection to the forefront, highlighting its significance in understanding the context of education systems where ILSAs are administered.

This research topic aims to discuss the process of country-level data collection efforts, challenges, and considerations. It seeks to unpack country-level accounts of data collection as a resource, discovery, and assumption of the operational side of ILSAs, which are crucial in creating, curating, and compiling ongoing systems of educational monitoring and benchmarking. By focusing on how data is collected, the research aims to provide insights into the design and implementation challenges that are often overlooked in scholarly work that tends to focus on data outcomes and results. Specific questions to be answered include the impact of sampling stratification decisions, system and school-level exclusions, response rates, modes of participation, translation of assessment materials, and scoring of open-ended responses.

To gather further insights into the boundaries of data collection in ILSAs, we welcome articles addressing, but not limited to, the following themes:

- Sampling stratification decisions and how these shape the assessment of the population
- Decisions about system and school-level exclusions
- Response rates and efforts to increase these within and across study cycles
- Choices in modes of participation (i.e., electronic vs. paper and pencil)
- Translation of assessment materials and efforts to quality assure these processes
- Scoring open-ended response efforts

By exploring these themes, the research aims to provide a comprehensive understanding of the complexities and nuances involved in the data collection process of ILSAs.

Keywords: ILSC, Large Scale Assessment, Data Collection, Benchmark Education

Important note: All contributions to this Research Topic must be within the scope of the section and journal to which they are submitted, as defined in their mission statements. Frontiers reserves the right to guide an out-of-scope manuscript to a more suitable section or journal at any stage of peer review.