AUTHOR=Roman Zachary Joseph , Brandt Holger , Miller Jason Michael 

TITLE=Automated Bot Detection Using Bayesian Latent Class Models in Online Surveys

JOURNAL=Frontiers in Psychology

VOLUME=Volume 13 - 2022

YEAR=2022

URL=https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2022.789223

DOI=10.3389/fpsyg.2022.789223

ISSN=1664-1078

ABSTRACT=Researchers have recently started to collect data on online platforms such as Amazon’s Mechanical Turk. The advantage of these platforms include an easy access to a potentially more representative sample, a huge population, and an easy to use implementation and data management. A major drawback that has recently been discovered is the existence of statistical bots that are used to complete such surveys because of the financial gain associated with it. These bots contaminate the data and need to be identified in order to draw any valid conclusion from data obtained from online platforms. In this article, we will provide a Bayesian latent class model that can be routinely applied to identify statistical bots. This method can be used to separate the bots’ response patterns from real human responses that were provided in line with the item content. The model has the advantage that it is very flexible and is based on plausible assumptions that are met in most empirical settings. We will provide a simulation study that investigates the performance of the model under several relevant scenarios including sample size, proportion of bots, and model complexity. We will show that ignoring bots will lead to severe parameter bias whereas the Bayesian latent class model results in unbiased estimates and thus controls this source of bias. We will illustrate the model and its capabilities with data from an empirical political ideation survey with known bots. We will discuss the implications of the findings with regard to future data collection via online platforms.