Edited by: Claudio De Stefano, University of Cassino, Italy
Reviewed by: Hugo Gamboa, New University of Lisbon, Portugal; Nicole Cilia, University of Cassino, Italy
This article was submitted to Cognitive Neuroscience, a section of the journal Frontiers in Human Neuroscience
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
We introduce a large-scale dataset of mouse cursor movements that can be used to predict user attention, infer demographics information, and analyze fine-grained movements. Attention is a finite resource, so people spend their time on things they find valuable, especially when browsing online. Objective measurements of attentional processes are increasingly sought after by researchers, advertisers, and other key stakeholders from both academia and industry. With every click, digital footprints are created and logged, providing a detailed record of a person's online activity. However, click data provide an incomplete picture of user interaction, as they inform mainly about a users' end choice. A user click is often preceded by several valuable interactions, such as scrolling, hovers, aimed movements, etc. and thus having access to this kind of data can lead to an overall better understanding of the user's cognitive processes. For example, previous work has evidenced that when the mouse cursor is motionless, the user is processing information (Hauger et al.,
Research in mouse cursor tracking has a long track record. Chen et al. (
For a long time, commercial search engines have been interested in how users interact with Search Engine Result Pages (SERPs), to anticipate better placement and allocation of ads in sponsored search or to optimize the content layout. Early work considered simple, coarse-grained features derived from mouse cursor data to be surrogate measurements of user interest (Goecks and Shavlik,
Mouse cursor tracking has been used to survey the visual focus of the user, thus revealing valuable information regarding the distribution of user attention over the various SERP components. Despite the technical challenges that may arise from this analysis, previous work has shown the utility of mouse movement patterns to measure within-content engagement (Arapakis et al.,
The connection between mouse cursor movements and the underlying psychological states has been a topic of research since the early 90s (Card et al.,
Prior work has linked age with motor control and pointing performance in tasks that involve the use of a computer mouse (Walker et al.,
Others have also examined the extent to which mouse cursor movements can help identify gender and age (Yamauchi and Bowman,
We ran an online crowdsourcing study that reproduced the conditions of a transactional search task. Participants were presented with a simulated information need that explained that they were interested in purchasing some product for them or a friend. Overall, the study consisted of three parts, to be described later: (1) pre-task guidelines, (2) the web search task, and (3) a post-task questionnaire.
We recruited participants from the F
Demographics information from our dataset.
18–23 | 380 | Male | 1,605 | USA | 1,755 | High school | 593 | <25K | 881 |
24–29 | 716 | Female | 1,118 | VEN | 251 | College | 472 | 25–34K | 446 |
30–35 | 590 | NA | 14 | GBR | 209 | Bachelor's | 704 | 35–49K | 367 |
36–41 | 417 | CAN | 66 | Graduate | 499 | 50–74K | 394 | ||
42–47 | 223 | EGY | 37 | Master's | 399 | 75–99K | 249 | ||
48–53 | 174 | UKR | 31 | Doctorate | 30 | 100–149K | 145 | ||
54–59 | 132 | IND | 29 | NA | 40 | 150–249K | 42 | ||
60–65 | 63 | SRB | 27 | >250K | 23 | ||||
+66 | 24 | RUS | 25 | NA | 190 | ||||
NA | 18 | … |
Starting from Google Trends
Using this final selection of search queries, we produced the static version of the corresponding Google SERPs and injected custom JavaScript code that allowed us to capture all client-side user interactions. For this, we used E
All queries triggered some form of advertisements on the SERPs, according to three different formats: “native” (organic ads) or “bundled” (direct display ads). All SERPs included one or more native ads together with one bundled ad. The native advertisements could appear either at the top or bottom position of the SERP, whereas the bundled ads could appear either at the top-left or top-right position. We ensured that only one ad was visible per condition and participant at a time. This was possible by instrumenting each downloaded SERP with custom JavaScript code that removed all ads excepting one that would be selected for a given participant. In any case, native bottom-most ads were not shown to the participants.
Participants were instructed to read carefully the terms and conditions of the study which, among other things, informed them that they should perform the task from a desktop or laptop computer using a computer mouse (and refrain from using a touchpad, tablet, or mobile device) and that their browsing activity would be logged. Moreover, participants consented to share their browsing data and their (anonymized) responses for later analysis.
Participants were asked to act naturally and choose anything that would best answer a given search query, since all “clickable” elements (e.g., result links, images, etc.) on the SERP were considered valid answers. The instructions were followed by a brief search task description using this template: “
Participants were allowed as much time as they needed to examine the SERP and proceed with the search task, which would conclude whenever they clicked on any SERP element. The payment for the participation was $0.20. Participants could also opt out at any moment, in which case they were not compensated. Each participant could take the study only once.
Each participant was presented with a search task description, then provided with a predefined search query (selected at random from our pool of queries) and the corresponding SERP, and they were asked to click on any element of the page that best solved the task. This way, we ensured that participants interacted with the same pool of web search queries and avoided any unaccounted systematic bias due to query quality variation. All possible combinations of query and ad style (i.e., format and position) were pre-computed so that whenever a new user accessed the study, they were assigned one of these combinations at random.
Participants accessed the instrumented SERPs through a dedicated web server that did not alter the look and feel of the original SERPs. This allowed us to capture fine-grained user interactions while ensuring that the content of the SERPs remained consistent with the original version. Each participant was allowed to perform the search task only once to avoid introducing possible carry over effects and, thus, altering their browsing behavior in subsequent search tasks. In sum, each participant was exposed only to a single condition; i.e., a unique combination of query and ad style. Finally, at the end of the study participants had to copy a unique code and paste it on F
Upon concluding the search task, participants were asked to answer a series of questions. The questions were forced-choice type and allowed multi-point response options.
The first question asked the degree to which the user noticed the advertisements shown on the SERP:
The questionnaire also comprised the following demographics-related questions:
Crowdsourcing studies offer several advantages over
We collected self-reported ground-truth labels in a similar vein to previous work (Feild et al.,
Starting from a set of 3,223 participants who initially accessed the study, we filtered automatically those who did not finish it (138 cases) as well as participants who did not move their mouse at all (176 cases). We concluded to a dataset with 2,909 observations comprising at least one mouse movement, together with their associated browser's and user's metadata. See
There are 92 unique combinations of query and ad style, each of which assessed by 32 users on average (
Excepting the automatic filtering procedure explained above, our data is in raw form and therefore some columns require further processing. For example, most columns pertaining demographics information are stored as integers, therefore researchers should consult
The Attentive Cursor dataset includes the following resources:
A folder with mouse tracking log files, as recorded by the E
Browser events: space-delimited files (CSV) with information about each event type (8 columns).
Browser metadata: XML files with information about the user's browser (e.g., viewport size).
A TSV file with ground-truth labels (4 columns).
A tab-delimited file (TSV) with user's demographics and stimulus condition (12 columns).
A folder with all SERPs in HTML format.
A README file with a detailed explanation of each resource.
File content samples (top) and SERP snapshots with mouse cursor trajectories (bottom). An ellipsis (
We have presented a large-scale, in-the-wild dataset of mouse cursor movements in web search, with associated ground-truth labels about user's attention and demographics attributes. The dataset represents real-world behavior of individuals completing a transactional web search task. What makes this dataset both unique and challenging is the fact that there is only one observation per user. It is not possible to leak information from any data splits; e.g., training, validation, and testing splits typically used in machine learning studies. It is our hope that the dataset will foster research in several scientific domains, Including, e.g., information retrieval, movement science, and psychology.
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. The patients/participants provided their written informed consent to participate in this study.
All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.
IA was employed by the company Telefonica Research, though no payment or services from the institution has been received or requested for any aspect of the submitted work. The remaining author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Two manuscripts using a post-processed version of this dataset have been recently published by the authors (Arapakis and Leiva,
1
2
3