<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD Journal Publishing DTD v2.3 20070202//EN" "journalpublishing.dtd">
<article article-type="research-article" dtd-version="2.3" xml:lang="EN" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">
<front>
<journal-meta>
<journal-id journal-id-type="publisher-id">Front. Comput. Sci.</journal-id>
<journal-title>Frontiers in Computer Science</journal-title>
<abbrev-journal-title abbrev-type="pubmed">Front. Comput. Sci.</abbrev-journal-title>
<issn pub-type="epub">2624-9898</issn>
<publisher>
<publisher-name>Frontiers Media S.A.</publisher-name>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="publisher-id">628634</article-id>
<article-id pub-id-type="doi">10.3389/fcomp.2021.628634</article-id>
<article-categories>
<subj-group subj-group-type="heading">
<subject>Computer Science</subject>
<subj-group>
<subject>Original Research</subject>
</subj-group>
</subj-group>
</article-categories>
<title-group>
<article-title>A Universal Screening Tool for Dyslexia by a Web-Game and Machine Learning</article-title>
<alt-title alt-title-type="left-running-head">Rauschenberger et&#x20;al.</alt-title>
<alt-title alt-title-type="right-running-head">Universal Screening Tool for Dyslexia</alt-title>
</title-group>
<contrib-group>
<contrib contrib-type="author" corresp="yes">
<name>
<surname>Rauschenberger</surname>
<given-names>Maria</given-names>
</name>
<xref ref-type="aff" rid="aff1">
<sup>1</sup>
</xref>
<xref ref-type="aff" rid="aff2">
<sup>2</sup>
</xref>
<xref ref-type="aff" rid="aff3">
<sup>3</sup>
</xref>
<xref ref-type="corresp" rid="c001">&#x2a;</xref>
<uri xlink:href="https://loop.frontiersin.org/people/1111219/overview"/>
</contrib>
<contrib contrib-type="author">
<name>
<surname>Baeza-Yates</surname>
<given-names>Ricardo</given-names>
</name>
<xref ref-type="aff" rid="aff3">
<sup>3</sup>
</xref>
<xref ref-type="aff" rid="aff4">
<sup>4</sup>
</xref>
</contrib>
<contrib contrib-type="author">
<name>
<surname>Rello</surname>
<given-names>Luz</given-names>
</name>
<xref ref-type="aff" rid="aff5">
<sup>5</sup>
</xref>
</contrib>
</contrib-group>
<aff id="aff1">
<sup>1</sup>
<institution>Technology, University of Applied Science Emden/Leer</institution>, <addr-line>Emden</addr-line>, <country>Germany</country>
</aff>
<aff id="aff2">
<sup>2</sup>
<institution>Max-Planck-Institute for Software Systems</institution>, <addr-line>Saarbr&#xfc;cken</addr-line>, <country>Germany</country>
</aff>
<aff id="aff3">
<sup>3</sup>
<institution>Universitat Pompeu Fabra</institution>, <addr-line>Barcelona</addr-line>, <country>Spain</country>
</aff>
<aff id="aff4">
<sup>4</sup>
<institution>Institute for Experiential AI, Northeastern University, Boston</institution>, <addr-line>MA</addr-line>, <country>United States</country>
</aff>
<aff id="aff5">
<sup>5</sup>
<institution>Department of Information Systems and Technology, IE Business School, IE University</institution>, <addr-line>Madrid</addr-line>, <country>Spain</country>
</aff>
<author-notes>
<fn fn-type="edited-by">
<p>
<bold>Edited by:</bold> <ext-link ext-link-type="uri" xlink:href="https://loop.frontiersin.org/people/158763/overview">Faustina Hwang</ext-link>, University of Reading, United&#x20;Kingdom</p>
</fn>
<fn fn-type="edited-by">
<p>
<bold>Reviewed by:</bold> <ext-link ext-link-type="uri" xlink:href="https://loop.frontiersin.org/people/187957/overview">Kai Kunze</ext-link>, Keio University, Japan</p>
<p>
<ext-link ext-link-type="uri" xlink:href="https://loop.frontiersin.org/people/1243327/overview">Gil Aguilar</ext-link>, Intuit, Eagle, Idaho, United States</p>
</fn>
<corresp id="c001">&#x2a;Correspondence: Maria Rauschenberger, <email>maria.rauschenberger@hs-emden-leer.de</email>
</corresp>
<fn fn-type="other">
<p>This article was submitted to Human-Media Interaction, a section of the journal Frontiers in Computer Science</p>
</fn>
</author-notes>
<pub-date pub-type="epub">
<day>03</day>
<month>01</month>
<year>2022</year>
</pub-date>
<pub-date pub-type="collection">
<year>2021</year>
</pub-date>
<volume>3</volume>
<elocation-id>628634</elocation-id>
<history>
<date date-type="received">
<day>12</day>
<month>11</month>
<year>2020</year>
</date>
<date date-type="accepted">
<day>02</day>
<month>11</month>
<year>2021</year>
</date>
</history>
<permissions>
<copyright-statement>Copyright &#xa9; 2022 Rauschenberger, Baeza-Yates and Rello.</copyright-statement>
<copyright-year>2022</copyright-year>
<copyright-holder>Rauschenberger, Baeza-Yates and Rello</copyright-holder>
<license xlink:href="http://creativecommons.org/licenses/by/4.0/">
<p>This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these&#x20;terms.</p>
</license>
</permissions>
<abstract>
<p>Children with dyslexia have difficulties learning how to read and write. They are often diagnosed after they fail school even if dyslexia is not related to general intelligence. Early screening of dyslexia can prevent the negative side effects of late detection and enables early intervention. In this context, we present an approach for universal screening of dyslexia using machine learning models with data gathered from a web-based language-independent game. We designed the game content taking into consideration the analysis of mistakes of people with dyslexia in different languages and other parameters related to dyslexia like auditory perception as well as visual perception. We did a user study with 313 children (116 with dyslexia) and train predictive machine learning models with the collected data. Our method yields an accuracy of 0.74 for German and 0.69 for Spanish as well as a F1-score of 0.75 for German and 0.75 for Spanish, using Random Forests and Extra Trees, respectively. We also present the game content design, potential new auditory input, and knowledge about the design approach for future research to explore Universal screening of dyslexia. universal screening with language-independent content can be used for the screening of pre-readers who do not have any language skills, facilitating a potential early intervention.</p>
</abstract>
<kwd-group>
<kwd>dyslexia</kwd>
<kwd>screening tool</kwd>
<kwd>game</kwd>
<kwd>machine learning</kwd>
<kwd>German</kwd>
<kwd>Spanish</kwd>
<kwd>study setup</kwd>
<kwd>online experiment</kwd>
</kwd-group>
</article-meta>
</front>
<body>
<sec id="s1">
<title>1 Introduction</title>
<p>Dyslexia is a <italic>specific learning disorder</italic> which affects 5&#x2013;15% of the global population (<xref ref-type="bibr" rid="B1">American Psychiatric Association, 2013</xref>; <xref ref-type="bibr" rid="B70">World Health Organization, 2010</xref>, <xref ref-type="bibr" rid="B69">2019</xref>). A person with dyslexia has difficulties with reading and writing that are independent from intelligence, mother tongue, social status, or education level. Hence, people with dyslexia understand the meanings of words, but do not always know how to spell or pronounce them correctly. However, children with dyslexia do not show any obvious difficulties in other areas. This is why <italic>dyslexia</italic> is considered to be a <italic>hidden</italic> disorder. This often results in bad grades in school and frustration for the children and parents over many years. Around 40&#x2013;60% of children with dyslexia show symptoms of psychological disorders (<xref ref-type="bibr" rid="B59">Schulte-K&#xf6;rne, 2010</xref>) such as negative thoughts, sadness, sorrow, or anxiety. A study showed that even if the child is diagnosed by the age of eight, they achieve lower school performance (<xref ref-type="bibr" rid="B11">Esser et&#x20;al., 2002</xref>). Also, according to the same study, the unemployment rate for adults with dyslexia is higher. Moreover, these are common indicators for detecting a person with dyslexia.</p>
<p>Generally, dyslexia manifestations can be observed when children reach a certain age and literary knowledge. Current approaches to screen (pre-)readers require expensive personnel, such as a professional therapist or special hardware such as fMRI scans (<xref ref-type="bibr" rid="B32">Paulesu et&#x20;al., 2014</xref>). Previous research have studied signs of dyslexia that are not related to reading and writing such as visual perception, short-term memory, executive functions or auditory perception (<xref ref-type="bibr" rid="B19">Goswami et&#x20;al., 2016</xref>). These signs could be used to screen potential dyslexia in pre-readers and our work shows a possible approach for doing this by using machine learning with data coming from a language-independent content integrated in a web-based game [The collected user data is online available (<xref ref-type="bibr" rid="B39">Rauschenberger et&#x20;al., 2021b</xref>)]. Our game has the potential of being easily accessible, making parents aware the potential risk of dyslexia to further look for more help, e.g., a medical doctor or therapist.</p>
<p>The game and the user study is designed with the <italic>human-centered design</italic> framework (ISO/TC 159/SC 4&#x20;<xref ref-type="bibr" rid="B10">Ergonomics of human-system interaction, 2010</xref>) to collect the data set. This is relevant since collecting personal data is challenging because of privacy and trust issues (<xref ref-type="bibr" rid="B2">Baeza-Yates, 2018</xref>; <xref ref-type="bibr" rid="B12">Faraway and Augustin, 2018</xref>; <xref ref-type="bibr" rid="B67">Weigand et&#x20;al., 2021</xref>). As a result, the final data sets are small and <italic>small data</italic> makes the prediction with machine learning models more difficult. That is, there is the risk of over-fitting or having a data set too small to be divided into meaningful test, training and validation sets. Hence, we followed our own recommendations coming from experience analyzing small data (<xref ref-type="bibr" rid="B36">Rauschenberger and Baeza-Yates, 2020a</xref>; <xref ref-type="bibr" rid="B37">Rauschenberger and Baeza-Yates, 2020b</xref>; <xref ref-type="bibr" rid="B67">Weigand et&#x20;al., 2021</xref>).</p>
<p>We use standard machine learning classifiers like Random Forest with and without class weights, Extra Trees and Gradient Boosting from the <italic>Scikit-learn</italic> library for the prediction of dyslexia. Our models yields an accuracy of 0.74 and F1-score of 0.75 in German using a Random Forest and an accuracy of 0.69 and F1-score of 0.75 in Spanish using Extra Trees (<xref ref-type="bibr" rid="B42">Rauschenberger et&#x20;al., 2020</xref>).</p>
<p>Historically, the rates of spelling mistakes and reading errors have been the most common way to detect persons with dyslexia, using the popular paper and pencil assessments in different languages (<xref ref-type="bibr" rid="B7">Cuetos et&#x20;al., 2002</xref>, <xref ref-type="bibr" rid="B8">2007</xref>; <xref ref-type="bibr" rid="B14">Fawcett and Nicolson, 2004</xref>; <xref ref-type="bibr" rid="B20">Grund et&#x20;al., 2004</xref>). Therefore, we compare our game measures and found in our pilot study (<italic>n</italic>&#x20;&#x3d; 178) four significant game measurements for Spanish, German, and English as well as eight significant game measurements for Spanish (<xref ref-type="bibr" rid="B46">Rauschenberger et&#x20;al., 2018b</xref>), e.g., total clicks or time to first&#x20;click.</p>
<p>Early, accurate prediction of dyslexia remains a challenge (<xref ref-type="bibr" rid="B3">Bandhyopadhyay et&#x20;al., 2018</xref>) because dyslexia is known for causing reading and writing problems but no obvious deficits in other areas. Therefore, we need to design language-independent content fit to differentiate between children with and without dyslexia.</p>
<p>Another challenge is finding language-independent content that can show measurable differences between children with and without dyslexia that are comparable to differences in reading and writing mistakes. Designing language-independent content is probably the greatest challenge [also according to a report from the <italic>National Center on Improving Literacy</italic> (<xref ref-type="bibr" rid="B33">Petscher et&#x20;al., 2019</xref>)] because the new indicators, though related to the reading and writing difficulties, are probably not the main causes.</p>
<p>Therefore, we also share here additional <xref ref-type="sec" rid="s14">Supplement Material</xref> of the content design, promising potential new auditory content, and knowledge about the design approach for others to use. To gather the data of this study, we had participants already diagnosed with dyslexia, instead of using pre-readers (younger children), since that would have required a long-term study. At this point, a long-term study with pre-readers would be very time-consuming, since the effort to find participants is high, participants are less likely to be diagnosed, and much time passes before results are available. An online study with readers has the advantage of reducing the effort and time required to design content, conduct various experiments for optimization, and increase the number of participants. Nevertheless, the language-independent content can be used to screen pre-readers who do not yet have any language skills. Additionally, we present the design decisions for the content creation for the auditory content and the new potential acoustic parameters that can be used in future applications. Our results show that the approach is feasible and that a higher prediction accuracy is obtained for German than for Spanish participants.</p>
<p>The rest of the paper is organized as follows: <xref ref-type="sec" rid="s2">Section 2</xref> covers the related work while <xref ref-type="sec" rid="s3">Section 3</xref> explains the rationale behind the game design. In <xref ref-type="sec" rid="s4">Section 4</xref> we cover the methodology and in <xref ref-type="sec" rid="s5">Sections 5, 6</xref> the predictive models and their results. We discuss the results in <xref ref-type="sec" rid="s7">Section 7</xref>, finishing with conclusions and future work in <xref ref-type="sec" rid="s8">Section&#x20;8</xref>.</p>
</sec>
<sec id="s2">
<title>2 Related Work</title>
<p>Over the last decades, dyslexia has been studied from different fields, but no scientific agreement of the causal origin has been achieved (<xref ref-type="bibr" rid="B5">Borleffs et&#x20;al., 2019</xref>). There are two main theories at this point (<xref ref-type="bibr" rid="B9">De Zubicaray and Schiller, 2018</xref>). One considers visual perception (<xref ref-type="bibr" rid="B66">Vidyasagar and Pammer, 2010</xref>) to be a key attribute for the cause of dyslexia depending on the information processing and memory, while the other considers it to be auditory perception (<xref ref-type="bibr" rid="B18">Goswami, 2011</xref>).</p>
<p>Various applications and games to support, detect and treat dyslexia have been developed (<xref ref-type="bibr" rid="B43">Rauschenberger et&#x20;al., 2019b</xref>). <italic>Gamification</italic> has been used to design various use cases, applications as well as frameworks (<xref ref-type="bibr" rid="B22">Hamari et&#x20;al., 2014</xref>; <xref ref-type="bibr" rid="B55">Ritzhaupt et&#x20;al., 2014</xref>; <xref ref-type="bibr" rid="B29">Mora et&#x20;al., 2015</xref>; <xref ref-type="bibr" rid="B62">Seaborn and Fels, 2015</xref>; <xref ref-type="bibr" rid="B72">Thomas et al., 2021</xref>). Gamification designs the <italic>game play</italic> of games with game elements to engage and motivate users (<xref ref-type="bibr" rid="B57">Rouse, 2004</xref>; <xref ref-type="bibr" rid="B50">Rauschenberger et&#x20;al., 2019c</xref>). Games are developed to screen readers (<xref ref-type="bibr" rid="B53">Rello et&#x20;al., 2020</xref>, <xref ref-type="bibr" rid="B54">2018</xref>) using linguistic content and to screen pre-readers (<xref ref-type="bibr" rid="B16">Gaggi et&#x20;al., 2017</xref>; <xref ref-type="bibr" rid="B17">Geurts et&#x20;al., 2015</xref>; <xref ref-type="bibr" rid="B44">Rauschenberger et&#x20;al., 2019a</xref>, <xref ref-type="bibr" rid="B45">2018a</xref>) focusing on the gameful experience. Apart from our own work (<xref ref-type="bibr" rid="B42">Rauschenberger et&#x20;al., 2020</xref>) only <italic>Lexa</italic> (<xref ref-type="bibr" rid="B34">Poole et&#x20;al., 2018</xref>) published an accuracy (89.2%) using features related to phonological processing. However, they did not include game elements, and features are collected with costly and long tests. In addition, the classification is carried out on a small sample (<italic>n</italic>&#x20;&#x3d; 56), without any validation and no discussion about over-fitting.</p>
<p>To the best of our knowledge, others have not published the details of the design decisions, or iterations of content design. Here we advance previous approaches by taking precautions on over-fitting, by not focusing on linguistic knowledge, and by using the same game content for every language, publishing our raw data (<xref ref-type="bibr" rid="B38">Rauschenberger et&#x20;al., 2021a</xref>; <xref ref-type="bibr" rid="B39">Rauschenberger et&#x20;al., 2021b</xref>; <xref ref-type="bibr" rid="B40">Rauschenberger et&#x20;al., 2021c</xref>; <xref ref-type="bibr" rid="B41">Rauschenberger et&#x20;al., 2021d</xref>). This will reduce the effort and time to design different content for different languages but more importantly, the content could be used and tested for pre-readers in different applications from different research&#x20;labs.</p>
</sec>
<sec id="s3">
<title>3 Game Design</title>
<p>The aim of our web-game called <italic>MusVis</italic> (<xref ref-type="fig" rid="F1">Figure&#x20;1</xref>) is to measure the reaction of children with and without dyslexia while playing, in order to find differences on their behavior. A video of MusVis is available at <ext-link ext-link-type="uri" xlink:href="http://bit.ly/MusVisContent">http://bit.ly/MusVisContent</ext-link>. We designed our game with the assumption that non-linguistic content like rhythm or frequency (<xref ref-type="bibr" rid="B34">Poole et&#x20;al., 2018</xref>) can represent the difficulties that a child with dyslexia has with writing and reading (<xref ref-type="bibr" rid="B71">Yuskaitis et&#x20;al., 2015</xref>; <xref ref-type="bibr" rid="B19">Goswami et&#x20;al., 2016</xref>), and dyslexia can be measured through the interaction of a person (<xref ref-type="bibr" rid="B53">Rello et&#x20;al., 2020</xref>, <xref ref-type="bibr" rid="B54">2018</xref>) like total number of clicks or play duration. We measure the reactions of children with and without dyslexia while playing in order to find differences in the groups&#x2019; behavior. The auditory (<xref ref-type="fig" rid="F2">Figure&#x20;2</xref>) and visual (<xref ref-type="fig" rid="F3">Figure&#x20;3</xref>) content refers mainly to one single acoustic or visual indicator, e.g., frequency or horizontal similarity. Participants need to find the visual or auditory cue that has been shown to them before.</p>
<fig id="F1" position="float">
<label>FIGURE 1</label>
<caption>
<p>Participants playing the visual part <bold>(left)</bold> and the musical part <bold>(right)</bold> of MusVis. Photos included with the adults&#x2019; permission.</p>
</caption>
<graphic xlink:href="fcomp-03-628634-g001.tif"/>
</fig>
<fig id="F2" position="float">
<label>FIGURE 2</label>
<caption>
<p>Example of the auditory part from the game <italic>MusVis</italic> for the first two clicks on two sound cards <bold>(left)</bold> and then when a pair of equal sounds is found <bold>(right)</bold>. The participant is asked to find two equal auditory cues by clicking on sound&#x20;cards.</p>
</caption>
<graphic xlink:href="fcomp-03-628634-g002.tif"/>
</fig>
<fig id="F3" position="float">
<label>FIGURE 3</label>
<caption>
<p>Example of the visual part of the game <italic>MusVis</italic> with the priming of the target cue <italic>symbol</italic> <bold>(left)</bold> and the nine-squared design including the distractors for each <italic>symbol</italic> <bold>(right)</bold>.</p>
</caption>
<graphic xlink:href="fcomp-03-628634-g003.tif"/>
</fig>
<p>The game is implemented as a web application using JavaScript, jQuery, CSS, and HTML5 for the front-end, and a PHP server plus a MySQL database for the back-end. One reason for this is simplicity for remote online studies. Another reason is the advantage of adapting the application for different devices in future research studies.</p>
<p>We designed the language-independent game content taking into account the knowledge of previous literature selecting the most challenging content for people with dyslexia that was also easy to design in a web-game, namely auditory and visual&#x20;cues.</p>
<p>The auditory part is shown in <xref ref-type="fig" rid="F2">Figure&#x20;2</xref> while the visual part is shown in <xref ref-type="fig" rid="F3">Figure&#x20;3</xref>. The game play is different due to the unequal perception of auditory and visual cues but both parts targets general skills, e.g., short-term memory (<xref ref-type="bibr" rid="B26">Johnson, 1980</xref>; <xref ref-type="bibr" rid="B31">Overy, 2000</xref>; <xref ref-type="bibr" rid="B19">Goswami et&#x20;al., 2016</xref>), the phonological similarity effect (<xref ref-type="bibr" rid="B19">Goswami et&#x20;al., 2016</xref>), or the correlation of acoustic parameters in speech (<xref ref-type="bibr" rid="B71">Yuskaitis et&#x20;al., 2015</xref>; <xref ref-type="bibr" rid="B19">Goswami et&#x20;al., 2016</xref>).</p>
<p>As is well-known, children have more difficulty paying attention over a longer period of time. Therefore, the two parts have four stages which are counter-balanced with <italic>Latin Squares</italic> (<xref ref-type="bibr" rid="B15">Field and Hole, 2003</xref>). Each stage has two rounds, which sums up to 16 rounds in total for the whole game. Each stage first has a round with four cards and then with six cards, needing less than 10&#xa0;min to play. We aim to address participants&#x2019; motivation for both game parts with the design of the following game mechanics frequently used in learning environments (<xref ref-type="bibr" rid="B50">Rauschenberger et&#x20;al., 2019c</xref>): rewards (points), feedback (instant feedback) or challenges (time limit), plus the game components (story for the game design).</p>
<p>The content design, user interface, game play, interaction and implementation for the auditory and visual parts of the game are described in the following sections. First, we describe the selection of content and follow with the description of the game <italic>MusVis</italic>, which already integrates the changes suggested after an usability test (<xref ref-type="bibr" rid="B48">Rauschenberger et&#x20;al., 2017b</xref>).</p>
<sec id="s3-1">
<title>3.1 Selection of Content</title>
<p>The selection of the content for the game is crucial, because the content links the key features extracted from previous literature connected to dyslexia into a game format. For this we need to design the game with the proper indicators (content) and game constraints in order to collect solid dependent measures that reveal differences between the participant groups. Our language-independent content to measure differences between children with and without dyslexia that represent reading and writing difficulties people with dyslexia have is shown in <xref ref-type="table" rid="T1">Table&#x20;1</xref>. Furthermore, this new content needs to be integrated into a game context, designed to be used as online experiment and pre-tested to avoid unintentional influences.</p>
<table-wrap id="T1" position="float">
<label>TABLE 1</label>
<caption>
<p>Description of the auditory attributes which show promising relations to the prediction of dyslexia.</p>
</caption>
<table>
<thead valign="top">
<tr>
<th align="center">Key</th>
<th align="center">Name</th>
<th align="center">Description</th>
</tr>
</thead>
<tbody valign="top">
<tr>
<td align="left">
<bold>CS</bold>
</td>
<td align="left">Complex <italic>vs.</italic> simple</td>
<td align="left">Children with dyslexia (DG) recall significantly fewer items correctly in a lab study for long memory spans <xref ref-type="bibr" rid="B19">Goswami et&#x20;al. (2016)</xref>. The rhythmic complexity did not have an effect on the difference between DG and children without dyslexia (CG) <xref ref-type="bibr" rid="B24">Huss et&#x20;al. (2011)</xref>.</td>
</tr>
<tr>
<td align="left">
<bold>Pi</bold>
</td>
<td align="left">Pitch</td>
<td align="left">Pitch perception is essential for prosodic performance <xref ref-type="bibr" rid="B24">Huss et&#x20;al. (2011)</xref>, is correlated to language development, and can be used as a predictor for language <xref ref-type="bibr" rid="B71">Yuskaitis et&#x20;al. (2015)</xref>.</td>
</tr>
<tr>
<td align="left">
<bold>SD</bold>
</td>
<td align="left">Sound duration</td>
<td align="left">Acoustic parameter differences in short tones (&#x3c;350&#xa0;ms) are difficult to distinguish for a person with language difficulties <xref ref-type="bibr" rid="B31">Overy, (2000)</xref>.</td>
</tr>
<tr>
<td align="left">
<bold>RT</bold>
</td>
<td align="left">Rise time</td>
<td align="left">Both groups showed significant differences when comparing <italic>rise time</italic> <xref ref-type="bibr" rid="B19">Goswami et&#x20;al. (2016)</xref>. Rise time and prosodic development are strongly connected and were shown to be most sensitive to dyslexia <xref ref-type="bibr" rid="B24">Huss et&#x20;al. (2011)</xref>.</td>
</tr>
<tr>
<td align="left">
<bold>Rh</bold>
</td>
<td align="left">Rhythm</td>
<td align="left">DG show deficits in recalling the patterns of auditory cues <xref ref-type="bibr" rid="B31">Overy, (2000)</xref>. However, rhythm modulations show no effect on the children performance <xref ref-type="bibr" rid="B24">Huss et&#x20;al. (2011)</xref>.</td>
</tr>
<tr>
<td align="left">
<bold>STM</bold>
</td>
<td align="left">Short-term memory</td>
<td align="left">DG show weaknesses in short-term memory tasks <xref ref-type="bibr" rid="B31">Overy, (2000)</xref> when more items are presented <xref ref-type="bibr" rid="B19">Goswami et&#x20;al. (2016)</xref>. Also, deficits can be frequently observed for the short-term auditory memory span <xref ref-type="bibr" rid="B26">Johnson. (1980)</xref>.</td>
</tr>
<tr>
<td align="left">
<bold>PSE</bold>
</td>
<td align="left">Phonological similarity effect</td>
<td align="left">DG have difficulties with similar sounds and the <italic>phonological neighborhood</italic> when long memory spans are used <xref ref-type="bibr" rid="B19">Goswami et&#x20;al. (2016)</xref>.</td>
</tr>
<tr>
<td align="left">
<bold>CAPS</bold>
</td>
<td align="left">Correlated acoustic parameters speech</td>
<td align="left">Since the <italic>phonological grammar</italic> of music is similar to the prosodic structure of language, music (i.e.,&#x20;a combination of acoustical parameters) can be used to imitate these features <xref ref-type="bibr" rid="B71">Yuskaitis et&#x20;al. (2015)</xref>. DG are &#x201c;<italic>reliably impaired in prosodic tasks</italic>&#x201d; <xref ref-type="bibr" rid="B19">Goswami et&#x20;al. (2016)</xref>.</td>
</tr>
</tbody>
</table>
</table-wrap>
<p>Previously studied language-independent indicators have been used in lab settings, which means these indicators have been tested in controlled environments. That is not the case for online experiments. Consequently, external factors must be controlled and influences made transparent for the analysis. For example, we asked all participants to use the <italic>Google Chrome</italic> browser since browsers behave differently.</p>
<p>We decided to use an approach similar to the <italic>Memory</italic>
<xref ref-type="fn" rid="fn1">
<sup>1</sup>
</xref> game for auditory content because of the easy and well-known gameplay for young children.</p>
<p>We describe the selection of content for auditory, to give an example how to inventory and select possible content (<xref ref-type="bibr" rid="B38">Rauschenberger et&#x20;al., 2021a</xref>). The design iterations and files are available at <italic>GitHub</italic> (<xref ref-type="bibr" rid="B47">Rauschenberger et&#x20;al., 2017a</xref>) and <italic>Researchgate</italic> (<xref ref-type="bibr" rid="B39">Rauschenberger et&#x20;al., 2021b</xref>)<xref ref-type="fn" rid="fn2">
<sup>2</sup>
</xref>. Our goal was to reduce ideas, gameplay and acoustic parameters due to the following main requirements.<list list-type="simple">
<list-item>
<p>&#x2022; The acoustic parameters integrated indicators strongly and significant connected to dyslexia.</p>
</list-item>
<list-item>
<p>&#x2022; The acoustic parameters need to be easily deployed in a web-game.</p>
</list-item>
<list-item>
<p>&#x2022; The acoustic parameters need to be easily deployed in the gameplay.</p>
</list-item>
<list-item>
<p>&#x2022; The game duration fits pre-readers attention&#x20;span.</p>
</list-item>
</list>
</p>
<p>We present the different iterations starting with the first iteration where we collected ideas (possible acoustic parameters connected to dyslexia) from literature (<xref ref-type="bibr" rid="B41">Rauschenberger et&#x20;al., 2021d</xref>). We selected with a semi structured literature review the literature relevant to find indicators related to auditory difficulties (<xref ref-type="bibr" rid="B38">Rauschenberger et&#x20;al., 2021a</xref>). We found the first core paper like (<xref ref-type="bibr" rid="B31">Overy, 2000</xref>) and looked into similar wording and publication for the second iteration. Next, we explored acoustic parameters with a strong connection to dyslexia in other lab studies and redefine the acoustic parameter collection (see iteration two in the <xref ref-type="sec" rid="s14">Supplementary Material</xref>). For example, we added new acoustic parameters ideas such as presenting one frequency at different times on one side of the ear and then ask &#x201c;<italic>Where did you hear the frequency first</italic>&#x201d;. Also, we included the first ideas of a gameplay and how to make game rounds. How we came up with the game round was a very unsupervised creative approach. In the third and last iteration included the main requirements mentioned&#x20;above.</p>
<p>The main parameters to describe our auditory features picked are detailed in <xref ref-type="table" rid="T2">Table&#x20;2</xref> and elaborate here on important decisions. Iteration three has promising indicators not tested in a game environment, yet (<xref ref-type="bibr" rid="B41">Rauschenberger et&#x20;al., 2021d</xref>). Example indicators are the lab sound (<xref ref-type="bibr" rid="B24">Huss et&#x20;al., 2011</xref>), volume level, or timing. Also, we describe promising game rounds such as &#x201c;Find the same sound&#x201d; behind a card but the card is making a sound from a certain direction and needs to be found. Another game round could be &#x201c;Which sound came first?&#x201d; that schedules the same sound on different timing for the left and right&#x20;ear.</p>
<table-wrap id="T2" position="float">
<label>TABLE 2</label>
<caption>
<p>Description of auditory parameters.</p>
</caption>
<table>
<thead valign="top">
<tr>
<th align="left">Participant features</th>
<th align="left">Description</th>
</tr>
</thead>
<tbody valign="top">
<tr>
<td align="left">1 Implementation Priority</td>
<td align="left">Our priority to implement this option.</td>
</tr>
<tr>
<td align="left">2&#x20;Main-Round ID</td>
<td align="left">ID for rounds with the same acoustic parameter, <italic>e.g.,</italic> frequency, rhythm.</td>
</tr>
<tr>
<td align="left">3&#x20;Sub-Round ID</td>
<td align="left">Additional ID to distinguish different main-rounds with different settings, <italic>e.g.,</italic> different <italic>difficulty level</italic> like <italic>easy</italic> vs. <italic>difficult</italic>.</td>
</tr>
<tr>
<td align="left">4 Difficulty Level</td>
<td align="left">It indicates depending mainly on the amount of cards how difficult this content is.</td>
</tr>
<tr>
<td align="left">5 Instructions</td>
<td align="left">Short description of what the participants should do in English and German</td>
</tr>
<tr>
<td align="left">6 Input Description</td>
<td align="left">Short description of the parameters of the auditory elements for this round.</td>
</tr>
<tr>
<td align="left">7 Auditory Parameters Part</td>
<td align="left">It indicates which acoustic parameters are considered.</td>
</tr>
<tr>
<td align="left">8 Feedback Loop Example</td>
<td align="left">It shows example feedbacks from the domain experts and researchs for the round.</td>
</tr>
<tr>
<td align="left">9 Interaction</td>
<td align="left">It is a description of how the child should interact with the game and content.</td>
</tr>
<tr>
<td align="left">10 Reason</td>
<td align="left">It is a short description why the researcher thinks this round will work for the goal.</td>
</tr>
<tr>
<td align="left">11 Citation Key</td>
<td align="left">It is an example of the citation key to point to the literature we use as baseline. The final connections are presented in <xref ref-type="sec" rid="s3-2">Section 3.2</xref>.</td>
</tr>
</tbody>
</table>
</table-wrap>
<p>The consideration of <italic>difficulty-level</italic> is important as we want to address pre-readers that can be easily overwhelmed by information. Hence, <italic>MusVis</italic> implements only single acoustic indicators and only up to three different choices (one choice equals 2 cards). We decided against a game round with only one choice (two cards) as this would increase finding the correct answer by chance (50% to be correct) and a change of gameplay.</p>
<p>We included a short description (<italic>instructions</italic>) in different languages already in the phase of <italic>collection of content</italic> to ensure the feasibility and consistency between user studies in different languages. We recommend to make first one draft of the game and description in one language and then the translation into other languages. Reason is the iterative creative process until a first draft is reached that would make also a lot of changes to the text. But due to the differences of languages designer should not wait too long as German has longer names compared to English and therefore titles and descriptions need more space in the game design. The space needed for different language needs to be taken into account for pre-testing.</p>
<p>The short description of the input and short description of acoustic parameters is to ensure a simple overview to other rounds when deciding between different sounds. It might be usefully to separate the information from these two columns in the future to reduce redundant information.</p>
<p>The feedback loop is used between researchers and an expert in the creation of MP3 files to verify the artifact with the goal we have: Finding differences in the interaction behavior of groups with and without dyslexia when playing our game with this content. We shortly explain here the dyslexia auditory theory that was also mentioned in <italic>Section</italic> 2. Researchers argue that dyslexia might be mainly based on phonological and perception differences (<xref ref-type="bibr" rid="B18">Goswami, 2011</xref>). Moreover, previous research has related speech perception difficulty to auditory processing, phonological awareness, and literacy skills (<xref ref-type="bibr" rid="B64">Tallal, 2004</xref>; <xref ref-type="bibr" rid="B56">Rolka and Silverman, 2015</xref>; <xref ref-type="bibr" rid="B9">De Zubicaray and Schiller, 2018</xref>). Phonological deficits of dyslexia have also been linked to basic auditory processing (<xref ref-type="bibr" rid="B21">H&#xe4;m&#xe4;l&#xe4;inen et&#x20;al., 2013</xref>). However, there are musicians with dyslexia who scored better on auditory perception tests than the general population (<xref ref-type="bibr" rid="B28">M&#xe4;nnel et&#x20;al., 2017</xref>). At the same time, these participants score worse on tests of auditory working memory, i.e.,&#x20;the ability to keep a sound in mind for seconds. This observation is in line with the results on perceptions for short duration sounds (<xref ref-type="bibr" rid="B24">Huss et&#x20;al., 2011</xref>) and the findings on the <italic>prosodic similarity effects</italic> of participants with dyslexia (<xref ref-type="bibr" rid="B19">Goswami et&#x20;al., 2016</xref>). Still, it is challenge to design auditory cues connection to the auditory perception that can be used in our gameplay and not measuring the musical knowledge or hearing range. Hence, we included knowledge from an domain expert and an example feedback is presented in <italic>feedback loop</italic>. An example of screenshots with the parameters for different stages is available on <italic>Researchgate</italic> (<xref ref-type="bibr" rid="B40">Rauschenberger et&#x20;al., 2021c</xref>).</p>
<p>We hope other research working on language-independent screening find our collection of auditory indicator, insight about the design decision and how we inventoried our content useful for their own research and present next our selection for the game <italic>MusVis</italic>.</p>
</sec>
<sec id="s3-2">
<title>3.2 Auditory Game Design</title>
<p>The auditory part is inspired in the traditional game <italic>Memory</italic> in which pairs of identical cards (face down) must be identified by flipping them over (<xref ref-type="bibr" rid="B68">Wikipedia, 2019</xref>). We chose this game play because it is a well-known children game and could be easily transformed to use auditory cues. To create the auditory cues, we used acoustic parameters; for example, to imitate the <italic>prosodic</italic> structure of language which is similar to the <italic>phonological grammar</italic> of music (<xref ref-type="bibr" rid="B35">Port, 2003</xref>).</p>
<p>Musicians with dyslexia score better on auditory perception tests than the general population, but not on auditory working memory tests (M&#xe4;nnel et&#x20;al., 2016). Auditory working memory helps a person to keep a sound in mind. We combined, for example, the deficits of children with dyslexia in auditory working memory with the results on the short duration of sounds (<xref ref-type="bibr" rid="B24">Huss et&#x20;al., 2011</xref>) while taking the precaution of not measuring hearing ability (<xref ref-type="bibr" rid="B13">Fastl and Zwicker, 2007</xref>). Each stage is assigned to one acoustic parameter like frequency or rhythm which is designed with the knowledge of the analysis from previous literature (<xref ref-type="bibr" rid="B46">Rauschenberger et&#x20;al., 2018b</xref>, <xref ref-type="bibr" rid="B38">2021a</xref>,<xref ref-type="bibr" rid="B41">d</xref>).</p>
<p>Therefore, we used the acoustic parameters <italic>frequency, length, rise time and rhythm</italic> as auditory cues. Each auditory cue was assigned to a game stage (<xref ref-type="table" rid="T3">Table&#x20;3</xref>), which we mapped to the attributes and literature references (<xref ref-type="table" rid="T1">Table&#x20;1</xref>) that provide evidence for distinguishing a person with dyslexia.</p>
<table-wrap id="T3" position="float">
<label>TABLE 3</label>
<caption>
<p>Mapping of the evidence from literature to distinguish a person with dyslexia, the attributes and general assumptions, and the stages of the auditory part of the game <italic>MusVis</italic>.</p>
</caption>
<table>
<thead valign="top">
<tr>
<th rowspan="2" align="left">Attributes</th>
<th colspan="5" align="center">Auditory</th>
<th colspan="3" align="center">General</th>
</tr>
<tr>
<th align="left">CS</th>
<th align="left">Pi</th>
<th align="left">SD</th>
<th align="left">RT</th>
<th align="left">Rh</th>
<th align="left">STM</th>
<th align="left">PSE</th>
<th align="left">CAPS</th>
</tr>
</thead>
<tbody valign="top">
<tr>
<td colspan="9" align="left">Literature</td>
</tr>
<tr>
<td align="left">&#xa0;<xref ref-type="bibr" rid="B19">Goswami et&#x20;al. (2016)</xref>
</td>
<td align="center">&#x2713;</td>
<td align="left"/>
<td align="left"/>
<td align="center">&#x2713;</td>
<td align="left"/>
<td align="center">&#x2713;</td>
<td align="center">&#x2713;</td>
<td align="center">&#x2713;</td>
</tr>
<tr>
<td align="left">&#xa0;<xref ref-type="bibr" rid="B24">Huss et&#x20;al. (2011)</xref>
</td>
<td align="center">&#x2713;</td>
<td align="center">&#x2713;</td>
<td align="left"/>
<td align="center">&#x2713;</td>
<td align="center">&#x2713;</td>
<td align="left"/>
<td align="left"/>
<td align="left"/>
</tr>
<tr>
<td align="left">&#xa0;<xref ref-type="bibr" rid="B26">Johnson. (1980)</xref>
</td>
<td align="left"/>
<td align="left"/>
<td align="left"/>
<td align="left"/>
<td align="left"/>
<td align="center">&#x2713;</td>
<td align="left"/>
<td align="left"/>
</tr>
<tr>
<td align="left">&#xa0;<xref ref-type="bibr" rid="B31">Overy, (2000)</xref>
</td>
<td align="left"/>
<td align="left"/>
<td align="center">&#x2713;</td>
<td align="left"/>
<td align="center">&#x2713;</td>
<td align="center">&#x2713;</td>
<td align="left"/>
<td align="left"/>
</tr>
<tr>
<td align="left">&#xa0;<xref ref-type="bibr" rid="B71">Yuskaitis et&#x20;al. (2015)</xref>
</td>
<td align="left"/>
<td align="center">&#x2713;</td>
<td align="left"/>
<td align="left"/>
<td align="left"/>
<td align="left"/>
<td align="left"/>
<td align="center">&#x2713;</td>
</tr>
<tr>
<td colspan="9" align="left">Stage</td>
</tr>
<tr>
<td align="left">&#xa0;Frequency</td>
<td align="center">&#x2713;</td>
<td align="center">&#x2713;</td>
<td align="center">&#x2713;</td>
<td align="left"/>
<td align="left"/>
<td align="center">&#x2713;</td>
<td align="center">&#x2713;</td>
<td align="center">&#x2713;</td>
</tr>
<tr>
<td align="left">&#xa0;Length</td>
<td align="left"/>
<td align="left"/>
<td align="center">&#x2713;</td>
<td align="left"/>
<td align="left"/>
<td align="center">&#x2713;</td>
<td align="center">&#x2713;</td>
<td align="center">&#x2713;</td>
</tr>
<tr>
<td align="left">&#xa0;Rise time</td>
<td align="center">&#x2713;</td>
<td align="left"/>
<td align="center">&#x2713;</td>
<td align="center">&#x2713;</td>
<td align="left"/>
<td align="center">&#x2713;</td>
<td align="center">&#x2713;</td>
<td align="center">&#x2713;</td>
</tr>
<tr>
<td align="left">&#xa0;Rhythm</td>
<td align="center">&#x2713;</td>
<td align="left"/>
<td align="center">&#x2713;</td>
<td align="left"/>
<td align="center">&#x2713;</td>
<td align="center">&#x2713;</td>
<td align="center">&#x2713;</td>
<td align="center">&#x2713;</td>
</tr>
</tbody>
</table>
</table-wrap>
<p>For example, our <italic>rhythm</italic> stage uses the following characteristics: <italic>complex vs. simple</italic> (<xref ref-type="bibr" rid="B24">
<italic>Huss et&#x20;al., 2011</italic>
</xref>; <xref ref-type="bibr" rid="B19">
<italic>Goswami et&#x20;al., 2016</italic>
</xref>)<italic>, sound duration, rhythm (</italic>
<xref ref-type="bibr" rid="B24">
<italic>Huss et&#x20;al., 2011</italic>
</xref>
<italic>), short-term memory</italic> (<xref ref-type="bibr" rid="B26">
<italic>Johnson, 1980</italic>
</xref>; <xref ref-type="bibr" rid="B19">
<italic>Goswami et&#x20;al., 2016</italic>
</xref>)<italic>, phonological similarity effect (</italic>
<xref ref-type="bibr" rid="B19">
<italic>Goswami et&#x20;al., 2016</italic>
</xref>
<italic>), and correlated acoustic parameters speech</italic> (<xref ref-type="bibr" rid="B71">
<italic>Yuskaitis et&#x20;al., 2015</italic>
</xref>; <xref ref-type="bibr" rid="B19">
<italic>Goswami et&#x20;al., 2016</italic>
</xref>).</p>
<p>Each acoustic stage has three auditory cues (we use MP3 for sound files). Each stage is assigned to one acoustic parameter of sound, which is designed with knowledge of the analysis from previous literature (e.g., frequency or rhythm).</p>
<p>The auditory cues are generated with a simple sinus tone using the free software <italic>Audacity</italic>
<xref ref-type="fn" rid="fn3">
<sup>3</sup>
</xref>. The exact parameters of each auditory cue are already published (<xref ref-type="bibr" rid="B46">Rauschenberger et&#x20;al., 2018b</xref>) and the auditory cues are available at <italic>GitHub</italic> (<xref ref-type="bibr" rid="B47">Rauschenberger et&#x20;al., 2017a</xref>)<xref ref-type="fn" rid="fn4">
<sup>4</sup>
</xref>. Each stage has two rounds, with first two and then three auditory cues that must be assigned by choosing the same sound (see <xref ref-type="fig" rid="F2">Figure&#x20;2</xref>). The arrangement of sounds (which auditory cue matches which card) is random for each&#x20;round.</p>
</sec>
<sec id="s3-3">
<title>3.3 Visual Game Design</title>
<p>The visual game play uses a Whac-A-Mole interaction similar to the first round of <italic>Dytective</italic> (<xref ref-type="bibr" rid="B53">Rello et&#x20;al., 2020</xref>). But instead of using letter recognition as does <italic>Dytective</italic>, we used language-independent visual cues. An example for letter recognition would be finding the graphical representation of the letter /e/. We adapted the interaction design and content for this purpose (<xref ref-type="fig" rid="F3">Figure&#x20;3</xref>). For the visual game, we designed cues that have the potential of making more cues with similar features and represent horizontal and vertical symmetries that are known to be difficult for a person with dyslexia in different languages (<xref ref-type="bibr" rid="B66">Vidyasagar and Pammer, 2010</xref>; <xref ref-type="bibr" rid="B52">Rello et&#x20;al., 2016a</xref>; <xref ref-type="bibr" rid="B49">Rauschenberger et&#x20;al., 2016</xref>).</p>
<p>To create the visual cues, we designed different visual representations similar to visual features of annotated error words from people with dyslexia (<xref ref-type="bibr" rid="B66">Vidyasagar and Pammer, 2010</xref>; <xref ref-type="bibr" rid="B52">Rello et&#x20;al., 2016a</xref>; <xref ref-type="bibr" rid="B49">Rauschenberger et&#x20;al., 2016</xref>) and designed the game as a simple search task, which does not require language acquisition.</p>
<p>In the beginning, participants are shown the target visual cues (see <xref ref-type="fig" rid="F3">Figure&#x20;3</xref>, left) for 3&#xa0;seconds. They are asked to remember this visual cue. After that, the participants are presented with a setting where the target visual cue and distractors are displayed (see <xref ref-type="fig" rid="F3">Figure&#x20;3</xref>, right). The participants try to click on the target visual cue as often as possible within a span of 15&#xa0;s. The arrangement of the target and distractor cues randomly changes after every&#x20;click.</p>
<p>The visual part has four stages, which are counter-balanced with <italic>Latin Squares</italic> (<xref ref-type="bibr" rid="B15">Field and Hole, 2003</xref>). Each stage is assigned to one visual type (<italic>symbol, z, rectangle, face</italic>) and four visual cues for each stage are presented. One visual cue is the target, which the participants need to find and click (see <xref ref-type="fig" rid="F4">Figure&#x20;4</xref>, top). The other three visual cues are <italic>distractors</italic> for the participants. Each stage has two rounds with first a 4-squared and then a 9-squared design (see <xref ref-type="fig" rid="F3">Figure&#x20;3</xref>, right). The target and all three distractors are displayed in the 4-squared design. In the 9-squared design, the target is displayed twice as well as distractors two and three. Only distractor one is displayed three&#x20;times.</p>
<fig id="F4" position="float">
<label>FIGURE 4</label>
<caption>
<p>Overview of the designed visual cues. The figure shows the target cue <bold>(top)</bold> and distractor cues <bold>(below)</bold> for the four different stages (<italic>z, symbol, rectangle, face</italic>) of the visual part of the game <italic>MusVis</italic>.</p>
</caption>
<graphic xlink:href="fcomp-03-628634-g004.tif"/>
</fig>
</sec>
</sec>
<sec id="s4">
<title>4 User Study Methodology</title>
<p>We use the human-centered design framework to design our study and to collect the data for the prediction of dyslexia. We conducted a within-subject design study (<italic>n</italic>&#x20;&#x3d; 313) which means that all participants played all game rounds (<xref ref-type="bibr" rid="B15">Field and Hole, 2003</xref>) with the same language-independent content. Only the game instructions were translated into each native language.</p>
<p>Spanish participants diagnosed with dyslexia were mainly recruited from public social media calls by non-profit organizations. We recruited German participants diagnosed with dyslexia mainly over support groups on social media. Also, some English speakers contacted us through this call as our location is international. The control groups for Spanish and German were recruited mostly with the collaboration of four schools, two in each country.</p>
<sec id="s4-1">
<title>4.1 Online Data Collection</title>
<p>Collecting data is costly in terms of time consumption and privacy issues, especially if the data is related to education and health. Therefore, we must make the best of the limited resource (<xref ref-type="bibr" rid="B2">Baeza-Yates, 2018</xref>; <xref ref-type="bibr" rid="B12">Faraway and Augustin, 2018</xref>; <xref ref-type="bibr" rid="B36">Rauschenberger and Baeza-Yates, 2020a</xref>; <xref ref-type="bibr" rid="B67">Weigand et&#x20;al., 2021</xref>). In our case, we need a certain age range to make sure a person with dyslexia is already diagnosed and has not been fully treated yet. Since our collected data is considered <italic>small data</italic> (<xref ref-type="bibr" rid="B2">Baeza-Yates, 2018</xref>; <xref ref-type="bibr" rid="B12">Faraway and Augustin, 2018</xref>), we need to analyze them accordingly, i.e.,&#x20;avoid over-fitting using cross-validation instead of training, test and validation sets as well as using classifiers configured to avoid over-fitting.</p>
</sec>
<sec id="s4-2">
<title>4.2 Procedure and Ethics Statement</title>
<p>First, the parents were informed about the purpose of the voluntary study. Next, only after the parents gave the consent, children were allowed to participate in this user study from home or from school, with the first author of this work present or always available through digital communication.</p>
<p>The data collection for this user study has been approved by the German Ministry of Education, Science and Culture in Schleswig-Holstein (<italic>Ministerium f&#xfc;r Bildung, Wissenschaft und Kultur</italic>) and Lower Saxony State Education Authority (<italic>Nieders&#xe4;chsische Landesschulbeh&#xf6;rde</italic>). In Spain governmental approval was not needed in addition to the school approval.</p>
<p>If the study was conducted in a school or learning center, the parents or the legal guardian consent was obtained in advance and the user study was supervised by a teacher or therapist. After the online consent form was approved, we collected demographic data which was completed by the participant&#x2019;s supervisor (e.g., parent/teacher), including the age of the participant, the dyslexia diagnosis (yes/no/maybe) and the native language. We ask the participant&#x2019;s supervisor to only say <italic>YES</italic> for a participant if the child had an official diagnosis, for example from an authorized specialist or a medical doctor.</p>
<p>After that participants played both parts of the game. At the end, two feedback questions are asked and the participant&#x2019;s supervisor could leave contact details to be informed about the results of the study. Personal information of the participant&#x2019;s supervisor such as name or email is not published and is stored separately from the participants data, if given. On the other hand, the name of the child is not collected and all data is stored on a password secured web server.</p>
</sec>
<sec id="s4-3">
<title>4.3 Participants</title>
<p>The data includes only participants that completed all 16 rounds of the web game using a computer or a tablet. Dropouts happened mostly because participants used a different browser (e.g., <italic>Internet Explorer</italic> instead of <italic>Google Chrome</italic>) or a different device (tablet instead of a computer).</p>
<p>For the predictive models, we took 313 participants into account, including the 178 participants from the pilot study (<xref ref-type="bibr" rid="B46">Rauschenberger et&#x20;al., 2018b</xref>). To have precise data, we took out participants that reported in the background questionnaire that they suspected of having dyslexia but did not have a diagnosis (<italic>n</italic>&#x20;&#x3d;&#x20;48).</p>
<p>The remaining participants were classified as diagnosed with dyslexia (DG) or not showing any signs of dyslexia (control group, CG), as reported in the background questionnaire.</p>
<p>We separated our data into three data sets: one for the Spanish participants (ES, <italic>n</italic>&#x20;&#x3d; 153), a second for the German participants (DE, <italic>n</italic>&#x20;&#x3d; 149), and one for all languages (ALL, <italic>n</italic>&#x20;&#x3d; 313) in which we included participants that spoke English (<italic>n</italic>&#x20;&#x3d; 11). Participants ranged in age from 7 to 12&#x20;years old. The users in the data sets are described in <xref ref-type="table" rid="T4">Table&#x20;4</xref>.</p>
<table-wrap id="T4" position="float">
<label>TABLE 4</label>
<caption>
<p>Overview of the participants per data&#x20;set.</p>
</caption>
<table>
<thead valign="top">
<tr>
<th rowspan="2" align="center">Data set</th>
<th rowspan="2" align="center">
<italic>N</italic>
</th>
<th colspan="4" align="center">Dyslexia (DG)</th>
</tr>
<tr>
<th align="left">
<italic>n</italic>
</th>
<th align="left">
<inline-formula id="inf1">
<mml:math id="m1">
<mml:mrow>
<mml:mover accent="true">
<mml:mrow>
<mml:mi>a</mml:mi>
<mml:mi>g</mml:mi>
<mml:mi>e</mml:mi>
</mml:mrow>
<mml:mo>&#x304;</mml:mo>
</mml:mover>
</mml:mrow>
</mml:math>
</inline-formula>
</th>
<th align="left">female</th>
<th align="left">male</th>
</tr>
</thead>
<tbody valign="top">
<tr>
<td align="left">DE</td>
<td align="center">149</td>
<td align="center">59</td>
<td align="char" char=".">10.22</td>
<td align="center">21</td>
<td align="center">38</td>
</tr>
<tr>
<td align="left">ES</td>
<td align="center">153</td>
<td align="center">49</td>
<td align="char" char=".">9.47</td>
<td align="center">26</td>
<td align="center">23</td>
</tr>
<tr>
<td align="left">ALL</td>
<td align="center">313</td>
<td align="center">116</td>
<td align="char" char=".">9.77</td>
<td align="center">50</td>
<td align="center">66</td>
</tr>
</tbody>
</table>
<table>
<thead>
<tr>
<td rowspan="2" align="center">
<bold>Data set</bold>
</td>
<td rowspan="2" align="center">
<italic>
<bold>N</bold>
</italic>
</td>
<td colspan="4" align="center">
<bold>Control (CG)</bold>
</td>
</tr>
<tr>
<td align="left">
<italic>
<bold>n</bold>
</italic>
</td>
<td align="left">
<inline-formula id="inf2">
<mml:math id="m2">
<mml:mrow>
<mml:mover accent="true">
<mml:mrow>
<mml:mi mathvariant="bold-italic">a</mml:mi>
<mml:mi mathvariant="bold-italic">g</mml:mi>
<mml:mi mathvariant="bold-italic">e</mml:mi>
</mml:mrow>
<mml:mo>&#x304;</mml:mo>
</mml:mover>
</mml:mrow>
</mml:math>
</inline-formula>
</td>
<td align="left">
<bold>female</bold>
</td>
<td align="left">
<bold>male</bold>
</td>
</tr>
</thead>
<tbody>
<tr>
<td align="left">DE</td>
<td align="center">149</td>
<td align="center">90</td>
<td align="char" char=".">9.58</td>
<td align="center">42</td>
<td align="center">48</td>
</tr>
<tr>
<td align="left">ES</td>
<td align="center">153</td>
<td align="center">104</td>
<td align="char" char=".">9.99</td>
<td align="center">58</td>
<td align="center">46</td>
</tr>
<tr>
<td align="left">ALL</td>
<td align="center">313</td>
<td align="center">197</td>
<td align="char" char=".">9.76</td>
<td align="center">103</td>
<td align="center">94</td>
</tr>
</tbody>
</table>
</table-wrap>
<p>Participants played the game either in English, German or Spanish depending on their native language. We had some bilingual participants (<italic>n</italic>&#x20;&#x3d; 48) in the Spanish data set (Spanish and Catalan) since the media call was done from the non-profit organization <italic>ChangeDyslexia</italic>
<xref ref-type="fn" rid="fn6">
<sup>6</sup>
</xref>. For these cases, we used the language they reported to be more comfortable with, which was used for the instructions of the game. We do not use the native language, but rather the language the game was played in as the criterion to split the data sets for three reasons. First, the definition of a native language or mother tongue can be made easily when a participant speaks only one language. But this is not the case for bilingual participants because they might not be able to choose, and then we cannot distinguish the mother tongue or native language clearly (<xref ref-type="bibr" rid="B27">Kecskes and Papp, 2000</xref>). Second, this question is a self-reported question and every participant&#x2019;s supervisor might define it differently for each child. Finally, some bilingual speakers spoke similar Latin languages (Spanish and Catalan). We consider these participants in the ES data set, as the instructions of the game were in Spanish.</p>
</sec>
<sec id="s4-4">
<title>4.4 Dependent Variables and Features</title>
<p>The participant features are detailed in <xref ref-type="table" rid="T5">Table&#x20;5</xref> while the dependent variables collected through the game are listed in <xref ref-type="table" rid="T6">Table&#x20;6</xref>. These variables were used for the statistical comparison of the pilot study and for the selection of the features for the predictive models. Feature three was set with the language selected for the instructions. Features 1, 2, 4 to 8 were answered with the online questions by the participants&#x2019; supervisor. Feature 9 was collected from the browser during the study experiment.</p>
<table-wrap id="T5" position="float">
<label>TABLE 5</label>
<caption>
<p>Description of participant features.</p>
</caption>
<table>
<thead valign="top">
<tr>
<th align="center">Participant features</th>
<th align="center">Description</th>
</tr>
</thead>
<tbody valign="top">
<tr>
<td align="left">1 Age</td>
<td align="left">It ranges from 7 to 12&#x20;years old.</td>
</tr>
<tr>
<td align="left">2 Gender</td>
<td align="left">It is a binary feature, either with a <italic>female</italic> or <italic>male</italic> value.</td>
</tr>
<tr>
<td align="left">3 Language</td>
<td align="left">It is either <italic>Spanish, German</italic> or <italic>English</italic>.</td>
</tr>
<tr>
<td align="left">4 Native Language</td>
<td align="left">It indicates if the language used for the instructions is the first language of the participants, being <italic>Yes</italic>, <italic>No</italic> or <italic>Maybe</italic>.</td>
</tr>
<tr>
<td align="left">5 Instrument</td>
<td align="left">It indicates if a participant plays a musical instrument, being <italic>No</italic>, <italic>Yes, less than 6&#x20;months</italic> or <italic>Yes, over 6&#x20;months.</italic>
</td>
</tr>
<tr>
<td align="left">6 Memory</td>
<td align="left">It indicates how well the participant knows the visual <italic>Memory</italic> game, being <italic>Participant gave no answer</italic>, <italic>Participant does not known the game</italic>, <italic>Played once</italic>, <italic>Played a few times</italic> or <italic>Played a lot</italic>.</td>
</tr>
<tr>
<td align="left">7 Rating Auditory Part</td>
<td align="left">It indicates the self-reported answer with a 6-level <italic>Likert scale</italic> <xref ref-type="bibr" rid="B15">Field and Hole, (2003)</xref> to the statement: &#x201c;the auditory part was easy for the participants.&#x201d; The values are <italic>Answer unknown</italic>, <italic>Strongly disagree</italic>, <italic>Disagree</italic>, <italic>Undecided</italic>, <italic>Agree</italic> or <italic>Strongly Agree</italic>.</td>
</tr>
<tr>
<td align="left">8 Rating Visual Part</td>
<td align="left">It indicates the self-reported answer of the statement: &#x201c;the visual part was easy for the participants.&#x201d; (same <italic>Likert scale</italic> from feature 7).</td>
</tr>
<tr>
<td align="left">9 Device</td>
<td align="left">It is the device the participants used and is a binary feature with the value <italic>Computer</italic> or <italic>Tablet</italic>.</td>
</tr>
</tbody>
</table>
</table-wrap>
<table-wrap id="T6" position="float">
<label>TABLE 6</label>
<caption>
<p>On the left are features 10 to 105 for the auditory part and on the right are features 106 to 201 for the visual part of the game MusVis.</p>
</caption>
<table>
<thead valign="top">
<tr>
<th align="left">Auditory features</th>
<th align="center">Visual features</th>
</tr>
</thead>
<tbody valign="top">
<tr>
<td align="left">
<bold>10&#x2013;17</bold> Time to click</td>
<td align="left">
<bold>106&#x2013;113</bold> Time to click</td>
</tr>
<tr>
<td align="left">
<bold>18&#x2013;25</bold> Total clicks</td>
<td align="left">
<bold>114&#x2013;121</bold> Total clicks</td>
</tr>
<tr>
<td align="left">
<bold>26&#x2013;33</bold> Duration per round</td>
<td align="left">
<bold>122&#x2013;129</bold> Correct answers</td>
</tr>
<tr>
<td align="left">
<bold>34&#x2013;41</bold> Duration interaction</td>
<td align="left">
<bold>130&#x2013;137</bold> Wrong answers</td>
</tr>
<tr>
<td align="left">
<bold>42&#x2013;49</bold> Average click time</td>
<td align="left">
<bold>138&#x2013;145</bold> Accuracy</td>
</tr>
<tr>
<td align="left">
<bold>50&#x2013;57</bold> Logic</td>
<td align="left">
<bold>146&#x2013;153</bold> Efficiency</td>
</tr>
<tr>
<td align="left">
<bold>58&#x2013;65</bold> 2nd click interval</td>
<td align="left">
<bold>154&#x2013;161</bold> 2nd click interval</td>
</tr>
<tr>
<td align="left">
<bold>66&#x2013;73</bold> 3rd click interval</td>
<td align="left">
<bold>162&#x2013;169</bold> 3rd click interval</td>
</tr>
<tr>
<td align="left">
<bold>74&#x2013;81</bold> 4th click interval</td>
<td align="left">
<bold>170&#x2013;177</bold> 4th click interval</td>
</tr>
<tr>
<td align="left">
<bold>82&#x2013;89</bold> 5th click interval</td>
<td align="left">
<bold>178&#x2013;185</bold> 5th click interval</td>
</tr>
<tr>
<td align="left">
<bold>90&#x2013;97</bold> 6th click interval</td>
<td align="left">
<bold>186&#x2013;193</bold> 6th click interval</td>
</tr>
<tr>
<td align="left">
<bold>98&#x2013;105</bold> Instructions</td>
<td align="left">
<bold>194&#x2013;201</bold> Time last click</td>
</tr>
</tbody>
</table>
</table-wrap>
<p>We used the following dependent variables for the statistical comparison:</p>
<sec id="s4-4-1">
<title>Auditory game part</title>
<p>
<list list-type="simple">
<list-item>
<p>&#x2022; <italic>Duration round</italic> (milliseconds) starts when round is initialized.</p>
</list-item>
<list-item>
<p>&#x2022; <italic>Duration interaction</italic> (milliseconds) starts after the player clicks the first time on a card in each&#x20;round.</p>
</list-item>
<list-item>
<p>&#x2022; <italic>Average click time</italic> (milliseconds) is the duration of a round divided by the total number of clicks.</p>
</list-item>
<list-item>
<p>&#x2022; <italic>Time interval</italic> (milliseconds) is the time needed for the second, third, fourth, fifth and sixth clicks.</p>
</list-item>
<list-item>
<p>&#x2022; <italic>Logic</italic> we define it as <italic>True</italic> when in a round the first three clicked cards are different, otherwise, it is <italic>False.</italic>
</p>
</list-item>
<list-item>
<p>&#x2022; <italic>Instructions</italic> is the number of times the game instructions were listened by the player.</p>
</list-item>
</list>
</p>
</sec>
<sec id="s4-4-2">
<title>Visual game&#x20;part</title>
<p>
<italic>Number of hits</italic> is the number of correct answers. <italic>Number of misses</italic> is the number of incorrect answers. <italic>Efficiency</italic> is the number of hits multiplied by the total number of clicks. <italic>Accuracy</italic> is the number of hits divided by the total number of clicks.</p>
</sec>
<sec id="s4-4-3">
<title>Both parts</title>
<p>
<list list-type="simple">
<list-item>
<p>&#x2022; <italic>Time to the first click</italic> (milliseconds) is the duration between the round start and the first user&#x20;click.</p>
</list-item>
<list-item>
<p>&#x2022; <italic>Total number of clicks</italic> is the number of clicks during a&#x20;round.</p>
</list-item>
</list>
</p>
<p>We would like to further elaborate on the game measurement <italic>Logic</italic>, which is based on the direct experience of the user study. Some children may not have <italic>really listened</italic> to the sounds and played <italic>logically</italic>. As each round is designed such that the first two clicks never match, if the participant chooses for the third click a different card, s/he is increasing the chances of finding a match independent of the total amount of&#x20;cards.</p>
<p>The descriptions of the participant features are in <xref ref-type="table" rid="T5">Table&#x20;5</xref>. The features for the data sets ALL, ES, and DE are the same. Each data set has 201 features per participant, where features 10 to 105 are the variables from the auditory part and features 106 to 201 are the variables from the visual&#x20;part.</p>
</sec>
</sec>
</sec>
<sec id="s5">
<title>5 Predictive Models</title>
<p>In this section we present the machine learning techniques used for the data sets ALL (<italic>n</italic>&#x20;&#x3d; 313), ES (<italic>n</italic>&#x20;&#x3d; 153), and DE (<italic>n</italic>&#x20;&#x3d; 149). First, we explain the choice of predictive models and then the feature selection.</p>
<sec id="s5-1">
<title>5.1 Model Selection</title>
<p>We used Random Forest (RF), Random Forest with class weights (RFW), Extra Trees (ETC), Gradient Boosting (GB), and the Dummy Classifier (Baseline), which are described in the Scikit-learn version 0.21.2 (<xref ref-type="bibr" rid="B61">Scikit-learn Developers, 2019</xref>). We address the risk of over-fitting on our small data sets with 10-fold cross-validation and the default parameters suggested in the Scikit-learn library to avoid training a model by optimizing the parameters specifically for our data (<xref ref-type="bibr" rid="B61">Scikit-learn Developers, 2019</xref>). As we have small data, we are not optimizing the input parameters of classifiers until we can hold out a test data set as proposed by scikit-learn 0.21.2 documentation to evaluate the changes (<xref ref-type="bibr" rid="B60">Scikit-learn, 2019</xref>) and to avoid biases (<xref ref-type="bibr" rid="B65">Varma and Simon, 2006</xref>). To explore the best prediction conditions, we used the feature selection as described in the next section.</p>
</sec>
<sec id="s5-2">
<title>5.2 Informative Features</title>
<p>We address the danger of selecting the correct features (<xref ref-type="bibr" rid="B25">Jain and Zongker, 1997</xref>) by taking into account the knowledge of previous literature about the differences of children with an without dyslexia. For example, since there are two theories of the cause of dyslexia [visual <italic>vs.</italic> auditory (<xref ref-type="bibr" rid="B9">De Zubicaray and Schiller, 2018</xref>)], we use subsets of visual and auditory features to explore the influence on the classifiers.</p>
<p>We rank the most informative features with <italic>Extra Trees</italic>. The results show a flat distribution for all three data sets and a step at the information score of 0.008: ALL (<italic>n</italic>&#x20;&#x3d; 33 features), ES (<italic>n</italic>&#x20;&#x3d; 41 features), and DE (<italic>n</italic>&#x20;&#x3d; 38 features). The comparison of the most informative features reveals that the data sets have only a few features in common, e.g., four features for Spanish and German (Logic, sixth click interval, total clicks, duration interaction) or only 16 features in ALL compared to Spanish and German. Visual and auditory features are equally represented in the ranking of the most informative features; for example, ALL has 16 auditory features and 14 visual features.</p>
<p>The biggest step in the informative ranking for all three data sets is between the fifth and sixth informative features, e.g., for ALL the step is between the visual part (cue <italic>Z</italic>, 4 cards) <italic>Efficiency</italic> with the informative score of 0.0128 and the auditory part (cue <italic>Rhythm</italic>, 6 cards), <italic>Time fifth click</italic> with a score of 0.0104. The only dependent variables with the same tendency are <italic>Number of misses</italic> and <italic>Total clicks</italic> from the visual game part, but the features from the different rounds for the different data sets are mainly not under the 33 informative features (ALL 2/16, ES 3/16 and DE 6/16).</p>
</sec>
</sec>
<sec id="s6">
<title>6 Results</title>
<p>We followed the same steps of the pilot study to compare the statistical findings before giving the machine learning results.</p>
<sec id="s6-1">
<title>6.1 Statistical Validation</title>
<p>The pilot study collected data from 178 participants (which were later included into our current data set, <italic>n</italic>&#x20;&#x3d; 313) to find significant differences on the game measurements (<xref ref-type="bibr" rid="B46">Rauschenberger et&#x20;al., 2018b</xref>). Therefore, we apply first the <italic>Shapiro-Wilk Test</italic> and then the <italic>Wilcoxon Test</italic> since all game measures are not normally distributed. We use the Bonferroni correction (<italic>p</italic>&#x20;&#x3c; 0.002) to avoid type I errors. We present the results of the statistical analysis for the validation data (<italic>n</italic>&#x20;&#x3d; 313) separated by language and for all languages (see <xref ref-type="table" rid="T7">Table&#x20;7</xref>). Additionally, we compare the statistical analysis results from the pilot-study (<italic>n</italic>&#x20;&#x3d; 178) with the new data set (<italic>n</italic>&#x20;&#x3d;&#x20;313).</p>
<table-wrap id="T7" position="float">
<label>TABLE 7</label>
<caption>
<p>Overview of dependent variables for visual (top) and auditory (below) features of <italic>MusVis</italic>.</p>
</caption>
<table>
<thead valign="top">
<tr>
<th rowspan="2" align="left">Part</th>
<th rowspan="2" align="center">Data set</th>
<th rowspan="2" align="center">Variable</th>
<th colspan="2" align="center">Control</th>
<th colspan="2" align="center">Dyslexia</th>
<th colspan="3" align="center">Mann-Whitney U</th>
</tr>
<tr>
<th align="center">Mean</th>
<th align="center">sd</th>
<th align="center">Mean</th>
<th align="center">d</th>
<th align="center">W</th>
<th align="center">
<italic>p</italic>-value</th>
<th align="center">Effect size</th>
</tr>
</thead>
<tbody valign="top">
<tr>
<td rowspan="16" align="left">Visual</td>
<td rowspan="2" align="left">ALL</td>
<td align="left">
<bold>Total clicks</bold>
</td>
<td align="char" char=".">6.8</td>
<td align="char" char=".">2.7</td>
<td align="char" char=".">7.2</td>
<td align="char" char=".">3.2</td>
<td align="left">670194</td>
<td align="center">
<bold>2e-04</bold>
</td>
<td align="char" char=".">0.14</td>
</tr>
<tr>
<td align="left">Misses</td>
<td align="char" char=".">1.2</td>
<td align="char" char=".">2</td>
<td align="char" char=".">1.3</td>
<td align="char" char=".">2.7</td>
<td align="left">713627</td>
<td align="center">0.14</td>
<td align="char" char=".">0.05</td>
</tr>
<tr>
<td rowspan="6" align="left">ES</td>
<td align="left">
<bold>Total clicks</bold>
</td>
<td align="char" char=".">6.8</td>
<td align="char" char=".">2.7</td>
<td align="char" char=".">7.7</td>
<td align="char" char=".">3</td>
<td align="left">132207</td>
<td align="center">
<bold>3e-08</bold>
</td>
<td align="char" char=".">
<bold>0.31</bold>
</td>
</tr>
<tr>
<td align="left">
<bold>First click</bold>
</td>
<td align="char" char=".">2.63s</td>
<td align="char" char=".">1.69s</td>
<td align="char" char=".">2.26s</td>
<td align="char" char=".">1.22s</td>
<td align="left">141938</td>
<td align="center">
<bold>1e-04</bold>
</td>
<td align="char" char=".">0.27</td>
</tr>
<tr>
<td align="left">
<bold>Hits</bold>
</td>
<td align="char" char=".">5.8</td>
<td align="char" char=".">3</td>
<td align="char" char=".">6.5</td>
<td align="char" char=".">2.9</td>
<td align="left">136904</td>
<td align="center">
<bold>2e-06</bold>
</td>
<td align="char" char=".">0.25</td>
</tr>
<tr>
<td align="left">Misses</td>
<td align="char" char=".">1</td>
<td align="char" char=".">1.7</td>
<td align="char" char=".">1.2</td>
<td align="char" char=".">2.7</td>
<td align="left">157086</td>
<td align="center">0.12</td>
<td align="char" char=".">0.07</td>
</tr>
<tr>
<td align="left">Accuracy</td>
<td align="char" char=".">0.82</td>
<td align="char" char=".">0.27</td>
<td align="char" char=".">0.85</td>
<td align="char" char=".">0.26</td>
<td align="left">153012</td>
<td align="center">0.03</td>
<td align="char" char=".">0.10</td>
</tr>
<tr>
<td align="left">
<bold>Efficiency</bold>
</td>
<td align="char" char=".">3.1s</td>
<td align="char" char=".">2.6s</td>
<td align="char" char=".">2.75</td>
<td align="char" char=".">2.4s</td>
<td align="left">142162</td>
<td align="center">
<bold>1e-04</bold>
</td>
<td align="char" char=".">0.14</td>
</tr>
<tr>
<td rowspan="6" align="left">DE</td>
<td align="left">Total clicks</td>
<td align="char" char=".">6.7</td>
<td align="char" char=".">2.6</td>
<td align="char" char=".">6.8</td>
<td align="char" char=".">3.3</td>
<td align="left">169439</td>
<td align="center">0.47</td>
<td align="char" char=".">0.03</td>
</tr>
<tr>
<td align="left">First click</td>
<td align="char" char=".">2.50s</td>
<td align="char" char=".">1.32s</td>
<td align="char" char=".">2.58s</td>
<td align="char" char=".">1.56s</td>
<td align="left">168932</td>
<td align="center">0.43</td>
<td align="char" char=".">0.06</td>
</tr>
<tr>
<td align="left">Hits</td>
<td align="char" char=".">5.4</td>
<td align="char" char=".">2.6</td>
<td align="char" char=".">5.3</td>
<td align="char" char=".">2.8</td>
<td align="left">164224</td>
<td align="center">0.16</td>
<td align="char" char=".">0.05</td>
</tr>
<tr>
<td align="left">Misses</td>
<td align="char" char=".">1.3</td>
<td align="char" char=".">2.1</td>
<td align="char" char=".">1.5</td>
<td align="char" char=".">2.8</td>
<td align="left">166140</td>
<td align="center">0.24</td>
<td align="char" char=".">0.09</td>
</tr>
<tr>
<td align="left">Accuracy</td>
<td align="left">0.81</td>
<td align="char" char=".">0.27</td>
<td align="char" char=".">0.78</td>
<td align="char" char=".">0.29</td>
<td align="left">165688</td>
<td align="center">0.22</td>
<td align="center">0.08</td>
</tr>
<tr>
<td align="left">Efficiency</td>
<td align="left">3.2s</td>
<td align="char" char=".">2.4s</td>
<td align="char" char=".">3.5s</td>
<td align="char" char=".">2.9s</td>
<td align="left">167288</td>
<td align="center">0.33</td>
<td align="center">0.10</td>
</tr>
<tr>
<td rowspan="11" align="left">Auditory</td>
<td rowspan="5" align="left">ES</td>
<td align="left">Total clicks</td>
<td align="char" char=".">11.3</td>
<td align="char" char=".">6</td>
<td align="char" char=".">10.9</td>
<td align="char" char=".">5.5</td>
<td align="left">157282</td>
<td align="center">0.15</td>
<td align="char" char=".">0.07</td>
</tr>
<tr>
<td align="left">
<bold>4th click</bold>
</td>
<td align="char" char=".">2.0s</td>
<td align="char" char=".">1.3s</td>
<td align="char" char=".">1.7s</td>
<td align="char" char=".">1.0s</td>
<td align="left">131228</td>
<td align="center">
<bold>1e-08</bold>
</td>
<td align="char" char=".">0.29</td>
</tr>
<tr>
<td align="left">6th click</td>
<td align="char" char=".">1.7s</td>
<td align="char" char=".">1.1s</td>
<td align="char" char=".">1.6s</td>
<td align="char" char=".">0.9s</td>
<td align="left">152772</td>
<td align="center">0.04</td>
<td align="char" char=".">0.15</td>
</tr>
<tr>
<td align="left">
<bold>Duration</bold>
</td>
<td align="char" char=".">32.6s</td>
<td align="char" char=".">69.9s</td>
<td align="char" char=".">24.7s</td>
<td align="char" char=".">18.2s</td>
<td align="left">142726</td>
<td align="center">
<bold>2e-04</bold>
</td>
<td align="char" char=".">0.19</td>
</tr>
<tr>
<td align="left">
<bold>Average</bold>
</td>
<td align="char" char=".">3.0s</td>
<td align="char" char=".">2.7s</td>
<td align="char" char=".">2.6s</td>
<td align="char" char=".">0.9s</td>
<td align="left">121966</td>
<td align="center">
<bold>5e-13</bold>
</td>
<td align="char" char=".">0.29</td>
</tr>
<tr>
<td rowspan="5" align="left">DE</td>
<td align="left">Total clicks</td>
<td align="char" char=".">11.1</td>
<td align="char" char=".">5.5</td>
<td align="char" char=".">11.5</td>
<td align="char" char=".">6.6</td>
<td align="left">166340</td>
<td align="center">0.27</td>
<td align="char" char=".">0.07</td>
</tr>
<tr>
<td align="left">4th click</td>
<td align="char" char=".">1.9s</td>
<td align="char" char=".">1.0s</td>
<td align="char" char=".">2.0s</td>
<td align="char" char=".">1.0s</td>
<td align="left">167184</td>
<td align="center">0.32</td>
<td align="char" char=".">0.01</td>
</tr>
<tr>
<td align="left">6th click</td>
<td align="char" char=".">1.8s</td>
<td align="char" char=".">0.8s</td>
<td align="char" char=".">1.9s</td>
<td align="char" char=".">1.3s</td>
<td align="left">163076</td>
<td align="center">0.12</td>
<td align="char" char=".">0.12</td>
</tr>
<tr>
<td align="left">Duration</td>
<td align="char" char=".">27.1s</td>
<td align="char" char=".">18.6s</td>
<td align="char" char=".">29.4s</td>
<td align="char" char=".">22.9s</td>
<td align="left">163994</td>
<td align="center">0.15</td>
<td align="char" char=".">0.11</td>
</tr>
<tr>
<td align="left">Average</td>
<td align="left">2.7s</td>
<td align="char" char=".">0.8s</td>
<td align="char" char=".">2.8s</td>
<td align="char" char=".">1.0s</td>
<td align="left">166194</td>
<td align="center">0.26</td>
<td align="char" char=".">0.11</td>
</tr>
</tbody>
</table>
<table-wrap-foot>
<fn>
<p>Significant results are in&#x20;bold.</p>
</fn>
</table-wrap-foot>
</table-wrap>
<p>The ES data set (<italic>n</italic>&#x20;&#x3d; 153) has seven dependent variables with significant differences between groups: <italic>fourth click interval, duration round, average click time, total number of clicks, time to the first click, number of hits, and efficiency.</italic> The ES data set (<italic>n</italic>&#x20;&#x3d; 153) confirmed the results of the pilot study (<italic>n</italic>&#x20;&#x3d; 178). All other game measurements decreased the significance by slightly increasing the <italic>p</italic>-value (visual efficiency from 4<italic>e</italic>&#x20;&#x2212; 5 to 1<italic>e</italic>&#x20;&#x2212; 4). The data set ES has seven significant variables that distinguish a person with or without dyslexia.</p>
<p>For the data set ALL (<italic>n</italic>&#x20;&#x3d; 313) we consider only dependent variables with the same tendency as for the pilot study (<italic>n</italic>&#x20;&#x3d; 178). We categorize the tendency (e.g., <italic>playing faster or having more clicks</italic>) by the group (dyslexia compared to control group) <italic>mean</italic> of the dependent variables within the same language. ALL (<italic>n</italic>&#x20;&#x3d; 313) has two visual game measurements (<italic>number of misses</italic> and <italic>total clicks</italic>) with the same tendency while the pilot study had five for the visual game (<italic>total clicks, time to the first click, hits, accuracy, and efficiency</italic>).</p>
<p>The DE data set (<italic>n</italic>&#x20;&#x3d; 149) confirmed the results of the pilot study (<italic>n</italic>&#x20;&#x3d; 57) with no significant dependent variables. The <italic>means</italic> of the dependent measurements for DE are all very close (e.g., the <italic>time to the first click</italic> is 2.58<italic>s</italic> for the control group and 2.50<italic>s</italic> for the dyslexia group).</p>
<p>We can confirm that misses did not reveal significant differences for German or Spanish, even though the tendency is now the same for both languages. On the other hand, the total number of clicks is still significant.</p>
<p>To sum up, we confirmed one significant dependent variable in ALL (<italic>n</italic>&#x20;&#x3d; 313), seven significant dependent variables for ES (<italic>n</italic>&#x20;&#x3d; 153), and no significant dependent variables for DE (<italic>n</italic>&#x20;&#x3d;&#x20;149).</p>
</sec>
<sec id="s6-2">
<title>6.2 Predictive Results</title>
<p>We processed our data sets with different classifiers and different subsets of features, following the description from the previous section. We follow our criteria for analyzing small (tiny) data to avoid wrong results as wrong results have a negative impact on a person such as missing out a person with dyslexia (<xref ref-type="bibr" rid="B36">Rauschenberger and Baeza-Yates, 2020a</xref>; <xref ref-type="bibr" rid="B37">Rauschenberger and Baeza-Yates, 2020b</xref>).</p>
<p>We computed the <italic>balanced accuracy</italic> for our binary classification problem to deal with imbalanced data sets; for example, the ALL data set has dyslexia 37% <italic>vs.</italic> control 63%. The Dummy Classifier is computed for our imbalanced data with the most frequent label and reported with the balanced accuracy (<xref ref-type="bibr" rid="B61">Scikit-learn Developers, 2019</xref>). We do not apply over- or under-sampling to address our imbalanced data because the variances among people with dyslexia are broad, for example, difficulty level or the individual causes for perception differences.</p>
<p>As described in the previous section the ranking of the informative features is different for the three data sets. Hence, we explore the influence of different subsets of features, namely: 1) all represented features (201 features); 2) the 5 most informative features; 3) the 33 most informative features, as this was the next natural informative subset; 4) 20 random features selected from (3); and 5) 27 features that have the same tendency and which have been answered by the participants&#x2019; supervisors, because they are mainly not under the most informative feature subsets (although <italic>total clicks</italic> is significant in the statistical comparison).</p>
<p>We report the two best F1-scores and <italic>balanced accuracy</italic> scores for each data set as well as the baseline, as can be seen in <xref ref-type="table" rid="T8">Table&#x20;8</xref>. We outperform our baseline for all data sets. The best F1-score, <italic>0.75</italic>, is achieved for both languages, the DE and ES data sets. DE uses 5 features with RF and ES uses ETC with 20 features. The second best F1-score, <italic>0.74</italic>, is achieved with the DE data set using 5 features and RFW. The best accuracy, <italic>0.74</italic>, is achieved with RF while the second best of <italic>0.73</italic> is achieved with RFW, both in the DE data set using just 5 features.</p>
<table-wrap id="T8" position="float">
<label>TABLE 8</label>
<caption>
<p>Best results of the different classifiers, features and data sets. Results are ordered by the best F1-score and accuracy.</p>
</caption>
<table>
<thead valign="top">
<tr>
<th align="left">Model</th>
<th align="center">Data</th>
<th align="center">Feat.</th>
<th align="center">Recall</th>
<th align="center">Precis.</th>
<th align="center">F1</th>
<th align="center">Acc.</th>
</tr>
</thead>
<tbody valign="top">
<tr>
<td align="left">
<bold>RF</bold>
</td>
<td align="left">
<bold>DE</bold>
</td>
<td align="left">5</td>
<td align="char" char=".">0.77</td>
<td align="char" char=".">0.78</td>
<td align="char" char=".">
<bold>0.75</bold>
</td>
<td align="char" char=".">
<bold>0.74</bold>
</td>
</tr>
<tr>
<td align="left">RFW</td>
<td align="left">DE</td>
<td align="left">5</td>
<td align="char" char=".">0.75</td>
<td align="char" char=".">0.75</td>
<td align="char" char=".">0.74</td>
<td align="char" char=".">0.73</td>
</tr>
<tr>
<td align="left">Baseline</td>
<td align="left">DE</td>
<td align="left"/>
<td align="char" char=".">0.60</td>
<td align="char" char=".">0.37</td>
<td align="char" char=".">0.46</td>
<td align="char" char=".">0.50</td>
</tr>
<tr>
<td align="left">
<bold>ETC</bold>
</td>
<td align="left">
<bold>ES</bold>
</td>
<td align="left">20</td>
<td align="char" char=".">0.76</td>
<td align="char" char=".">0.76</td>
<td align="char" char=".">
<bold>0.75</bold>
</td>
<td align="char" char=".">
<bold>0.69</bold>
</td>
</tr>
<tr>
<td align="left">RF</td>
<td align="left">ES</td>
<td align="left">5</td>
<td align="char" char=".">0.74</td>
<td align="char" char=".">0.73</td>
<td align="char" char=".">0.72</td>
<td align="char" char=".">0.65</td>
</tr>
<tr>
<td align="left">Baseline</td>
<td align="left">ES</td>
<td align="left"/>
<td align="char" char=".">0.68</td>
<td align="char" char=".">0.46</td>
<td align="char" char=".">0.55</td>
<td align="char" char=".">0.50</td>
</tr>
<tr>
<td align="left">
<bold>GB</bold>
</td>
<td align="left">
<bold>ALL</bold>
</td>
<td align="left">20</td>
<td align="char" char=".">0.66</td>
<td align="char" char=".">0.65</td>
<td align="char" char=".">
<bold>0.65</bold>
</td>
<td align="char" char=".">
<bold>0.61</bold>
</td>
</tr>
<tr>
<td align="left">GB</td>
<td align="left">ALL</td>
<td align="left">5</td>
<td align="char" char=".">0.64</td>
<td align="char" char=".">0.64</td>
<td align="char" char=".">0.63</td>
<td align="char" char=".">0.59</td>
</tr>
<tr>
<td align="left">Baseline</td>
<td align="left">ALL</td>
<td align="left"/>
<td align="char" char=".">0.63</td>
<td align="char" char=".">0.40</td>
<td align="char" char=".">0.49</td>
<td align="char" char=".">0.50</td>
</tr>
</tbody>
</table>
</table-wrap>
<p>For ES, the best F1-score is also <italic>0.75</italic> with ETC and the selection of 20 features. The second best F1-score for ES is <italic>0.72</italic> with RF and a selection of 5 features. The F1-score is reduced by 0.1 when combining the two data sets (DE and ES), since the best F1-score for ALL is <italic>0.65</italic> using GB and 20 features. The second best F1-score for ALL is <italic>0.63</italic> with GB and 5 features. For ES, the best accuracy is <italic>0.69</italic> with ETC and the selection of 20 features. The second best accuracy for ES is <italic>0.65</italic> with RF and a selection of 5 features. The accuracy is reduced by nearly 0.1 when combining the two data sets (DE and ES), since the best accuracy for ALL is <italic>0.61</italic> using GB and 20 features. The second best accuracy for ALL is <italic>0.59</italic> with GB and 5 features. This shows that there are differences across languages.</p>
<p>The normalized confusion matrix (see <xref ref-type="fig" rid="F5">Figure&#x20;5</xref>) does not show over-fitting for the best results for DE, ES and ALL. The fact that the best results are with few features imply that the rest are highly correlated or&#x20;noisy.</p>
<fig id="F5" position="float">
<label>FIGURE 5</label>
<caption>
<p>Normalized confusion matrix for the three best results (F1-score and accuracy): <bold>(A)</bold> <italic>DE, 5 features with RF</italic>; <bold>(B)</bold> <italic>ES, 20 features with ETC</italic>; and <bold>(C)</bold> <italic>ALL, 20 features with GB</italic>.</p>
</caption>
<graphic xlink:href="fcomp-03-628634-g005.tif"/>
</fig>
<p>The reduction of features improves the accuracy for DE but not consistently for ES and ALL, as can be seen for the different classifiers and data sets in <xref ref-type="fig" rid="F6">Figure&#x20;6</xref>. For example, reducing the features for DE improves the accuracy for ET, RF, and RFW, but not for GB. For ES, the accuracy improves only for RF and stagnates for RFW when reducing the number of features, otherwise the accuracy inverts for ETC and GB. For the data set ALL, RFW and RF improve but ETC and GB decrease.</p>
<fig id="F6" position="float">
<label>FIGURE 6</label>
<caption>
<p>The plot shows the relation of accuracy to features for all classifiers in the data set ALL <bold>(left)</bold>, ES <bold>(middle)</bold> and DE <bold>(right)</bold>.</p>
</caption>
<graphic xlink:href="fcomp-03-628634-g006.tif"/>
</fig>
</sec>
</sec>
<sec id="s7">
<title>7 Discussion</title>
<p>Most children with dyslexia show a varying severity of deficits in more than one area (<xref ref-type="bibr" rid="B4">Black et&#x20;al., 2016</xref>), which makes dyslexia more a spectrum than a binary disorder. Additionally, we rely on current diagnostic tools (e.g., DRT (<xref ref-type="bibr" rid="B20">Grund et&#x20;al., 2004</xref>; <xref ref-type="bibr" rid="B63">Steinbrink and Lachmann, 2014</xref>)) to select our participant groups, which do not yet represent the diversity of people with dyslexia. We accept that our participants have a high variance because of the measurement of our current diagnostic tools and the spectrum that dyslexia&#x20;have.</p>
<sec id="s7-1">
<title>7.1 Group Comparison</title>
<p>The measurement data taken from the game <italic>MusVis</italic> show that Spanish participants with dyslexia behave differently than their control group. Differences can be reported for the auditory game part for: <italic>fourth click interval, duration, and average click time</italic>. For the visual part, the following measurements can be reported as indicators: <italic>total clicks, time to the first click, hits</italic>, and <italic>efficiency</italic>.</p>
<p>We can show with our results over all languages that the effect for each measurement is confirmed even if we cannot draw strong conclusions about our sample size on the comparison of German <italic>vs.</italic> Spanish speaking participants. Spanish had eight significant indicators in the pilot study and we expected to reproduce the same number of significant indicators with more German participants.</p>
<p>In general, all participants found the game easy to understand, and only children at the age of 12 complained about missing challenges. The amount of positive feedback and engagement of all age groups let us conclude that the game mechanics and components applied are also positive for perceiving <italic>MusVis</italic> as a game and not as a&#x20;test.</p>
<p>Dyslexia is known to be present across different languages and cultures (<xref ref-type="bibr" rid="B1">American Psychiatric Association, 2013</xref>). The assumption that the tendencies for the indicators are similar over all languages cannot (yet) be proven for all indicators in our study (e.g., German participants with dyslexia start to click faster than the Spanish participants compared to their language control group in the auditory part). We can exclude external factors such as different applications or study setups as possible influences on this opposite tendency. According to the results, we may have to assume that not all indicators for dyslexia are language-independent and that some have cultural dependencies, or we have <italic>omitted variable bias</italic>. To confirm this assumption, we will need to obtain larger numbers of participants for both language groups (Spanish and German) or investigate further measurements (indicators).</p>
<p>The variables <italic>time to first click (visual and auditory)</italic> and <italic>total number of clicks (visual and auditory)</italic> provide dependencies of the game content and game design. Otherwise, we could not explain the trend difference between the auditory and visual parts for <italic>total number of clicks</italic> (i.e.,&#x20;<italic>total clicks</italic> for visual is significantly different than for auditory). Additionally, the analysis of the auditory game part presents one limitation: participants could select a correct pair by chance, e.g., participants could click through the game board without listening to the sounds.</p>
<p>Children with dyslexia are detected by their slower reading or spelling <bold>error rate</bold> (<xref ref-type="bibr" rid="B58">Schulte-K&#xf6;rne et&#x20;al., 1996</xref>; <xref ref-type="bibr" rid="B6">Coleman et&#x20;al., 2008</xref>). Therefore, we designed our game with content that is known to be difficult for children with dyslexia to measure the errors and duration. Nevertheless, from previous literature we knew that children with dyslexia do not make more mistakes in games than the control group (<xref ref-type="bibr" rid="B53">Rello et&#x20;al., 2020</xref>). We can confirm that <italic>misses</italic> did not reveal significant differences for German or Spanish either. It might be possible that we cannot compare errors in reading and writing with errors in this type of game. Then, we cannot explain (yet) why the Spanish control group made more mistakes than the Spanish group with dyslexia. It might also be possible that participants with dyslexia show generally different behavior that is separated from the content but depends on the <italic>game&#x20;play</italic>.</p>
<p>Spanish children without dyslexia take significantly more time to find all pairs and finish the auditory game part. Children without dyslexia take more time before they <italic>click the first time</italic> (visual) for all languages. This might be due to the time they need to process the given auditory information (<xref ref-type="bibr" rid="B64">Tallal, 2004</xref>) or recall the auditory and visual information from short-term memory (<xref ref-type="bibr" rid="B19">Goswami et&#x20;al., 2016</xref>). However, participants with dyslexia from the German group are nearly as fast as the control group in finding all pairs (auditory) which might be due to cultural differences (e.g., more musical training).</p>
<p>The auditory and visual cues are designed on purpose to be more difficult to process for people with dyslexia than without. Therefore, children with dyslexia are expected to need more time (duration), which might be due to a less distinctive encoding of prosody (<xref ref-type="bibr" rid="B19">Goswami et&#x20;al., 2016</xref>) and is in line with the indicator of slower reading. Considering that children with dyslexia need more time to process information, we observe this behavior as well for our indicators. For example, participants with dyslexia from the Spanish group take more time on the <italic>fourth click interval</italic> and also on the <italic>average click time</italic> compared to the control group. Both results are significant and have medium effect sizes of 0.29, so we can estimate what the effects would be in the whole population (<xref ref-type="bibr" rid="B15">Field and Hole, 2003</xref>).</p>
<p>A person with dyslexia has difficulties with reading and writing independent of the mother tongue, which also appear when learning a second language (<xref ref-type="bibr" rid="B23">Helland and Kaasa, 2005</xref>; <xref ref-type="bibr" rid="B30">Nijakowska, 2010</xref>). The analysis of errors from children with dyslexia show similar error categories for Spanish, English (<xref ref-type="bibr" rid="B52">Rello et&#x20;al., 2016a</xref>), and German (<xref ref-type="bibr" rid="B49">Rauschenberger et&#x20;al., 2016</xref>), revealing similarities of perception between the languages.</p>
<p>Our results from the pilot study (<xref ref-type="bibr" rid="B46">Rauschenberger et&#x20;al., 2018b</xref>) suggest that we can measure a significant difference on four indicators for the visual game with the same tendency between Spanish, German, and English. With all our data (<italic>n</italic>&#x20;&#x3d; 313), we can confirm just one significant dependent variable with the same tendency for Spanish and German.</p>
<p>Still this means that people with dyslexia might perceive our visual game content similarly, independent of the mother tongue. Further research needs to be done to confirm the results, but this validation study provides strong evidence that it will be possible to screen dyslexia with our content, approach, and game design using the same language-independent content for different languages.</p>
</sec>
<sec id="s7-2">
<title>7.2 Screening Differences</title>
<p>Our approach aims to screen dyslexia with indicators that do not require linguistic knowledge. These indicators are probably not as strong or visible as the reading and spelling mistakes of children with dyslexia. Therefore, we consider our results (highest accuracy of 0.74 and highest F1-scores of 0.75) for German with Random Forest as a promising way to predict dyslexia using language-independent auditory and visual content for pre-readers.</p>
<p>Having an early indication of dyslexia before spelling or reading errors appear can have a positive impact on the child&#x2019;s development, as we can intervene earlier in her/his education. Therefore, we aim to optimize the recall and F1-score by finding as many participants with dyslexia as possible.</p>
<p>We have set ourselves this goal because early detection in a person with dyslexia has a greater positive effect on the person with dyslexia than a misjudgment in a person without dyslexia. However, to avoid over-fitting we did not modify the default value for the threshold (typically 0.5), something that we plan to study in the near future as we need to increase recall for the dyslexia class keeping a reasonable number of false positives.</p>
<p>If a person with dyslexia is not discovered (early), they are prone to face additional issues such as anxiety, sadness and decreased attention (<xref ref-type="bibr" rid="B59">Schulte-K&#xf6;rne, 2010</xref>). Also, a person with dyslexia needs around 2&#xa0;years to compensate for their reading and spelling difficulties. Early treatment among children at risk of dyslexia as well as children without dyslexia can serve, both, as a preventive measure and as early stimulation of literacy skills.</p>
<p>Our results support the hypothesis that dyslexia cannot be reduced to one cause, but is rather a combination of characteristics (<xref ref-type="bibr" rid="B9">De Zubicaray and Schiller, 2018</xref>). The equal distribution of auditory and visual features in the informative features ranking supports the hypothesis of dyslexia being related to auditory and visual perception in different people. We might be able to measure stronger effects when we design visual and auditory cues that have more attributes related to dyslexia, including some that favor the latter.</p>
<p>The ALL data set reached <italic>only</italic> an accuracy of 0.61, which might be due to the following reasons. First, the informative features for each data set are different from each other, which indicates different informativeness in German and Spanish. Combining the data sets into ALL probably adds noise for the prediction, which results in a lower accuracy. The noise might be that features are not as informative anymore because they cancel each other out as they are highly correlated.</p>
<p>In addition, reducing the features only to the features with the same tendency as used for the statistical analysis did not reveal any improvement, which supports the hypothesis that features in ALL cancel each other&#x20;out.</p>
<p>The results of our current game measures with 313 participants confirm differences in the behavior of Spanish <italic>vs.</italic> German participants (i.e.,&#x20;1) seven significant dependent variables in Spanish <italic>vs.</italic> none in German and 2) only two dependent variables with the same tendency over all languages).</p>
<p>These results might be explained by bilingualism. It is argued that a person who speaks more than one language has more knowledge of their first language than a monolingual person (<xref ref-type="bibr" rid="B27">Kecskes and Papp, 2000</xref>), and it is unclear whether this also has an influence on &#x201c;how people perceive differences as well&#x201d;. Additionally, dyslexia detection differences are reported for transparent (like Spanish) <italic>vs.</italic> deep (like English) orthographies [quoted after (<xref ref-type="bibr" rid="B51">Rello et&#x20;al., 2019</xref>)]. In a transparent orthography mainly a single grapheme (letter) corresponds to a single phoneme (sound) and dyslexia is reported to be more distinct in deep orthographies.</p>
<p>If so, this might explain the difference we have in the significance for the statistical analysis as well as the tendency of values, and the need for separate models to predict dyslexia for our German <italic>vs.</italic> Spanish data set (Spanish has bilingual participants).</p>
<p>Overall, having fewer features improves the accuracy, but this is less so when we run experiments for ALL or ES. There, the influence of the different informative features for ES and DE seem to cancel each other out. The high correlation between features would explain why, for example, taking into account 27 features (GB) performs no better than using 20 features (GB) for the ALL data set. The fact that the accuracy does not increase when more features are used supports the argument that features are highly correlated.</p>
<p>As described before, small data can help to understand the data and results better. In our case, we see that ALL does not perform as well as ES or DE. This is probably due to the facts described above (e.g., bilingualism, features canceling each other, English-speaking participants). The prediction for dyslexia is therefore possible with the data taken from the same game, but needs different models for the prediction in different languages as was proposed by (<xref ref-type="bibr" rid="B3">Bandhyopadhyay et&#x20;al., 2018</xref>), something that made sense in retrospect.</p>
</sec>
</sec>
<sec id="s8">
<title>8 Conclusions and Future Work</title>
<p>We processed our game data with Extra Trees, Random Forest without and with class weights, and Gradient Boost to predict dyslexia using a data set of 313 participants. We reached the best accuracy of 74% for the German case using RF while the best accuracy for Spanish was 69% using&#x20;ETC.</p>
<p>Our approach can optimize resources for detecting and treating dyslexia, however, it would need at the beginning more personnel to screen many more children at a young age to enlarge our training data. As children with dyslexia need around 2&#xa0;years to compensate their difficulties, our approach could help to decrease school failure, late treatment and most importantly, to reduce suffering for children and parents.</p>
<p>The main advantage of our language-independent content approach is that has the potential to screen pre-readers in the near future. Indeed, we aim to collect more data with younger children to improve our results, use different input related to more characteristics of dyslexia, and other game design.</p>
<p>Future work includes improving our machine learning models and do further feature analysis. More explainable models should also be considered.</p>
</sec>
</body>
<back>
<sec id="s9">
<title>Data Availability Statement</title>
<p>The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found in the article/<xref ref-type="sec" rid="s14">Supplementary Material</xref>.</p>
</sec>
<sec id="s10">
<title>Ethics Statement</title>
<p>The studies involving human participants were reviewed and approved by the institutions responsible. The data collection for this user study has been approved by the German Ministry of Education, Science and Culture in Schleswig-Holstein (Ministerium f&#xfc;r Bildung, Wissenschaft und Kultur) and Lower Saxony State Education Authority (Nieders&#xe4;chsische Landesschulbeh&#xf6;rde). In Spain, governmental approval was not needed in addition to the school approval. Written informed consent to participate in this study was provided by the participants&#x27; legal guardian/next of&#x20;kin.</p>
</sec>
<sec id="s11">
<title>Author Contributions</title>
<p>All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.</p>
</sec>
<sec sec-type="COI-statement" id="s12">
<title>Conflict of Interest</title>
<p>The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.</p>
</sec>
<sec sec-type="disclaimer" id="s13">
<title>Publisher&#x2019;s Note</title>
<p>All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.</p>
</sec>
<ack>
<p>This is an extension of the previously published conference paper from W4A &#x2019;20: Proceedings of the 17th International Web for All Conference in Tapei. This paper and content were partially funded by the <italic>fem:talent Scholarship</italic> from the <italic>Applied University of Emden/Leer</italic> as well as by the <italic>Deutschen Lesepreis 2017</italic> from the <italic>Stiftung Lesen</italic> and the <italic>Commerzbank-Stiftung</italic>. First, we would like to thank all teachers, students, and parents for their participation and time as well as all supporters to distribute my participation calls! Special thanks goes to one class and one teacher which cannot be named due to the anonymous regulations. We deeply thank for their support L. Alb&#xf3;, Barcelona; <italic>ChangeDyslexia</italic>, Barcelona; M. Jes&#xfa;s Blanque and R. No&#xe9; L&#xf3;pez, school <italic>Hijas de San Jos&#xe9;</italic>, Zaragoza; A. Carrasco, E. M&#xe9;ndez and S. Tena, innovation team of school <italic>Leonardo da Vinci</italic>, Madrid; in Spain, and L. Niemeier, <italic>Fr&#xf6;bel Bildung und Erziehung gemeinn&#xfc;tzige GmbH</italic>, Berlin; E. Prinz-Burghardt, <italic>Lerntherapeutische Praxis</italic>, Duderstadt; L. Klaus, <italic>Peter-Ustinov-Schule</italic>, Eckernf&#xf6;rde; H. Marquardt, <italic>Gorch-Fock-Schule</italic>, Eckernf&#xf6;rde; M. Batke and J.&#x20;Thomaschewski, <italic>Hochschule Emden/Leer</italic>, Emden; N. Tegeler, <italic>Montessori Bildungshaus Hannover gGmbH</italic>, Hannover; Y. Schulz, <italic>Grundschule Heidgraben</italic>, Heidgraben; T. Westphal, <italic>Leif-Eriksson-Gemeinschaftsschule</italic>, Kiel; F. Goerke, <italic>Grundschule Luetjensee</italic>, Luetjensee; B. Wilke, <italic>Schule am Draiberg</italic>, Papenburg; P. St&#xfc;mpel, <italic>AncoraMentis</italic>, Rheine; A. Wendt, <italic>Grundschule Seth</italic>, Seth; K. Usemann, <italic>OGGS Meyerstra&#xdf;e</italic>, Wuppertal; in Germany. We also thank all parents and children for playing <italic>MusVis</italic>. Finally, thanks to H. Witzel for his advice during the development of the visual part and to M. Blanca, and M. Herrera for the translation of the Spanish version.</p>
</ack>
<sec id="s14">
<title>Supplementary Material</title>
<p>The Supplementary Material for this article can be found online at: <ext-link ext-link-type="uri" xlink:href="https://www.frontiersin.org/articles/10.3389/fcomp.2021.628634/full#supplementary-material">https://www.frontiersin.org/articles/10.3389/fcomp.2021.628634/full&#x23;supplementary-material</ext-link>
</p>
<supplementary-material xlink:href="DataSheet4.CSV" id="SM1" mimetype="application/CSV" xmlns:xlink="http://www.w3.org/1999/xlink"/>
<supplementary-material xlink:href="DataSheet3.XLSX" id="SM2" mimetype="application/XLSX" xmlns:xlink="http://www.w3.org/1999/xlink"/>
<supplementary-material xlink:href="DataSheet2.DOCX" id="SM3" mimetype="application/DOCX" xmlns:xlink="http://www.w3.org/1999/xlink"/>
<supplementary-material xlink:href="DataSheet1.XLSX" id="SM4" mimetype="application/XLSX" xmlns:xlink="http://www.w3.org/1999/xlink"/>
</sec>
<fn-group>
<fn id="fn1">
<label>1</label>
<p>Pairs of identical cards (face down) must be identified by flipping them over (<xref ref-type="bibr" rid="B68">Wikipedia, 2019</xref>)</p>
</fn>
<fn id="fn2">
<label>2</label>
<p>The musical content used in the final game MusVis is available at <ext-link ext-link-type="uri" xlink:href="https://github.com/Rauschii/DysMusicMusicalElements">https://github.com/Rauschii/DysMusicMusicalElements</ext-link>.</p>
</fn>
<fn id="fn3">
<label>3</label>
<p>
<italic>Audacity</italic> is available at <ext-link ext-link-type="uri" xlink:href="http://audacity.es/">http://audacity.es/</ext-link>, Last access: May&#x20;2019</p>
</fn>
<fn id="fn4">
<label>4</label>
<p>
<ext-link ext-link-type="uri" xlink:href="https://github.com/Rauschii/DysMusicMusicalElements">https://github.com/Rauschii/DysMusicMusicalElements</ext-link>
</p>
</fn>
<fn id="fn5">
<label>5</label>
<p>We used the standard linguistic conventions: &#x2018;&#x3c;&#x3e;&#x2032; for graphemes, &#x2018;//&#x2019; for phonemes and &#x2018;[ ]&#x2019; for phones</p>
</fn>
<fn id="fn6">
<label>6</label>
<p>
<ext-link ext-link-type="uri" xlink:href="https://changedyslexia.org/">https://changedyslexia.org/</ext-link>
</p>
</fn>
</fn-group>
<ref-list>
<title>References</title>
<ref id="B1">
<citation citation-type="book">
<collab>American Psychiatric Association</collab> (<year>2013</year>). <source>Diagnostic and Statistical Manual of Mental Disorders</source>. <publisher-loc>London, England</publisher-loc>: <publisher-name>American Psychiatric Association</publisher-name>. <pub-id pub-id-type="doi">10.1176/appi.books.9780890425596.744053</pub-id> </citation>
</ref>
<ref id="B2">
<citation citation-type="web">
<person-group person-group-type="author">
<name>
<surname>Baeza-Yates</surname>
<given-names>R.</given-names>
</name>
</person-group> (<year>2018</year>). <article-title>Big, Small or Right Data: Which Is the Proper Focus?</article-title> <comment>Available at: <ext-link ext-link-type="uri" xlink:href="https://www.kdnuggets.com/2018/10/big-small-right-data.html">https://www.kdnuggets.com/2018/10/big-small-right-data.html</ext-link>
</comment>. </citation>
</ref>
<ref id="B3">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Bandhyopadhyay</surname>
<given-names>A.</given-names>
</name>
<name>
<surname>Dey</surname>
<given-names>D.</given-names>
</name>
<name>
<surname>Pal</surname>
<given-names>R. K.</given-names>
</name>
</person-group> (<year>2018</year>). <source>Prediction of Dyslexia Using Machine Learning &#x2014;&#x20;A Research Travelogue</source>. <publisher-name>Springer Singapore</publisher-name>. <pub-id pub-id-type="doi">10.1007/978-981-10-6890-4</pub-id> </citation>
</ref>
<ref id="B4">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Black</surname>
<given-names>D. W.</given-names>
</name>
<name>
<surname>Grant</surname>
<given-names>J.&#x20;E.</given-names>
</name>
</person-group>
<collab>American Psychiatric Association</collab> (<year>2016</year>). <source>DSM-5 Guidebook: The Essential Companion to the Diagnostic and Statistical Manual of Mental Disorders</source>. <edition>fifth edition</edition>. <publisher-name>American Psychiatric Association</publisher-name>. </citation>
</ref>
<ref id="B5">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Borleffs</surname>
<given-names>E.</given-names>
</name>
<name>
<surname>Maassen</surname>
<given-names>B. A. M.</given-names>
</name>
<name>
<surname>Lyytinen</surname>
<given-names>H.</given-names>
</name>
<name>
<surname>Zwarts</surname>
<given-names>F.</given-names>
</name>
</person-group> (<year>2019</year>). <article-title>Cracking the Code: The Impact of Orthographic Transparency and Morphological-Syllabic Complexity on Reading and Developmental Dyslexia</article-title>. <source>Front. Psychol.</source> <volume>9</volume>, <fpage>1</fpage>&#x2013;<lpage>19</lpage>. <pub-id pub-id-type="doi">10.3389/fpsyg.2018.02534</pub-id> </citation>
</ref>
<ref id="B6">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Coleman</surname>
<given-names>C.</given-names>
</name>
<name>
<surname>Gregg</surname>
<given-names>N.</given-names>
</name>
<name>
<surname>McLain</surname>
<given-names>L.</given-names>
</name>
<name>
<surname>Bellair</surname>
<given-names>L. W.</given-names>
</name>
</person-group> (<year>2008</year>). <article-title>A Comparison&#x20;of Spelling Performance across Young Adults with and without Dyslexia</article-title>. <source>Assess. Eff. Intervention</source> <volume>34</volume>, <fpage>94</fpage>&#x2013;<lpage>105</lpage>. <pub-id pub-id-type="doi">10.1177/1534508408318808</pub-id> </citation>
</ref>
<ref id="B7">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Cuetos</surname>
<given-names>F.</given-names>
</name>
<name>
<surname>Ramos</surname>
<given-names>J.&#x20;L.</given-names>
</name>
<name>
<surname>Ruano</surname>
<given-names>E.</given-names>
</name>
</person-group> (<year>2002</year>). <source>PROESC. Evaluaci&#xf3;n de los procesos de escritura (Writing processes assessment)</source>. <publisher-loc>Madrid</publisher-loc>: <publisher-name>TEA</publisher-name>. </citation>
</ref>
<ref id="B8">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Cuetos</surname>
<given-names>F.</given-names>
</name>
<name>
<surname>Rodr&#xed;guez</surname>
<given-names>B.</given-names>
</name>
<name>
<surname>Ruano</surname>
<given-names>E.</given-names>
</name>
<name>
<surname>Arribas</surname>
<given-names>D.</given-names>
</name>
</person-group> (<year>2007</year>). <article-title>PROLEC-R: Bater&#xed;a de Evaluaci&#xf3;n de los Procesos Lectores, Revisada (Battery of reading processes assessment&#x2014;Revised)</article-title>. </citation>
</ref>
<ref id="B9">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>De Zubicaray</surname>
<given-names>G.</given-names>
</name>
<name>
<surname>Schiller</surname>
<given-names>N. O.</given-names>
</name>
</person-group> (<year>2018</year>). <source>The Oxford Handbook of Neurolinguistics</source>. <publisher-loc>New York, NY</publisher-loc>: <publisher-name>Oxford University Press</publisher-name>. </citation>
</ref>
<ref id="B10">
<citation citation-type="book">
<collab>Ergonomics of human-system interaction</collab> (<year>2010</year>).<article-title>Part 210: Human- Centred Design for Interactive Systems</article-title>. In <conf-name>Ergonomics of human-system interaction</conf-name>. <publisher-loc>Brussels</publisher-loc>: <publisher-name>International Organization for Standardization</publisher-name>, <fpage>132</fpage>. </citation>
</ref>
<ref id="B11">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Esser</surname>
<given-names>G.</given-names>
</name>
<name>
<surname>Wyschkon</surname>
<given-names>A.</given-names>
</name>
<name>
<surname>Schmidt</surname>
<given-names>M. H.</given-names>
</name>
</person-group> (<year>2002</year>). <article-title>Was wird aus Achtj&#xe4;hrigen mit einer Lese- und Rechtschreibst&#xf6;rung</article-title>. <source>Z. f&#xfc;r Klinische Psychol. Psychotherapie</source> <volume>31</volume>, <fpage>235</fpage>&#x2013;<lpage>242</lpage>. <pub-id pub-id-type="doi">10.1026/0084-5345.31.4.235</pub-id> </citation>
</ref>
<ref id="B12">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Faraway</surname>
<given-names>J.&#x20;J.</given-names>
</name>
<name>
<surname>Augustin</surname>
<given-names>N. H.</given-names>
</name>
</person-group> (<year>2018</year>). <article-title>When Small Data Beats Big Data</article-title>. <source>Stat. Probab. Lett.</source> <volume>136</volume>, <fpage>142</fpage>&#x2013;<lpage>145</lpage>. <pub-id pub-id-type="doi">10.1016/j.spl.2018.02.031</pub-id> </citation>
</ref>
<ref id="B13">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Fastl</surname>
<given-names>H.</given-names>
</name>
<name>
<surname>Zwicker</surname>
<given-names>E.</given-names>
</name>
</person-group> (<year>2007</year>). <source>Psychoacoustics</source>. <edition>third edn.</edition> <publisher-loc>Berlin, Heidelberg</publisher-loc>: <publisher-name>Springer Berlin Heidelberg</publisher-name>. </citation>
</ref>
<ref id="B14">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Fawcett</surname>
<given-names>A.</given-names>
</name>
<name>
<surname>Nicolson</surname>
<given-names>R.</given-names>
</name>
</person-group> (<year>2004</year>). <source>The Dyslexia Screening Test: Junior (DST-J)</source>. <publisher-loc>London, UK</publisher-loc>: <publisher-name>Harcourt Assessment</publisher-name>. </citation>
</ref>
<ref id="B15">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Field</surname>
<given-names>A.</given-names>
</name>
<name>
<surname>Hole</surname>
<given-names>G.</given-names>
</name>
</person-group> (<year>2003</year>). <source>How to Design and Report Experiments</source>. <publisher-loc>London</publisher-loc>: <publisher-name>SAGE Publications</publisher-name>. </citation>
</ref>
<ref id="B16">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Gaggi</surname>
<given-names>O.</given-names>
</name>
<name>
<surname>Palazzi</surname>
<given-names>C. E.</given-names>
</name>
<name>
<surname>Ciman</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Galiazzo</surname>
<given-names>G.</given-names>
</name>
<name>
<surname>Franceschini</surname>
<given-names>S.</given-names>
</name>
<name>
<surname>Ruffino</surname>
<given-names>M.</given-names>
</name>
<etal/>
</person-group> (<year>2017</year>). <article-title>Serious Games for Early Identification of Developmental Dyslexia</article-title>. <source>Comput. Entertain.</source> <volume>15</volume>, <fpage>1</fpage>&#x2013;<lpage>24</lpage>. <pub-id pub-id-type="doi">10.1145/2629558</pub-id> </citation>
</ref>
<ref id="B17">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Geurts</surname>
<given-names>L.</given-names>
</name>
<name>
<surname>Vanden Abeele</surname>
<given-names>V.</given-names>
</name>
<name>
<surname>Celis</surname>
<given-names>V.</given-names>
</name>
<name>
<surname>Husson</surname>
<given-names>J.</given-names>
</name>
<name>
<surname>Van den Audenaeren</surname>
<given-names>L.</given-names>
</name>
<name>
<surname>Loyez</surname>
<given-names>L.</given-names>
</name>
<etal/>
</person-group> (<year>2015</year>). &#x201c;<article-title>DIESEL-X: A Game-Based Tool for Early Risk Detection of Dyslexia in Preschoolers</article-title>,&#x201d; in <source>Describing and Studying Domain-specific Serious Games</source> (<publisher-loc>Switzerland</publisher-loc>: <publisher-name>Springer</publisher-name>), <fpage>93</fpage>&#x2013;<lpage>114</lpage>. <pub-id pub-id-type="doi">10.1007/978-3-319-20276-1_7</pub-id> </citation>
</ref>
<ref id="B18">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Goswami</surname>
<given-names>U.</given-names>
</name>
</person-group> (<year>2011</year>). <article-title>A Temporal Sampling Framework for Developmental&#x20;Dyslexia</article-title>. <source>Trends Cogn. Sci.</source> <volume>15</volume>, <fpage>3</fpage>&#x2013;<lpage>10</lpage>. <pub-id pub-id-type="doi">10.1016/j.tics.2010.10.001</pub-id> </citation>
</ref>
<ref id="B19">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Goswami</surname>
<given-names>U.</given-names>
</name>
<name>
<surname>Barnes</surname>
<given-names>L.</given-names>
</name>
<name>
<surname>Mead</surname>
<given-names>N.</given-names>
</name>
<name>
<surname>Power</surname>
<given-names>A. J.</given-names>
</name>
<name>
<surname>Leong</surname>
<given-names>V.</given-names>
</name>
</person-group> (<year>2016</year>). <article-title>Prosodic Similarity Effects in Short-Term Memory in Developmental Dyslexia</article-title>. <source>Dyslexia</source> <volume>22</volume>, <fpage>287</fpage>&#x2013;<lpage>304</lpage>. <pub-id pub-id-type="doi">10.1002/dys.1535</pub-id> </citation>
</ref>
<ref id="B20">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Grund</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Naumann</surname>
<given-names>C. L.</given-names>
</name>
<name>
<surname>Haug</surname>
<given-names>G.</given-names>
</name>
</person-group> (<year>2004</year>). <source>Diagnostischer Rechtschreibtest F&#x00FC;r 5. Klassen: DRT 5 (Diagnostic Spelling Test for Fifth Grade: DRT 5)</source>. <edition>aktual edn</edition>. <publisher-loc>G&#x00F6;ttingen, Germany</publisher-loc>: <publisher-name>Deutsche Schultests G&#x00F6;ttingen: Beltz Test</publisher-name>, <fpage>2</fpage>. </citation>
</ref>
<ref id="B21">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>H&#xe4;m&#xe4;l&#xe4;inen</surname>
<given-names>J.&#x20;A.</given-names>
</name>
<name>
<surname>Salminen</surname>
<given-names>H. K.</given-names>
</name>
<name>
<surname>Lepp&#xe4;nen</surname>
<given-names>P. H. T.</given-names>
</name>
</person-group> (<year>2013</year>). <article-title>Basic Auditory Processing Deficits in Dyslexia</article-title>. <source>J.&#x20;Learn. Disabil.</source> <volume>46</volume>, <fpage>413</fpage>&#x2013;<lpage>427</lpage>. <pub-id pub-id-type="doi">10.1177/0022219411436213</pub-id> </citation>
</ref>
<ref id="B22">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Hamari</surname>
<given-names>J.</given-names>
</name>
<name>
<surname>Koivisto</surname>
<given-names>J.</given-names>
</name>
<name>
<surname>Sarsa</surname>
<given-names>H.</given-names>
</name>
</person-group> (<year>2014</year>). <article-title>Does Gamification Work? -- A Literature Review of Empirical Studies on Gamification</article-title>. In <conf-name>2014 47th Hawaii International Conference on System Sciences</conf-name>. <publisher-name>IEEE</publisher-name>, <fpage>3025</fpage>&#x2013;<lpage>3034</lpage>. <pub-id pub-id-type="doi">10.1109/HICSS.2014.377</pub-id> </citation>
</ref>
<ref id="B23">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Helland</surname>
<given-names>T.</given-names>
</name>
<name>
<surname>Kaasa</surname>
<given-names>R.</given-names>
</name>
</person-group> (<year>2005</year>). <article-title>Dyslexia in English as a Second Language</article-title>. <source>Dyslexia</source> <volume>11</volume>, <fpage>41</fpage>&#x2013;<lpage>60</lpage>. <pub-id pub-id-type="doi">10.1002/dys.286</pub-id> </citation>
</ref>
<ref id="B24">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Huss</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Verney</surname>
<given-names>J.&#x20;P.</given-names>
</name>
<name>
<surname>Fosker</surname>
<given-names>T.</given-names>
</name>
<name>
<surname>Mead</surname>
<given-names>N.</given-names>
</name>
<name>
<surname>Goswami</surname>
<given-names>U.</given-names>
</name>
</person-group> (<year>2011</year>). <article-title>Music, Rhythm, Rise Time Perception and Developmental Dyslexia: Perception of Musical Meter Predicts reading and Phonology</article-title>. <source>Cortex</source> <volume>47</volume>, <fpage>674</fpage>&#x2013;<lpage>689</lpage>. <pub-id pub-id-type="doi">10.1016/j.cortex.2010.07.010</pub-id> </citation>
</ref>
<ref id="B25">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Jain</surname>
<given-names>A.</given-names>
</name>
<name>
<surname>Zongker</surname>
<given-names>D.</given-names>
</name>
</person-group> (<year>1997</year>). <article-title>Feature Selection: Evaluation, Application, and Small Sample Performance</article-title>. <source>IEEE Trans. Pattern Anal. Machine Intell.</source> <volume>19</volume>, <fpage>153</fpage>&#x2013;<lpage>158</lpage>. <pub-id pub-id-type="doi">10.1109/34.574797</pub-id> </citation>
</ref>
<ref id="B26">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Johnson</surname>
<given-names>D. J.</given-names>
</name>
</person-group> (<year>1980</year>). <article-title>Persistent Auditory Disorders in Young Dyslexic Adults</article-title>. <source>Bull. Orton Soc.</source> <volume>30</volume>, <fpage>268</fpage>&#x2013;<lpage>276</lpage>. <pub-id pub-id-type="doi">10.1007/BF02653723</pub-id> </citation>
</ref>
<ref id="B27">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Kecskes</surname>
<given-names>I.</given-names>
</name>
<name>
<surname>Papp</surname>
<given-names>T. n.</given-names>
</name>
</person-group> (<year>2000</year>). <source>Foreign Language and Mother&#x20;Tongue</source>.&#x20;<edition>1&#x20;edn</edition>. <publisher-loc>New York</publisher-loc>: <publisher-name>Psychology Press</publisher-name>. <pub-id pub-id-type="doi">10.4324/9781410606464</pub-id> </citation>
</ref>
<ref id="B28">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>M&#xe4;nnel</surname>
<given-names>C.</given-names>
</name>
<name>
<surname>Schaadt</surname>
<given-names>G.</given-names>
</name>
<name>
<surname>Illner</surname>
<given-names>F. K.</given-names>
</name>
<name>
<surname>van der Meer</surname>
<given-names>E.</given-names>
</name>
<name>
<surname>Friederici</surname>
<given-names>A. D.</given-names>
</name>
</person-group> (<year>2017</year>). <article-title>Phonological Abilities in Literacy-Impaired Children: Brain Potentials Reveal Deficient Phoneme Discrimination, but Intact Prosodic Processing</article-title>. <source>Dev. Cogn. Neurosci.</source> <volume>23</volume>, <fpage>14</fpage>&#x2013;<lpage>25</lpage>. <pub-id pub-id-type="doi">10.1016/j.dcn.2016.11.007</pub-id> </citation>
</ref>
<ref id="B29">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Mora</surname>
<given-names>A.</given-names>
</name>
<name>
<surname>Riera</surname>
<given-names>D.</given-names>
</name>
<name>
<surname>Gonzalez</surname>
<given-names>C.</given-names>
</name>
<name>
<surname>Arnedo-Moreno</surname>
<given-names>J.</given-names>
</name>
</person-group> (<year>2015</year>). <article-title>A Literature Review of Gamification Design Frameworks</article-title>. In <conf-name>7th International Conference on Games and Virtual Worlds for Serious Applications</conf-name>. <pub-id pub-id-type="doi">10.1109/VS-GAMES.2015.7295760</pub-id> </citation>
</ref>
<ref id="B30">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Nijakowska</surname>
<given-names>J.</given-names>
</name>
</person-group> (<year>2010</year>). <source>Dyslexia in the Foreign Language Classroom</source>. <publisher-loc>Bristol, United Kingdom</publisher-loc>: <publisher-name>Multilingual Matters</publisher-name>. </citation>
</ref>
<ref id="B31">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Overy</surname>
<given-names>K.</given-names>
</name>
</person-group> (<year>2000</year>). <article-title>Dyslexia, Temporal Processing and Music: The Potential of Music as an Early Learning Aid for Dyslexic Children</article-title>. <source>Psychol. Music</source> <volume>28</volume>, <fpage>218</fpage>&#x2013;<lpage>229</lpage>. <pub-id pub-id-type="doi">10.1177/0305735600282010</pub-id> </citation>
</ref>
<ref id="B32">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Paulesu</surname>
<given-names>E.</given-names>
</name>
<name>
<surname>Danelli</surname>
<given-names>L.</given-names>
</name>
<name>
<surname>Berlingeri</surname>
<given-names>M.</given-names>
</name>
</person-group> (<year>2014</year>). <article-title>Reading the Dyslexic Brain: Multiple Dysfunctional Routes Revealed by a New Meta-Analysis of PET and fMRI Activation Studies</article-title>. <source>Front. Hum. Neurosci.</source> <volume>8</volume>, <fpage>830</fpage>. <pub-id pub-id-type="doi">10.3389/fnhum.2014.00830</pub-id> </citation>
</ref>
<ref id="B33">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Petscher</surname>
<given-names>Y.</given-names>
</name>
<name>
<surname>Fien</surname>
<given-names>H.</given-names>
</name>
<name>
<surname>Stanley</surname>
<given-names>C.</given-names>
</name>
<name>
<surname>Gearin</surname>
<given-names>B.</given-names>
</name>
<name>
<surname>Fletcher</surname>
<given-names>J.&#x20;M.</given-names>
</name>
<name>
<surname>Johnson</surname>
<given-names>E.</given-names>
</name>
<etal/>
</person-group> (<year>2019</year>). <source>Screening for Dyslexia</source>. <publisher-loc>Washington, DC</publisher-loc>: <publisher-name>U.S. Department of Education, Office of Elementary and Secondary Education, Office of Special Education Programs, National Center on Improving Literacy</publisher-name>. <comment>Retrieved from: <ext-link ext-link-type="uri" xlink:href="improvingliteracy.org.">improvingliteracy.org.</ext-link>
</comment> </citation>
</ref>
<ref id="B34">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Poole</surname>
<given-names>A.</given-names>
</name>
<name>
<surname>Zulkernine</surname>
<given-names>F.</given-names>
</name>
<name>
<surname>Aylward</surname>
<given-names>C.</given-names>
</name>
</person-group> (<year>2017</year>). <article-title>Lexa: A Tool for Detecting Dyslexia through Auditory Processing</article-title>. In <conf-name>2017 IEEE Symposium Series on Computational Intelligence, SSCI 2017 - Proceedings</conf-name> <conf-date>2018-January</conf-date>, <fpage>1</fpage>&#x2013;<lpage>5</lpage>. <pub-id pub-id-type="doi">10.1109/SSCI.2017.8285191</pub-id> </citation>
</ref>
<ref id="B35">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Port</surname>
<given-names>R. F.</given-names>
</name>
</person-group> (<year>2003</year>). <article-title>Meter and Speech</article-title>. <source>J.&#x20;Phonetics</source> <volume>31</volume>, <fpage>599</fpage>&#x2013;<lpage>611</lpage>. <pub-id pub-id-type="doi">10.1016/j.wocn.2003.08.001</pub-id> </citation>
</ref>
<ref id="B36">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Rauschenberger</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Baeza-Yates</surname>
<given-names>R.</given-names>
</name>
</person-group> (<year>2021a</year>). <article-title>How to Handle Health-Related Small Imbalanced Data in Machine Learning?</article-title> <source>i-com</source> <volume>19</volume>, <fpage>215</fpage>&#x2013;<lpage>226</lpage>. <pub-id pub-id-type="doi">10.1515/icom-2020-0018</pub-id> </citation>
</ref>
<ref id="B37">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Rauschenberger</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Baeza-Yates</surname>
<given-names>R.</given-names>
</name>
</person-group> (<year>2020b</year>). &#x201c;<article-title>Recommendations to Handle Health-Related Small Imbalanced Data in Machine Learning</article-title>,&#x201d; in <source>Mensch und Computer 2020 - Workshopband (Human and Computer 2020 - Workshop proceedings)</source>. Editor <person-group person-group-type="editor">
<name>
<surname>Hansen</surname>
<given-names>B.</given-names>
</name>
</person-group> (<publisher-loc>Bonn</publisher-loc>: <publisher-name>Gesellschaft f&#xa8;ur Informatik e.V.</publisher-name>), <fpage>1</fpage>&#x2013;<lpage>7</lpage>. <comment>Christian AND N&#xa8;urnberger, Andreas AND Preim</comment>. <pub-id pub-id-type="doi">10.18420/muc2020-ws111-333</pub-id> </citation>
</ref>
<ref id="B38">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Rauschenberger</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Baeza-Yates</surname>
<given-names>R.</given-names>
</name>
<name>
<surname>Rello</surname>
<given-names>L.</given-names>
</name>
</person-group> (<year>2021a</year>). <article-title>Data: Documentation Semi-structure Literature Review 0.1</article-title>. <pub-id pub-id-type="doi">10.13140/RG.2.2.19378.94401</pub-id> </citation>
</ref>
<ref id="B39">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Rauschenberger</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Baeza-Yates</surname>
<given-names>R.</given-names>
</name>
<name>
<surname>Rello</surname>
<given-names>L.</given-names>
</name>
</person-group> (<year>2021b</year>). <article-title>MusVis User Dataset - as Csv (Resource, Content)</article-title>. <pub-id pub-id-type="doi">10.13140/RG.2.2.20633.95846</pub-id> </citation>
</ref>
<ref id="B40">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Rauschenberger</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Baeza-Yates</surname>
<given-names>R.</given-names>
</name>
<name>
<surname>Rello</surname>
<given-names>L.</given-names>
</name>
</person-group> (<year>2021c</year>). <article-title>Protocol of the Generated Audio Files Selected for A Universal Screening Tool for Dyslexiaby a Web-Game and Machine Learning</article-title>. <pub-id pub-id-type="doi">10.13140/RG.2.2.27348.12162</pub-id> </citation>
</ref>
<ref id="B41">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Rauschenberger</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Baeza-Yates</surname>
<given-names>R.</given-names>
</name>
<name>
<surname>Rello</surname>
<given-names>L.</given-names>
</name>
</person-group> (<year>2021d</year>). <article-title>Protocol of the Iterations to Select the Content for A Universal Screening Tool for Dyslexiaby a Web-Game and Machine Learning</article-title>. <pub-id pub-id-type="doi">10.13140/RG.2.2.17281.79209</pub-id> </citation>
</ref>
<ref id="B42">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Rauschenberger</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Baeza-Yates</surname>
<given-names>R.</given-names>
</name>
<name>
<surname>Rello</surname>
<given-names>L.</given-names>
</name>
</person-group> (<year>2020</year>). <source>Screening Risk of Dyslexia through a Web-Game Using Language-independent Content and Machine Learning</source>. <publisher-loc>Taipei</publisher-loc>: <publisher-name>ACM Press</publisher-name>, <fpage>1</fpage>&#x2013;<lpage>12</lpage>. <pub-id pub-id-type="doi">10.1145/3371300.3383342</pub-id> </citation>
</ref>
<ref id="B43">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Rauschenberger</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Baeza&#x2013;Yates</surname>
<given-names>R.</given-names>
</name>
<name>
<surname>Rello</surname>
<given-names>L.</given-names>
</name>
</person-group> (<year>2019b</year>). &#x201c;<article-title>Technologies for Dyslexia</article-title>,&#x201d; in <source>Web Accessibility Book</source>. Editors <person-group person-group-type="editor">
<name>
<surname>Yesilada</surname>
<given-names>Y.</given-names>
</name>
<name>
<surname>Harper</surname>
<given-names>S.</given-names>
</name>
</person-group>. <edition>2 edn</edition> (<publisher-loc>London</publisher-loc>: <publisher-name>Springer-Verlag London</publisher-name>), <volume>1</volume>, <fpage>603</fpage>&#x2013;<lpage>627</lpage>. <pub-id pub-id-type="doi">10.1007/978-1-4471-7440-010.1007/978-1-4471-7440-0_31</pub-id> </citation>
</ref>
<ref id="B44">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Rauschenberger</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Lins</surname>
<given-names>C.</given-names>
</name>
<name>
<surname>Rousselle</surname>
<given-names>N.</given-names>
</name>
<name>
<surname>Fudickar</surname>
<given-names>S.</given-names>
</name>
<name>
<surname>Hain</surname>
<given-names>A.</given-names>
</name>
</person-group> (<year>2019a</year>). <article-title>A Tablet Puzzle to Target Dyslexia Screening in Pre-readers</article-title>. In <conf-name>Proceedings of the 5th EAI International Conference on Smart Objects and Technologies for Social Good - GOODTECHS</conf-name>. <publisher-loc>Valencia</publisher-loc>, <fpage>155</fpage>&#x2013;<lpage>159</lpage>. </citation>
</ref>
<ref id="B45">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Rauschenberger</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Rello</surname>
<given-names>L.</given-names>
</name>
<name>
<surname>Baeza-Yates</surname>
<given-names>R.</given-names>
</name>
</person-group> (<year>2018a</year>). <publisher-loc>Barcelona</publisher-loc>: <publisher-name>ACM Press</publisher-name>, <fpage>306</fpage>&#x2013;<lpage>312</lpage>. <pub-id pub-id-type="doi">10.1145/3236112.3236156</pub-id> </citation>
</ref>
<ref id="B46">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Rauschenberger</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Rello</surname>
<given-names>L.</given-names>
</name>
<name>
<surname>Baeza-Yates</surname>
<given-names>R.</given-names>
</name>
<name>
<surname>Bigham</surname>
<given-names>J.&#x20;P.</given-names>
</name>
</person-group> (<year>2018b</year>).<article-title>Towards Language Independent Detection of Dyslexia with a Web-Based Game</article-title>. In <conf-name>W4A &#x2019;18: The Internet of Accessible Things</conf-name>. <publisher-loc>Lyon, France</publisher-loc>: <publisher-name>ACM</publisher-name>, <fpage>4</fpage>&#x2013;<lpage>6</lpage>. <pub-id pub-id-type="doi">10.1145/3192714.3192816</pub-id> </citation>
</ref>
<ref id="B47">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Rauschenberger</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Rello</surname>
<given-names>L.</given-names>
</name>
<name>
<surname>Baeza-Yates</surname>
<given-names>R.</given-names>
</name>
<name>
<surname>Gomez</surname>
<given-names>E.</given-names>
</name>
<name>
<surname>Bigham</surname>
<given-names>J.&#x20;P.</given-names>
</name>
</person-group> (<year>2017a</year>). <article-title>Supplement: DysMusicMusicalElements: Towards the Prediction of Dyslexia by a Web-Based Game with Musical Elements</article-title>. <pub-id pub-id-type="doi">10.5281/zenodo.809783</pub-id> </citation>
</ref>
<ref id="B48">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Rauschenberger</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Rello</surname>
<given-names>L.</given-names>
</name>
<name>
<surname>Baeza-Yates</surname>
<given-names>R.</given-names>
</name>
<name>
<surname>Gomez</surname>
<given-names>E.</given-names>
</name>
<name>
<surname>Bigham</surname>
<given-names>J.&#x20;P.</given-names>
</name>
</person-group> (<year>2017b</year>). <article-title>Towards the Prediction of Dyslexia by a Web-Based Game with Musical Elements</article-title>. In <conf-name>The Web for All conference Addressing information barriers &#x2013; W4A&#x2019;17</conf-name>. <publisher-loc>Western Australia</publisher-loc>: <publisher-name>PerthACM Press</publisher-name>, <fpage>4</fpage>&#x2013;<lpage>7</lpage>. <pub-id pub-id-type="doi">10.1145/3058555.3058565</pub-id> </citation>
</ref>
<ref id="B49">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Rauschenberger</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Rello</surname>
<given-names>L.</given-names>
</name>
<name>
<surname>F&#x00FC;chsel</surname>
<given-names>S.</given-names>
</name>
<name>
<surname>Thomaschewski</surname>
<given-names>J.</given-names>
</name>
</person-group> (<year>2016</year>). &#x201c;<article-title>A Language Resource of German Errors Written by Children with Dyslexia</article-title>,&#x201d; in <conf-name>Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016)</conf-name> (<publisher-loc>Paris, France</publisher-loc>: <publisher-name>European Language Resources Association</publisher-name>). </citation>
</ref>
<ref id="B50">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Rauschenberger</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Willems</surname>
<given-names>A.</given-names>
</name>
<name>
<surname>Ternieden</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Thomaschewski</surname>
<given-names>J.</given-names>
</name>
</person-group> (<year>2019c</year>). <article-title>Towards the Use of Gamification Frameworks in Learning Environments</article-title>. <source>J.&#x20;Interactive Learn. Res.</source> <volume>30</volume>. </citation>
</ref>
<ref id="B51">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Rello</surname>
<given-names>L.</given-names>
</name>
<name>
<surname>Baeza-Yates</surname>
<given-names>R.</given-names>
</name>
<name>
<surname>Ali</surname>
<given-names>A.</given-names>
</name>
<name>
<surname>Bigham</surname>
<given-names>J.&#x20;P.</given-names>
</name>
<name>
<surname>Serra</surname>
<given-names>M.</given-names>
</name>
</person-group> (<year>2019</year>). <article-title>Predicting Risk of Dyslexia with an Online Gamified Test</article-title>. <comment>arXiv preprint arXiv:1906.03168 V.1</comment>, <fpage>1</fpage>&#x2013;<lpage>13</lpage>. </citation>
</ref>
<ref id="B52">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Rello</surname>
<given-names>L.</given-names>
</name>
<name>
<surname>Baeza-Yates</surname>
<given-names>R.</given-names>
</name>
<name>
<surname>Llisterri</surname>
<given-names>J.</given-names>
</name>
</person-group> (<year>2016a</year>). <article-title>A Resource of Errors Written in Spanish by People with Dyslexia and its Linguistic, Phonetic and Visual Analysis</article-title>. <source>Lang. Resour. Eval.</source> <volume>51</volume>, <fpage>379</fpage>&#x2013;<lpage>408</lpage>. <pub-id pub-id-type="doi">10.1007/s10579-015-9329-0</pub-id> </citation>
</ref>
<ref id="B53">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Rello</surname>
<given-names>L.</given-names>
</name>
<name>
<surname>Baeza-Yates</surname>
<given-names>R.</given-names>
</name>
<name>
<surname>Ali</surname>
<given-names>A.</given-names>
</name>
<name>
<surname>Bigham</surname>
<given-names>J. P.</given-names>
</name>
<name>
<surname>Serra</surname>
<given-names>M.</given-names>
</name>
</person-group> (<year>2020</year>).<article-title>Predicting risk of dyslexia with an online gamified test</article-title>. <source>PLoS ONE</source> <volume>15</volume> (<issue>12</issue>), <fpage>e0241687</fpage>. <pub-id pub-id-type="doi">10.1371/journal.pone.0241687</pub-id> </citation>
</ref>
<ref id="B54">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Rello</surname>
<given-names>L.</given-names>
</name>
<name>
<surname>Romero</surname>
<given-names>E.</given-names>
</name>
<name>
<surname>Rauschenberger</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Ali</surname>
<given-names>A.</given-names>
</name>
<name>
<surname>Williams</surname>
<given-names>K.</given-names>
</name>
<name>
<surname>Bigham</surname>
<given-names>J.&#x20;P.</given-names>
</name>
<etal/>
</person-group> (<year>2018</year>).<article-title>Screening Dyslexia for English Using HCI Measures and Machine Learning</article-title>. In <conf-name>Proceedings of the 2018 International Conference on Digital Health - DH &#x2019;18</conf-name>. <publisher-loc>New York, New York, USA</publisher-loc>: <publisher-name>ACM Press</publisher-name>, <fpage>80</fpage>&#x2013;<lpage>84</lpage>. <pub-id pub-id-type="doi">10.1145/3194658.3194675</pub-id> </citation>
</ref>
<ref id="B55">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Ritzhaupt</surname>
<given-names>A. D.</given-names>
</name>
<name>
<surname>Poling</surname>
<given-names>N. D.</given-names>
</name>
<name>
<surname>Frey</surname>
<given-names>C. A.</given-names>
</name>
<name>
<surname>Johnson</surname>
<given-names>M. C.</given-names>
</name>
</person-group> (<year>2014</year>). <article-title>A Synthesis on Digital Games in Education: What the Research Literature Says from 2000 to 2010</article-title>. <source>Jl. Interactive Learn. Res.</source> <volume>25</volume>, <fpage>263</fpage>&#x2013;<lpage>282</lpage>. </citation>
</ref>
<ref id="B56">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Rolka</surname>
<given-names>E. J.</given-names>
</name>
<name>
<surname>Silverman</surname>
<given-names>M. J.</given-names>
</name>
</person-group> (<year>2015</year>). <article-title>A Systematic Review of Music and Dyslexia</article-title>. <source>The Arts in Psychotherapy</source> <volume>46</volume>, <fpage>24</fpage>&#x2013;<lpage>32</lpage>. <pub-id pub-id-type="doi">10.1016/j.aip.2015.09.002</pub-id> </citation>
</ref>
<ref id="B57">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Rouse</surname>
<given-names>R.</given-names>
</name>
</person-group> (<year>2004</year>). <article-title>Game Design: Theory and Practice</article-title> in <conf-name>Theory and Practice</conf-name>. <edition>Second Edition Second Edition</edition> (<publisher-loc>Plano, TX</publisher-loc>: <publisher-name>Wordware Publishing, Inc.</publisher-name>). </citation>
</ref>
<ref id="B58">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Schulte-K&#xf6;rne</surname>
<given-names>G.</given-names>
</name>
<name>
<surname>Deimel</surname>
<given-names>W.</given-names>
</name>
<name>
<surname>M&#xfc;ller</surname>
<given-names>K.</given-names>
</name>
<name>
<surname>Gutenbrunner</surname>
<given-names>C.</given-names>
</name>
<name>
<surname>Remschmidt</surname>
<given-names>H.</given-names>
</name>
</person-group> (<year>1996</year>). <article-title>Familial Aggregation of Spelling Disability</article-title>. <source>J.&#x20;Child. Psychol. Psychiat</source> <volume>37</volume>, <fpage>817</fpage>&#x2013;<lpage>822</lpage>. <pub-id pub-id-type="doi">10.1111/j.1469-7610.1996.tb01477.x</pub-id> </citation>
</ref>
<ref id="B59">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Schulte-K&#xf6;rne</surname>
<given-names>G.</given-names>
</name>
</person-group> (<year>2010</year>). <article-title>The Prevention, Diagnosis, and Treatment of Dyslexia</article-title>. <source>Deutsches &#x00C4;rzteblatt Int.</source> <volume>107</volume>, <fpage>718</fpage>&#x2013;<lpage>727</lpage>. <pub-id pub-id-type="doi">10.3238/arztebl.2010.0718</pub-id> </citation>
</ref>
<ref id="B60">
<citation citation-type="web">
<collab>Scikit-learn</collab> (<year>2019</year>). <article-title>3.1. Cross-Validation: Evaluating Estimator Performance</article-title>. <comment>Available at: <ext-link ext-link-type="uri" xlink:href="https://scikit-learn.org/stable/modules/cross&#x2d9;validation.html">https://scikit-learn.org/stable/modules/cross&#x2d9;validation.html</ext-link>
</comment>. </citation>
</ref>
<ref id="B61">
<citation citation-type="web">
<collab>Scikit-learn Developers</collab> (<year>2019</year>). <article-title>Scikit-learn Documentation</article-title>. <comment>Available at: <ext-link ext-link-type="uri" xlink:href="https://scikit-learn.org/stable/documentation.html">https://scikit-learn.org/stable/documentation.html</ext-link>
</comment>. </citation>
</ref>
<ref id="B62">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Seaborn</surname>
<given-names>K.</given-names>
</name>
<name>
<surname>Fels</surname>
<given-names>D. I.</given-names>
</name>
</person-group> (<year>2015</year>). <article-title>Gamification in Theory and Action: A Survey</article-title>. <source>Int. J.&#x20;Human-Computer Stud.</source> <volume>74</volume>, <fpage>14</fpage>&#x2013;<lpage>31</lpage>. <pub-id pub-id-type="doi">10.1016/j.ijhcs.2014.09.006</pub-id> </citation>
</ref>
<ref id="B63">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Steinbrink</surname>
<given-names>C.</given-names>
</name>
<name>
<surname>Lachmann</surname>
<given-names>T.</given-names>
</name>
</person-group> (<year>2014</year>). <source>Lese-Rechtschreibst&#xa8;orung (Dyslexia)</source>. <publisher-name>Springer Berlin Heidelberg</publisher-name>. <pub-id pub-id-type="doi">10.1007/978-3-642-41842-6</pub-id>
<article-title>Lese-Rechtschreibst&#xf6;rung</article-title> </citation>
</ref>
<ref id="B64">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Tallal</surname>
<given-names>P.</given-names>
</name>
</person-group> (<year>2004</year>). <article-title>Improving Language and Literacy Is a Matter of Time</article-title>. <source>Nat. Rev. Neurosci.</source> <volume>5</volume>, <fpage>721</fpage>&#x2013;<lpage>728</lpage>. <pub-id pub-id-type="doi">10.1038/nrn1499</pub-id> </citation>
</ref>
<ref id="B72">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Thomas</surname>
<given-names>A.</given-names>
</name>
<name>
<surname>Bader</surname>
<given-names>F.</given-names>
</name>
<name>
<surname>Thomaschewski</surname>
<given-names>J.</given-names>
</name>
<name>
<surname>Rauschenberger</surname>
<given-names>M.</given-names>
</name>
</person-group> (<year>2021</year>). <article-title>Integrating Gamification: The Human-Centered Gamification Process</article-title>. in<conf-name>Proceedings of the 17th International Conference on Web Information Systems and Technologies</conf-name>. <comment>Online, 2021</comment>, <fpage>430</fpage>&#x2013;<lpage>435</lpage>. <pub-id pub-id-type="doi">10.1038/nrn1499</pub-id> </citation>
</ref>
<ref id="B65">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Varma</surname>
<given-names>S.</given-names>
</name>
<name>
<surname>Simon</surname>
<given-names>R.</given-names>
</name>
</person-group> (<year>2006</year>). <article-title>Bias in Error Estimation when Using Cross-Validation for Model Selection</article-title>. <source>BMC Bioinformatics</source> <volume>7</volume>, <fpage>91</fpage>. <pub-id pub-id-type="doi">10.1186/1471-2105-7-91</pub-id> </citation>
</ref>
<ref id="B66">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Vidyasagar</surname>
<given-names>T. R.</given-names>
</name>
<name>
<surname>Pammer</surname>
<given-names>K.</given-names>
</name>
</person-group> (<year>2010</year>). <article-title>Dyslexia: a Deficit in Visuo-Spatial Attention, Not in Phonological Processing</article-title>. <source>Trends Cogn. Sci.</source> <volume>14</volume>, <fpage>57</fpage>&#x2013;<lpage>63</lpage>. <pub-id pub-id-type="doi">10.1016/j.tics.2009.12.003</pub-id> </citation>
</ref>
<ref id="B67">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Weigand</surname>
<given-names>A. C.</given-names>
</name>
<name>
<surname>Lange</surname>
<given-names>D.</given-names>
</name>
<name>
<surname>Rauschenberger</surname>
<given-names>M.</given-names>
</name>
</person-group> (<year>2021</year>). <article-title>How Can Small Data Sets Be Clustered</article-title> in <conf-name>Mensch und Computer 2021}{Workshopband}{Workshop on User-Centered Artificial Intelligence (UCAI &#x2019;21)</conf-name>, <volume>1</volume>. <publisher-loc>Bonn, Germany</publisher-loc>: <publisher-name>Association for Computing Machinery</publisher-name>. <pub-id pub-id-type="doi">10.18420/muc2021-mci-ws02-284</pub-id> </citation>
</ref>
<ref id="B68">
<citation citation-type="journal">
<collab>Wikipedia</collab> (<year>2019</year>). <article-title>Memory (Spiel) (Memory Game)</article-title>. </citation>
</ref>
<ref id="B69">
<citation citation-type="book">
<collab>World Health Organization</collab> (<year>2019</year>). <source>International Classification of Diseases 11th Revision</source>. <publisher-name>World Health Organization</publisher-name>. </citation>
</ref>
<ref id="B70">
<citation citation-type="book">
<collab>World Health Organization</collab> (<year>2010</year>). <source>International Statistical Classification of Diseases and Related Health Problems 10th Revision</source>. <publisher-loc>Geneva, Switzerland</publisher-loc>: <publisher-name>World Health Organization</publisher-name>. </citation>
</ref>
<ref id="B71">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Yuskaitis</surname>
<given-names>C. J.</given-names>
</name>
<name>
<surname>Parviz</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Loui</surname>
<given-names>P.</given-names>
</name>
<name>
<surname>Wan</surname>
<given-names>C. Y.</given-names>
</name>
<name>
<surname>Pearl</surname>
<given-names>P. L.</given-names>
</name>
</person-group> (<year>2015</year>). <article-title>Neural Mechanisms Underlying Musical Pitch Perception and Clinical Applications Including Developmental Dyslexia</article-title>. <source>Curr. Neurol. Neurosci. Rep.</source> <volume>15</volume>, <fpage>51</fpage>. <pub-id pub-id-type="doi">10.1007/s11910-015-0574-9</pub-id> </citation>
</ref>
</ref-list>
</back>
</article>