Citizen Seismology in Taiwan: Development, Outreach, and Formative Assessment of Near-Real Time Earthquake Game Competition Activities

Citizen seismology encourages the public involvement in data collection, analysis, and reporting, and has the potential to greatly improve the emergency response to seismic hazard. We believe in the value of citizen seismology and started with distributing Quake-Catcher Network (QCN) sensors at schools in Taiwan. Unlike most of western countries, Earth Sciences is a required course in junior and senior high schools of Taiwan (Ministry of Education, 2014). Low-cost seismometer is potentially a powerful tool in classrooms to teach earthquake science, which makes school teachers our targeted users. We work closely with school teachers and students to understand how a citizen seismology project can help them to encourage successful engagement. In this study, we establish the achievements gained and problems encountered in different phases of the project since 2013. The main tasks carried out here include (1) refinement of Citizen Seismology Literacy (CSL) into three dimensions: Awareness, and Willingness, and Technology fluency (2) development of an interactive competition platform called the Near-Real Time Earthquake Game Competition (EGCnrt) that allows citizen seismologists to report earthquake information by processing P- and S-wave arrivals, peak ground motion, and first motion of P-waves for every inland ML≥4 earthquake in Taiwan; (3) development of the formative assessment for the 1.5 h long, non-lecture-based game activities. Based on 565 student surveys completed after our 2016 summer outreach activity, we found that all three CSL dimensions have an influence on students’ score in the training activity. The final score in EGCnrt is also found to significantly correlate with the performance in the training sections. We therefore, propose that the CSL can act as a powerful indicator for the performance and engagement in earthquake learning activities. The game-based, non-lecture-based learning approach can be effective in promoting citizen seismology in the future.

HIGHLIGHT -How to motivate the long-term engagement in citizen seismology remains a global challenge. In this paper we review different phases of citizen seismology in Taiwan since 2013. The highlight of the project is the development of a near-real time earthquake game competition. This competition platform allows citizen seismologist to report earthquake information by processing P-and S-wave arrivals (Finding Earthquakes game), peak ground motion (Measuring Earthquake Shaking and Sizing Up Earthquakes games), and first motion of P waves (Measuring How a Fault Moves game). The users are allowed to process the real data, to estimate the location, seismic intensity, magnitude, and focal mechanism of any magnitude greater than 4 earthquake in Taiwan. Through the formative assessment during the outreach activities, we propose a 1.5 h long, game-based, non-lecture-based learning approach that can be effective in promoting citizen seismology in the future.

PROJECT BACKGROUND
In the past decade, citizen seismology has become more popular due to the rise of internet, the development of social media, and the low-cost sensor technologies that allow for building networks of non-scientists for collecting and analyzing seismic data. The successful projects for rapid earthquake information include DYFI (Did You Feel It) developed by U.S. Geological Survey (Wald et al., 2011), multichannel rapid information system comprising websites developed by European Mediterranean Seismological Centre (EMSC) (e.g., Bossu et al., 2008Bossu et al., , 2015Bossu et al., , 2018, and other social networking tools (e.g., Twitter, facebook, Instagram, LinkedIn) that allow the citizens to provide the first-hand accounts of ground conditions for situational awareness and emergence response Guy et al., 2010). For places where the seismic station coverage is sparse, crowdsourcing approaches also play a crucial role in understanding the shaking level of an earthquake. There exist some projects of citizen seismology that aim at distributing low-cost sensors for early warning or data sharing purposes. With a collection of seismic signals from a denser seismic network, they provide accurate and useful seismic signals for scientific investigation and hazard awareness. For example, QCN and Community Seismic Network provide citizens with microelectro-mechanical systems (MEMS) accelerometers that are installed at home and school, connected to the computer, and sending continuous data to the server (Cochran et al., 2009). MyShake makes use of the data from phone motion sensors, to provide early warning of earthquakes (Kong et al., 2016;Allen et al., 2019). P-Alert distributes the MEMS sensors at schools that are equipped with P wave alarm technology (Wu et al., 2013). These sensor-based citizen networks provide precise and real-time information, which allow the science community to collect and analyze data. However, the participation of citizens has been minimal due to several reasons: (1) the devices are black-boxes to the users, (2) the recorded signals are difficult for non-seismologists to read and understand, (3) application does not encourage the instant feedback from users, (4) no routine exercises or activities to attract users' attention. How to motivate citizens to participate is a big challenge.
Taiwan is situated at a complicated plate boundary zone between the Eurasian plate (EP) and the Philippine Sea Plates (PSP), which exhibits a unique interaction between the EP and PSP. In northeast Taiwan, the PSP subducts beneath the rifted Eurasian plate margin along the Ryukyu Trench at a rate of 8 cm/yr to the north-west (Seno, 1977;Yu et al., 1977;Seno et al., 1993), whereas in southwest Taiwan, the Eurasian plate subducts underneath PSP along the Malina trench. As a result, approximately 21,000 earthquakes strike the island with an average rate of 39 ML ≥ 5 events [equivalent to Mw ≥ 4.48 using Mw -ML relation derived from local seismicity by Huang et al. (2000)] every year (in the past 29 years since 1991). In the next 30 years, the probability of Mw ≥ 6.5 earthquakes is predicted to be higher than 87% based on probabilistic seismic hazard analysis (PSHA) (Wang et al., 2016). Note that the moment magnitude is used in PSHA to avoid the ML saturation at a large magnitude. To better prepare Taiwanese citizens for future impacts of seismic hazard, it is important that they understand why earthquakes happen, how they occur, and how to best prepare for an earthquake. Citizen seismology in Taiwan was motivated by the need for volunteers to host QCN sensors . Unlike most western countries, Earth Sciences is a required course in junior and senior high schools of Taiwan (Ministry of Education, 2014). Can QCN be potentially a powerful tool in classrooms for teaching earthquake science? Given that very dense seismic networks in Taiwan (e.g., 5 km spacing for P-Alert network) already existed, our main goal was not to increase the number of sensors. Instead, we work closely with school teachers and students to understand how a citizen seismology project can help them to encourage successful engagement. The key questions are: how to motivate the teachers and their students to interact with the seismograms, how to transform their contribution to useful information, and how to plan a series of activities for a long-term engagement of citizens? In this paper we detail the scopes and tasks since 2013, the result of formative assessment, and the problems encountered in different phases of the project.

PHASE I: QCN NETWORK
Motivated by the collaboration with QCN project (Cochran et al., 2009) for promoting citizen seismology in Asia, the Citizen Seismologists in Taiwan Project (CSTaiwan) was initiated in 2012. The purpose is to build a cloud-based computing service incorporating an earthquake school where the volunteers (teachers and students) can contribute to QCN data collection, analysis, and reporting with the potential of improving the emergency response to earthquakes. Up to 2019, 149 volunteers in total installed QCN sensors with Internet-enabled computers at schools. The population is highest at the places where we had intensive Professional Teacher Development workshops held by Taiwan Rotary club. Figure 1 shows the current QCN site distribution in Taiwan. Most of the stations are located in southwestern Taiwan (Tainan and Chiayi counties). The time evolution from 2012 January to 2016 October of active QCN sensors is shown in Figure 2. System logs indicate that the sensors are on average active ∼50-60% of the time. Note that to keep the QCN project active (i.e., seismic data can be continuously sent to the servers), the local host needs to check "project manager window" of the BOINC server software from time to time. It is a difficult mission for the hosts who are not familiar with the software in the English interface. Another problem is the switching role of local QCN host. In schools, the QCN sensor is usually hosted by the computer/information system manager, which is a temporary position with only a few years in service. Thus from time to time, the QCN hosts lost contact with the administration server, leading to the low "active" rate.
The recorded waveforms for magnitude greater than 4 earthquakes in Taiwan are shared in the platform of School QCN network (Figure 3). The volunteers can view and download the waveforms with Seismic Analysis Code (SAC) format (Goldstein et al., 2003) and further, analyze the seismic signal through the program of Seisgram2K. Continuous signals collected at homes/schools are streamed to Academia Sinica of Taiwan for data archiving. The QCN hosts who make effort to maintain the "active QCN, " often request teachable materials on "what the recorded signal means" and "what can be learned from the signals." That's where the next phase began.

PHASE II: NEAR-REAL TIME EARTHQUAKE GAME COMPETITION AND TEACHING RESOURCES
To make the recorded signals useful in classrooms and to allow the citizen to learn basic seismology, we developed an interactive tool called "the Near-Real Time Earthquake Game Competition (EGCnrt) 1 . This platform was announced along with online teaching resources in Nov. 2014. Within 10 min of a M≥4 earthquake event in Taiwan, the near-real time seismic data from the P-alert strong motion network (Wu et al., 2013 are released and the competition begins. The original purpose of the P-alert network is to provide on-site earthquake early warning and near real-time ground motion intensity measurements. Such island-wide earthquake early warning network could be beneficial for earthquake science learning (e.g., Kong et al., 2016), especially since the P-alert sensors are mostly installed in schools. 1 http://qcntw.earth.sinica.edu.tw/games/competitionV3/index.php The EGCnrt competition platform allows citizen seismologists to report earthquake information by processing P-and S-wave arrivals (Finding Earthquakes game), peak ground motion (Measuring Earthquake Shaking and Sizing Up Earthquakes games), and first motion of P-waves (Measuring How a Fault Moves game). A series of simplified certificate games are designed in a training activity, to familiarize beginners with data processing steps. As shown in Figure 4A, four different processing skills are required in this training stage: (1) Finding the earthquakeby picking P-and S-wave arrivals at more than 3 stations, the epicenter of the earthquake can be found. (2) Measuring earthquake shaking -by picking the maximum amplitude in three components of seismograms, peak ground motion can be measured. (3) Sizing up earthquake -by picking the maximum amplitude in the horizontal component with the previously defined epicenter, earthquake magnitude can be measured. (4) Measuring how a fault moves -by picking initial motion polarities of P wave first arrival (up or down), the fault type (normal, thrust, or strike slip) can be determined. The certificate games corresponding to the above four components require the processing of at least three, five, ten, and 20 stations, respectively ( Figure 4A). Once the certificates are achieved, the citizen seismologists can challenge themselves by taking part in the near-real time competition. As shown in Figure 5, the three components waveforms in each station can be viewed and processed when the users click the red circles in Figure 4B. The processing flow is listed in the left side of Figure 4B as (1) picking P and S arrivals at more than 3 stations (2) measuring FIGURE 3 | Website of School QCN network. Circles represent the recorded earthquakes colored coded by focal depth. The list of earthquakes on the right panel shows the date and magnitude for M L ≥4 events, which allows the volunteers to access the waveforms in each QCN site.
peak ground acceleration (PGA) and magnitude at more than 7 stations (3) determining fault plane solution at more than 10 stations (4) creating the shake map at more than 15 stations. One can click the "locating earthquake" function to determine the hypocenter once three stations are processed. The competition steps are in a different order than the training steps. Here the processing of at least seven and ten stations are needed, to determine earthquake magnitude and fault plane solution, respectively. The score of the near-real time game is determined based on the precision of the location of the epicenter, magnitude, and fault plane solution. The algorithms are detailed in Liang et al. (2016) and also briefly summarized below.
1. Score for "Finding Earthquake": (5 < Score_1 < 20) X = Epicenter difference = (location determined by the user) -(location announced by Central Weather Bureau) Score_1 = −0.6X + 20 if X < 25 km Score_1 = 5, if X ≥ 25 km 2. Score for "Measuring earthquake shaking" Game: (5 < Score_2 < 20) PGA difference Xi = (PGA determined by the user) -(PGA measured by the densest P-alert seismic network) where n is the number of stations processed. 3. Score for "Sizing up earthquake" Game: (5 < Score_3 < 20) Magnitude difference X = (Magnitude determined by the user) -(magnitude announced by Central Weather Bureau) Score_3 = −10X + 20 if X < 1.5 Score_3 = 5 if X ≥ 1.5 4. Score for "Measuring how a fault moves" Game: (5 < Score_4 < 20) Score_4 = 20 if the fault type is the same with the one announced by Real-Time Moment Tensor Monitoring System in Taiwan by Lee et al. (2013) 2 . 5. Additional score for number of stations processed: (5 < Score_5 < 20) X = Number of stations processed Note that the final score is computed and accumulated when the above (1)-(4) are completed. Six teachable units were also prepared, to closely link with different functions in EGCnrt. The summary of teachable units is listed below. (1) Orphan Tsunami: A story of mysterious tsunami in 1700 that stroke Japan along the coastline over a distance 1000 km without an apparent cause. The parent earthquake was found to locate in the western United States and Canada, in a region that was not known to have experienced an earthquake greater than Mw 7.5 in recorded times. Through geological evidence, the students will learn how the magnitude 8 (or higher) earthquake had written its own history. (2) Finding Earthquakes: An introduction to the skills needed for locating earthquakes -picking P-and S-wave arrivals at more than three stations. (3) The 2004 Sumatra Earthquake and Tsunami: A story of how the serious fatalities could happen during a disastrous tsunami, following the 2004 Mw 9.3 earthquake that occurred in northern Sumatra. The students will learn what are the precursory signals before a massive tsunami, how the amplitude of the seismic signal correlates with earthquake magnitude, and what information is required for effective tsunami warning. (4) Sizing Up Earthquakes: An introduction to the skills needed for determining earthquake magnitude and intensity -picking the maximum amplitude on three-component seismograms. (5) Forensic Seismology: A story of how the seismic signals help us to discriminate explosion/mine collapse from tectonic earthquake. (6) Making Fault Motions: An introduction to the skill needed for focal mechanism determination -picking the initial motion polarities of the first arrival of the P waves. These teaching materials are put into a several-mins-long youtube video, to prepare the citizen seismologist for EGCnrt. The original ppt and video clips are available on the website of https://katepili2003. wixsite.com/future-eq-school (in Chinese). Note that since students' level may vary greatly from third to twelfth grade, the educational materials were designed differently to fit the student's background knowledge. This was done during eight seed-teacher workshops where we worked with teachers from elementary, junior, and senior high schools in October 2013 to July 2014.

PHASE III: TEACHER WORKSHOPS AND DEVELOPMENT OF CITIZEN SEISMOLOGY LITERACY (CSL)
Shortly after Phase II, we introduced the teaching materials in six teacher professional development workshops. With 15-80 participants in each workshop, we reached 121 first-to twelfthgrade teachers across the nation in 2014 and 2015. To collect feedback of the participants for future directions of this project, we developed 49 items to evaluate participants' preknowledge, attitude, and skills in earthquake sciences, called Citizen Seismological Literacy (CSL). The CSL is compared with participants' personality categorized into three sub-factors: Conscientiousness (a desire of personality to complete a task well), Intellect/Imagination (the ability of personality to understand or depict abstract ideas) and Grit (perseverance and passion for long-term goals). As detailed in Liang et al. (2017), we found that all three sub-factors in Personality are positively related to CSL dimensions of Attitude and Skills.
This indicates that that Conscientiousness, Intellect/imagination, and Grit may have an effect when developing one's seismology literacy and that the CSL can act as a powerful approach to citizens' learning paths for promoting citizen seismology in the future. Using Exploratory Factor Analysis (EFA), we also found some items in CSL questions are redundant. By removing the items with low factor loadings from the original 49 survey items, 15 items were extracted and re-categorized into new dimensions of Willingness, Awareness, and Technology fluency ( Table 1), referring to the willingness to be educated and trained as contributors, the awareness of earthquakes and disaster prevention, and the self-efficacy on technical proficiency. The reliability can be indicated by Cronbach's alpha that is commonly used to evaluate the internal consistency of the measurements within the same dimension. The value of higher than 0.7 and lower than 0.5 represent acceptable and unacceptable reliability, respectively. Here the alpha is calculated to be 0.756, 0.778, and 0.858 for Willingness, Awareness, and Technology fluency, respectively. The modified CSL model is now embedded in the

CSL dimensions Sample survey items
Awareness I often take part in the community outreach for disaster prevention and preparedness When the earthquake occurs, I attempt to search for earthquake info due to the potential impact from aftershocks and other natural hazards that may come together. The modern technology allows me to upload the recorded seismic data when an earthquake occurs FIGURE 6 | Flowchart of a 1.5 h long EGCnrt program at schools. This program is composed of CSL survey, training activity (four certificate games), and near-real time earthquake game competition. The tutorial in training activity only takes 5 min for introducing the structure and function of the EGCnrt platform.
Frontiers in Earth Science | www.frontiersin.org

Section Items
Basic knowledge (total score = 3) 1. For a given station, when the P-and S-arrivals are picked for, the circle appears on the map. What does the size of the circle mean? (1) magnitude of earthquake (2) distance between earthquake and station (3) how far the seismic energy can travel.
2. In theory, earthquake occurred at the point where three circles intersect. However, which point you would choose in the practice if the intersect looks like the figure below?
3. In this game the circle can be enlarged or reduced through a function called "velocity parameter." This is because the real speed at which a seismic wave travels to the station is different from the default. Presumably, when you increase the velocity parameter you increase the travel speed. In this case the circles (1) become bigger (2) become smaller (3) remain the same.
Advanced knowledge (total score = 3) 4. What is the main difference between earthquake magnitude and intensity (1) The magnitude can only go up to 7.0 (2) Seismic intensity does not have a unit (3) The location of max. intensity always corresponds to the epicenter (4) Magnitude defines how much energy is released, the value remains the same at different places.
5. The max. amplitude of a seismogram tells us the peak ground motion. In the certificate game, you learned that max. amplitude is NOT controlled by (1) travel distance (2) magnitude (3) particle motion of waves.
6. In this certificate game, you determine the magnitude using the relationship between magnitude, distance, and max. amplitude. However, at the different stations you obtained different magnitude due to (1) different seismometer at different stations (2) that the amplitude is not only controlled by travel distance and earthquake magnitude (3) different elevation for different stations.
Comprehensive knowledge (total score = 3) 7. If the fault ruptured to the surface, the direction of fault trace (as a line) relative to North is referred as faults strike. The fault trace could be indicated by two straight lines in the figure -either strike N-S or E-W. When the area is compressed during the fault motion (coinciding with the direction of arrows), the P first motion detected is "upward" motion as illustrated by filled circles. On the other hand, dilation coincides with "downward" first motion. Judging from the direction of motion (arrows), the fault is (1) N-S striking, right-lateral (2) N-S striking, left-lateral.
9. Through stereographic projection, the fault plane (area encircled by red line) can be projected into an equatorial plane (yellow area) as the green line. This green line is a projection of a fault plane with a dip angle of 45 • . If the fault plane has a steep angle of 70 • , the projected line will be located to the (1) right or (2) of the current green line.
EGCnrt game, which allows us to collect and analyze the players' demographical information, pre-knowledge of seismology, and personality self-portraits.
In addition, we found that the most efficient learning happens when the participants are engaged in a competitive environment without a priori introduction of seismological background, indicating that the EGCnrt itself can be treated as self-learning tool. Despite the excellent learning progress through the EGCnrt, the degree of willingness of the participants to use EGCnrt in their own classrooms is found to be low. The feedback gathered in the individual interviews reveals the lack of confidence in teaching the earthquake science behind the game. The barriers for the teachers to make EGCnrt (with six teachable units) useful in the classroom mainly come from the limited teaching hours in Earth Sciences. In the exam-oriented education system of Taiwan, all the teaching hours are divided into five topics including astronomy, oceanography, atmospheric sciences, geology, and geophysics. The teachers often find it impossible to arrange additional hours to run all the teaching units associated with EGCnrt. We therefore changed our target audiences from teachers to students.

PHASE IV: EGCNRT PROGRAM IN SCHOOLS AND THE IMPACT ASSESSMENT OF EARTHQUAKE GAME COMPETITION
The outreach activities called EGCnrt program were initiated by an online registration system that allows the host teachers to sign in and arrange the time for the EGCnrt program in their classrooms. In summer 2016, we visited 12 schools selected by geographical diversity to run the pilot teaching. We reached 406 citizen seismologists (hereafter referred as players) who successfully participated in the game. Based on the experience in the previous workshops with teachers, the outreach activities were designed to facilitate the learning and thinking. As illustrated by the flowchart in Figure 6, the students are assigned to fill *Correlation is significant at the 0.05 level (2-tailed). **Correlation is significant at the 0.01 level (2-tailed). In the total number of 425 samples, 14 and 19 went missing due to incorrect coordination with EGCnrt activity and training activity, respectively.
FIGURE 7 | The relationship between training score and EGCnrt game performance. The number of students belonging to groups of low, medium, and high performance in EGCnrt is 230, 112, 69, respectively. In the total number of 425 samples, 14 went missing due to incorrect coordination. The number within each horizontal bar represent the coefficient of variation (the ratio of standard deviation to the mean).
out the CSL survey before entering EGCnrt platform. The 1.5 h long EGCnrt program is composed of half an hour training activity and 1 h EGCnrt activity, as detailed in sections Training Activity (Certificate Game) and EGCnrt Activity.

Training Activity (Certificate Game)
The training activity requires the students to complete the four levels of certificates without lecture. A 5 min tutorial allows the students to (1) login to the system, (2) briefly understand the structure of this platform, and (3) learn how to view and process the seismic signals. Once they successfully got the certificates, nine questions will appear on the screen for bridging the game to some basic earthquake knowledge. Different from CSL, these questions are strongly tied with the knowledge behind the game. They can be classified into three sections: basic, advanced, and comprehensive knowledge. The three questions in each section are listed in Table 2. When the three questions are completed after each section, the correct answers will be displayed on the screen. The section of basic knowledge (S Basic ) refers to the recognition of the P and S first arrivals and their association with earthquake location, which is listed in high school curriculum. The section of advanced knowledge (S Adv ) refers to an understanding of the relationship between maximum amplitude of seismogram and seismic intensity/earthquake magnitude. Although not listed in the high school curriculum, this concept introduced in this section is straightforward. The section of comprehensive knowledge (S Comp ) refers to the determination of fault plane solution using first motion of seismic waves, which requires an integration of seismic source and slip on the fault. It is a section bringing a new concept associating P wave radiation pattern and stereographic projection. The total score in this training activity (S train ) is 9, as S Train = S Basic + S Adv + S Com . Our experience shows that during the activity, getting the certificates requires only learning a specific skill or set of steps rather than requiring a deeper understanding. The knowledge behind each skill is only introduced in the pop-out questions that requires critical thinking. From our observation, in each classroom there exists only a handful of students who asked questions associated with logical connection between the skill and knowledge. These active learners can be identified during the activity.

EGCnrt Activity
The following 1 h EGCnrt activity requires the skills learned in the training stage to compete with each other using real data. The total score of up to 100 (S Game ) is determined by the precision of the location of the epicenter, magnitude, and fault-plane solution. By examining the statistical significance of their correlation, we used Pearson Correlation to measure the linear association of two variables and k-mean clustering analysis to compare the difference between grouped variables. Table 3 show the result of Pearson Correlation. We found that S Game is better correlated with S Adv and S Comp than with S Basic , as revealed by r = 0.098, p = 0.049 for S Adv , r = 0.11, p = 0.027 for S Comp , and p > 0.05 for S Basic . Here r and p represent Pearson correlation and statistical significance (pvalue, > 0.05 means no significant correlation). This suggests that the knowledge gained during the training activity reflect their performance in EGCnrt activity. The performance in EGCnrt activity is also categorized into three groups as low (S Game = 0), medium (0 < S Game < 42), and high scores (S Game > 62) using k-means method, to compare with the scores in training activity. The k-means clustering method is commonly used due to its simplicity (MacQueen, 1967), which minimizes the distance between the cluster center and each point in the data set. As illustrated in Figure 7, it appears that the high score group in EGCnrt shows highest score in the training activity, which is consistent with the observation in Table 3. The increasing rate FIGURE 8 | The relationship between the training scores and CSL. Low, medium, and high CSL is now divided into three dimensions (Technology fluency, awareness, and willingness), to correlate with training score. The number of samples that belong to groups of low, medium, and high CSL is 38, 177, and 210, respectively. In the bottom, the numbers in the first row (within the brackets) denote the clusters center for individual CSL group using k-means. The numbers in the second row denote the range of coefficient of variation in Sbasic, Sadv, and Scomp. tends to be highest in advanced knowledge (Figure 7), which could be due to the fact that the skill learned for S Adv is heavily used in the EGCnrt activity. Note that the S Game is computed and accumulated once enough of stations are processed to obtain the basic earthquake information of location, magnitude, and fault plane solution (minimum requirement is 15 stations). From our observation, the large number of S Game = 0 happened when the students failed to stay focused on the EGCnrt activity. The coefficient of variation (cov) is a measurement of precision, smaller cov leads to greater precision. The cov of greater than 1 is considered to be high variance. In Figure 7, the largest cov (up to 0.82) appears in low game performance group, suggesting a less precise measure comparing with high and medium S Game group. Table 3 also shows the impact of EGCnrt on change in knowledge, attitude, and skill in earthquake sciences for students, as represented by CSL. The large p-value (0.167-0.827) indicates that the CSL does not reveal significant correlation with S Game and S train . When the CSL is categorized into three levels from low, medium, and high for computing the corresponding S Game in Table 4, we found that the low CSL group tends to have lower S Game comparing with high and medium CSL groups. If comparing with the training score using the same CSL groups, the medium and high CSL also tend to show higher training scores, while the difference between medium and high CSL appears to be very small (Figure 8). This indicates that the correlation between scores in the EGCnrt activity (S Game and S train ) and CSL exists but is unlikely to follow a linear relation. Among three training sections, S Comp and S Basic reveal strong dependency with CSL level, as the training score denoted by blue and gray symbols significantly increases from low CSL to medium/high CSL. S Adv , however, does not show such a tendency, implying that the core knowledge in S Adv may not be discriminable between different level of CSL groups. The effect of different CSL dimensions on the performance in training activity can be also seen in Figure 8. We found that when learners' technology fluency and awareness increases, their performance and engagement in basic and comprehensive knowledge increases. The player's willingness may not play an important role in the performance in the training activity, as the training score stands high even for lowwillingness. In summary, we found that CSL could be regarded as an important indicator that reflects the students' learning efficiency in seismology. Compared to willingness, technology fluency and awareness appear to play a more important role in the performance of training activity. The advanced knowledge gained during the training activity significantly correlates with the performance in EGCnrt activity, but no correlation with CSL scale is evident. This may be due to the fact that the advanced knowledge is relatively easy, so that even the low CSL group has fairly high training score. The coefficient of variation is found to be large (up to 0.97) in Figure 8, as a result of large cov in Scomp (gray triangles in low CSL group). Comparing with the dispersion in Sgame vs. Strain (Figure 7), the dispersion in CSL vs. Sgame (Table 4) and CSL vs. Strain (Figure 8) tends to be relatively large (up to > 0.90). This indicates that the association between CSL and training score/game performance is not as strong as the association between training score and game performance.

DISCUSSION AND CONCLUSION
By attempting to distribute the QCN in schools, the project of "Citizen Seismology in Taiwan" started with developing a nearreal time earthquake game competition, followed by preparing the corresponding teachable units, and operating a series of FIGURE 10 | Location of the M L > 4 inland earthquakes in the earthquake game competition (the same events listed in Figure 9). White stars indicate events with magnitude greater than 4 but smaller than 6. Red stars indicate events with magnitude greater than 6. outreach activities with formative assessment. The purposes are to encourage volunteers to interact with the real earthquake data and improve learning outcomes in earthquake science. Through professional development workshops with teachers, we found that teachers tend to lack confidence to run EGCnrt in their own classrooms. The barriers are mainly from the limited teaching hours in Earth Sciences (required course). In the examoriented education system in Taiwan, the teaching hours in high school are split into the five core knowledge units of astronomy, oceanography, atmospheric sciences, geology, and geophysics. The teachers often find it impossible to arrange additional hours to run all the teaching units associated with EGCnrt. Therefore, other than collaborating with primary and secondary educators (Liang et al., 2017), we also conducted pilot teaching in high school classrooms. Up to 2019, the total number of players reached 1319 since the EGCnrt was first online in late 2014. The time evolution of the number of players for all M L ≥4 earthquakes on inland regions of Taiwan is shown in Figure 9. The corresponding location of earthquakes is shown in Figure 10. The largest number of players, 565, is reached for the 2016 ML6.4 Meinong earthquake, which is reinforced by a series of pilot courses intensely conducted during the summer of 2016 (as detailed in section 5), several months after the Meinong mainshock. Although the top-three players are announced real time on the EGCnrt platform and rewarded, only 23 out of 425 players who joined the 2016 summer outreach activities came back to the EGCnrt, to challenge with more earthquake data. These 23 players have an average score of 2, 1.96, and 1.96 for S Basic , S Adv and S Comp , respectively, which are classified into the "high" performance in training activity illustrated by Figure 7. This again confirms the strong connection between the performance in training and EGCnrt activities. However, so far the number of players who returned to EGCnrt (outside the classroom) is too small to show a statistically meaningful trend in their CSL scale. In the future, more outreach activities are needed to investigate the personal characteristics for this group of volunteers. It is worthwhile to further study whether the long-term engagement is associated to the attributes proposed for CSL (e.g., awareness technology fluency, and willingness). To increase the returnrate and facilitate the long-term engagement, follow-up activity is necessary (Figure 6). We strongly encourage the host teachers/schools to run a "earthquake hunter" competition. For example, hunt for a major earthquake archived in the EGCnrt platform to report how much we know about this event. The nationwide competition with a strategic reward system can be expected as well to raise the public awareness to the EGCnrt platform.
Different from the traditional classroom, the EGCnrt program allows that students interact with each other across the room by discussing the skills needed in the activities and helping each other to get the certificates in training stage. From our observation, the most active discussion occurred when the questions appear on the screen after completing the certificate game(s) in each section. The students often asked around for the meaning of the questions and their connection with the certificate game(s). While the EGCnrt scores and ranking are shown online, a fun and competitive atmosphere can be naturally formed where the players attempt to outdo others and put each other into good humor. With such competition and lighter moments in class, the concentration in EGCnrt usually lasted longer than 30 mins. The host teachers were encouraged to participate in the activity as the observer. The interviews with the host teachers were conducted during and after the activity to learn (1) how do they teach earthquake science in the classroom, (2) what is the general behavior of students toward learning, (3) what is the problem encountered in teaching earthquake science, (4) how do the students behave differently when the competition platform is introduced in the classroom. Based on their observations, those players who showed excellent performance in EGCnrt (i.e., frequently helped others and attempted to understand the meaning of each game) are found to generally achieve low academic grades. Such an observation implies that the EGCnrt program has potential to foster learning and behavioral changes for the students who are less motivated in a traditional instruction classroom. In fact, mounting evidence has supported that the game-based elements in learning activities improve the learners' motivation and engagement in educational environment (Dichev and Dicheva, 2017;Chen et al., 2019;Herrera et al., 2019). In this study, we present a game-based approach that uses real seismic data on an online digital platform, to extract information from students and assess the learning outcomes. This approach is composed of (1) preclass Citizen Seismology Literacy (CSL) survey (2) half hour training activity with formative assessment (3) 1 h near-real time earthquake game competition (EGCnrt). Through statistical analysis, the above three elements are found to correlate with each other. We found that students with high CSL tend to have higher scores in game performance and training activity (especially for basic and comprehensive knowledge), however, such correlation does not follow a linear pattern. The score in the training activity is found to be more strongly correlated with performance in the EGCnrt, whereby the most significant factor is advanced knowledge. Such a result suggests that the CSL survey has potential to act as an indicator for the learning outcomes in seismology, while training program may efficiently improve the students' performance in EGCnrt activity. In the near future, it is worthwhile to develop more dedicated CSL dimensions that might better depict the long-term engagement in citizen seismology throughout the programs and activities we offer.

DATA AVAILABILITY STATEMENT
The datasets generated for this study are available on request to the corresponding author.

ETHICS STATEMENT
Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements (Center for Research Ethics and Research Ethics Committee in National Taiwan Normal University). The written consent was not required to participate in this study in accordance with the national legislation and the institutional requirement.

AUTHOR CONTRIBUTIONS
W-TL developed the platform of near-real time earthquake game competition (EGCnrt) and maintained the QCN in Taiwan, while LW developed the Citizen Seismology Literacy. C-HL carried out the formative assessment. KC conducted the EGCnrt outreach activities in high schools and wrote the manuscript.