Knowledge, perceptions, and behavioral responses to earthquake early warning in Aotearoa New Zealand

them, what actions they took in response to the warning, and their overall judgment of the system including its usefulness. We undertook surveys following two widely received alerts from the Android Earthquake Alert System to assess public knowledge, perceptions, and responses to these alerts with a total sample size of 3,150. Results: While most participants who received the alert found it useful, knowledge of both EEW generally and the Android System speciﬁcally was low and few participants used the time to protect themselves from shaking. Discussion: These ﬁndings reiterate the importance of education and communication around a warning system, so that the public know how to act when they receive an alert.


Introduction
Earthquake early warning (EEW) systems have been active in some parts of the world for decades; these systems typically use sensors to detect the primary earthquake waves (which move faster but tend to be non-damaging) and then send an alert to users before the dangerous secondary waves (which tend to cause the damage) reach them.While these systems have potential benefits, they also have considerable costs (Strauss and Allen, 2016).Recent technological advancements mean these systems are becoming more common and reaching a larger number of people.It is therefore important to explore how people respond to these warnings.In particular, given the speed with which some systems have been rolled out, including little to no public education in some instances, it is especially valuable to understand how people respond to EEW systems with which they are not familiar.To this end, the recent rollout of the Android Earthquake Alert System in Aotearoa New Zealand, a highly-seismic country but one without a national, public earthquake early warning system, provided an opportunity to address this question.

Aotearoa New Zealand's earthquake context
Aotearoa New Zealand (NZ) experiences over 20,000 earthquakes a year, with an average of 360 earthquakes between M4.0 to 4.9 and 30 between M5.0 and 5.9 (GeoNet, 2022).Several damaging earthquakes have occurred recently, including the Canterbury Earthquake Sequence (CES) with large events in 2010 and 2011 and the 2016 Kaikōura earthquake.With the exception of the 2011 Christchurch event, earthquakes in NZ tend to cause few fatalities.However, around 15,000 people were injured in earthquakes in the short timeframe between 2010 and 2014 (Basharati et al., 2020).Many of the injuries seen in NZ could be prevented by people undertaking appropriate protective actions (Horspool et al., 2020) such as "Drop, cover, and hold" (DCH), which is taught in the annual ShakeOut earthquake drill.Although people who participate in this drill are more likely to know and use DCH (Vinnell et al., 2020), use of these actions is still relatively low (e.g., Johnston et al., 2014;Lambie et al., 2017;Vinnell et al., 2022b), an issue not just in NZ (Bernardini et al., 2019).Many people in NZ report that they wait to see if an earthquake will continue or get stronger before deciding to take protective action (Vinnell et al., 2022b).Part of the problem might be a normalization bias (Solberg et al., 2010) where people assume that the current earthquake will be similar to ones they have felt before (i.e., not dangerous, for most people).Given that people tend to be slow to take protective actions, if they take them at all, even when they feel actual earthquake shaking, there is a significant question whether a warning will be able to override these existing barriers to behavior.
The state of earthquake early warning in NZ NZ does not currently have an official, national EEW system.Some private in-house EEW services exist in NZ but these are not available to the public (Becker et al., 2020a;Prasanna et al., 2022).Similar to other countries which experience frequent damaging earthquakes such as Italy (Ladina et al., 2021), work is underway to assess the feasibility of a national EEW system in NZ.This includes technological research, for example on the feasibility of a low-cost EEW system (Prasanna et al., 2022), as well as social science research exploring the public expectations and appetite for a national EEW system in NZ.There is general support for EEW among both the NZ public (Becker et al., 2020b) and sectors including emergency management, health, critical infrastructure, and engineering (Becker et al., 2020a).People perceive benefits to physical and emotional safety, plan activation and assessment, and organization specific actions.However, this work highlighted the need to consider people's confidence and trust in the system, where to set the warning threshold, and what content to include in public messaging.Overall, the possibility of a national, official EEW system in NZ is complex and work is ongoing to calculate the cost to benefit ratio, from both a financial and a social perspective.
Currently, there is no legislation in NZ covering responsibility for earthquake warning, as previously there was no existing means for a reliable warning system (Becker et al., 2020a).GeoNet is a science and government collaboration responsible for earthquake monitoring, providing real-time information when an earthquake occurs but not EEW.The National Emergency Management Agency (NEMA) of NZ is responsible for providing national level warning and advisories to local civil defense and emergency management groups, central and local authorities, emergency services, lifeline utilities, and broadcasters (National Civil Defence Emergency Management Plan Order 2015, 2022); however, this mandate does not include earthquakes.The local civil defense emergency management groups then have the responsibility to distribute alerts to their respective communities.Moreover, NEMA oversees the function of the National Warning System in NZ, which can be used to issue national warnings about hazards (National Civil Defence Emergency Management Plan Order 2015, 2022).Monitoring agencies GNS Science (geological survey) and the MetService (meteorological agency) also provide public alerting with regards to their respective hazards (Wright et al., 2015;Becker et al., 2020a).
In April of 2021, Google rolled out its Android Earthquake Alert System (AEA; Google Crisis Response, n.d.; Voosen, 2021;Allen and Stogaitis, 2022) in NZ.This feature uses the accelerometers in Android phones to detect earthquakes.Alerts are then sent to those who are expected to experience weak or light (Modified Mercalli Index III to IV) shaking ("Be Aware" alert; Figure 1) or stronger (MMI V to X) ("Take Action" alert; Figure 1).In the former case, a notification is displayed on the screen of the mobile phone similar to other applications or message notifications and settings such as silent mode and Do Not Disturb are not overridden.In the latter case, the full screen is taken over by the warning, which displays the DCH recommendation, and the phone makes a unique sound regardless of settings.Google anticipated sending a few Be Aware alerts each year; in the last 7 months of 2021, at least five alerts were sent in the Wellington Region of NZ alone.While it is impossible to know how many people in NZ have at least one Android phone, an estimated 92% of New Zealanders have a smartphone and approximately 55% of the smartphones in NZ used the Android mobile operating system (Statista, 2023); it is therefore likely that about half of the NZ population do, and do not, have access to the alerts.

Behavioral responses to warnings
Response to earthquake warnings has been studied previously, although before EEW these studies looked at aspects like aftershock "warnings", which are more similar to forecasts than an EEW (Mileti and O'Brien, 1992;Doyle et al., 2020).Previous research has explored how people respond to warnings for hazards such as nuclear attacks (Wood et al., 2018), tsunami (Sutton et al., 2018), and tornadoes (Demuth et al., 2022).While these hazards differ from EEW in terms of the time people have to respond, this research has highlighted several considerations relevant to the earthquake context.For example, many people who feel an earthquake will seek further information before undertaking actions such as evacuating the building (Gu et al., 2016) or moving uphill/inland in case of tsunami (Vinnell et al., 2022a).Warning response times can therefore be decreased by providing more information in the warning itself (Sutton et al., 2018;Wood et al., 2018).Other key considerations include the design of the message ./fcomm. .

FIGURE
Examples of Be Aware and Take Action alerts.Note: The "Earthquake demo" can be shown on an Android device at any time (assuming the user is aware of its existence and how to find it in the settings menu).

FIGURE
Location of earthquakes included in this study.Images from GeoNet ( a; b) showing the epicenter of the two earthquakes and the location of people who reported feeling the shaking (darker colors indicate stronger reported shaking).Date, time, magnitude, and felt report captions added by the authors.(Dallo et al., 2022b), the process people undergo between receiving a warning and acting on it (understanding, believing, personalizing, deciding), and the type of information which should be included (hazard, impacts, threat, actions, location, time, message source etc.; Wood et al., 2018).
Recent research in the tornado context found that rather than people not responding because they were complacent or had had too many false alerts, the main reason was lack of efficacy (e.g., nowhere safe to shelter; Demuth et al., 2022).It is clearly important not to assume why people do or not do respond to warnings, and what influences how they choose to respond.Many recommendations for effective warning were not developed in earthquake contexts, where the time between warning and action is vastly shorter than many hazards.It is unclear, therefore, which if any of the recommendations are useful for an EEW context.

Behavioral responses to earthquake early warning
The exploration of people's responses to EEW is still new (Bossu et al., 2021) and often based on anecdotes (Cochran and Husker, 2019), so the effects of EEW are not yet well known (Fallou et al., 2022;Vaiciulyte et al., 2022).The first system was in Mexico in 1991, followed by Japan in 2007, Taiwan and South Korea in 2018, and ShakeAlert in some states of the US in 2019 (Allen and Stogaitis, 2022).Some systems are public and governmentrun, while others are independent, private, and/or commercial with varied features.For example, the Earthquake Network initiative (Bossu et al., 2021;Fallou et al., 2022) is independent, but is optin, where the public can download an app to receive alerts unlike the AEA System which is opt-out.These differences highlight the importance of considering not just the alerts themselves, but decisions around their implementation (e.g., opt-in or optout), the technical environment (e.g., smartphone ownership for systems reliant on this platform), the social environment (e.g., public knowledge of early warning, levels of earthquake experience, desired and expected behavioral response), and implications of these considerations.
With any EEW system, it is important to find the balance between what the technology can do with what will actually help (Allen and Stogaitis, 2022).For example, a study using a simulated EEW found that icons (i.e., images showing the recommended protective action) had little effect on the ability to undertake those actions (Sutton et al., 2020) while another study found that behavioral images increased intention to act (Dallo et al., 2022a,b).A further study found that a warning time is necessary for ideal response (Santos-Reyes, 2019).Dallo et al. (2022a), in a study of the Swiss public, found that the message designs preferred by their participants were not necessarily the most effective designs for prompting action, highlighting the importance of evidence and scientific evaluation of EEW messages rather than relying on positive sentiment from the public or intuitive ideas of what will be effective based on user preferences.
Evidence of the effectiveness of EEW is mixed, with an unclear impact on reducing death and severe injury (Huggins et al., 2021).For example, only a quarter of people in Peru who received alerts before a recent earthquake took protective action (Bossu et al., 2021).Similarly, most people in Japan tend to do nothing (54%) or very little (e.g., stopping: 21%) in response to an EEW (Nakayachi et al., 2019).Key to the success of any technological intervention designed for societal use and benefit is communication.For example, people in Mexico tend to be tolerant of false alerts because they are seen as good practice (Allen et al., 2018).New Zealanders who participated in EEW scoping research similarly indicated that they would be tolerant of false and missed alerts if there was communication about why those errors had occurred (Becker et al., 2020a;Brown et al., 2021), reflecting international experience of the importance of post-alert messaging (McBride et al., 2020).This is not currently included in the AEA System.
Vaiciulyte et al. (2022) found little difference in reactions between those in Mexico who did and did not get a warning, demonstrating the need to educate about what to do in response to a warning, but also to continue efforts to teach people how to respond when an earthquake starts without a warning.EEW, therefore, can be additional to, but not a replacement for, existing educational measures.As well as existing education, there needs to be thorough and clear communication about EEW systems specifically.This was well acknowledged in Japan, where a detailed preparatory process was followed to incrementally introduce the system and EEW principles, processes, limitations, and desired responses were communicated to the public (Kamigaichi et al., 2009).

NZ earthquake early warning research
A 2020 workshop with a community-of-practice of NZ-based EEW researchers, experts, and potential institutional users highlighted the need for a collaborative framework for research and practice (Tan et al., 2021b).The results of the workshop showed that, for the community-of-practice, the success of EEW in NZ will not be based solely on its technical feasibility but must consider whether end-users (e.g., the public) will use the system effectively.Technical research has shown that a cost-effective system for NZ is possible through the integration of low-cost sensors and innovative architecture: for example, an experimental participatory sensor network in Wellington (Prasanna et al., 2022).However, despite technical feasibility, for a system to be ready to issue public alerts, it must overcome several challenges.A sectoral analysis highlighted social considerations of implementing EEW, including concerns for users' confidence in the EEW system and the comprehension of the alert that would lead to people taking appropriate or inappropriate action (Becker et al., 2020a).For a public-facing EEW system to be successful, the public should be able to comprehend and act on the information that is given out (Tan et al., 2021b).
A 2019 survey of over 3,000 New Zealanders explored expectations of, preferences for, and intended responses to a hypothetical EEW, before the Android system launched (Becker et al., 2020b).The vast majority of people (97%) thought an EEW would be useful or somewhat useful, although it is unlikely that people were aware of limitations of the system such as false and missed alert rates.Most (83%) thought a warning would be useful for mentally preparing and that they thought the threshold for a warning should be moderate shaking (MMI V) or stronger; this is similar to Japan but higher than the US (Cochran and Husker, 2019) and higher than the threshold for Google's "Be Aware" alerts (MMI III to IV).Large proportions of participants wanted information about actions to take, additional geohazards, and earthquake characteristics, suggesting that they were overestimating the amount of time they would have to act, underestimating how long it would take them to process the warning, or both.This study also found that people's intentions to undertake various actions differed based on their experiences in previous earthquakes as well as where in NZ they live, suggesting that the same information or system may not be used in the same way (Becker et al., 2022).
Since the AEA System rolled out in April 2021, multiple alerts have been sent.Few earthquakes have been centered near populated areas, so the number of people who have received the "Take Action" alerts is limited.However, a large number of people have received "Be Aware" alerts (i.e., for expected weak or light shaking, MMI III to IV).Little to no public communication has been undertaken around this system.Only a few media reports covered the introduction of the system (Guesgen, 2021;McDonald, 2021) or briefly commented on the alerts after they had been sent (Hall, 2021;Kilpatrick, 2021), including limited information about how the system works and no information on how the public should react.This situation, therefore, presented a useful context to explore how people respond to a novel EEW without prior education or communication by the provider.We have kept this study as similar as possible to that of Becker et al. (2020b) which asked New Zealanders how they think they would react to an EEW when a public system was hypothetical; this means we can roughly compare what people thought they would do and what they actually did in response to these Android EEWs.

Particularities of the Aotearoa New Zealand context
One of the challenges to developing a successful EEW system (i.e., one which produces more benefit than the cost incurred) is the contextual variations between areas of the world prone to severe seismic impacts.Many systems use historical earthquakes to determine alerting thresholds, balancing the need to reduce both unnecessary alerts and missed alerts (Saunders et al., 2022).This highlights the importance of considering the unique geological and paleoseismological history of the area in which the system is being used, along with other factors including site conditions to assess the expected frequency, accuracy, and consistency of performance (Minson et al., 2022).Performance of an EEW might differ in NZ based on current aspects of the population and seismic profile.For example, most of the population is within a small number of cities and many earthquakes are epicentred offshore (Voosen, 2021).
Further, public preferences for alerting thresholds differ.Public reaction to alerts not being sent in the 2019 Ridgecrest, US earthquake (in areas where EEW was available) led to the intensity threshold being lowered (Cochran and Husker, 2019).However, evidence suggests that New Zealanders want a higher threshold for alerts (Becker et al., 2020b) than that used by US-developed systems such as Google's AEA.
Other aspects of EEW which might vary between contexts include recommendations for protective actions, which should take into account social, cultural, and demographic factors as well as previous earthquake experiences of the public (Becker et al., 2022;McBride et al., 2022).For example, people in Canterbury who experienced an extended earthquake sequence, including highly damaging aftershocks, were more likely to intend to use an EEW to mentally prepare than people in other regions of NZ (Becker et al., 2022).
A further consideration is the likelihood of aftershocks.In the 2019 Ridgecrest event, many alerts were sent for aftershocks (although one Mw 7.1 main shock was missed; Chung et al., 2020).Particularly if damage did not occur in the mainshock, normalization bias raises another issue of how people might respond to an EEW in an ongoing earthquake sequence (Mileti and O'Brien, 1992).Authors have established that information needs change during extended aftershock sequences (Wein et al., 2016;Becker et al., 2019).

Research questions
This study is the first to explore how New Zealanders react to an actual earthquake early warning.It is important to explore a range of responses to newly introduced EEWs, including what actions they took in response to the warning, how much people know about them, and their overall judgment of the system including its usefulness.The primary aim of this research is to explore the actions people in NZ take when they receive an EEW.The novelty of the system at the time of the two alerts considered in this study (October 2021) means that this research demonstrates the responses of a public largely unaware of what EEW is, how it works, and how they should react.The findings of this study should, therefore, ideally represent a baseline to identify improvement over time as more people become aware of EEWs and key areas for education.
RQ1a: How do people respond to an EEW in NZ? RQ1b: What are common reasons for lack of response to an EEW?
We were also interested in the level of awareness and knowledge about EEW held by the NZ public, given the relative novelty of not just EEW systems generally but the Android alerting system studied here.
RQ2a: How much do people in NZ (think they) know about EEW? RQ2b: Who do New Zealanders think is responsible for sending the EEW they received?
Finally, given the novelty of this system, we explored perceptions of usefulness, preferences for delivery channels, and preferences for information included in the message.
RQ3a: How useful did people find the EEW that they received?RQ3b: What delivery channels are preferred for sending EEW? RQ3c: What information would people like to receive in an EEW?
This study was designed to be as similar as possible to the previous study conducted by Becker et al. (2020b) which used a hypothetical context (i.e., before any EEW system was operating in NZ).We were therefore also able to explore to what extent the opinions and intentions expressed regarding a hypothetical system reflect the opinions and behavior demonstrated in response to an actual EEW.

Method Design
This study used a cross-sectional design.Data was collected at two time points, following two separate alerts on the 12th of October (M5.3, 2.55 pm) and the 22nd of October (M5.9, 10.58 pm), 2021 (Figure 2).While we do not have access to data from Android regarding how many alerts were, or should have been, sent, approximately one million people live in the area shown to be covered in the second alert (slightly fewer for the first alert due to the lower magnitude).The intention of this study was to examine how knowledge and behavior changed between the first and second alert.However, given that the latter alert was received over a much larger geographical area, the population from which the samples were drawn were too different for statistical comparison (particularly in terms of previous experience with both earthquakes and EEW, such that it would be difficult to determine if any differences were due to population-level changes between the two events, pre-existing differences between the two populations from which the samples were drawn, or a combination of both).Instead, both datasets are presented concurrently.Further, comparisons are made to the previous survey of Becker et al. (2020b) but given the unavoidable differences in questions resulting from adapting from a hypothetical to a real-world context, these comparisons are descriptive rather than inferential.

Materials
As stated above, the questionnaire used in this study was based on that used in Becker et al. (2020b).Additional questions were added to assess the participants' experience of the shaking, including intensity and duration.All questions reported here were multiple choice (rather than open text) with some allowing multiple selections.The survey was pilot tested by a small group of people within and beyond the project team.However, analysis of the findings from the first public survey identified some minor improvements which were made for the second survey, such as adding a question to check that participants have an Android phone and the addition of a "Not Applicable" answer to the question assessing behavioral responses to the alert.
Participants were asked: -who they thought was responsible for delivering the earthquake alert, -how much they knew about earthquake early warning and Google Alerts prior to the recent alert, -if they received an alert, -if they felt shaking, -how strong that shaking was, -how much time there was between receiving the alert and feeling shaking, -what they did in response to the warning, -why they did not do anything in response (if they answered "Nothing" to the previous question), -how useful they found the warning, -why they found the warning useful/useless (depending on their answer for the previous question), -how they felt if they received the warning but did not feel shaking (i.e., a false alert), -how they would prefer to receive an earthquake early warning, and -what information they would like to have in an earthquake early warning message.
Response options varied between the questions, so to avoid repetition are presented in the following results section when and as relevant.

Procedure
The link to the Qualtrics survey was shared on social media by the research team and emergency management-related organizations and posted in geographically-focused online groups (e.g., Facebook community noticeboards) to maximize the spread of participants over the alerting area.The first survey ran from the 18th of October until the 22nd.This survey was closed after a second earthquake on the 22nd (unrelated to the one on the 12th of October) occurred and triggered a widespread EEW, to avoid confusion about the warning being assessed.The survey assessing this second earthquake ran from the 23rd of October until the 31st of October.The data collection method for this study received peer-reviewed approval under Massey University's code of ethical conduct for "low risk" research, teaching, and evaluations involving human participants (Application ID 4000025159).

Participants th October survey
Two participants indicated that they did not consent, three were under 18, and 30 did not answer either the consent or age check questions.All 35 of these participants were not presented with the actual survey questions and were excluded from the dataset.A further 69 participants who did not answer any questions beyond the consent and age check were also excluded.This left 1,087 participants who were included in the analyzed data for this survey.Of these, a majority received the alert (73%) and felt shaking (79%).Most were women (66%) and 21-39 years old (62%), and Māori (New Zealand's indigenous people) were underrepresented (7.5% compared to a national proportion of 17%; StatsNZ, 2021).

nd October survey
Given the wider distribution of the alert, more participants responded to the second survey.In total 246 participants were excluded from the dataset (for the reasons above, or not having an Android phone).These exclusions left 2,063 included in the dataset.Again, most felt shaking (73%) but a smaller proportion received an alert (58%).A larger majority of this sample were women (78%), participants were older (49% between 30 and 49 years old), and Māori were better represented at 12.2% (though still lower than the population proportion).
Across both samples, those who did not receive an alert were retained in the survey, so that we could look for differences in message preferences (including delivery channel as well as provided information).Questions that relied on people having received the alert (e.g., behavioral response questions) only included the relevant participants.

Earthquake characteristics
In both studies, the majority of people who felt the earthquake reported weak (12th: 24.6%; 22nd: 20.8%) or light shaking (62.6%; 58.3%; Figure 3).Most participants reported between 0 and 20 s of warning, with up to one fifth receiving the alert after shaking had already started (Figure 4).

Research question : behavioral responses
RQ1a: How do people respond to an EEW in NZ?
Figure 5 shows the frequency of different behaviors undertaken in response to the alert.Participants were able to indicate multiple actions.
Few people undertook protective actions such as drop, cover, and hold or other safety actions such as turning off gas appliances.Few people stopped their car, although this mostly reflects the low number who were driving at the time of getting the alert (1.7% and 2.4%).The most common actions included looking for information and telling others.Consistent with how people intended to act in response to a hypothetical EEW, many people took the time to mentally prepare themselves for the shaking.Many others stopped and waited for the shaking to start, or did nothing in response to the alert.
The main difference in the "Other" responses between the 12th and 22nd surveys was due to over half of the participants in the former saying they did not receive or see the alert until after the shaking; the Not Applicable (warning after earthquake) option was added for the latter survey.Across the two surveys, the next most common "Other" responses were feeling confused, not knowing what the alert was, or not having time to react.These responses are not strictly answering the question, suggesting changes for future surveys to clarify what is being asked (i.e., what people did, not why).

RQ1b: What are common reasons for lack of response to an EEW?
Participants who indicated that they did nothing in response to the EEW were asked to provide reasons for not acting (Figure 6).In both surveys, common reasons included not having enough time and expecting the shaking to be light.Many other participants also thought they were already relatively safe, felt that they had been okay in previous earthquakes so did not need to take action, and that they did not understand the message or did not know what to do.
A considerable number of participants (66 and 65) also chose the "Other" option.Among those who provided other reasons, most commonly people said they did not see or notice the alert until after the shaking had started or ended.

Research question : knowledge of EEW
RQ2a: How much do people in NZ (think they) know about EEW?
In both surveys, more people knew nothing about EEW generally than Google Alerts specifically (Figure 7).It is possible that this counterintuitive finding is due to ambiguity in the question.
Although the introduction to the survey specified earthquake alerts, some participants may have assumed this question was asking about all types of Google Alerts.However, both patterns across both surveys show a strong lack of knowledge in NZ.
RQ2b: Who do New Zealanders think is responsible for sending the EEW they received?
Consistent with this low level of knowledge, only a quarter to a third of participants correctly identified Google as responsible for sending the alert (Figure 8).More people believed that GeoNet, NZ's earthquake monitoring agency, was responsible for the warning.Fewer, but still a significant proportion of people, thought that the National Emergency Management Agency (NEMA) was responsible for the warning.These findings are, again, unsurprising given GeoNet's role in disseminating post-earthquake information and NEMA's role in sending warnings for other hazards such as tsunami, but concerning as erroneous attributions of responsibility could lead to erroneous attributions of blame when the system does not operate correctly (e.g., false alerts).The importance of public perceptions of responsibility is discussed later.

Research question : perceptions of usefulness and message preferences
RQ3a: How useful did people find the EEW that they received?
Consistent with previous findings that most New Zealanders thought an EEW system would be useful or somewhat useful (95%; Becker et al., 2020b), the vast majority of participants in both surveys found the alert useful or somewhat useful (84.0% and 83.4%).Most participants who found the alert useful said it was because it gave them time to mentally prepare, followed by that it allowed them to get information about the earthquake, they were able to avoid panicking and move quickly, and they could prepare to take protective actions (Figure 9).Few participants indicated that the alert was somewhat useless or useless.Among these participants, most felt that the alert was useless because they received it after the shaking started, they did not notice the warning, they were not able to respond in time, or they did not experience any shaking.Participants who received the warning but did not feel any shaking were asked how that experience made them feel.Most people were still happy with the alert and did not see a problem in it arriving after the shaking (65.1%; 63.5%), perhaps reflecting a known desire among New Zealanders to receive information about an earthquake after one has occurred (e.g., Vinnell et al., 2022a).Most participants who chose "Other" said they found the alert useful because it made them aware of the existence of the alert so they would know what they are for next time.Other common responses included people who saw how the alert could be useful in future earthquakes even though it was not useful for the current event, that it confirmed there had been an earthquake, and that it was interesting.
RQ3b: What delivery channels are preferred for sending EEW?
The vast majority of participants (91%) indicated that a mobile phone alert was their preferred delivery system.However, social media (16%), TV messages (22%), radio messages (17%), and PA or sirens (14%) were also relatively well supported.These responses reinforce the earlier findings of lack of knowledge, given that few if any of these latter options are viable for an EEW.
RQ3c: What information would people like to receive in an EEW?
Generally, most participants indicated that all suggested information would be useful in an alert, with the most supported

FIGURE
Proportion of di erent behaviors undertaken in response to receiving the alert.The NA (not applicable) option was only available in the second survey.

FIGURE
Reasons for not taking any action in response to receiving the alert.
information being additional geohazards and information about the earthquake.While previous research has demonstrated that more information in an alert can reduce response time by limiting milling and information seeking (Sutton et al., 2018;Wood et al., 2018), this research tends to be in contexts with minutes rather than seconds of warning.These findings demonstrate the importance of communicating the purpose of an EEW, and in particular the two-tiered system used by Android.While the alert received by most people (the lower tier, "Be Aware" alert) does contain some information about the earthquake characteristics and some additional geohazards (namely tsunami), it is important that people should understand that the priority is to undertake protective actions in response to the alert.This communication should include where and how to find the additional information which people clearly want but which is not necessary until after the shaking has stopped.

Discussion
Aotearoa New Zealand currently does not have an official public EEW system.While recent work has identified general public support for such a system (Becker et al., 2020a,b), other factors such as intended responses suggest that effort is needed to communicate how such a system might work (including why it might go wrong) as well as the appropriate action to take in response to receiving a warning.Since April of 2021, Google's Android Earthquake Alert System has operated independently in NZ; however, there has been little communication around this system.Consistent with recommendations to evaluate aspects of EEWs such as how people respond and what information to include, we undertook surveys following two widely received Android alerts to assess public knowledge, perceptions, and responses to these alerts.

Research question : behavioral responses
Of primary interest was exploring how people responded to the alerts, as well as reasons for not responding.A large number of participants used the time to mentally prepare themselves, consistent with reported intended use (Becker et al., 2020b).This mental preparation may produce psychological benefits such as reducing fear or reassuring people that they did in fact experience an earthquake.However, it is also possible that this mental preparation might backfire, particularly in the event of false alerts where shaking does not occur.Further, AEAs use a lower shaking threshold than the NZ public want, meaning they could be causing unnecessary stress.
Very few people took protective actions for the earthquake alerts, despite generally high knowledge of what to do in response to earthquake shaking among the NZ population (Vinnell et al., 2020;Colmar Brunton, 2021).More commonly, people stopped and waited, told others, or looked for more information.These behaviors likely reflect the novelty of the system as individuals who received the alerts, as well as those around them, were unfamiliar with it.However, similar low rates of protective actions are seen in other countries with established systems (e.g., Japan; Becker et al., 2020b).Other explanations could be lack of trust in the message (e.g., people are not convinced that they will feel shaking so do not need to act), lack of experience of earthquakes and in particular injury such that they do not see the need to protect themselves, or confidence in the safety of aspects of their environment meaning they do not feel action is necessary.
While milling behaviors are common in response to warnings (Wood et al., 2018), the nature of EEW means that there is little to no time for these actions before shaking arrives.It is  Agency or organizations participants believe is responsible for the earthquake early warning system which produced the alert.
possible that behavior will naturally improve over time as people become familiar with the alerts, unless outweighed by the negative impacts of false or missed alerts.However, public education or communication should be implemented so that people know what the alert is, why they have received one, and what to do, increasing the likelihood that they will be able to protect themselves before the shaking starts.
A large number of people did nothing in response to the alert.Most participants who did not respond said it was because they did not have enough time, they expected the shaking to be light, they were in a safe place, or they had been okay before.
All of these reasons raise concerns.In particular, a common reason for not acting once an earthquake starts is because the shaking is too weak to bother.However, earthquakes can increase rapidly in strength such that safely undertaking protective actions, once they are necessary, is difficult and some of the benefit has been missed (Vinnell et al., 2022b).It is also possible for EEW systems to underestimate the magnitude (Chung et al., 2020), and therefore likely the shaking intensity, of an earthquake, so ideally people would not judge the need to act on the estimates of the alert.Further, even when in a "safe" place, it is recommended to "drop, cover, and hold".The influence of having been okay before,

FIGURE
Reasons for finding the warning useful.
reflecting a normalization bias (Solberg et al., 2010), is a particular issue in the NZ earthquake context and perhaps could be addressed specifically in any communication which might happen around this system.

Research question : knowledge of EEW
Secondly, we explored knowledge of EEW, including responsibility for sending the alerts received.Unsurprisingly given the small amount of information available to the public, the vast majority of participants had little to no knowledge of EEW generally and the AEA System specifically.Previous research in areas with EEW systems has identified the importance of messaging before, during, and after the alert or earthquake (Cochran and Husker, 2019;Valbonesi, 2021).When people know what to do in response to a warning, as well as when they might or might not receive one, it is likely the best outcomes will be achieved.For example, it is important for the public to understand critical limitations in EEW such as the low probability of a warning for those at the epicenter who are at the most risk as well as the expected rate of false and missed alerts (Minson et al., 2019).Persistent false alerts without communication could lead to a "crying wolf " effect and cause unnecessary fear (Reddy, 2020).
Across the two samples, roughly equal numbers thought that GeoNet, Google, and NEMA were responsible for the AEAs, with a further, meaningful proportion identifying GNS Science as the agency responsible.Trust in official information is a key factor in successful disaster responses, so the large proportion of New Zealanders who believe that organizations such as NEMA, GeoNet, and GNS Science are responsible is a concern, as people might lose trust in these sources when they experience incorrect, false, or missed alerts from the Android system.There are therefore ethical implications of launching an EEW system, particularly without surrounding public education, given the possibility for inappropriate behavioral responses, undue fear, and loss of trust in key official agencies.
As well as issues around trust, there are potential legal implications of an EEW system, particularly when those systems produce errors (Valbonesi, 2021).The ubiquity of smartphones means that EEW is becoming transboundary, and as such is no longer necessarily limited to geographical jurisdictions (Tan et al., 2022), particularly when those systems are not run by national or official organizations.This development raises the important, and as yet unanswered, question of who is responsible and liable when a system leads to harm.

Research question : perceptions of usefulness and message preferences
Finally, we assessed perceptions of the usefulness of the alerts and message preferences.Consistent with previous research in NZ concerning a hypothetical EEW (Becker et al., 2020b), the vast majority of participants said the system was useful or somewhat useful.It is notable that participants seemed to find the alert useful as it allowed them to prepare to respond, rather than because it allowed them to respond pre-emptively.This could be due to the combination of longer processing times due to the novelty of the system and relatively short lead times.It is possible that with increased familiarity and education more people would use the warning time to actually act, rather than to prepare to act.
While these perceptions of usefulness are overall encouraging for EEW in NZ, the hypothetical nature of the system in the previous work as well as the novelty and lack of knowledge of the AEA System raises the distinct possibility that the public are largely unaware of limitations in EEW.Given the large proportion who reported little to no knowledge of EEW, it is highly likely that people are not aware of the possibility of false, missed, or incorrect alerts from EEW systems.Further work will need to assess how perceived usefulness changes as people become familiar with these limitations.Relatedly, many of the assumptions here about limitations of the AEA System are based on limitations of other, non-smartphone based systems; while these other systems work differently, there is little to no public evaluation of the AEA System available so the assumptions were necessary.
There is a tendency to confound different alerting systems, hazard information, delivery methods, and the source of information, which can be an issue for understanding and responding to warnings.When EEW was issued in Japan in its early days, many people did not differentiate between EEW and standard earthquake information-which is sent out after events have occurred (Fujinawa and Noda, 2013).The number of participants who thought the warning came from official providers of earthquake information (GeoNet, GNS, and NEMA) suggests that the same might be happening in NZ.In Mexico, multiple parties can issue alerts through apps outside the official agency, but the public does not differentiate between the official and unofficial alerts (Reddy, 2020).A similar problem is observed in the findings of this paper, where respondents attributed the alert to different parties, which may cause misunderstanding over capabilities and jurisdictions in alerting for different hazards.Further, having multiple sources of information might present inconsistent messages, leading to confusion.For example, earlier on in the introduction of the Android system in New Zealand, one alert indicated the earthquake was large (M6.0) and offshore, whereas GeoNet (correctly) identified the earthquake as smaller (M5.1) and epicentred onshore.Such discrepancies could impact important response behaviors such as tsunami evacuation.
New Zealand uses multiple channels for public alerting.One alerting system uses cell-broadcast technology that delivers Emergency Mobile Alerts (EMAs) to the public's mobile phones, which are sent out by official public alerting agencies (Ministry of Civil Defence Emergency Management, 2017).EMAs are relatively new to New Zealand and have been used to inform the public of lockdowns in response to COVID-19, and other hazards; yet a study in 2021 showed that some people do not fully understand when they might receive EMAs and further education is needed around their use (Vinnell et al., 2022a).Issuing AEAs without supporting public education may hamper the effort of ongoing effort to improve understanding and response to existing alerting systems.
The primary purpose of an EEW is to prompt people to anticipate incoming ground shaking.Managing people's expectation as to what the alert can deliver must be communicated clearly, especially when the alerts include information about potential additional geohazards such as tsunami.The ongoing public education in New Zealand on tsunami is "Long or strong, get gone", encouraging the public to use the natural warning signs as the prompt for evacuation for local source tsunami.As official warning may not be possible for local source tsunami, over-reliance on alerting systems may lead to people waiting for an alert which will not arrive, and therefore staying in harm's way (Tan et al., 2021a).It should be made clear to the public that the scope of the AEA System is to warn people for possible anticipated shaking but not for any other associated hazards.

Particularities of the Aotearoa New Zealand context
One of the challenges to developing a successful EEW system (i.e., one which produces more benefit than the cost incurred) is the contextual variations between areas of the world prone to severe seismic impacts.
The vast majority of participants in this study reported weak (MMI III) or light (MMI IV) shaking for both alerts; previous research found that most New Zealanders only want an alert for moderate (MMI V) or stronger shaking.While "Be Aware" alerts for light shaking is consistent with the threshold adopted by Google for the AEA System, the number of such alerts sent in the Wellington Region in the last 8 months of 2021 was double the expected number of yearly alerts, suggesting that as well as public preferences, this threshold could better take into consideration NZ's unique seismic profile.Alerting at too low a threshold could lead to warning fatigue, meaning that people are more likely to ignore alerts when they receive them.Other aspects which might differ between NZ and other countries with the same or similar system include seismic profile, population density, previous experience of earthquakes, and information in the alert.For example, the AEAs include "safety tips"; however, this information is not necessarily appropriate for NZ users.Suggestions to move to higher ground in case of tsunami is listed third, after using ice cubes for water and cleaning up spilled medicine.All of these factors should be appropriately considered when launching an EEW system in a new location.

Limitations
As with similar online studies using primarily social media for recruitment, the demographic profile of the participants did not precisely reflect that of the study population.In particular, women were overrepresented in both samples while Māori were underrepresented.However, there is limited evidence of impacts of demographic factors such as gender and ethnicity in earthquakerelated judgments and behavior (Becker et al., 2015).Further, this study relied on self-report, so the accuracy of some responses such as timing between alert and shaking and intensity of shaking could not be validated.Given the novelty of this system, however, the limitations of such data are outweighed by the benefit of obtaining a large amount of data relatively rapidly in response to unpredictable events.Future work could use representative sampling methods, although these tend to cost more and take longer which can mean that the data might become less reliable, or target specific communities of particular interest or concern.For example, older people and those with disability who may not be able to protect themselves quickly might benefit more from the warning time.Such studies could use more intensive data collection methods such as focus groups, but again these tend to take longer to organize and risk memory deterioration.

Future studies
The findings of this study raise several potential avenues for future studies.In particular, future work should explore how perceptions of usefulness and use of the AEA System changes over time.This might occur as people become more familiar with the system but also potentially experience false and missed alerts or have different earthquake experiences.While a true longitudinal design would be difficult to implement, partly due to the unpredictability of earthquakes, quasi-longitudinal methods could be used to explore whether, at a general level, public knowledge and behavioral response improves over time as people become more familiar with the system.
Second, given the strong evidence of this study and previous work around the use of EEW to mentally prepare, work could examine whether this action is beneficial.While people might perceive the ability to mentally prepare as helpful, it is also possible that this emotional bracing could have a negative psychological impact, particularly in instances of false alerts when the warned shaking does not occur.Further, if people are mentally preparing themselves instead of taking protective actions, rather than as well as, then this action might not achieve the most benefit.EEW could potentially reduce the high injury burden of earthquakes in NZ if people are using the time to physically protect themselves, so it might be that mentally preparing is something that should be discouraged.
Third, this study should be repeated to assess the perceptions and responses to the higher tier, "Take Action" alerts.These alerts are notably different to the "Be Aware" alerts studied here, and are much more similar to the alerts sent by other systems, so it would be informative to explore how New Zealanders respond.There are two key challenges to such work.It is logistically difficult given the country's population density, as few people are typically close enough to the epicenter to receive this alert.While a large earthquake under an urban center would reduce this logistic challenge, it raises an ethical one.Such an earthquake would likely at least have psychological impacts which means that not only would responding to a survey not be a priority, it could be potentially distressing and therefore should be avoided.

Conclusion and recommendations
The findings of this study, in the context of existing knowledge, leads to several recommendations for EEW in NZ.First, and perhaps primarily, is the importance of education and communication.This should be before alerts-so that people when and why they might or might not receive one, as well as what they should do in response-and after, particularly so that people understand why any errors may have occurred and trust in the system can be developed.Educational information should be consistent across multiple channels, including in the first instance information from the EEW provider itself, alongside any national, regional, and local material that might support people's understanding about and response to earthquakes.
Secondly, opting out of receiving alerts should be easy and accessible or users should be prompted to choose in advance if they want to receive alerts.The existing evidence for potential physical harm from inappropriate responses to an EEW should be extended to include potential psychological harm, particularly in cases of false alerts, or where an extended earthquake sequence has the potential to affect psychological wellbeing.
Third, as much as possible the warnings themselves should be tailored to the NZ context.This could include preferences for alerting threshold, the information provided in the initial alert itself, and the information provided in the full message following the notification.For example, suggested actions to take after shaking should be aligned with official information and likely hazards and links to official sources of information should be provided.Privately-run but publicly accessible EEW systems operate in several countries, and in some instances in varying levels of alignment with public or government run systems.For example, Google has collaborated with USGS and California emergency services in the ShakeAlert EEW system (Stogaitis, 2020).Lessons from that collaboration could be useful for other countries with corporate and government systems.
In line with the above suggestions, and with recommendations from other researchers in the field, public responses to EEW systems such as the one considered in this study should be constantly evaluated.This will allow the identification of ways in which they could better achieve the intended outcome of reducing injuries and fatalities, as well as ensuring they are not producing excessive negative outcomes.
In conclusion, despite the public perceiving EEW as useful, people held limited knowledge of EEW in general and the AEA System specifically and were unlikely to take protective action, instead either undertaking milling behavior or doing nothing.The reasons for these, as discussed earlier are varied, and lead to several recommendations for EEW in New Zealand.

FIGURE
FIGUREThe intensity of shaking felt by survey participants.

FIGURE
FIGUREParticipants' estimations of seconds between receiving the alert and first feeling shaking.

FIGURE
FIGURESelf-assessed knowledge of earthquake early warning generally and AEA specifically.

FIGURE
FIGURE