Developing meaningful qualitative data regarding critical facilities presents several challenges, such as the sensitivity and security of the information, and the parameters of its use (Rinaldi, 2004). This work was thus conceived in collaboration with the Rhode Island Emergency Management Agency (RIEMA) as part of a larger effort to evaluate and improve preparedness in Rhode Island. Clarity regarding the purposes, scope, and auspices of the work was essential to the perceived legitimacy of the inquiry. This allowed investigators to gain access to FMs within the case study that might otherwise be hesitant to participate because of security threat concerns (Rinaldi, 2004).

Westerly Rhode Island was chosen as the pilot community because of its relatively small number of critical facilities and high level of hazard exposure. Westerly is a coastal community on Rhode Island’s southern coast with a population of about 18,000 as of 2010 (United States Census Bureau, 2018). The Hurricane of 1938, 1954, and the “Floods of 2010” (a series of rain events that impacted Westerly during March of 2010) all impacted the town and it lost many buildings during Hurricane Sandy due to storm surge and winds that reached 86 mph (sustained at 64.4 mph) in Westerly (Manning et al., 2014). The combination of hazard exposure and limited number of facilities allowed for investigation of interdependencies and cascading consequences which are of particular concern to RIEMA.

Gaining access to infrastructure vulnerability information is challenging because it is normally proprietary in nature (Rinaldi, 2004). Partnership with RIEMA and clarity regarding the use of outputs facilitated introductions to local emergency managers (EMs). By the nature of their work, local EMs are highly informed about how a storm affects their community (Newkirk, 2001) and are well connected to the FMs in their community. EMs played an essential role in selecting FMs to interview since FMs are highly informed about their facility (Mendonça and Wallace, 2006) and external resources their facilities rely on (Rinaldi, 2004). Critical facilities were defined in collaboration with EMs based on FEMA’s definition as facilities that, “if severely damaged, would reduce the availability of essential community services necessary to cope with an emergency” and facilities “associated with utilities that are required to protect the health and safety of a community” according to the local EM (Federal Emergency Management Agency [FEMA], 2012). These critical facilities include fire departments, police stations, hospitals, and waste water treatment plants among others.

Selected FMs were invited to participate via email and telephone. Investigators explained the overall goal of the research while scheduling interviews (Paul et al., 2018). Additional FMs were interviewed as opportunities presented themselves (Patton and Appelbaum, 2003). In total, 13 FMs from 11 of 30 critical facilities in Westerly were interviewed. This included four fire departments, the police station, the dispatch center, the ambulance corps, the waste water treatment plant, the water department, the largest electrical distribution substation and the department of public works (Figure 1). FMs from Westerly’s hospital, school system, telephone networks, and natural gas facilities declined to participate, in most cases due the perceived security threat posed by sharing sensitive information. As this method has been developed, tested and demonstrated in Westerly, interest in participation in subsequent processes across the state has increased. Demonstrating the utility of outputs and trust-building through iterative processes is thus essential to building effective databases.

An open-ended interview instrument was designed to collect a wide range of FM’s storm concerns. Questions were adapted from the Department of Homeland Security’s Critical Infrastructure vulnerability assessment (Department of Homeland Security [DHS], 2013), the UKCIP’s threshold identification methods (United Kingdom Climate Impacts Programme [UKCIP], 2013), the global consulting firm ICF’s climate vulnerability assessment (ICF International, 2017), and results from the Sandy Mitigation Assessment Team for critical facilities (Federal Emergency Management Agency [FEMA], 2013) (see Appendix I). Three members of RIEMA’s Critical Infrastructure Program reviewed the survey instrument along with two members of URI Marine Affairs Department, and one ex-FM from Rhode Island.

In three cases, an interviewee’s colleagues joined the interviews, which helped interviewees feel more comfortable and is a recommended practice for risk communication meetings (Chess et al., 1988). To begin, we explained that the purpose of the interview was to collect interviewee’s storm concerns for incorporation into storm impact models (Chess et al., 1988). We used an illustration of a fictional CT for a petroleum facility triggered by storm surge from Hurricane Carol at 4:30 pm on August 5th, 1954 overlaid on a modern day map at the Port of Providence as a thought prompt to show the interviewee how the information could ultimately be used (Figure 2; White et al., 2010). We explained to interviewees that their names would be kept confidential and quotes from their responses would not be identified or attributed to them individually but might be discovered due to the limited number of critical facilities in Westerly.

We then asked for the interviewee’s immediate concerns to a storm and encouraged them to consider storm consequences on other parts of the community (e.g., roads) that could affect their facility. We listened to interviewee’s concerns (Chess et al., 1988), and then worked to identify remaining CT components for each concern interviewees identified (Figure 3).

We asked the interviewee to identify the location of his/her concern on a navigable Google Map satellite (15 m to 15 cm resolution) and/or street view using a laptop computer (Figure 4). We also asked the interviewee to identify the concern threshold using an open response threshold-identification method similar to the ICF and United Kingdom Climate Impacts Programme (UKCIP) methods (United Kingdom Climate Impacts Programme [UKCIP], 2013; Monioudi et al., 2018). For example, we asked, “What inundation level would cause the consequence you mentioned?” Although interviewees were encouraged to identify the storm forces that the underlying storm models used (rain, wind, storm surge, standing inundation, and wave height), interviewees also identified other weather and geological concerns, for example lightning strikes, soil moisture content, and microbursts.

Interviewee’s usually began by identifying storm consequences they had previously experienced. Once the interviewee exhausted these historical reference, asking the interviewee about unprecedented storms enabled the collection of more consequences. If interviewees had trouble identifying concerns, follow-up prompts were used to stimulate the conversation (see Appendix I). For example, questions like, “What are the consequences of one foot of flooding where we are standing now?” or “What concerns do you have if a storm with 150 mph winds passed over this facility?” both prompted additional consequences. The precise wording and order in which the questions came up were not constrained (Merriam, 1988), since interviews were focused on subjects that matched the interviewee’s knowledge and points that the interviewee brought up (Lewis and Sheppard, 2006). Interviewees identified the location of the concern using the navigable Google Maps satellite or street view during the interview.

Respondents also described their career experience and responsibilities at the facility. Interviews lasted between 1 and 2 h each.

We digitally recorded and transcribed the interviews in full with the help of a hired transcription service (200 pages). In order to answer our three research questions, we identified and analyzed all CT components mentioned by interviewees using Microsoft Excel. We also coded interviews line by line and identified themes in the data through an analytic induction method using NVivo, a form of grounded theory described by Ratcliff (1994) as an iterative process. We standardized CT components before incorporating them into a storm impact model (Table 2).

The storm impact model employed high-resolution physics-based simulations of surface winds during hurricane landfall using the track, intensity and size parameters of historic or hypothetical storm events. This model adapts Gao and Ginis (2016) model for open ocean hurricanes to account for changes in surface roughness at landfall in order to better simulate landfalling storms (Gao and Ginis, 2016). This model was developed as part of a larger project for the Department of Homeland Security Coastal Resilience Center. Storm surge response to the wind model was computed using the ADvanced CIRCulation (ADCIRC) model (Luettich et al., 1992), coupled with the Simulating Waves Nearshore (SWAN) model (Booij et al., 1999). An all numerical storm impact model combining outputs from the wind and hydrodynamic models was programmed to test the collected CTs.

The hazards available for testing CTs included:

The use of the storm impact models allowed investigators to test the effectiveness of the whole process, and to verify whether the five components identified (the concern, the location, the modeled hazard, the threshold, and the consequence) were sufficient for modeling. Testing the gathered CTs also allowed investigators to provide feedback regarding specific CTs to participating FMs and RIEMA. The use of the full range of available time increments allowed for the order of events to be investigated (e.g., when radio communications would be likely to fail or when a particular road would be blocked).

The CT collection method resulted in identification of 201 concerns from 13 interviewees representing 11 critical facilities (see Supplementary Table 1). Many concerns and associated consequences were based on the interviewee’s experiences with the Floods of 2010 and Hurricane Sandy.

Since geographical coordinates are necessary to integrate CTs into storm models, the accurate location of each concern was noted by the respondent. Most concerns could be tied to one location, but the interviewees explained that 40 of the 201 concerns had multiple locations in Westerly that were difficult to geospatially locate. For example, one interviewee was concerned that a storm would flood fire hydrants and prevent his firemen from reaching them. However, the interviewee did not know the locations of the fire hydrants that were at risk to flooding and therefore the concern was not incorporated into the storm impact model. Though a GIS database of fire hydrants for Westerly probably exists, bringing those additional datasets in was beyond the scope of this project. In other examples, the interviewee could identify several locations, but was not aware of which in particular would be vulnerable to a storm impact. Other concerns interviewees were not able to immediately geospatially identify included:

One Hundred-Ninety three FM consequences could be triggered by either a hydrodynamic, wind, or precipitation model. FMs explained that 90 CTs were at risk to standing inundation, 74 to wind, 13 to rain, seven to storm surge, and five to wave height (Table 3). However, the clearest challenge in gathering modelable CTs from FMs was the incompatibility of terms from the wind and ocean models which are based on physical simulations and the observations of FMs. Whereas water velocity and direction can be modeled in meters per second, these forces are seldom measured or discussed in those terms. For example, one fire chief was concerned that storm surge could cause a bridge his crew relied on to collapse, but could only estimate that a “Category 5 hurricane” would cause this to happen. Without a quantifiable water velocity or direction that would cause the collapse, we could not include this consequence into the models. Validation of modeled surge velocities often relies upon forensic studies of damage (Pistrika and Jonkman, 2010). As will be subsequently discussed, the incompatibility of terms makes it difficult for FMs to identify thresholds. Even those who have witnessed events have no practical way to estimate forces.

Several consequences could be triggered by “hazards” that were not part of the underlying storm simulation. For example, ten consequences could be triggered by hurricane forecasts. If the National Weather Service (NWS) warned that a category 2 hurricane was going to hit Rhode Island, an ambulance corpsman would station an ambulance on the other side of the Pawcatuck River to maintain access to certain neighborhoods. Also, when combined with other storm forces, time was mentioned as a hazard for a few CTs. For example, an ambulance corpsman explained that if roads were blocked by fallen trees for 9 days, his facility would run out of fuel because fuel deliveries could not arrive. Both of these hazards would be challenging to model because they rely on the behaviors of individuals (Aerts et al., 2018).

Highly specific thresholds combined with time incremented events provided valuable feedback to EMs and made it possible to discuss the order consequences from the modeled storm would occur (the unfolding of these events can be seen in Figure 5).

Interviewees most commonly gave thresholds for flooding in feet and wind speed in miles per hour. 86 of the 201 CTs could not be incorporated into storm impact models because the interviewee did not know the threshold, it was not explicitly asked for, or the interviewee gave the threshold in unmodelable units (Table 3). Almost every threshold given by interviewees was coded as uncertain, which means the interviewee was willing estimate the threshold, but was unsure of its accuracy. 41 of the thresholds were coded as “unknown,” which means the interviewee was not willing to estimate the threshold. One interviewee illustrated this dilemma when he said, “for an antenna to break, each one has a rated wind speed velocity– and you have to look up each and every antenna because there’s multiple types of antennas, different manufacturers.” Only one threshold was modelable for rainfall and no threshold was modelable for storm surge because interviewee’s do not know numerical values of rainfall rates or storm surge velocities (Table 3). One interviewee illustrated this when he said, “When it rains, it rains hard! That’s all I know.” Similarly, for storm surge, another interviewee explained that he would not drive his fire truck through water, “If you could see a decent current with any type of like a ripple to it.”

When asked for the threshold that would cause the impact, interviewees asked the researcher for wind speed/storm surge levels of Hurricane Sandy and the rainfall amounts of the Floods of 2010. Interviewees also tried to look up these thresholds online during the interview. One interviewee illustrated a commonly given un-modelable threshold with the quote, “Whatever the 2010 floods were, how much rain we got.”

This method collected a range of FM’s storm concerns. 195 consequences were immediate, e.g., when water reaches one foot at a generator, the generator short-circuits. Eight consequences were long term, e.g., sand deposits prevent fire trucks from accessing certain areas after a storm. 50 of the consequences were related to mobility (roads/vehicles), 41 of the consequences were related to power (power lines/electrical panels/generators), 40 were related to communications (telephone lines/antennas), 34 were related to specific equipment operated by waste water and water purification plants (pump stations/wells), 22 were structural (windows/roofs) and five were related to personnel. 26 of the consequence components included an interviewee‘s immediate response to the consequence and 34 included an interviewee’s long-term response to the consequence. For example, the water department’s wells need to be shut down immediately if water reaches them and then they require a long-term chemical treatment process once the floods recede. Most consequences mentioned by emergency response FMs (firemen, policemen, and ambulance corpsmen) were related to mobility and communications.

Participatory mapping using Google Maps and a set of exploratory open-ended questions was an effective way to collect interviewee’s storm concerns. Many of the concerns mentioned at the beginning of interviews had been impacted during Rhode Island’s Floods of 2010 and Hurricane Sandy. Also, beginning interviews with storm consequences that have already occurred may have made the research more relevant and realistic to FMs, which likely made it easier for interviewees to identify CT components. Interviewees readily identified location of concerns on the navigable Google Map satellite view during the interview. When the satellite view was limited, Google Map’s Street View, which includes a 3D visualization, helped identify location of concerns. Identifying the location of concern on a Google map made it easier to record interviewee’s concerns and identify remaining CT components. For example, identifying locations of concern helped identify thresholds because we could explain to the interviewee that thresholds should be for the location the interviewee identified on the map. This was useful because interviewees tended to identify inundation thresholds in feet above sea level instead of feet above ground level at the location of the concern.

Consequence Thresholds can be integrated with static and dynamic models. This is a key feature of the CT approach, as it allows the audience to understand how consequences of storms unfold over time. Once the CTs are integrated with a storm model, the parameters of the model (e.g., surge, wind speed, and flooding) trigger concern thresholds upon running the model. To illustrate, we incorporated the CTs from this project into a storm impact model for a hypothetical storm, called Hurricane Rhody (Ginis et al., 2017). Hurricane Rhody is a plausible hurricane scenario created to simulate the effects of a high-impact storm on the Rhode Island coast in order to provide state and local agencies with better understanding of the hazards associated with extreme hurricanes. The characteristics of the hurricane are not arbitrarily chosen, but are based on those of several historical storms that have impacted the region. In this example, 23 of the CTs identified in interviews were triggered by wind and 21 CTs were triggered by storm inundation. Text Box 1 describes some of the 44 CTs triggered by Rhody in chronological order.

The most common challenges when collecting interviewee’s concerns for incorporation into storm impact models are identifying modelable thresholds and identifying accurate locations of concern.

Incorporating qualitative consequences of storms can improve storm impact models by increasing their accuracy and relevance to participants. Traditional DRR assessment outputs struggle to provide actionable data regarding relevant, specific local concerns for communities to use to prepare for disasters (Paul et al., 2018). For example, caution needs to be used when analyzing HAZUS hurricane damage outputs for a community’s facilities because results are based on average damages to similar facilities under similar circumstances (Vickery et al., 2006). Traditional DRR outputs like HAZUS also do not take into account intangible consequences of storms like losses of cultural assets (Messner and Meyer, 2006) even though these types of losses are what people normally mourn the most after a flood (Becker et al., 2015). Therefore, including FMs concerns using a participatory process allows storm impact models to output more specific local concerns than relying on generalized damage curves and may make the models more credible, actionable and relevant to participating FMs. However, additional research is needed to quantify how developing CTs through participatory processes influences a storm impact model’s credibility, actionability and relevance for participating FMs.

As storm simulations increase in accuracy (Aerts et al., 2018), on-the-ground vulnerability information will need to be collected with increasing precision using a participatory method to use these models most effectively. Incorporating CTs into many storm impact models through a Montecarlo analysis may help decision makers more objectively prioritize storm adaptations (see for example (Bosma et al., 2015). The use of these methods goes some distance to addressing concerns about uncertainty. Storm impact models may also assist FMs prepare for real storms in the days and hours leading up to landfall (Stempel et al., 2018).

That investigators had to make consequential decisions in implementing the data that was collected (e.g., choosing the most conservative estimate) points to the need for consistency and standardization in processes. Anchoring and other methods previously elaborated will help that process. There is a larger question regarding the utility and perceived legitimacy of CTs if they are utilized outside of local processes in which thresholds are gathered, tested, and reflected to participants. Participation not only enhances the perceived legitimacy of the outputs (White et al., 2010), there are real questions as to the validity of the gathered thresholds where individual judgments on the part of FMs are applied outside of the particular facilities and situations with which they are familiar. If CTs are to function as more than boundary objects—points of common communication between FMs, EMs, and scientists—attention must be paid to standardizing procedures for gathering data. This may include, for instance, standardizing anchor points and prompts among different researchers and agreeing that the conservative estimate will be utilized. If sufficient data can be gathered for common types of facilities using consistent methods, it may be possible to aggregate and generalize CTs once enough have been collected to situations for which no new data is gathered.

This methodology was piloted with some of the FMs of one community and should be repeated with other FMs in other regions in order to both validate the utility of the approach and refine it based on the findings described in this paper. Modeling storm surge impacts involves compounding uncertainties. In addition to, the uncertainty of the scenario itself (probability of occurrence), there are uncertainties regarding the interaction between the landfalling hurricane, landform, and physical infrastructure that are not accounted for in numerical simulations (Kostelnick et al., 2013). Uncertainty on the part of interviewees regarding the thresholds are compounded on top of those existing uncertainties, and thus not accounted for.

Precisely quantifying the storm force at which a consequence occurs makes the output appear more certain than it is (Shackley and Deanwood, 2003). When experts identify a point at which a certain piece of infrastructure will fail, it is likely that the point has a good deal of uncertainty, which is not easily shown by the point (Cooke and Goossens, 2004). Adding an additional query regarding uncertainty may address some of these concerns, however, many of these interactions are fundamentally unknowable (Couclelis, 2003). Until sufficient external validation takes place, in part through engineering based structural analyses that address newly identified concerns, the utility of CTs is thus best understood as a tool for DRR used with participating EMs and FMs (Schroth et al., 2011). Also, selecting appropriate FMs is an important additional step if the method presented here is to be used to prepare communities for storms.

An essential next step in developing and assessing the utility of CTs is a comparison between CTs and fragility curves for impacts that are conducive to such an analysis. This comparison can underscore the complimentary nature of the approaches.