Comparing behavioral risk assessment strategies for quantifying biosecurity compliance to mitigate animal disease spread

1. Introduction

Understanding how human behavior impacts the spread of disease is crucial in strengthening agricultural industries. Our research focuses on building interactive games and simulations to emulate complex decision-mechanisms that can impact the well-being of agricultural production networks. This can be especially useful for testing how people respond to risk communication strategies in preparation for infectious disease outbreaks. Here, we compare risk classifications using a traditional survey-based risk assessment to response data collected using an experimental game with the goal of helping to inform decision support during potential disease outbreak scenarios.

In order to combat disease spread, biosecurity protocols and the practices that they detail have become prominent tools, integrated into standard operating procedures among the agricultural industry (1, 2). These may include forms of decontamination like truck washes, line of separation protocols, and other disease preventative measures like vaccinations, and feed treatment. However, the use and compliance with biosecurity protocols implemented within farms can vary (3). Some of these procedures may be onerous or cumbersome and at times bypassed by workers (4, 5). Motivators of biosecurity decision making are unclear, but decision ramifications can be substantial.

Experimental gaming simulations can be used as digital tools for studying factors that influence behavior and decision-making, including in various sectors of agriculture (6–12). Since risk communication is an important tool for education and nudging behaviors for crisis mitigation (13), several of our experimental games were designed to test how various risk communication strategies impact decision-making. In another experimental game, we simulated biosecurity investment amidst several outbreak scenarios along pig production supply chains (6). This allowed us to identify prominent strategies and behaviors that were used to address conflicts associated with disease in agriculture (9, 10). In Clark et al. (9), clustering algorithms (14) were shown to be a useful tool for categorizing behavioral observations from the experimental game. We've also shown how non-monetary awards and incentives can affect conservation decisions in farms (11). Experimental gaming simulations can be applied to quantify situational risk aversion associated with agricultural decision-making. This may be especially useful for identifying and accounting for biosecurity non-compliance among worker populations within agricultural production networks.

There are many avenues used to assess behavior associated with confronting risk. One such tool ,MPL risk assessment (15, 16), has shown potential in quantifying economic risk preferences. We compare our simulation results with a well-tested MPL risk assessment strategy, Holt and Laury (16) in which 10 disjoint binary lotteries are presented, and participants choose between a safe or risky option. The payouts are structured such that the probability of success for choosing the more risky, higher paying option sequentially increases. Participants generally move from the less to more risky option as the probability spread between the two choices becomes more favorable. The proportion of safe choices, and the probability point at which participants switch from the safe to risky option can provide a metric for comparing risk aversion profiles. This general context-free approach has been correlated to real world behaviors in finance (17). Risk aversion via the MPL was found to be associated with less participation in negative health practices, including smoking, excessive alcohol consumption, and lack of seatbelt compliance (18). Since the MPL-style risk aversion metrics are context-free, it is unclear how context-related conflict will influence behavior, given the possibility that risk aversion may evolve throughout an individual's experience with a complex decision mechanism.

Our study uses an experimental game to emulate potential rule breaking behavior (i.e, non-compliance) with a line of separation shower protocol within a pig production facility. The aim of our simulation is to understand under what circumstances participants will engage in risky behaviors during an outbreak scenario. We then compared observed behavioral risk to a well-tested MPL risk assessment survey. We hypothesized that risk aversion observed within the experimental simulation would correspond to risk aversion measured in the MPL assessment.

2. Methods

The experimental game was originally designed and featured in Merrill et al. (7). An additional, larger sample of online participants was recruited for this follow-up study. The game emulates working in a pig production facility during an infectious disease outbreak. To begin the experiment, participants engaged with an introductory slide show to provide contextual background information and frame the simulated decision mechanism. Participants then completed a practice round, featuring an interactive tutorial, in order to acquaint them with the simulation mechanics. These practices were accepted by the University of Vermont Institutional Review Board concerning experiments using human participants (University of Vermont IRB # CHRBSS-16-232-IRB).

The experimental game was designed using Unity game development software (Unity Technologies, Version 5.6.3) and was built for online deployment using webGL (19). Data were compiled using a relational database. Participants were recruited using the Amazon Mechanical Turk (MTurk) survey marketplace (20–22). We motivated participants with performance based incentives, as a means to bolster immersion and increase effort throughout the experiment (23). Player earnings from the experimental simulation were converted at a rate of $350 simulation dollars to $1 USD. A demo of the experimental game is hosted here: https://segs.w3.uvm.edu/demos/compliance/.

During the game, participants controlled a digital avatar throughout several simulated workdays. Participants were directed to collect coins, which symbolized indoor tasks, as well as completion of a single outdoor task per day. When the outdoor task was initiated, a score timer began ticking down (see Figure 1). The player then decided to either comply with the shower-in shower-out biosecurity practice or exit via the emergency exit. The emergency exit was faster (and thus more profitable), but risked the possibility that the animals in the facility could become infected with a disease. Choosing the shower-in, shower-out biosecurity practice exit effectively prevented animals from becoming sick. However, the shower-in, shower-out practice required that the participant wait 5 s before both exiting and returning to the facility. The time spent “showering” resulted in lost potential earnings both with the outdoor task losing value, and with not being able to collect coins (as inside tasks) upon re-entering the facility. Using observations from each participant's gameplay, we can explicitly calculate this opportunity cost to describe the economic motivations for breaking compliance. On average participants earned $18.29 (σ = $2.92) simulation dollars per round when using the shower-in, shower-out biosecurity practice, and $22.42 (σ = $3.56) when using the emergency exit, if their animals did not become sick, equating to an estimated $4.13 opportunity cost for using the biosecurity practice. This accounts to approximately $99.12 simulation dollars ($0.28 USD) over the course of the 24 experimental scenarios, or “rounds” of gameplay. However, this opportunity cost was contrasted with the monetary penalty of contracting an infection. If the player's facility became infected, the round ended, and they incurred a monetary penalty ($50 simulation dollars) and lost any experimental dollars they had accrued during that round. Thus, players were expected to balance the economic tradeoff regarding completing their tasks quickly but with the risk of contaminating their facility, or completing their tasks more slowly (opportunity costs) yet safely. At the end of each round, or simulated workday, the next round begins with a new set of experimental treatment parameters. This continues until all 24 rounds of gameplay are completed.

FIGURE 1

Figure 1. Screenshot of the compliance game decision mechanism. Participants can choose to comply with a biosecurity shower protocol carrying an associated time cost, or bypass via the emergency exit which can subject the facility to infection.

Each simulation round was designed to test how participants will behave under a set of treatments, or initialization factors, which act as our experimental variables. This allows us to quantify behavior with respect to a set of predefined epidemiological parameters and interface modifications to compare rates of compliance. The risk communication prompts included one of three types of messages describing the risk of infection associated with non-compliance with the shower protocol: 1) a linguistic (i.e., Very Low, Low, Medium, High) infection probability message, 2) a graphical threat gauge (see Figure 1), or 3) a discrete numeric or percentile value (i.e., 25%). These messages corresponded to four probabilities of infection if participants used the emergency exit: We tested a 1% (Very Low), 5% (Low), 15% (Medium), and 25% (High) probability of infection. Uncertainty in the decision making process was also tested by specifying that the disease was described as having known infection or contagion rates, or unknown characteristics prompting estimates of infection or contagion rates. If the disease was described as unknown then a range of contagion risk rates (e.g., infection risk is very low to medium with a best estimate of low risk) was provided instead of a discrete value.

The focus of the experimental simulation was to observe a participant's willingness to comply with the shower biosecurity protocol throughout various risk messaging strategies. Behavioral risk was quantified using the rate of compliance throughout the experimental simulation. This was calculated as the proportion of shower exits throughout each participant's gameplay. We then normalized by the total number of decisions to obtain each participant's risk score on a scale of 0 to 1: with 0 being risky (all emergency exits) and 1 as risk averse (all shower exits). The coin collection (indoor tasks) were designed to immerse participants within the simulated decision mechanism, so these were not pertinent for quantifying risk.

To quantify behavioral strategies within the experimental game, we applied the K-means clustering algorithm (24) to the observed rates of compliance for each participant. We administered K-means to the one-dimensional set of observations calculated using the average rate of compliance across all 24 experimental scenarios. Although we could have split these observations by treatment in order to perform a multi-dimensional analysis, we found the one dimensional case produced a straightforward distribution of behavioral strategies. We chose K = 3, using the elbow method by inspection, as this value optimized the sum of the square errors across each clustered interval (25, 26) and generated a rational division of observed behavioral compliance.

Upon completing the experimental simulation, participants were directed to an exit survey containing a demographic questionnaire and the MPL assessment derived from Holt ann Laury (16). Participants were informed that their choices in the experimental lottery and resultant winnings would be added to their final compensation. Figure 2 provides the MPL directions and interface. Here, participants were instructed to choose their preference across ten disjoint paired lotteries: a safer “Option A” vs. a more risky “Option B.” The payout between option A is either $0.60 or $0.50 USD while the payout for the more risky Option B was $1.10 or $0.05. For example, in the first choice both “Option A” and “Option B” have a 1/10 chance for the high payout and 9/10 chance for the low payout. Here, it best to choose Option A, as Option B has a 90% chance at only earning a nickel. The probability for the high payout increases by 10% sequentially per lottery question. This makes the risky “Option B” more favorable toward the end of the survey. Participants at some point generally crossover between the safe Option A and more risky Option B, as the probability spread becomes more favorable. The final question features a 100% chance for the high payout, in which “Option B” becomes the optimal and rationally should be chosen over “Option A.” Participants could revise their decision up until their final choice, after which a random number generator selected one of their choices and issued their corresponding reward. This payout is then added to their experimental game earnings.

FIGURE 2

Figure 2. Multiple price list (MPL) survey structure.

Risk in the MPL was measured using the proportion of safe “Option A” choices throughout the survey. Following, Holt and Laury (16) participants with 4 out of 10 safe choices were classified as risk neutral; more than 4 safe choices were classified as risk averse; and less than 4 safe choices were considered risk tolerant. We excluded participants who didn't choose at least one safe and one risky option during the MPL portion of the experiment, as this was indicative of a lack of understanding of the paired lottery choice mechanics or a sign of potential survey fatigue. Data from these participants were also excluded from the experimental game analysis.

The goal of our study was to compare risk classifications using the experimental game and the MPL risk assessment. In particular we aimed to answer the question: Does more compliance in the experimental game correspond to more risk aversion in the MPL? To compare distributions of risk between the experimental game and MPL, we used standard Pearson linear correlation (27). We compared distributions of simulated compliance and MPL risk assessment using Mann Whitney (MW) U-tests (28). We chose a non-parametric test, as both distributions of compliance and MPL risk failed D'Agostino and Pearson's test for normality (29, 30). This allowed us to infer whether groups of participants categorized by the MPL as risk tolerant, risk neutral, or risk averse complied more or less with the simulated biosecurity shower protocol. We also compared the categorical distributions from the clustering analysis of game data to MPL risk classification using a Chi Square (χ²) test (31). The strength of this association was then quantified with Cramer (32), which is measured on a scale of 0 (no association) to 1 (strong association). Statistics were calculated using Python (v3) with the SciPy stats module (33).

3. Results

We recruited 1,284 participants using the MTurk online survey marketplace between March 5th and March 31st, 2021. Participants were compensated with a base wage of $3.00 for completing the task and earned an average bonus of $2.72 which included their game performance and MPL earnings. The median completion time for both simulation and survey was 32 min.

In both the simulation and the MPL assessment, participant choices were generally skewed toward more risk averse behaviors, which in the case of the MPL is supported by previous research (16). Participants in the MPL were classified as 67.2% risk averse, 17.46% neutral, and 15.41% risk tolerant. Overall, participants complied with the shower-in, shower-out biosecurity practice on average 63% of their game play. Within the simulation, the communicated infection probability had the strongest impact on compliance with the biosecurity practices, with 87% average compliance during rounds where the probability of infection was high vs. 81% during the medium infection probability rounds, 54% during low infection probability rounds, and 33% during very low probability rounds. Risk distributions for each set of experimental treatments as well as the MPL assessment are given in Figure 3.

FIGURE 3

Figure 3. Violin plots showing risk distributions calculated for each experimental game treatment: probability of infection (p_inf) and disease information uncertainty. The overall risk observed from the full simulation (i.e., all experimental scenarios) as well as the MPL calculated risk are given in the final violin plots.

To quantify behavioral strategies within the experimental game, we applied the K-means clustering algorithm with K = 3. Cluster 1 contained the most risk averse participants (38%), who complied most often with the shower protocol throughout each experimental scenario. In particular, these participants had the highest compliance rate even when the probability of infection was both “Low” and “Very Low.” Cluster 2 consisted of the most risk tolerant individuals (13.8%). They had the least rate of compliance throughout all experimental scenarios. Participants in cluster 3, which we refer to as opportunists (48.2%), adapted their behavior depending on the risk of infection. During “Medium” and “High” risk rounds, they complied more often with the shower protocol yet took more risks during rounds with “Low” and “Very Low” probabilities of infection. The rates of compliance of each behavioral cluster for each treatment are given in Table 1.

TABLE 1

Table 1. Experimental game risk cluster compliance comparison.

We found very weak positive correlation between the compliance rate within the experimental simulation and the MPL risk assessment (Pearson r = 0.203, p < 0.001, N = 1, 168) . Risk measured using the MPL was not a strong predictor of participant's behavior in the experimental game. That is, a participant's rate of compliance in the experimental game did not directly relate to their risk aversion score within the MPL. In this way, we could not adequately determine if a participant may comply more or less with the shower protocol given their choices within the MPL.

Participants who did not choose both the low and high risk option at least once during the MPL assessment were excluded from the analysis. However, including the data of these 116 participants (9.03% of the sample) would not have changed the overall findings of our statistical analysis. Specifically, the correlation between the simulation and MPL risk aversion metric was still positive, yet slightly lower (Pearson r = 0.173, p < 0.001, N = 1, 284).

We also investigated how compliance rates differed when grouping participants by their risk classification within the MPL. In other words, we compiled rates of compliance from participant's classified as risk tolerant, neutral, and risk averse in the MPL to compare whether or not they, as a group, behaved differently within the game. We found that participants classified as risk averse via the MPL, complied more often with the biosecurity shower protocol vs. those classified as risk tolerant (MW: U = 85, 874.5;p < 0.001). Risk neutral participants also complied with the shower protocol less than the risk averse group (MW: U = 90, 655.0, p < 0.01) and more often than the risk tolerant group (MW: U = 16524.5, p < 0.05). These results are summed up in Table 2. This shows that the MPL classification can be useful to capture group behavioral dynamics (i.e., belonging to one of these groups may allow us to infer whether the individual may be more prone to risky behaviors comparatively).

TABLE 2

Table 2. Experimental game average compliance rates stratified by MPL risk classification (N = 1,168).

Finally, we compare the 3 risk clusters found within the experimental game to the MPL risk categories: Risk Averse, Neutral, and Risk Tolerant. Using a Chi-Square test, we found that these categorical distributions were related [χ² = 47.33, p < 0.0001 ]. To measure the strength of the association between distributions we calculated Cramér's V as 0.142. This shows that although these categorical distributions are related, the association is relatively weak. The contingency table for each of the observed counts of these categorical distributions is given in Table 3.

TABLE 3

Table 3. Contingency Table comparing participants who were categorized using the game data clustering analysis (Rows) and classification using the MPL assessment (columns).

4. Discussion and conclusion

Experimental gaming simulations can provide a unique lens for quantifying behavioral risk. Contextual framing allows for these simulations to elicit particular behaviors of interest in comparison to generalized survey methods. Combining economic risk aversion metrics derived from traditional survey methodologies along with behaviors captured via simulation may help build risk profiles that better capture how human behavior may impact the resiliency of these systems.

Risk communication strategies are important tools for mitigating disease spread among other crisis situations (13). Experimental games are well suited to test the influence of risk communication strategies on behavior in these types of contrived instances. Our simulations have shown that the type of risk communication strategy can have a large impact on behavior (7). Our clustering analysis identified the most prominent behavioral strategies driving non-compliance. Participants classified as risk averse complied with the shower protocol most often, while the risk tolerant and opportunists took more liberties with risking infection for a potential higher payout. All groups increased their compliance with the communicated probability of infection, albeit at different rates. The risk averse cluster bolstered very high rates of compliance, even when the risk of infection was very low. The risk tolerant were the least responsive to risk messaging, exhibiting high rates of non-compliance even during scenarios with high rates of infection. The opportunists were very responsive to the risk messaging and changed their strategy based upon the rate of contagion; adopting more risk as the probability of infection decreased. Experimental games can be used to identify how risk messaging impacts behavior and can help us adjust communication strategies accordingly. These malleable, digital tools are well suited for observing behaviors that may be difficult to capture in the real-world.

MPL risk assessment was not a strong predictor of simulated biosecurity compliance at the individual level. However, we did find when grouping participants via their MPL assessment, the risk averse participants complied with the biosecurity shower-in, shower-out practice more often than those classified as risk tolerant. We also found that the compliance rate of MPL-classified risk neutral participants was sandwiched between the risk averse and risk tolerant participants' rate of compliance observed during game play. This is interesting as it indicates that a portion of the behavioral signal regarding biosecurity compliance is preserved within the MPL assessment. It is also noteworthy that the MPL risk assessment can be administered without computers and thus may presently be better equipped for sampling low income populations, especially in countries lacking internet access. This shows, from a broad perspective, the MPL risk aversion assessment can be a useful option as it is lightweight and easily deployable in comparison to experimental games.

We should also note that the MPL risk assessment was administered directly after the experimental game. It is possible that the experience within the game may have had a priming effect on our participants. Though in our case, the MPL and experimental game were very different thematically; one testing pure economic risk aversion while the other simulated risk associated with animal disease spread. It is still possible that the profits and losses experienced within the game could have had some influence on the behavioral risk exhibited within the MPL. However, since the risk distributions of the MPL were found to align with previous research, this suggests the order of administration did not have a large impact on our findings, but should be considered in future experiments.

Although MPL risk assessment has many useful applications, the contextual framing provided by experimental games may be better equipped for targeting situational risk and capturing nuanced behaviors at the individual level. Our results show that group behaviors may be inferred using the MPL classification, however could not be reliably applied to the individual. This may suggest that pure economic risk does not fully translate to situational risk, even when the monetary drivers may be comparable. This may be a consequence of the contextual framing embedded within the experimental game. One's actions within the simulated world had an effect on the well-being of a farm and livestock health. This immersion factor may have played a role in motivating individuals to behave differently as opposed to purely economic motivators to achieve a higher payout. This substantiates the use of experimental games for studying complex decision mechanisms and for building circumstantial risk profiles. Identifying behavioral risk in this way can help us better understand the decision-making process, and allow us to study how best to intervene or nudge human behavior toward safer practices.

Identifying risk profiles is pertinent for modeling disease spread across agricultural production networks. Agent based modeling approaches (34–37) can be implemented to simulate disease spread scenarios and economic impacts across supply chain networks (38). Incorporating human behavior components into these models can vastly impact the projections of disease spread (39). Experimental simulations can help inform these models to make more realistic epidemiological forecasts. In particular, using outcomes from experimental games (i.e., rates of compliance) to inform modeling approaches can be useful in projecting potential damages to the supply chain. Within the model, agents' decision heuristics can be adapted from behavioral strategies observed within the experimental game. Experimental games can help us identify realistic and prominent behavioral strategies along with distributions of risk profiles to parameterize modeling initiatives. This can help industry professionals account for these outcomes and work toward strengthening our systems against disease spread.

Quantifying emergent strategies using experimental games can help us better understand the broad behaviors most prominently driving outcomes within these scenarios. Identifying a risk metric, in our case rates of biosecurity compliance, and then applying a clustering algorithm can be a straightforward solution for quickly identifying prominent behaviors. Here, we found three overall strategies: risk tolerant, risk averse, and opportunist. Interestingly, we found the same categorical behaviors from another experimental game that focuses on managing a pig farm's biosecurity investment during various outbreak scenarios (9). Although mechanically these games were very different, as were the risk metric and clustering strategy, the emergent group behaviors were analogous. In the biosecurity adoption game, the risk tolerant invested very little in biosecurity across all scenarios; the risk averse invested the most in biosecurity throughout, and the opportunists changed their investment strategy based upon the rate of contagion. These three broad behavioral groups are thematic and can likely can be applied to a wide variety of decision mechanisms. Identifying membership within these groups can help us work toward individualizing our risk messaging strategies and our ability to influence positive changes in behavior.

Our study supports using experimental games for studying contextualized risk situations. These digital tools can provide nuanced insight into how context influences risk aversion using immersive, complex environments to explore adaptive and dynamic decision-making. Experimental games have a large potential for expanding upon traditional survey instruments by enveloping participants within a complex decision mechanism, allowing us to capture dynamic and evolving behavioral signals. By combining these simulations with survey instruments, we can work toward generating stronger risk profiles with more predictive power for modeling how human behavior impacts crisis situations. More research should be conducted to explore how experimental gaming simulations can be used for quantifying behavior and the resultant impact on the well-being of our agricultural industry. These digital applications can provide valuable insights that may allow us to nudge social ecological systems, such as swine production, to improve our ability to keep our systems disease resilient and promote herd health.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving human participants were reviewed and approved by the University of Vermont Institutional Review Board concerning experiments using human participants (University of Vermont IRB # CHRBSS-16-232-IRB). Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

Author contributions

EC contributed to conceptualization, data curation, formal analysis, software, visualization, and writing—original draft. SM contributed to conceptualization, investigation, methodology, project administration, supervision, and writing—review and editing. LT contributed to software and writing—review and editing. T-LL, AO'K, GB, NC, OL-C, and TS contributed to review and editing. CK contributed to funding acquisition, project administration, and writing—review and editing. AZ contributed to conceptualization and writing—review and editing and funding acquisition. JS contributed to investigation, project administration, funding acquisition, and writing—review and editing. All authors contributed to the article and approved the submitted version.

Funding

This work was supported by the USDA National Institute of Food and Agriculture, under Award No. 2015-69004-23273 and USDA NIFA Award No. 2021-67015-35236.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Author disclaimer

The contents are solely the responsibility of the authors and do not necessarily represent the official views of the USDA or NIFA.

References

1. Waage J, Mumford J. Agricultural biosecurity. Philos Trans R Soc B Biol Sci. (2008) 363:863–76. doi: 10.1098/rstb.2007.2188

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Kim Y, Yang M, Goyal SM, Cheeran MC, Torremorell M. Evaluation of biosecurity measures to prevent indirect transmission of porcine epidemic diarrhea virus. BMC Vet Res. (2017) 13:89. doi: 10.1186/s12917-017-1017-4

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Mankad A. Psychological influences on biosecurity control and farmer decision-making. A review. Agron Sustain Dev. (2016) 36:40. doi: 10.1007/s13593-016-0375-9

CrossRef Full Text | Google Scholar

4. Racicot M, Venne D, Durivage A, Vaillancourt JP. Evaluation of strategies to enhance biosecurity compliance on poultry farms in Québec: effect of audits and cameras. Prev Vet Med. (2012) 103:208–18. doi: 10.1016/j.prevetmed.2011.08.004

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Racicot M, Venne D, Durivage A, Vaillancourt JP. Evaluation of the relationship between personality traits, experience, education and biosecurity compliance on poultry farms in Québec, Canada. Prev Vet Med. (2012) 103:201–7. doi: 10.1016/j.prevetmed.2011.08.011

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Merrill SC, Koliba CJ, Moegenburg SM, Zia A, Parker J, Sellnow T, et al. Decision-making in livestock biosecurity practices amidst environmental and social uncertainty: evidence from an experimental game. PLoS ONE. (2019) 14:e0214500. doi: 10.1371/journal.pone.0214500

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Merrill SC, Moegenburg S, Koliba CJ, Zia A, Trinity L, Clark E, et al. Willingness to comply with biosecurity in livestock facilities: evidence from experimental simulations. Front Vet Sci. (2019) 6:156. doi: 10.3389/fvets.2019.00156

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Trinity L, Merrill SC, Clark E, Koliba CJ, Zia A, Bucini G, et al. Effects of social cues on biosecurity compliance in livestock facilities: evidence from experimental simulations. arXiv preprint arXiv:191012978. (2019)

PubMed Abstract | Google Scholar

9. Clark EM, Merrill SC, Trinity L, Bucini G, Cheney N, Langle-Chimal O, et al. Using experimental gaming simulations to elicit risk mitigation behavioral strategies for agricultural disease management. PLoS ONE. (2020) 15:e0228983. doi: 10.1371/journal.pone.0228983

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Clark EM, Merrill SC, Trinity L, Bucini G, Cheney N, Langle-Chimal O, et al. Emulating agricultural disease management: comparing risk preferences between industry professionals and online participants using experimental gaming simulations and paired lottery choice surveys. Front Vet Sci. (2021) 7:556668. doi: 10.3389/fvets.2020.556668

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Schattman RE, Trinity L, Clark EM, Merrill SC. AwardsUntapped motivation for agricultural conservation behavior. Elementa. (2021) 9:120. doi: 10.1525/elementa.2021.00120

CrossRef Full Text | Google Scholar

12. Zia A, Ding S, Messer KD, Miao H, Suter JF, Fooks JR, et al. Characterizing heterogeneous behavior of non-point-source polluters in a spatial game under alternate sensing and incentive designs. J Water Resour Plann Manag. (2020) 146:04020054. doi: 10.1061/(ASCE)WR.1943-5452.0001242

CrossRef Full Text | Google Scholar

13. Sellnow DD, Lane DR, Sellnow TL, Littlefield RS. The IDEA model as a best practice for effective instructional risk and crisis communication. Commun Stud. (2017) 68:552–67. doi: 10.1080/10510974.2017.1375535

CrossRef Full Text | Google Scholar

14. Xu R, Wunsch D. Survey of clustering algorithms. IEEE Trans Neural Netw. (2005) 16:645–78. doi: 10.1109/TNN.2005.845141

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Harrison GW, Rutström EE. Risk aversion in the laboratory. Res Exp Econ. (2008) 12:41–196. doi: 10.1016/S0193-2306(08)00003-3

CrossRef Full Text | Google Scholar

16. Holt CA, Laury SK. Risk aversion and incentive effects. Am Econ Rev. (2002) 92:1644–55. doi: 10.1257/000282802762024700

CrossRef Full Text | Google Scholar

17. Fellner G, Maciejovsky B. Risk attitude and market behavior: evidence from experimental asset markets. J Econ Psychol. (2007) 28:338–50. doi: 10.1016/j.joep.2007.01.006

CrossRef Full Text | Google Scholar

18. Anderson LR, Mellor JM. Predicting health behaviors with an experimental measure of risk preference. J Health Econ. (2008) 27:1260–74. doi: 10.1016/j.jhealeco.2008.05.011

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Parisi T. WebGL: Up and Running. Sebastopol: O'Reilly Media, Inc. (2012).

Google Scholar

20. Rand DG. The promise of Mechanical Turk: how online labor markets can help theorists run behavioral experiments. J Theor Biol. (2012) 299:172–9. doi: 10.1016/j.jtbi.2011.03.004

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Mason W, Suri S. Conducting behavioral research on amazon's mechanical turk. Behav Res Methods. (2012) 44:1–23. doi: 10.3758/s13428-011-0124-6

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Buhrmester M, Kwang T, Gosling SD. Amazon's mechanical turk: a new source of inexpensive, yet high-quality, data? Perspect Psychol Sci. (2011) 6:3–5. doi: 10.1177/1745691610393980

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Cheong L, Bleisch S, Kealy A, Tolhurst K, Wilkening T, Duckham M. Evaluating the impact of visualization of wildfire hazard upon decision-making under uncertainty. Int J Geogr Inf Sci. (2016) 30:1377–1404. Available from: doi: 10.1080/13658816.2015.1131829

CrossRef Full Text | Google Scholar

24. Hartigan JA, Wong MA. Algorithm AS 136: a k-means clustering algorithm. J R Stat Soc C. (1979) 28:100–8. doi: 10.2307/2346830

CrossRef Full Text | Google Scholar

25. Ng A. Clustering with the k-means algorithm. In: Machine Learning. Stanford. (2012).

Google Scholar

26. Kodinariya TM, Makwana PR. Review on determining number of cluster in K-means clustering. Int J. (2013) 1:90–95.

PubMed Abstract | Google Scholar

27. Benesty J, Chen J, Huang Y, Cohen I. Pearson correlation coefficient. In: Noise Reduction in Speech Processing. New York, NY: Springer (2009). p. 1–4.

Google Scholar

28. Mann HB, Whitney DR. On a test of whether one of two random variables is stochastically larger than the other. Ann Math Stat. (1947) 18:50–60. doi: 10.1214/aoms/1177730491

CrossRef Full Text | Google Scholar

29. d'Agostino RB. An omnibus test of normality for moderate and large size samples. Biometrika. (1971) 58:341–8. doi: 10.1093/biomet/58.2.341

CrossRef Full Text | Google Scholar

30. D'Agostino R, Pearson ES. Tests for departure from normality. Empirical results for the distributions of b₂ and $\sqrt{b_1}$. Biometrika. (1973) 60:613–22. doi: 10.1093/biomet/60.3.613

CrossRef Full Text | Google Scholar

31. McHugh ML. The chi-square test of independence. Biochem Med. (2013) 23:143–9. doi: 10.11613/BM.2013.018

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Cramer H. Mathematical Methods of Statistics. Princeton, NJ: Princeton Univ. Press (1946).

Google Scholar

33. Virtanen P, Gommers R, Oliphant TE, Haberland M, Reddy T, Cournapeau D, et al. SciPy 1.0: fundamental algorithms for scientific computing in python. Nat Methods. (2020) 17:261–72. doi: 10.1038/s41592-020-0772-5

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Galán JM, López-Paredes A, Del Olmo R. An agent-based model for domestic water management in Valladolid metropolitan area. Water Resour Res. (2009) 45:6536. doi: 10.1029/2007WR006536

CrossRef Full Text | Google Scholar

35. Schreinemachers P, Berger T. An agent-based simulation model of human-environment interactions in agricultural systems. Environ Model Software. (2011) 26:845–59. doi: 10.1016/j.envsoft.2011.02.004

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Berger T. Agent-based spatial models applied to agriculture: a simulation tool for technology diffusion, resource use changes and policy analysis. Agric Econ. (2001) 25:245–60. doi: 10.1111/j.1574-0862.2001.tb00205.x

CrossRef Full Text | Google Scholar

37. Farmer JD, Foley D. The economy needs agent-based modelling. Nature. (2009) 460:685. doi: 10.1038/460685a

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Wiltshire S, Zia A, Koliba C, Bucini G, Clark E, Merrill S, et al. Network meta-metrics: using evolutionary computation to identify effective indicators of epidemiological vulnerability in a livestock production system model. J Artif Societies Soc Simulat. (2019) 22:1–8. doi: 10.18564/jasss.3991

CrossRef Full Text | Google Scholar

39. Bucini G, Merrill SC, Clark E, Moegenburg S, Zia A, Koliba C, et al. Risk attitudes affect livestock biosecurity decisions with ramifications for disease control in a simulated production system. Front Vet Sci. (2019) 6:196. doi: 10.3389/fvets.2019.00196

PubMed Abstract | CrossRef Full Text | Google Scholar