Edited by: Mary M. Christopher, University of California, Davis, United States
Reviewed by: Diana K. Davis, University of California, Davis, United States; Malathi Raghavan, Purdue University, United States; Tom Johnston, University of York, United Kingdom
Specialty section: This article was submitted to Veterinary Humanities and Social Sciences, a section of the journal Frontiers in Veterinary Science
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
Disease maps are important tools in the management of disease. By communicating risk, disease maps can help raise awareness of disease and encourage farmers and veterinarians to employ best practice to eliminate the spread of disease. However, despite the importance of disease maps in communicating risk and the existence of various online disease maps, there are few studies that explicitly examine their usability. Where disease maps are complicated to use, it seems that they are unlikely to be used effectively. The paper outlines an attempt to create an open access, online, searchable map of incidents of bovine tuberculosis in England and Wales, and analyzes its usability among veterinarians. The paper describes the process of creating the map before describing the results of a series of usability trials. Results show the map to score highly on different measures of usability. However, the trials also revealed a number of social and technical limitations and challenges facing the use of online disease maps, including reputational dangers, role confusion, data accuracy, and data representation. The paper considers the challenges facing disease maps and their potential role in designing new methodologies to evaluate the effectiveness of disease prevention initiatives.
Disease maps are important tools in the management of disease. On the one hand, disease mapping is used to detect relationships between human and animal diseases (
Despite the apparent importance of using maps to communicate animal disease risks, there are few attempts to examine their use. Where disease maps have been created, it seems that they are complicated to use, require extensive training and only used by those responsible for data entry (
The paper examines the development and usability of the ibTB disease map through a series of usability trials with veterinarians. In doing so, the paper also explores the social and technical limitations and challenges facing the use of online disease maps to enhance and encourage greater levels of biosecurity on cattle farms.
In order to make bTB incident data available, a searchable website was designed.
A series of usability trials were conducted to assess the ease of use of ibTB and users’ perceptions of the role ibTB could play in managing bTB. Usability is defined in ISO 9241-11 as comprising three key dimensions: satisfaction (i.e., users’ subjective reactions), efficiency (i.e., the level of resource consumed in performing tasks), and effectiveness (i.e., the ability and quality of users to complete tasks) (
To assess the usability of ibTB, a task-based approach was adopted in which users were asked to complete five different tasks reflecting a range of activities that ibTB could be used for, varying in difficulty and geography (see Table
Tasks used for user testing of ibTB.
Task | Description | Function | Geographical scale of search |
---|---|---|---|
1 | Find out the bTB status of the last farm you visited? If they are under-restrictions, when did it happen? | General knowledge | Local |
2 | A farmer is worried because he has heard in the pub that one of his neighbors has gone down with bTB. Is the farmer right? | Checking neighbors | Local |
3 | One of your clients asks you about some cattle he’s interested in near XXXXX. What can you tell the farmer about the bTB situation around XXXXX? How many farms are currently under-restriction, and how many came off Tb restrictions in 2014? | Informed buying | Regional |
4 | A client is thinking of renting some ground near CPHH XXXX, but he doesn’t know the TB situation. What can you find out for the farmer? Are there any ongoing breakdowns in the area? | Advising farmers | Regional |
5 | You are writing a paper on bTB in the Low risk Area. How many ongoing bTB breakdowns are there in Norfolk, and how many farms had their restrictions lifted in 2014? | Epidemiology advocacy | National |
Results of the SUS survey (
Survey item | Dimension of usability ( |
Type of veterinarian (mean score) |
||
---|---|---|---|---|
APHA | Private sector | All veterinarians | ||
I think I would like to use this system frequently | Usability | 1.67 | 2.94 | 2.48 |
I found the system unnecessarily complex | Usability | 3.22 | 3.63 | 3.48 |
I thought the system was easy to use | Usability | 2.89 | 3.44 | 3.24 |
I think that I would need the support of a technical person to be able to use this system | Learnability | 4.00 | 3.88 | 3.92 |
I found the various functions in this system were well integrated | Usability | 2.67 | 3.00 | 2.88 |
I thought there was too much inconsistency in the system | Usability | 2.78 | 2.94 | 2.88 |
I would imagine that most people would learn to use this system very quickly | Usability | 3.44 | 3.31 | 3.36 |
I found the system very cumbersome to use | Usability | 3.22 | 3.5 | 3.4 |
I felt very confident using the system | Usability | 2.67 | 3.56 | 3.24 |
I needed to learn a lot of things before I could get going with this system | Learnability | 3.11 | 3.88 | 3.6 |
SUS score | 74.17 | 85.16 | 81.2 |
Usability trials were conducted with farmers and veterinarians. This paper reports only on veterinarians’ views of ibTB. Veterinarians were included in usability testing because ibTB has the potential to provide veterinarians with a complete epidemiological picture of bTB in their area. As influential experts, veterinarians also play an important role in advising farmers on bTB (
Participants were provided with a laptop with external mouse to complete each task. Each participant’s activity was recorded using “Silverback”—a screen capture application that records screen activity, mouse clicks, and audio. Following each usability trial, the video was reviewed and instances of user frustration (drawing mouse circles and double clicking) were noted, along with the number of mistakes made during the task and the time taken to complete it. These were cross-checked with separate observation notes taken during each usability trial. On three occasions Silverback failed to record screen and/or audio activity. In a further two cases, there was no accessible WiFi or 3G signal available to connect to ibTB. Instead, the participant’s own computer was used to conduct the usability tests, but without the ability to record using Silverback. These users are excluded from the analysis of task completion. Responses to open-ended questions were recorded using a digital voice recorder. Transcripts were prepared and analyzed thematically to draw out shared uses, concerns and limitations relating to ibTB. Survey data were entered into SPSS to calculate the SUS scores, and conduct descriptive analysis and statistical tests of association and difference.
Veterinarians were asked about their use of maps and computers in their daily work. Use of computers was rated as very frequent (mean 4.92, 1–5 scale) and veterinarians generally responded that they felt comfortable using a computer (mean 4.44, 1–5 scale). Veterinarians’ use of maps (mean 3.92) and online maps (mean 3.68) was less frequent.
All veterinarians welcomed the development of ibTB. Private veterinarians in particular were pleased to be able to see these data, suggesting that the information was vital for them to work with their clients to help them manage bTB. These veterinarians argued that previous data restrictions reflected a perceived lack of trust in private veterinarians’ epidemiological skills by Government, but that ibTB could now help them engage with farmers and government veterinarians on an equal footing. In democratizing bTB information, veterinarians therefore saw three clear uses for ibTB:
Veterinarians also commented on the potential use of ibTB among farmers. While opinions varied on the extent to which farmers would use ibTB and the impact it would have on their behavior, the following three main uses were identified:
Results of the usability trials are shown in Table
Completion of tasks set in ibTB.
Completion rate | Mean completion time (s) | Mouse circles ( |
Double clicks ( |
Mistakes ( |
Ease of completion (1–7 scale) | |
---|---|---|---|---|---|---|
Task 1 | 19/20 | 121.55 | 2 | 6 | 12 | 5.88 |
Task 2 | 18/20 | 101.65 | 2 | 4 | 4 | 4.76 |
Task 3 | 20/20 | 159.25 | 3 | 11 | 11 | 4.92 |
Task 4 | 19/20 | 103.05 | 3 | 0 | 0 | 5.76 |
Task 5 | 20/20 | 144.65 | 9 | 8 | 1 | 5.32 |
All Tasks | 96/100 | 126.03 | 19 | 29 | 28 | 5.32 |
The average time to complete a task was 126 s. Task 2 was completed quickest on average, while Task 3 had the slowest average although was completed by all participants. It is worth pointing out that variation in the time taken to complete the tasks was also due to the users’ level of interest in bTB. For example, some users were keen to explore in detail the incidence of bTB in surrounding herds for some tasks (e.g., Tasks 3 and 4) in order to form epidemiological judgments.
There were few visible signs of frustration among participants. Mouse circles were counted on 19 occasions, and double clicking on 29. One sign of frustration not recorded but which became evident during testing was respondents rapidly zooming in and out of the map. Sometimes this was due to poor mouse control but it also occurred when users were trying to find a specific location (such as during Tasks 3 and 5). One problem with the use of ESRI maps was that place names could disappear at different magnification levels making navigation awkward.
Mistakes were made by 17/20 users, although only a total of 28 individual errors were recorded across all five tasks. Twelve mistakes were recorded for Task 1. This could have been due to the fact that this was the first task. A common mistake was the incorrect use of the search function. On being asked to search for a farm using its CPHH, veterinarians frequently left out the final part of the code, resulting in a failed search. Task 3 also had a significant number of mistakes due to inappropriate searches. On being asked to search for a specific place, 10 users attempted to search by typing the place name into the search box. When this failed to work, users completed a Google search to find a postcode with which to search. The remainder navigated to the location where they thought the town was located before finding it. These mistakes highlighted the importance of including a place name search function in future versions of ibTB.
When asked to rate the ease of completing each task, all responses were positive. The average ease of completion for the five tasks was rated at 5.3 out of 7. Task 1 was rated easiest despite having most mistakes. Tasks 2 and 3 were rated lowest. In both cases, users experienced problems of identifying which farm they had searched for because of a failure of ibTB to show a placeholder for all farms. Users who rated the tasks as easy were also more likely to complete the tasks in less time (
Overall, ibTB scored highly on the System Usability Scale (see Table
Items on the SUS to which users agreed most included the ease of use, the ease of learning how to use ibTB, and users’ confidence while using ibTB. Items on the SUS that received most negative feedback included the belief that the system was very cumbersome, inconsistent, and unnecessarily complex. This may reflect some of the mistakes and problems highlighted in “Task completion” section, such as the inability to search for places and not highlighting farms searched for.
Usability was rated highest among private veterinarians whose average score was 85/100. Government veterinarians working for APHA scored slightly lower (74/100), but both scores indicated above average usability. The lower scores for government veterinarians are likely to be due to the fact that other bTB resources are available to them. While private veterinarians were generally enthused by the availability of “new” data, veterinarians working in APHA commented that other mapping and data management software was more suitable.
While the results of the SUS suggest ibTB to be highly usable, comments by veterinarians during and after the tasks revealed a number of social and technical limitations to the system. These are described as follows:
Despite the significance of maps and mapping to epidemiology and disease management, it is surprising to find few studies of their usability in animal disease management. The usability trials conducted for this research have proved useful in helping to further refine ibTB to enhance its usefulness. This has included the following: making changes to the appearance of the map; including a placeholder to identify the location of each search; changing clustering breakdowns by dynamically creating nested grids depending on the map resolution, counting the breakdowns in each cell and clustering if > 20; expanding the range of clusters to 5 (1 breakdown, 2–20, 21–150, 151–500, >500); and making changes to the search facility. The website has also been made fully compatible with mobile devices. Technical limitations have meant that the ability to search for place names—one of the main problems faced by users—has not been included.
The usability trials demonstrated that veterinarians had high levels of satisfaction, could complete tasks easily, and rated ibTB highly on the SUS. However, despite the high levels of usability, veterinarians still identified key technical and social challenges that affected their use of ibTB. For some, these challenges suggest the need for further research on the way maps represent and communicate disease data. The usability trials suggested the need for a broader consideration of how different representations of bTB can influence farmer and vet behavior. Studies on behavior change argue that the way a phenomenon is framed can impact upon subsequent behavior. For example, research suggests that avoiding losses is often a greater behavioral driver than winning (
On the other hand, thinking about how disease maps represent data should also direct disease mappers to consider the underlying assumptions and power relations within their maps. Veterinarians’ responses to ibTB revealed tensions within the assumption that they should play an important in using and promoting disease maps to farmers. For example, while previous research has highlighted the importance of veterinarians in encouraging farmers to implement biosecurity or advise them in animal health (
Similarly, the usability trials of ibTB also highlighted the contested boundaries of who or what counts as epidemiology—what critical cartographers refer to as the underlying political choices and meanings hidden within maps (
Despite the importance of maps in animal disease epidemiology, there have been few studies analyzing their effectiveness of communicating animal disease risks to farmers, veterinarians, and the public. This paper describes the use of animal disease surveillance data for bTB to create a freely available, searchable online map of current and historic incidents of bTB in England and Wales. In usability tests with veterinarians, the paper shows that epidemiological tasks could be completed quickly and easily, and usability was rated excellent. In doing so, the paper shows the potential role for disease maps in evaluating farmer behavior and devising effective mechanisms for risk-based trading. However, the paper also shows that the creation of disease maps can challenge traditional sets of power relations in disease management, question what counts as epidemiology, and require choices to be made over who can and cannot participate in the management of disease. As open data initiatives become more common as governments seek innovative digital solutions to intractable problems, disease mappers will need to find ways of resolving these challenges in order for disease maps to make effective contributions to the management of animal disease.
This study was carried in accordance with the recommendations from the Research Ethics Committee at the School of Geography and Planning, Cardiff University. All research participants provided their consent following an information briefing on the nature of the research. Participants have been anonymized in the research findings.
AM, WW, and NT designed and created the ibTB website. GE designed, conducted, and analyzed results from the usability trial. GE wrote the manuscript which was agreed by all authors.
ibTB was produced with funding from the Department for Environment, Food and Rural Affairs (Defra). AM is employed by the Animal and Plant Health Agency (APHA), an agency of Defra. Interpretation of the results of the usability trial rests with the authors. The reviewer DD and handling Editor declared their shared affiliation.
1Attempts to make maps of disease outbreaks available on-line include the OIE’s World Animal Health Information Database (WAHIS) (http://www.oie.int/wahis_2/public/wahid.php/Wahidhome/Home). Other examples of online maps of animal disease include:
2Animal disease policy is a responsibility for the devolved administrations in Wales, Scotland and Northern Ireland. Wales altered legislation in 2015, whilst Scotland and Northern Ireland have chosen not to make this information available.
3The website was commissioned by the Department for Environment, Food and Rural Affairs (Defra) who are responsible for bTB policy in England.