KAZNET: An Open-Source, Micro-Tasking Platform for Remote Locations

Introduction

The importance of timely and accurate data cannot be overemphasized when it comes to decision making (World Bank, 2018). Whether these decisions take place in the social or environmental domains, at the household-, community-, or policy-level, decision makers require accurate information. Gathering such data is often expensive and time consuming (Bitso et al., 2020), which especially holds true in remote and difficult to reach regions. The result is that these areas are often poorly-sensed, which leads to mis-representation and sometimes oversight all together.¹ The resulting data scarcity has slowed the achievement of development goals aimed at improving the livelihoods of poor communities (Bitso et al., 2020).

Conventional field surveys are the workhorse of many environmental and social science fields that require data from outside of the lab. Primary data collection from enumerator-to-respondent surveys, focused group discussions, sensor readings, and key informant interviews collected through on-site trained data monitors or enumerators, is the norm for those working on research or development in rural settings (Nyariki, 2009). These approaches usually either involve experts collecting data directly or experts training others to collect data in the expert way, both of which can generate very accurate data but can also be extremely expensive. In addition to the large transportation, lodging, feeding, training, salary and maintenance requirements, such approaches are not flexible or dynamic, and therefore struggle to respond changing environments or dynamic data needs. Equally notable is that the large, fixed costs of launching such field campaigns can reduce the frequency of data collection even when higher frequencies are clearly preferred. And, because the campaigns are infrequent, they can represent an effectively unique opportunity for a researcher or statistics office, which can lead to larger, longer, and more expensive data collection efforts, thereby further reducing the frequency of those activities.

Micro-tasking leverages advances in digital and mobile technologies to draw on a large pool of in-situ data collectors. Complex data collection processes are commonly broken down into a series of smaller and less complex tasks so that contributors do not have to go through costly onboarding processes typical of conventional methods (Minet et al., 2017; Durward, 2020; Sveen et al., 2020; Van De Gevel et al., 2020). The expansion in access to, and use of, smartphones simultaneously grow the pool of data collectors available for micro-tasking, provides ICT training, provides a channel to recruit and remotely train potential contributors, and provides a platform for the micro-tasking software itself (Mtsweni and Modiba, 2020).

In any data collection activity, limited supervision can generate low quality data and micro-taskers often operate with little individual supervision (Gadiraju et al., 2015), but there are several mechanisms for improving data quality that are available to micro-tasking that are less available for conventional approaches (Neto and Santos, 2018). For example, the low cost of data collection provides a high density of observations that can be used to cross-validate data between observations and flag outliers. Further, ICT-related features, such as photo verification, geo-fencing, which only allows tasks to be completed in a specific location, temporal gates, which only allow tasks to be completed on a specific time or date, and dynamic feedback, can be easily integrated into product design. Such features reduce the risk of lower data quality that data-collectors might generate when working without on-site supervision (Robert, 2019). In addition, when these features are combined with dynamics incentives, they can be used to direct sampling, thereby addressing concerns that contributors' preferences will drive sampling and bias the resultant data (Jensen et al., 2017).

Data collection through micro-tasking offers real-time flexibility that is not possible under conventional methods (Kittur et al., 2008). For example, networks of existing contributors can be activated or deactivated, and/or new contributors brought on board in response to changing data needs. New tasks targeting different subjects can also be launched on the same platform and performed together or independently with other tasks (Kittur et al., 2008). Further, data collection forms, question, and related parameters can be adjusted and re-deployed with little effort from platform administrators. The real-time adjustments on the type and content of tasks are a particularly relevant and important feature that can be used to quickly activate information gathering in response to infrequent and acute events, such as drought or pandemic, in ways that conventional approaches could not because the of lag time associated with recruiting, training and transporting appropriate data collectors. While this feature further reduces the cost of setting up data collection, it also allows multidisciplinary approaches to projects by pooling together expertise in different subjects to use a single platform (Cuccolo et al., 2021).

One key feature of micro-tasking is that contributors continue with their daily lives, while also occasionally completing tasks when convenient. The result is that local data collectors are usually the favored because they have the least transportation time and expenses to incur, and often preferred by the project because they are likely to have local knowledge and access that can improve data quality and further reduce transaction costs (e.g., time spent locating respondents). Further, such arrangements avoid many of the costs incurred to support enumerators during conventional data collection campaigns, which are fulltime for the enumerators and therefore require food, lodging, transportation and on-site management (Edgar et al., 2016). Once the network of contributors is activated, rewards for completed tasks are the main cost incurred. Dynamic reward systems can be set up that incentivize quality and quantity. This flexibility in rewards avoids paying for, and disincentivises poor quality data. Contributors also have the freedom to perform tasks that give them maximum reward for their effort, while administrators can use a value of information framework and adjust rewards to incentivize increased or reduced collection of specific tasks to reflect the marginal value of an additional observation (Allahbakhsh et al., 2013; Jensen et al., 2017). In the case where incentives are monetary, the activity also provides the contributors with an additional source of income. Other types of rewards for participating, such as providing contributors with access to processed data, can also further reduce costs and increase access to information by contributors in ways that conventional survey methods rarely do (Chelanga et al., 2021).

Despite the overarching benefits and extensive use in many fields, there are mixed views on the value of micro-tasking (Liu, 2017; Phuttharak and Loke, 2018). Critics argue that the data generated through micro-tasking has more errors and is more likely to be biased than data collected by scientists, technicians, or enumerators, using conventional instruments. However, proponents of micro-tasking argue that projects using non-expert contributors continue to produce high-quality data, equivalent to and sometimes surpassing trained enumerators (Eklund et al., 2019). Other scholars and practitioners argue that each dataset generated through micro-tasking should be judged individually, based on the context in which the project is implemented, as it could strongly complement traditional methods (Uhlmann et al., 2019).

In this study, we investigate the potential for using micro-tasking to monitor environmental and socio-economic indicators in remote and underrepresented pastoral regions in Africa, one of the most challenging contexts for data collection. To this end, we first present a phone-based micro-tasking platform named KAZNET that was explicitly developed in and for the extensive pastoral systems of the Horn of Africa, where credible data on household nutrition, rangeland conditions, market prices, consumer product availability and quality, disease outbreaks, conflict, forage availability, and other welfare and environmental indicators are dearly lacking (Tollens, 2006; Meena and Singh, 2013). Importantly, KAZNET was developed to suit the pastoral context, both in its user-interface that pastoralists use to browse, select, and complete tasks and its offline capabilities, which are unique in the micro-tasking field but necessary for this environment. We then provide several examples of high-frequency data collected with KANZET and discuss them in terms of quality and potential value for long-term high-frequency monitoring. Finally, we discuss ongoing research on improving the long-term sustainability of the platform, with notable advances in dynamic task allocation, reward optimization, and engagement with the private sector.

The remainder of this manuscript is organized as follows: the next section provides more details on the platform and its development; this is followed by a section on results which demonstrates KAZNET's functionality and value through a series of three applications. The Discussion section closes the manuscript, highlighting the platform's scalability and limitations.

Methods

The KAZNET platform was designed to operate as a micro-tasking platform to be used by pastoralists in rural and remote locations. Its origins lie in the demand for lower-cost options for collecting structured and high-quality data from remote regions by the International Livestock Research Institute (ILRI) research team. This demand led to a search for existing viable alternatives to the standard survey approaches that the team was using at the time, which resulted in a review of the literature and multiple discussions with ICT-for-development experts. The efforts revealed that, while crowdsourcing approaches might meet the objective, there were few micro-tasking or crowdsourcing platforms targeting the agricultural sector and those that existed all focused on crop farming. There were no platforms developed specifically for pastoral systems or even with pastoral systems in mind. Importantly, none of the platforms functioned offline, making them effectively unusable in most pastoral settings.

While the review did not offer an existing solution, it did inform the design and strategy for a scoping mission focused on assessing the potential for micro-tasking in dryland pastoral settings. The scoping mission took place in 2016 with the objective of assessing the demand for improved data from and to the dryland pastoral settings and the infrastructure available for ICT-based solutions to meet that demand. It was carried out across different sectors operating in the drylands, including the private sector, international development organizations, government institutions, and pastoralists.² The study showed that there was high and unmet demand in the public and private sectors for a reliable system that could collect and disseminate relevant information at high-frequency and low-cost. Further, smartphone penetration was observed to be high and seemingly offered an opportunity for micro-tasking or citizen science approaches to data collection.³

While the types of information demanded varied across stakeholders in correspondence to their diverse areas of operation, the need for improved livestock market information was identified by multiple stakeholders. A second scoping mission, this time specifically targeting livestock market information, was undertaken in 2017. The objective of this activity was to develop and pilot the tasking process to be used in remote livestock markets. The results of this second scoping mission and literature on incentive infrastructure used in other micro-tasking platforms reinforced the need for flexibility in the platform.

In 2017, ILRI engaged Ona, a software engineering firm located in Kenya, with a background that includes, among other products, developing Ona Data, a mobile data collection platform based on Open Data Kit (ODK). ODK is an open-source survey software that was developed to function in no/low bandwidth environments—thereby meeting our first requirement of functioning off-line. Because ODK, and Ona's version of it, were developed to make it easy for researchers to develop, launch, and update surveys with little or no coding experience, it met our second requirement—flexibility—in that users can easily adjust or change survey tools in near-real-time. Ona has gone on to develop the entire KAZNET platform, which is completely open source and relies on the same tool-building approaches that all ODK users will be familiar with.

Micro-Tasking Platform Process

The KAZNET platform consists of two main components: a web application, and a mobile application. The web application is used by the administrators of the platform to design and manage tasks, approve or reject submitted tasks, calculate payments, and access the submitted data. The mobile application provides contributors with a menu of available tasks, with descriptions of parameters and filtering options for geofences and temporal gates. It allows the registered contributors to download tasks for completion offline, perform tasks, submit tasks, and receive feedback on the quality of their submissions (e.g., the reason that a task was rejected).

Figure 1 demonstrates the sequence of actions between an identified demand for data and the delivery of that data. A detailed description of the two components follows.

FIGURE 1

Figure 1. The design flow of the KAZNET platform activities.

Web Application

The KAZNET web application is custom built by Ona to provide an interface for developing, deploying, managing, and approving tasks. In the current deployment, tasks are defined as an ODK form with a related set of parameters that define protocols. Task development then includes two steps—authoring forms and defining parameters. In our case, forms are authored using the Ona Data platform, but other platforms (e.g., Kobo Collect, Survey CTO, ODK Cloud) could feasibly be linked to the KAZENT web application. Importantly, all the standard features of ODK forms are available for the form development, including question types, time stamps, geo-stamps, photo capture, video/audio playback, skip logic/branching, as well as application features such as remote updating of forms. The task's parameters are then defined within the KAZNET web application. The parameters include (1) where the task can be completed—the geofence⁴—if there are restrictions, (2) when tasks can be completed—the temporal gate⁵—if needed, (3) the maximum frequency that the task can be completed, (4) the qualification requirements of the task, and (5) the reward for accepted task completions. The task is then defined as the ODK form and the set of parameters. Administrators can also provide a set of auxiliary instructions to the contributor. Figure 2 includes a screen shot of the web page used to parameterize a task. Note that parameters and notes are communicated to the contributor through their mobile application, which we discuss below.

FIGURE 2

Figure 2. The KAZNET web application is used to define task parameters.

Contributors are registered, allocated login credentials, and progressively categorized by performance and experience. Those with sufficient experience and consistently high performance (experts) could have access to some tasks that are deemed too challenging or sensitive for unproven beginners. Rewards are set to reflect the data needs and the complexity of the tasks—tasks requiring more effort are priced higher than those that require less effort. The rewards can be dynamic to respond to incoming data, for example, reducing rewards for tasks as data goals are met.

All submitted tasks are managed using the Ona Data web application. Here, administrators can individually or bulk accept or reject submitted tasks. In our experience, most rejections are either automatic, because they violated a parameter, or are because the photo does not meet the requirements. Figure 3 provides a screenshot of a task being validated. Here, the photo of a camel, the location and time of the task, and the domain of prices all help us check for data quality. Further, we could cross-validate this submission using data from other submissions in the same market on the same day. Rejected submissions are accompanied by justifications, whereas accepted submissions are coupled with applauding statements. The review outcomes and notes are communicated to the contributors at the mobile application interface. The reviewed data are then available for aggregation, generation of information outputs, reward calculations, and retrieval.

FIGURE 3

Figure 3. Using KAZNET's web application to validate a submitted task by checking its photo.

Mobile Platform

The KAZNET mobile application is an integration of a tasking wrapper on top of the ODK mobile client software. The wrapper performs two main functions. The first is to create a user interface, which allows contributors to browse available tasks and related protocols (e.g., locational requirements, rewards for completion), download tasks for completion offline, manage tasks, receive feedback on tasks, and track profile-level attributes. The second is to check for conformity to task parameters, for example, that the device is within the livestock market before the contributor can complete the livestock market task.

Once the contributor selects a task that she would like to complete, the task itself opens as an ODK form. This allows KAZNET to leverage the years of investment by ODK in form development for mobile devices. The forms are then completed according to the instructions. As mentioned earlier, ODK forms have a wide variety of functions, including recording geo-points, a diverse set of question types, taking photos, and playing audio or video files. Completed tasks are saved on the device for submission when the mobile device has connectivity.

Figure 4 provides a series of screenshots of screens that a contributor would see while using the mobile application. In Panel 1, the contributor is online and uses the Explore tab to browse available tasks, filtering by location if they choose. To learn more about a task, the contributor can select the task. Panel 2 has a screenshot from the livestock market task information screen as an example of what the contributor might see when selecting a task for more information. Here, the administrators can provide details on the reward, location, and frequency of the task, as well as any other instructions. Contributors then download tasks that they would like to save on their device for completion later. Panel 3 shows a contributor in the My Tasks tab, which shows that she has three tasks available for completion, either on-line or off-line. Note that a single task can be completed as many times as the parameters allow. So, for example, once downloaded, the contributor can complete the Livestock Price and Quality task multiple times in each market, each week.

FIGURE 4

Figure 4. The KAZNET platform front end schema.

To complete a task, the contributor selects the task, at which time it is launched in the ODC Collect mobile client (Panel 4). Once completed, tasks are stored on the device until they are submitted, which takes place in the background when the device has connectivity. The Submitted tab documents the status of each completed task (Panel 5). Submitted tasks are pending, until they have been approved or rejected by an administrator, which can be done individually, task-by-task, or using spot checks and bulk acceptance/rejection, or using an automated process. When the administrator provides a comment on a task, this is indicated by a small blue comment icon on the task in the Submitted tab and the contributor can view that comment by selecting the task. The Profile tab provides a summary of the contributor's performance (Panel 6).

Both the wrapper (KAZNET) and ODK Collect are available on the Google Play Store for download. Login credentials, which includes a username and a password provided by the system administrator, are used to log in to KAZNET, which will then provide credentials to the ODK client.

Results

The KAZNET platform was rolled out in 2018 to collect information on livestock markets in northern Kenya. In 2021, the network of contributors was expanded to collect data for a pilot network of ‘Sentinel Zones’ aimed at monitoring the impacts of climate shocks on the rangelands and the pastoralists who depended on them. Micro-tasks were developed to regularly collect data on markets, livestock production, rangelands, and human health, and, as of writing in 2022, data collection is ongoing in Kenya and Ethiopia. The next subsections use three data acquisition activities to illustrate the platform's value and flexibility.

Livestock Market Information

Livestock market data has historically been highly demanded by policy makers but has proven to be challenging and costly to collect (Stuth et al., 2006). Market information asymmetry adversely affects producers and traders, leading to losses and restricting the growth of markets (Roba et al., 2018). Indeed, there have been several large government and donor programmes in the pastoral regions aimed at collecting livestock market data, but none have provided consistent data from remote markets, which suggests demand for the data but the need for improved processes for collecting it (Kariuki et al., 2009).

In late 2018, ILRI was commissioned by the Kenyan State Department of Livestock to use KAZNET to collect livestock market information as a complement to its existing national livestock market information system.⁶ This was done in recognition of the fact that a micro-tasking approach might be able to overcome many of the challenges facing the existing system, which was struggling to maintain a consistent, accurate, and up-to-date database on market information from the pastoral areas.

A set of tasks was designed to address the challenges associated with conventional data collection methods while also capturing the same types of information collected by other livestock market surveys in the region. Survey forms were developed to collect information on the same indicators collected by the market monitors employed by the National Drought Management Authority in Kenya and geofencing and temporal gates were set to ensure that the data was collected only from within the livestock markets and during market hours on market days. These checks and processes are performed by the mobile application and do not require connectivity. To provide an addition opportunity for data verification, photos of the animal or livestock market in question must accompany tasks related to livestock price or market volume.

Submitted tasks first go through an automated screening process, during which some submissions are rejected automatically, at which time the contributor is notified of the rejection and reason. Those that pass the automated screening can be checked manually by an administrator, for example, checking to see if the price of a goat is accompanied by a photograph of a goat. As such, several other task-level protocols combine with web application infrastructure as well as financial incentives to define the quality and validity of the data.

It is worth mentioning that the contributors were also able to disseminate the information collected by them to their community and their own networks. For example, a task can be easily developed by asking the contributor to complete a short extension activity, such as providing information on how to respond to a current livestock disease outbreak. For more information on this use of KAZNET (see Chelanga et al., 2021).

Since 2018, data collection through KAZNET has expanded to new markets in Kenya and Ethiopia in response to demand from addition stakeholders. A network of contributors has been established in a total of 14 pastoral livestock markets (Figure 5). Unlike the conventional methods for collecting livestock market information, by which an individual is assigned to collect data, the network of contributors collect data throughout the day, providing a dense cloud of data that captures the great deal of variation that exists within livestock markets.

FIGURE 5

Figure 5. Sampled livestock markets in pastoral drylands of Kenya and Ethiopia.

As a demonstration of the data collected, we use the example of goat prices. Figure 6 illustrates weekly mean goat prices from across 10 livestock markets in northern Kenya, disaggregated by contributor-assessed body type. As expected, prices are related to body type; fat goats are more expensive than moderate, and the latter are more expensive than thin or emaciated goats. Alulu et al. (2020) use this data to show that more than half of the variation in the price of goats is explained by body type. Here we note that the red vertical line between March and April 2020 marks the onset of restrictions in Kenya related to the COVID-19 pandemic. During the period to the right of the vertical red line, movement was restricted in Kenya and most field operations, including most field-based data collection activities, were disrupted, highlighting another advantage of crowd-based data collection processes. Interestingly, the COVID-19 pandemic has had no discernible impact on prices.

FIGURE 6

Figure 6. Weekly mean goat price by body condition in 10 livestock markets in the Kenyan drylands. The red vertical line indicates when COVID-19 related restrictions started in Kenya.

Child MUAC and Household Milk Production

Tracking indicators of household nutrition is important for informing policies aimed at reducing malnutrition and improving the welfare of households. Effective policy formulation requires high quality and high spatial-temporal resolution data and is a great need for household-level nutrition data to better understand priorities in tackling malnutrition (Hawkes and Fanzo, 2017). In many instances, data used to compute regional and global malnutrition estimates are obtained from a single or annual survey, which limits our ability to distinguish between seasonal variation and annual averages, although those figures are rarely presented as a seasonal estimate (World Health Organization, 2019). Other nationally representative surveys are representative at low resolution, so appropriately aggregating their data can often mask variations in sub-regions, especially in hard-to-reach areas (Akombi et al., 2017). Further, pastoralists in remote regions are often underrepresented in such supposedly nationally representative surveys (Wild et al., 2019).

Micro-tasking has been applied to this data gap. Here, micro-tasking offers an opportunity to collect high-frequency data from households with little burden to the household, because the data collector is a local individual that can schedule data acquisition at the convenience of the respondent and there is little or no transportation, lodging, or food costs to support the contributor. The network of contributors can collect data on a weekly basis or even daily for target households, depending on how the tasks are parameterized. Such high-resolution data can capture the spatial and temporal dynamics of households' food consumption and nutrition that can be missed by infrequent surveys or cross-sectional surveys (Lepariyo et al., 2020).

To test this approach, KAZNET was used to collect a set of indicators on child consumption and nutritional status. Specifically, Mid-Upper Arm Circumference (MUAC) measurements were collected from the children between 6 and 59 months from 64 households in the drylands of Kenya and Ethiopia, as part of the ‘Sentinel Zones’ monitoring network. MUAC measurements are commonly used for tracking the nutritional status of children because the measurements are both sensitive to nutritional status and the materials required to make the measurement, a simple MUAC tape, are much less expensive than those used to measure weight and height (Myatt et al., 2006).

Contributors living within the communities measured, recorded, and submitted data from eligible children. A one-time training on the platform and how to measure MUAC was provided to contributors.⁷ Longitudinal data was collected on weekly cycles from the eligible children in target households. The recorded numeric measurements are complemented by images of the tapes' final reading position on the child's arm. These photos are used to reject submissions, encourage better measurements, and identify issues with measurement techniques. This process of using photos to verify and learn from submissions has proven useful for our team in other circumstances.

Figure 7 presents the weekly mean MUAC data from the youngest child in the sampled pastoralist household in Kenya and Ethiopia.

FIGURE 7

Figure 7. MUAC measurement of the youngest child in the household.

The results demonstrate that KAZNET can generate plausible MAUC values and progression over time, weekly, which is extremely uncommon to see from standard field surveys. Naturally, the data can be further disaggregated, for example, by location, household wealth, and other factors that one might believe are important for learning about the progression of nutritional status.

With this level of detail, the dynamics of MUAC could be compared with those of other relevant indicators tracked over the same period to better understand the dynamics effecting nutrition. Household milk production data are among the feasible and relevant indicators that could drive child nutritional status in this region. Milk is one of the primary outputs of the pastoral production system, is consumed frequently and in substantial quantities by many pastoral households, and has been shown to play a large role in nutritional status of children (Grace et al., 2018). At the same time, milk production and consumption in pastoral households is notoriously difficult to track accurately with conventional field surveys because it various dramatically within a household over time and because many different types of containers are used for milking. Figure 8 includes 24-hour milk production, collected once per week, from the households of the children whose MUAC was measured and presented in Figure 7. For context, the long rainy season usually starts in March and continues through May. Here we see an increase in milk production, which is consistent with the build-up and availability of forage in the rangelands. We look forward to determining if the increase in milk production has any (lagged) impact on the MUAC of children in these households. Such an analysis would not be possible with cross-sectional or annual longitudinal surveys, and such high frequency data would be cost-prohibitive with conventional field survey methods.

FIGURE 8

Figure 8. Twenty-four-hour milk production of sampled pastoral households in Kenya and Ethiopia.

Rangeland Data

Understanding the spatial and temporal dynamics of rangeland conditions in pastoral drylands is of critical importance because of the tight link between vegetation availability, resource-use patterns, herd management practices, pastoralist livelihoods and household welfare (Briske, 2017; Liao et al., 2020). But monitoring rangeland conditions is hampered by the extensiveness and remoteness of the pastoral region, the heterogeneity of vegetation communities, the short-term and quick vegetation response to rainfall, the complex grazing patterns linked to herd mobility practiced by the pastoral communities (Pickup et al., 1998). This makes accurate ground monitoring efforts extremely costly and time consuming, thus largely unfeasible on a regular basis. As a result, the availability of data on rangelands is extremely scarce and, when available, ground datasets are of limited quality or spatial/temporal resolution (Zezza et al., 2016).

To overcome this challenge, Earth Observation (EO) is seen as the only viable solution to regularly monitor indicators of rangelands condition in pastoral regions. Long term datasets of vegetation indices measured from satellite optical sensors onboard satellites are currently operationally used for continental and regional monitoring of rangelands (Fava and Vrieling, 2021), but they lack the spatial resolution for informing local-scale application and pastoralist tactical decisions on grazing management. Novel EO sensors and data, such as the one collected by the Copernicus Sentinel fleet or Planet Scope, provide the opportunity to improve rangeland monitoring at high frequency and high resolution (Zhang et al., 2019; Cheng et al., 2020). The Sentinel 2 mission, for example, acquires data globally every 2–5 days at up to 10 meters resolution. However, management-oriented applications require quantitative estimation of rangeland condition indicators such as herbaceous biomass, bare ground/vegetation cover or vegetation composition, and this can be achieved only by calibrating and testing EO-based models using ground datasets. Thus, ground data collection remains of paramount importance to improving our understanding of rangelands and developing models to support pastoral communities.

Cost-effective data collection processes for the rangelands have been developed, such as the Land-Potential Knowledge System (Herrick et al., 2013) or the VegMeasure tool (Louhaichi et al., 2018). These digital applications allow collecting data via mobile phones with relatively simple protocols and have been proven valuable by several applied research studies. However, they are not designed with a micro-tasking approach for agile data collection, and this limits their flexibility of use for multiple data collection objectives, including high-frequency rangeland monitoring.

Here we illustrate an example of how the KAZNET platform has been used for monitoring 32 pasture sites across two regions of Kenya and Ethiopia, as part of the “Sentinel Zones” monitoring network. The network and data collection protocols have been designed not only to gain an understanding of rangeland dynamics in the specific locations, but also to evaluate the relationship between ground observations and Sentinel 2 satellite data for remote sensing model calibration and testing. To this end, representative pasture locations were selected by local communities and the starting and ending point of 100 m linear transects for vegetation monitoring were marked. A KAZNET task was then developed to collect: (i) geo-tagged nadiral down-looking pictures every 10 m along the transect for the assessment of vegetation and bare soil covers, (ii) landscape pictures in the four-cardinal direction for site characterization, and (iii) supporting information about animals grazing in the area. Data are collected with a frequency of 7–10 days. The nadiral pictures, after quality check and validation of the task, are processed automatically through unsupervised image classification techniques using the Canopeo open tool (Patrignani and Ochsner, 2015) to estimate the green vegetation cover (Figure 9).

FIGURE 9

Figure 9. Digital image collected via mobile phone using the KAZNET platform. On the right, the classified image used to calculate the green grass cover.

Figure 10 provides two examples of vegetation cover time series collected over dry (S1) and wet (S2) season pastures in Ethiopia between March and June 2021, which corresponds to the first vegetation growing season (Belg) in the region. The data well captures the growing season's late start at the beginning of April and the end of the season toward the end of May.

FIGURE 10

Figure 10. Percent green grass cover estimated from KAZNET for a dry season (S1) and a wet season (S2) pasture in Ethiopia between March and June 2021. Red dots indicate the Sentinel 2 NDVI values acquired over the same sites.

The figure also includes time series of Normalized Vegetation Differenced Index (NDVI), an indicator of green biomass, for the same pastures. The coherence between the percent of grass cover from KAZNET and NDVI is very good for both sites, supporting the overall quality of micro-tasked ground observations. Interestingly, rapid dynamics of green grass availability, likely associated with rainfall or grazing events, are captured very well by KAZNET data, and can be confirmed by ancillary information collected by contributors on livestock presence and recent rainfall events. These sporadic events, for example during late April at the Harweyu S1 transect, are instead very difficult to capture and correctly interpreted by satellite data analysis because of potential uncertainties caused by cloud cover and other potential noise in satellite data time series that cannot be resolved without ground truth information.

Overall, the KAZNET micro-tasking approach coupled with a simple data collection protocol based on geotagged pictures has the potential to address critical information gaps in pastoral drylands related to rangeland vegetation dynamics. The protocol requires minimal training and can be easily executed by a large number of contributors that can conduct repeated observations over the same area with high frequency. Contributors can visit pasture sites close to their village or satellite camp during seasonal transhumance, thus minimizing the costs of reaching locations that are often remote and not easily accessible without local knowledge. Finally, while the illustrated example focuses on using the platform for monitoring purposes, the flexibility of the micro-tasking approach allows introducing additional questions or data collection points when necessary, for example, to better understand the cause of a rapid change in vegetation cover (e.g., locust invasion, fire), to evaluate the impact of a restoration intervention, or to better evaluate the effect of climate shocks (i.e., drought and floods) on forage resources over specific locations.

Cost Assessment

Reducing costs and thereby increasing the frequency of data that can be collected is one of the main objectives of KAZNET and micro-tasking more generally. To assess costs, it is helpful to first identify those activities that are more or less identical for KAZNET and conventional methods of data collection. Both approaches require survey tool development, developing a sampling protocol, recruiting, and training individuals to collect the data, paying data collectors and respondents for their time, setting up processes for data screening and cleaning, and general project management. The main difference is that for KAZNET, the data collectors are engaged continuously so that there are no additional recruitment and training rounds needed for additional rounds of data collection, and the data collectors live in the areas that they collect data from so that there are few of the costs related to transporting, feeding, and lodging teams of enumerators away from their home. To put these costs in perspective, total direct field expenditure by ILRI to collect a household survey in 2020 from 1,800 participants in northern Kenya was about USD185,000.⁸ Of that, USD35,000 was directly related to enumerator training, USD62,000 was for feeding and lodging the field teams during data collection, and USD 21,000 was spent on transportation for the enumerator teams during data collect. Together, these costs account for about 64% of the total budget of that activity. In 2022, ILRI is about to collect an additional round of data from that same households. The level and profile of the expenses for this additional round are nearly identical to that of the 2020 round.⁹

The micro-tasking approach does require the initial training for the data collectors, but then uses that training to its full potential by engaging the trained data collectors for many rounds of data collection. There are additional costs to collecting many rounds of high-frequency data through KAZNET though, specifically the payments to the data collectors for completing tasks and any payments to respondents. Back of the envelope calculations show that for the sentinel zone sample, the budget required to set up and collect the existing tasks weekly from the 64 households, 8 markets, and 32 pasture transects for a year, would be sufficient to collect two annual panels (e.g., baseline and endline) of a more conventional survey from the same 64 households, 8 markets, and 32 pasture transects. We note that conventional data collection approaches usually try to offset the extremely high fixed cost of reaching each survey respondent by asking many questions. In KAZNET, those fixed costs have been reduced dramatically, which also means that there is more focus on the marginal cost of additional questions, leading to the use of much shorter and more focused survey instruments. The result is that conventual methods usually collect information on many more aspects of individuals and households than does the KAZNET approach. This cost-neutral comparison highlights that the objectives of data collection that should determine its approach. Are those objectives better met by conventional detailed surveys of households at low frequency or higher frequency and focused data?

Discussion

Micro-tasking is one feasible option for overcoming many of the challenges and costs of data collection that conventional methods face, especially in regions difficult to reach or when targeting individuals that are mobile. The data collected through the KAZNET platform supports the notion that micro-tasking can be used to collect accurate, high frequency data on various dynamic indicators that have proven difficult to collect using conventional field survey methods. The breadth of tasks and related data collected through KAZNET demonstrates the flexibility of the platform to meet diverse needs. In all cases employed, the cost per datapoint were much smaller using KAZNET than if the data had been collected using conventional methods. While it is important to monitor the quality of data generated by micro-taskers and have mechanisms for ensuring quality, the issue is no different than that faced by conventional methods, except that, with micro-tasking, the data-user is likely to have a richer dataset that can be used for intertemporal and cross-sectional validation and cleaning.

Furthermore, there are many benefits to being able to update and adjust what, when, and where data are being collected, in near real time. The flexibility of rapidly expanding the network, through remote and/or peer-to-peer training and remote onboarding, allows managers to respond to the dynamic data needs of their clients. This flexibility also minimizes data gaps across space and time. For instance, gaps in market price data are less likely to occur if markets have several contributors collecting data from them simultaneously, and the absence of any particular contributor does not interrupt the flow of data. Flexibility in data type and the ability to change tasks is a huge benefit in some situations. For example, at the onset of the COVID-19 pandemic in 2020, it was quick and easy to create a set of shock-specific tasks that were rolled out and completed despite the restrictions imposed on movement between the location where researchers and managers were, and where the data were (see Chelanga et al., 2020; Graham, 2021).

The micro-tasking approach is also very conducive to learning objectives. Randomization of access to tasks or treatments through the platform is possible, as is adjusting and experimenting with rewards structures, contributor quality ranking, and access to additional resources, such as information. For example, Chelanga et al. (2021) tested if access to market price information, which was generated by processing data from the contributors, was an incentive for those very same contributors to provide more or higher quality data.

None of this is to suggest that the micro-tasking approach is always, or even often, a preferred substitute for other methods. In cases where other approaches are working well, there may be no reason to consider other options. Indeed, large socio-economic field surveys (e.g., Living Standards and Metrics Survey—LSMS -of the World Bank Group) are the workhorse of several academic fields, and those surveys fit the needs of many researchers in those fields well. Such long and in-depth assignments require a great deal of training and logistical precision that would almost certainly be difficult to replicate through a micro-tasking platform. However, the high costs of these long household surveys and other field methods have also clearly led to suboptimal availability of data in many cases, much of which is in difficult-to-reach places or among difficult to reach people, which is not coincidental. The above findings suggest that there can be advantages to using micro-tasking and that a diverse set of data can be collected well in this way. Broadly, it seems that micro-tasking should be considered in situations where near-real-time, high-frequency, difficult-to-collect information is needed or if flexibility in scale and scope is important.

In addition to micro-tasking not being appropriate for all types of data collection exercises, there are also some other factors that implementors should be aware of. First, while the implementation of data collection is relatively simple, the process of setting-up and managing a micro-tasking network of agents is a considerable undertaking. Furthermore, there are fees related to server space and payments directly to the contributors, which add up to the operational expenses. Many of the expenses, such as server rental and staff-time for network management and platform maintenance, are mostly fixed, so that they can be spread over several projects, but contributor payments, which in our experience are best made on a per-task basis, scale directly with the amount of data collected, and this can be a risk to sustaining long-term, high-frequency, data collection objectives. That is, while micro-tasking can have a very low cost per data point, the fundamental trade-offs between sampling density/frequency and costs remain so that researchers should continue to use careful sampling strategies consistent with their specific data collection goals and budgetary constraints.

Moving forward, work continues to create additional functionality and streamline processes. Lessons learned during management of the platform, evolving data needs, and clients feedback provides valuable inputs for further improvement of the KAZNET platform and learning related to micro-tasking. For instance, the recent round of feedback and development led to the new in-app feature by which contributors receive submission-level feedback on the quality of their submission and its status (pending, accepted, rejected)—which improves transparency of the accept/reject process and, in turn, payments, acting as an immediate source of feedback and training. Contributors and managers have found this additional feature extremely useful, and it has translated into improved data quality and improved contributor satisfaction.

While KAZNET has proven to be a valuable tool for a number of projects in pastoral regions of Kenya and Ethiopia and public institutions have expressed interest in adopting the platform for data collection and monitoring purposes, it is currently managed by a research organization with no mandate for service provision and limited expertise in managing such networks or software development. For micro-tasking to realize its full potential, it is likely that the private sector will need to take it up, perhaps with public support, and offer data collection services for a fee. Such firms may be able to manage teams of contributors more efficiently than ILRI has been able to and would hopefully be able to draw a wider client-base, including public institutions, which could help increase the breadth, diversity, and skills of contributors by aggregating demand. Such services could be extremely helpful in increasing data availability from data scarce regions.

Data Availability Statement

The data analyzed in this study is subject to the following licenses/restrictions: the data are under a 12 month embargo. The data will then be placed in the public domain. Requests to access these datasets should be directed to n.jensen@cgiar.org.

Ethics Statement

The studies involving human participants were reviewed and approved by Institutional Research Ethics Committee (IREC), ILRI. Written informed consent to participate in this study was provided by the participants' legal guardian/next of kin.

Author Contributions

PC and NJ lead the development of this manuscript. VA, PC, FF, and NJ provided original material for this manuscript. VA, RB, MB, PC, FF, DG, WG, NJ, WL, ON, KM, and MT contributed equally to the conceptualization and development of the KAZNET platform. All authors contributed to manuscript revision, read, and approved the submitted version.

Funding

The work described in this manuscript was made possible with support from USAID as part of the Feed the Future Initiative, UK aid as part of the Science for Humanitarian Emergencies and Resilience (SHEAR) Research Programme, the Accelerating Impacts of CGIAR Climate Research for Africa (AICCRA) project, which is supported by a grant from the International Development Association (IDA) of the World Bank, and the Supporting Pastoralism and Agriculture in Recurrent and Protracted Crisis (SPARC) Programme, which is supported by the United Kingdom's Foreign, Commonwealth and Development Office (FCDO). It was conducted as part of the CGIAR Research Program on Livestock and is supported by contributors to the CGIAR Trust Fund. The contents are the responsibility of the authors and do not necessarily reflect the opinion of its sponsors or the project's participants.

Author Disclaimer

The contents are the responsibility of the authors and do not necessarily reflect the opinion of its sponsors or the project's participants.

Conflict of Interest

MB was employed by Ona.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

We thank the KAZNET contributors for their enthusiasm and the reviewers for their constructive feedback.

Footnotes

1. ^For very relevant examples in conflict regions and pastoral regions, see Hoogeveen and Utz (2020) and Wild et al. (2019), respectively.

2. ^While it was clear that there was great demand at ILRI for improved data from these regions, the feasibility study focused on demand for data by other stakeholders in the region.

3. ^See Gesare et al. (2017) for more details on the scoping mission and its findings.

4. ^Geofence this is a function that defines a location using coordinates (latitudes and longitudes) with the help of satellites connection with location-enabled smartphones. No tasks are allowable to be submitted outside these coordinates.

5. ^Temporal gate is a time function embedded in a task to define time ranges which tasks are allowable to be completed by a contributor. Tasks performed outside theses time limits are rejected.

6. ^Information on Kenya's national livestock market system can be found at http://www.lmiske.go.ke.

7. ^Certainly, all standard survey protocols were performed, including community entries, project introductions to the households, contributor introductions to households, and consent was collected.

8. ^The authors recognize that these amounts may seem high to many readers familiar with collecting data in rural but less remote or less arid regions. There are a number of special features to pastoral regions that drive costs up substantially including: a low population density, mobile households, extremely poor roads requiring rental of expensive 4 x 4 vehicles, lack of connectivity so that mobilization is challenging, interruptions to data collection from conflict, and households that can only be reached by walking several hours.

9. ^Due to COVID and related restrictions on gatherings, the cost of training will actually increase by about 30%.

References

Akombi, B. J., Agho, K. E., Merom, D., Renzaho, A. M., and Hall, J. J. (2017). Child malnutrition in sub-Saharan Africa: a meta-analysis of demographic and health surveys (2006-2016). PLoS ONE 12, e0177338. doi: 10.1371/journal.pone.0177338

PubMed Abstract | CrossRef Full Text | Google Scholar

Allahbakhsh, M., Benatallah, B., Ignjatovic, A., Motahari-Nezhad, H. R., Bertino, E., and Dustdar, S. (2013). Quality control in crowdsourcing systems: issues and directions. IEEE Internet Comput. 17, 76–81. doi: 10.1109/MIC.2013.20

CrossRef Full Text | Google Scholar

Alulu, V. H., Abay, K. A., and Jensen, N. D. (2020). Feed the Future Accelerated Value Chain Development (AVCD) Program: KAZNET pilot data report. Nairobi: International Livestock Research Institute (ILRI).

Google Scholar

Bitso, C., Makori, E. O., and Kapondera, S. K. (2020). “Research data management and scientific evidence: a strategic imperative for SDGs,” in Africa and the Sustainable Development Goals. Sustainable Development Goals Series, eds M. Ramutsindela and D. Mickler (Cham: Springer), 103–112.

Google Scholar

Briske, D. D. (2017). Rangeland Systems: Processes, Management And Challenges. Texas: Springer Nature.

Google Scholar

Chelanga, P., Indira, T., and Lyons, E. (2020). Pastoralism in the COVID-Era. CEGA Blogs. Available online at: https://medium.com/center-for-effectiveglobal-action/pastoralism-in-the-covid-19-era-5ae64c92514e

Chelanga, P., Kavoi, M., and Jensen, N. D. (2021). Incentivizing Pastoralists to Micro-Task in Drylands of Kenya. Nairobi: Working Paper, International Livestock Research Institute (ILRI).

Cheng, Y., Vrieling, A., Fava, F., Meroni, M., Marshall, M., and Gachoki, S. (2020). Phenology of short vegetation cycles in a Kenyan rangeland from PlanetScope and Sentinel-2. Remote Sens. Environ. 248, 112004. doi: 10.1016/j.rse.2020.112004

CrossRef Full Text | Google Scholar

Cuccolo, K., Irgens, M. S., Zlokovich, M. S., Grahe, J., and Edlund, J. E. (2021). What crowdsourcing can offer to cross-cultural psychological science. Cross Cult. Res. 55, 3–28. doi: 10.1177/1069397120950628

CrossRef Full Text | Google Scholar

Durward, D. (2020). The Future of Digital Labor: Exploring Crowd Work as a New Phenomenon in Information Systems. Kassel: BoD–Books on Demand.

Google Scholar

Edgar, J., Murphy, J., and Keating, M. (2016). Comparing traditional and crowdsourcing methods for pretesting survey questions. Sage Open 6, 2158244016671770. doi: 10.1177/2158244016671770

CrossRef Full Text | Google Scholar

Eklund, L., Stamm, I., and Liebermann, W. K. (2019). The crowd in crowdsourcing: crowdsourcing as a pragmatic research method. First Monday 4. doi: 10.5210/fm.v24i10.9206

PubMed Abstract | CrossRef Full Text | Google Scholar

Fava, F., and Vrieling, A. (2021). Earth observation for drought risk financing in pastoral systems of sub-Saharan Africa. Curr. Opin. Environ. Sustain. 48, 44–52. doi: 10.1016/j.cosust.2020.09.006

CrossRef Full Text | Google Scholar

Gadiraju, U., Demartini, G., Kawase, R., and Dietze, S. (2015). Human beyond the machine: challenges and opportunities of microtask crowdsourcing. IEEE Intell. Syst. 30, 81–85. doi: 10.1109/MIS.2015.66

CrossRef Full Text | Google Scholar

Gesare, A., Chelanga, P., and Banerjee, R. (2017). Feasibility of Establishing a Market Information System in the Horn of Africa: Insights From Northern Kenya. Nairobi: ILRI Research brief, 79. ILRI.

Google Scholar

Grace, D., Dominguez-Salas, P., Alonso, S., Lannerstad, M., Muunda, E., Ngwili, N., et al. (2018). The Influence of Livestock-Derived Foods on Nutrition During the First 1,000 Days of Life. ILRI Research Report 44. Nairobi.

Google Scholar

Graham, M. W., Chelanga, P., Jensen, N. D., Leitner, S. M., Fava, F., and Merbold, L. (2021). A framework for assessing the effects of shock events on livestock and environment in sub-Saharan Africa: The COVID-19 pandemic in Northern Kenya. Agri. Syst. 192:103203. doi: 10.1016/j.agsy.2021.103203

CrossRef Full Text | Google Scholar

Hawkes, C., and Fanzo, J. (2017). Nourishing the SDGs: Global Nutrition Report 2017. Bristol.

Google Scholar

Herrick, J. E., Urama, K. C., Karl, J. W., Boos, J., Johnson, M. V. V., Shepherd, K. D., et al. (2013). The global Land-Potential Knowledge System (LandPKS): supporting evidence-based, site-specific land use and management through cloud computing, mobile applications, and crowdsourcing. J. Soil Water Conserv. 68, 5A−12A. doi: 10.2489/jswc.68.1.5A

CrossRef Full Text | Google Scholar

Hoogeveen, J., and Utz, P. (2020). Data Collection in Fragile States: Innovations From Africa and Beyond. Cham: Palgrave Macmillan.

Google Scholar

Jensen, N., Toth, R., Xue, Y., Bernstein, R., Chebelyon, E., Mude, A., et al. (2017). Don't follow the crowd: incentives for directed spatial sampling. Presented at the Annual Meeting of the Applied and Agricultural Economics Association (Chicago, IL).

Google Scholar

Kariuki, G., Mwangi, M., Maina, J., Wario, Q., and Kaitho, R. (2009). From LINKS to NLMIS: Issues, Challenges and Lessons Learned. GL-CRSP Research Brief;09-01-LINKS. Davis, CA: GL-CRSP; University of California.

Google Scholar

Kittur, A., Chi, E., and Suh, B. (2008). “Crowdsourcing for usability: Using micro-task markets for rapid, remote, and low-cost user measurements,” in Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI 2008) (California), 453–456.

Google Scholar

Lepariyo, W., Alulu, V., and Jensen, N. (2020). Innovation for Capturing and Tracking Temporal Changes in Health and Nutrition Indicators of Households: Samburu Pilot. ILRI Research Brief 98. Nairobi: ILRI.

Google Scholar

Liao, C., Agrawal, A., Clark, P. E., Levin, S. A., and Rubenstein, D. I. (2020). Landscape sustainability science in the drylands: mobility, rangelands and livelihoods. Landsc. Ecol. 35, 2433–2447. doi: 10.1007/s10980-020-01068-8

CrossRef Full Text | Google Scholar

Liu, H. K. (2017). “Crowdsourcing design: A synthesis of literatures,” in Proceedings of the 50th Hawaii International Conference on System Sciences. Available online at: https://scholarspace.manoa.hawaii.edu/bitstream/10125/41488/1/paper0339.pdf

Google Scholar

Louhaichi, M., Hassan, S., Clifton, K., and Johnson, D. E. (2018). A reliable and non-destructive method for estimating forage shrub cover and biomass in arid environments using digital vegetation charting technique. Agroforestry Syst. 92, 1341–1352. doi: 10.1007/s10457-017-0079-4

CrossRef Full Text | Google Scholar

Meena, H. R., and Singh, Y. P. (2013). Importance of information and communication technology tools among livestock farmers: a review. Sci. J. Pure Appl. Sci. 2, 57–65.

Google Scholar

Minet, J., Curnel, Y., Gobin, A., Goffart, J. P., Mélard, F., Tychon, B., et al. (2017). Crowdsourcing for agricultural applications: a review of uses and opportunities for a farmsourcing approach. Comput. Electron. Agric. 142, 126–138. doi: 10.1016/j.compag.2017.08.026

CrossRef Full Text | Google Scholar

Mtsweni, J. S., and Modiba, F. S. (2020). Inclusive development using a micro-tasking approach for building smart communities. Presented at the 5th Annual International Conference on Public Administration and Development Alternatives, 07 - 09 October 2020, Virtual Conference (Limpopo).

Google Scholar

Myatt, M., Khara, T., and Collins, S. (2006). A review of methods to detect cases of severely malnourished children in the community for their admission into community-based therapeutic care programs. Food Nutr. Bull. 27, S7–S23. doi: 10.1177/15648265060273S302

PubMed Abstract | CrossRef Full Text | Google Scholar

Neto, F. R. A., and Santos, C. A. (2018). Understanding crowdsourcing projects: a systematic review of tendencies, workflow, and quality management. Inf. Process. Manag. 54, 490–506. doi: 10.1016/j.ipm.2018.03.006

CrossRef Full Text | Google Scholar

Nyariki, D. M. (2009). Household data collection for socio-economic research in agriculture: approaches and challenges in developing countries. J. Soc. Sci. 19, 91–99. doi: 10.1080/09718923.2009.11892696

CrossRef Full Text | Google Scholar

Patrignani, A., and Ochsner, T. E. (2015). Canopeo: a powerful new tool for measuring fractional green canopy cover. Agronomy J. 107, 2312–2320. doi: 10.2134/agronj15.0150

CrossRef Full Text | Google Scholar

Phuttharak, J., and Loke, S. W. (2018). A review of mobile crowdsourcing architectures and challenges: toward crowd-empowered internet-of-things. IEEE Access 7, 304–324. doi: 10.1109/ACCESS.2018.2885353

CrossRef Full Text | Google Scholar

Pickup, G., Bastin, G., and Chewings, V. (1998). Identifying trends in land degradation in non-equilibrium rangelands. J. Appl. Ecol. 35, 365–377 doi: 10.1046/j.1365-2664.1998.00319.x

CrossRef Full Text | Google Scholar

Roba, G. M., Lelea, M. A., Hensel, O., and Kaufmann, B. (2018). Making decisions without reliable information: the struggle of local traders in the pastoral meat supply chain. Food Policy 76, 33–43. doi: 10.1016/j.foodpol.2018.01.013

CrossRef Full Text | Google Scholar

Robert, L. P. (2019). Crowdsourcing controls: a review and research agenda for crowdsourcing controls used for macro-tasks. Macrotask Crowdsourc. 45–126. doi: 10.1007/978-3-030-12334-5_3

CrossRef Full Text | Google Scholar

Stuth, J., Jama, A., Kaitho, R., Wu, J., Ali, A., Kariuki, G., et al. (2006). “Livestock market information systems for East Africa: the case of LINKS/GL-CRSP,” in Pastoral Livestock Marketing in Eastern Africa: Research and Policy Challenges, eds J. G. McPeak and P. L. Little (Rugby: Intermediate Technology Group), 203–226.

Google Scholar

Sveen, A. F., Erichsen, A. S. S., and Midtb,ø, T. (2020). Micro-tasking as a method for human assessment and quality control in a geospatial data import. Cartogr. Geogr. Inf. Sci. 47, 141–152. doi: 10.1080/15230406.2019.1659187

CrossRef Full Text | Google Scholar

Tollens, E. F. (2006). Market information systems in sub-Sahara Africa challenges and opportunities. International Association of Agricultural Economists (IAAE) Annual Meeting, Queensland, Australia. Brisbane, QLD.

PubMed Abstract | Google Scholar

Uhlmann, E. L., Ebersole, C. R., Chartier, C. R., Errington, T. M., Kidwell, M. C., Lai, C. K., et al. (2019). Scientific utopia III: crowdsourcing science. Perspect. Psychol. Sci. 14, 711–733. doi: 10.1177/1745691619850561

PubMed Abstract | CrossRef Full Text | Google Scholar

Van De Gevel, J., van Etten, J., and Deterding, S. (2020). Citizen science breathes new life into participatory agricultural research. A review. Agronomy Sustain. Dev. 40, 1–17 doi: 10.1007/s13593-020-00636-1

CrossRef Full Text | Google Scholar

Wild, H., Glowacki, L., Maples, S., Mejía-Guevara, I., Krystosik, A., Bonds, M. H., et al. (2019). Making pastoralists count: geospatial methods for the health surveillance of nomadic populations. Am. J. Trop. Med. Hyg. 101, 661. doi: 10.4269/ajtmh.18-1009

PubMed Abstract | CrossRef Full Text | Google Scholar

World Bank (2018). Information and Communications for Development 2018: Data-Driven Development. Washington, DC: World Bank.

Google Scholar

World Health Organization (2019). Levels and Trends in Child Malnutrition: Key Findings of the 2019 Edition (No. WHO/NMH/NHD/19.20). Geneva: World Health Organization.

Google Scholar

Zezza, A., Federighi, G., Kalilou, A. A., and Hiernaux, P. (2016). Milking the data: measuring milk off-take in extensive livestock systems. Experimental evidence from Niger. Food Policy 59, 174–186. doi: 10.1016/j.foodpol.2016.01.005

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhang, W., Brandt, M., Wang, Q., Prishchepov, A. V., Tucker, C. J., Li, Y., et al. (2019). From woody cover to woody canopies: how Sentinel-1 and Sentinel-2 data advance the mapping of woody plants in savannas. Remote Sens. Environ. 234, 111465. doi: 10.1016/j.rse.2019.111465

CrossRef Full Text | Google Scholar