Ethical Issues with Using Internet of Things Devices in Citizen Science Research: A Scoping Review

Introduction

This review seeks to identify and address the ethical issues arising from a collision between two innovation trends in scientific research. First, citizen science, or science conducted by non-professional scientists has long been a feature of scientific research. However, in the past 3 decades, an increasing amount of research is being carried out by non-professional scientists cooperating with professional scientists (Cooper, 2016; Irwin, 2018). We understand citizen science to be “an open collaboration where members of the public engage in the scientific process as active contributors, collaborators, or co-creators, undertaking activities similar to scientists” (Shirk et al., 2012, in; Cooper et al., 2019). Secondly, because of the rise in portable and networked computers (henceforth referred to as “Internet of Things”), researchers now have low cost data gathering devices at their disposal. The widespread availability of these Internet of Things tools increases the capacity of researchers to collect and process enormous amounts of data (Rothstein et al., 2015; Auffray et al., 2016). However, scientific projects involving citizen participants may carry a number of ethical complications, including those that may not be immediately apparent to the research team (Cooper et al., 2019). These ethical considerations may be further exacerbated by the ubiquity and massive data gathering potential of Internet of Things devices. Yet, it is unclear how ethical issues arising in such projects are addressed in practice, and whether they are addressed at all. A brief literature research of published studies did not reveal any review of ethical issues in citizen science related to the use of Internet of Things devices.

This paper therefore contains a scoping review of the literature. Its purpose was to analyze whether, and how, ethical challenges for citizen science research involving Internet of Things devices are communicated and handled. Further, this review aims to identify whether researchers in the field are reporting ethical issues and, if yes, what strategies they use to resolve them and what legal implications they mention. Accordingly, this article is split into three sections. The first part centers on the methodology used for this article and describes the scoping review protocol that was used to identify relevant sections of the literature. The second part offers an analysis of the results that address ethical issues in studies combining citizen science and Internet of Things devices, including the legal and social innovation implications. The third part discusses these results in conjunction with existing theoretical frameworks designed to help guide citizen science projects, and offers recommendations for future research.

Part 1: Scoping Review Protocol

In spring 2020, the authors designed and conducted a scoping review with the goal of retrieving and identifying scholarly literature of studies at the intersection of citizen science and Internet of Things that mention ethical issues. The authors endeavored to include articles describing or discussing an empirical study or project involving citizen science and Internet of Things devices, even if they may be using a different nomenclature. The authors designed and carried out a scoping review by retrieving potentially relevant literature, selecting eligible articles and analyzing the relevant sections (Arksey and O’Malley, 2005).

Retrieval

Based on this research question, the authors defined the following three relevant root keywords: “citizen science”, “ethics”, and “Internet of Things”. These root keywords were used to generate a number of synonymous keywords based on a qualitative exploration of terms used in citizen science research papers (cf. Table 1).

TABLE 1

TABLE 1. List of root keywords and synonyms used to search databases.

The following five databases were selected to search for relevant articles containing a combination of these keywords in any field: IEEEXplore, ACM Digital Library, Scopus, Web of Science, and PubMed. Table 2 contains the search strings used for each database as well as the number of results returned from each database:

TABLE 2

TABLE 2. List of databases, search strings used for each database and results returned for each search string.

These search strings returned 631 matches in total, which was reduced to 608 results once duplicates were removed. Each of these results was screened by manually examining the title and abstract using the criteria for inclusion and exclusion described in Table 3. The inclusion criteria were not applied automatically and the authors did not search to see whether the text contained the words “citizen science” or “citizen participation”. For example, a project that described volunteer collaborators was not removed because it simply did not contain a mention of citizen science in the abstract. Instead, the authors manually read each of the titles and abstracts to see whether they matched the screening in or screening out criteria.

TABLE 3

TABLE 3. Eligibility criteria for abstract screening phase.

Selection and Eligibility

The authors then worked together to assess whether the list of records that they had prepared were congruent with one another and achieved mutual agreement through reflective equilibrium (Daniels, 1996). This resulted in 133 articles screened in. The authors then retrieved the full text and proceeded to the eligibility assessment (cf. table 4).

TABLE 4

TABLE 4. Eligibility criteria for full text eligibility phase.

For the eligibility criteria defined above, a substantive discussion includes everything beyond a simple mention of an issue’s existence. Even short paragraphs were included to be as expansive as possible with the search criteria (Crampton et al., 2016). To this end, articles were eligible if they described a specific study design involving active participants. In contrast, study designs where the sole involvement of citizens consisted of them passively contributing data about themselves as part of a survey were not included. Articles that described case studies, or synthesized a research protocol from existing studies, were also included.

After full text eligibility assessment, a total of 34 articles were included as part of the full text analysis. These articles were published across a range of fields between the years 2009 and 2020. All articles were then coded inductively and deductively to identify ethical issues. These ethical issues were then grouped into clusters for an in-depth analysis.

Limitations

The lack of unified definitions for the terms “citizen science” and “Internet of Things” represents a first difficulty when conducting a literature review on these novel topics. One of the limitations of identifying published literature at the crossroad of several novel topics is that the search terms may not have retrieved all citizen science studies that utilized Internet of Things devices and described ethical issues. Further, the decision on whether to exclude or include articles for a “substantive discussion of ethical issues” were not always unambiguous. To mitigate the inherent subjectivity, two of the authors assessed the retrieved articles and reconciled any differences in their eligibility assessment.

Part 2: Analysis of Results

In this section we address the ethical, legal and social factors raised by the articles identified via this scoping review. Inductive-deductive coding revealed the occurrence of three overarching categories of ethical issues: participant autonomy and privacy, data quality, and intellectual property and labor. This section will discuss each of these issues in turn.

Participant Autonomy and Privacy

Existing ethical frameworks require scientific researchers to guarantee the autonomy and safety of all participants in research. This maxim is usually expressed by the default requirements for researchers to seek explicit, informed and free consent from participants prior to research. A number of results in this sample explicitly addressed this question or sought to guarantee participant consent (Seitzinger et al., 2019a, 2019b; Sousa et al., 2020). For example, Denefleh, in using a sensor device for measuring consumption in a share house, considered whether consent would be affected by the need for housing (Denefleh et al., 2019). Likewise, English et al. discuss the importance of ensuring that citizen science studies do not “fall through the cracks” and avoid ethics review or the need for consent (English et al., 2018). It is also important to recall that much of the existing ethics frameworks for scientific research, such as the Nuremberg Code and the Belmont Report, were developed following unethical and harmful research involving minority populations. Therefore, it is important that scientific researchers working with citizen scientists from minority communities avoid repeating the errors of the past. In particular, Pejovic and Skarlatidou highlight the importance of obtaining free, prior and informed consent when working with indigenous populations. This consent includes a requirement that not only should consent be obtained, but the research goals are conveyed to the community (Pejovic and Skarlatidou, 2020).

Unless the participant has expressly indicated otherwise, it is also important to ensure that the confidentiality of participants is protected. Therefore, a number of studies in this scoping review recommended strategies to maintain participant privacy, including anonymizing or encrypting participant data (Guerrero et al., 2016; Katapally et al., 2018; Acer et al., 2019; Komninos, 2019). As an alternative but complementary strategy, some studies recommended also aggregating personal data submitted by citizen scientists. By using aggregate data, the scientific researchers ensured that individual participants could not be reidentified from their contributions. Further, statistical disclosure controls should be used following the release of anonymized or aggregate data to prevent re-identification from inference attacks (Havinga et al., 2020). Finally, Drosatos et al. and Havlik et al. describe specific algorithmic platforms to guarantee data protection for citizen scientists involved in research. These platforms rely on novel privacy enhancing technologies, such as homomorphic encryption, to protect the identity of participants included in research (Havlik et al., 2013; Drosatos et al., 2014).

Some studies reported excluding some forms of participant data where it was judged to be an inappropriate encroachment upon participant privacy. For example, in Acer et al., the research team supplied Belgian postal workers with Android Wear devices to track their movements upon their rounds. However, these devices not only captured geolocational data but also ambient audio data, which the authors acknowledged represented a privacy concern for both the postal workers and their customers. Therefore, as their study was part of a pilot project, the authors determined to disable this continuous audio sensing functionality as part of future research projects (Acer et al., 2019). Conversely, it may not be possible to obtain explicit consent for all forms of data, such as crowd sourced or volunteered geographic information, or social media data. Havinga et al. suggest that researchers establishing citizen science projects consider whether mechanisms such as geotagging opt in on a social media platform, represents adequate consent (Havinga et al., 2020).

Another issue related to privacy and raised by Sousa et al. is the right to access information about the processing of their personal data enshrined under data protection and privacy law. Several of the studies included in this scoping review suggest extending these rights further to accomodate for specific features of citizen science research. In discussing the results of participants collecting data via smartphones from mosquito traps, Sousa et al. suggest participants should have the capacity to request data about their contributions (Sousa et al., 2020). Likewise, Katapally et al. provide functionality to allow scientific research participants to exercise their right to withdraw from a smartphone based public mHealth study (Katapally et al., 2018). Finally, two of the results, in providing a series of case studies of citizen science projects, defined specific protocols for dealing with sensitive data. These sensitive forms of data can include political opinions or the identity of park rangers investigating controversial ecological issues such as cattle invasions or poaching (Heiss and Matthes, 2017; Pejovic and Skarlatidou, 2020). In a similar fashion, Acer et al. note the importance of ensuring that activity data from workers will not be used against them by their employer (Acer et al., 2019).

Some of the studies included in this scoping review also addressed the more abstract question of autonomy, agency, and why citizen scientists participate in research. Vesnic-Alujevic et al. note that citizen scientists recruited for experiments designed to fine tune wearables for health monitoring are also personalizing devices and actively engaging in their healthcare (Vesnic-Alujevic et al., 2018). Likewise, Seitzinger et al. report how a mobile health app for patients to self-report data on foodborne illness study allowed for more sensitive forms of data collection (such as information on milder illness). Further, the authors describe how this approach helped them to avoid complicating factors around privacy and security for the volume of data usually accompanying big data research (Seitzinger et al., 2019a).

Data Quality and Integrity of Citizen Science Research

Another fundamental principle of scientific ethics pertains to the quality and integrity of research. The vested interests of citizen scientists may intentionally or coincidentally undermine the accuracy and reliability of the data they contribute. A number of the studies included in this sample reported discarding or questioning data due to data quality issues (Aoki et al., 2009; Andersson and Sternberg, 2016; Theunis et al., 2017; Barzyk et al., 2018; Vesnic-Alujevic et al., 2018). The nature of volunteered geographic or crowdsourced information means there can be substantial variances in data quality that are difficult to calibrate in the laboratory (Elwood et al., 2012; Ferster et al., 2013; Havlik et al., 2013; Wylie et al., 2014; Wiggins and He, 2016; Komninos, 2019; Weir et al., 2019).

The retrieved articles also addressed a number of strategies to resolve these issues and guarantee the quality of data. For example, Black and White, as part of an interview study with individuals who contribute air quality readings, note that researchers should consider the implications of “data empowered global citizens”. Black and White then report on how interviewees pondered whether they would decide to move from a particularly polluted area if they suffer from respiratory diseases (Black and White, 2016). Another example is the question of how government policy and government-citizen relations may be influenced by citizen science studies. Carton and Ache note that despite criticisms about data quality undermining the integrity of citizen science, citizen sensor networks provide residents with increased “information power” to confront governments (Carton and Ache, 2017). To legitimize this feedback between governments and citizens, Barzyk et al. recommend that government agencies publish guidelines on data quality (Barzyk et al., 2018). Some studies already integrated government standards for data quality into their reporting. Aoki et al. note that in the context of air quality data, California’s Clear Air Act 1967 creates the regulatory framework for air pollution management and standards.

Related to issues about the political nature of data are concerns regarding data bias. Acer et al. note that a majority of data contributions are made by a minority of contributors, which can decrease the representative nature of a sample (Acer et al., 2019). Further, the availability of Internet of Things devices may be comparatively less among older, regional, and minority populations, introducing a demographic or geographic skew in data (Havinga et al., 2020). Likewise, in Yu et al. an entire study was built around addressing deficiencies in data about socioeconomic features of agricultural land systems (Yu et al., 2017). Bias may also be an inherent feature of the data itself, or even exist with the scientific research team processing the data. Heiss and Matthes note that data bias is a particular problem for qualitative social sciences research data, which is based on human perception (Heiss and Matthes, 2017). For crowdsourced data, Wiggins and He note that data from contributors who have previously donated high-quality data may be prioritized over other sources (Wiggins and He, 2016).

In addition to individual and systematic bias, there may be data quality issues associated with the devices used to collect data. In describing how low-cost smartphones and wearables can be used to collect air quality data, Theunis et al. point out strategies that can be used to enhance the usability of this data. These strategies can include charging the battery of the measuring device or turning off the measuring software after use. Further, Theunis et al. describe how more of these measuring errors arise during the later stages of the project, possibly due to decreasing participant motivation (Theunis et al., 2017). Drawing on the literature from human computer interaction, Budde et al. describe how rewards, similar to those used for computer games, can increase participant motivation and guarantee data quality (Budde et al., 2016).

Conversely, the authors in some of the studies included in this review recognized that stringent technical standards of data quality could undermine the purposes of the study. To this end, Aoki et al. report that in assessing air quality, less accurate but cheaper data collection methods could provide useful information on dramatic regional variances in pollution (Aoki et al., 2009). Likewise, Dema et al. suggest that rather than focusing on study protocols, other strategies could be used to improve data quality. These include using tools that collect longitudinal data, as well as more closely integrating participants into the research protocol (Dema et al., 2019). Further, Ferster et al. and Heiss and Matthews both note that data quality can be improved through suitable training for volunteers and through focusing on particular areas (Ferster et al., 2013; Heiss and Matthes, 2017). Finally, Drosatos et al. note that privacy enhancing technologies for preserving participant confidentiality may necessitate compromising on data quality (Drosatos et al., 2014).

Intellectual Property, Data Rights and Confidential Information

Intellectual property and data ownership may refer to a number of overlapping rights. Each of these rights may apply to different aspects of citizen science research driven by Internet of Things devices. First, a prevailing ethos in citizen science research is the importance of open science (Wiggins and He, 2016; Weir et al., 2019). This principle requires open access to and licensing of publications, methodologies, tools, software, research guidelines, and data (Wylie et al., 2014; Theunis et al., 2017; Yu et al., 2017; Komninos, 2019; Harlow et al., 2020; Pejovic and Skarlatidou, 2020). In particular, Komninos reports that ensuring data was made openly available was an incentive for citizen scientists to participate in the project (Komninos, 2019). Further, a number of the studies included in this sample described the benefits of using low cost open access technologies for ubiquitous research (Black and White, 2016; Carton and Ache, 2017).

However, the presence of intellectual property and moral rights over data can impact whether data is made openly available. Further, the lack of guidance in this area can present a challenge for researchers planning to use both open data and open source technology. Often, these issues must be resolved on a case by case basis. For example, Wylie et al. describe how a collective for environmental citizen science encouraged the hosting research institute to update their policies on licensing for open source technology (Wylie et al., 2014). Verma et al. report on how the ownership of data and images about wildlife could not be transferred across borders due to the potential of identifying endangered species (Verma et al., 2016). Conversely, the absence of intellectual property or rules governing sharing can also have an impact on open access to data, Yu et al. note that the ethics of crowdsourcing big data from farmers as part of agricultural research may depend on who is collecting this data. In particular, industrial agricultural businesses such as Monsanto may gain a significant informational advantage over farmers if they freely benefit from such open research (Yu et al., 2017). Guaranteeing privacy for participants and ensuring data quality, particularly for the reproducibility of research, represent two further competing considerations militating against the use of open data without licensing requirements (Drosatos et al., 2014; Denefleh et al., 2019).

An incidental finding to the identification of ethical issues that indirectly relates to intellectual property concerns the type of devices used for research purposes. The most frequently used terms to describe tools for citizen science projects were smartphone (n = 27), sensor (n = 22) and wearable (n = 13). Less than a third of the results included in this scoping review refer to “Internet of Things” (n = 10) as the class of devices used in their research. By contrast, the use of terms associated with customisable devices (“Internet Connected”, “Connected Devices”, “Ubiquitous Computing” and “Pervasive Computing”) is relatively low.

Part 3: Discussion

This scoping review has identified the occurrence of the three overarching categories of ethical issues mentioned in current literature; privacy, data quality, and intellectual property. Accordingly, this section will discuss the legal and ethical factors raised by these issues. Moreover, recommendations will be offered on how to construct citizen science projects involving Internet of Things devices that address potential challenges in this regard.

First, the preceding analysis reveals that a number of ethical considerations must be integrated into the project design in a very early stage. Notably, all citizen science projects should have a protocol that adequately protects participant autonomy and privacy. A number of existing theoretical and case study derived frameworks have defined privacy protocols for Internet of Things devices in citizen science research projects (Rothstein et al., 2015; Evans, 2020). These frameworks focus on specific ethical and legal issues that may arise from using Internet of Things devices in citizen science projects, including how citizen science projects can comply with privacy legislation in particular jurisdictions. However, the authors of these frameworks note that privacy legislation may not apply to all citizen science projects. For example, these frameworks use the Health Insurance Portability and Accountability Act (HIPAA) from the United States as a reference point for privacy law. Nevertheless, HIPAA only applies to personal health information shared by healthcare providers or health insurers, and manufacturers of Internet of Things devices may not be required to necessarily comply with HIPAA.

Although privacy, like intellectual property, are regulated by specific legislation, and have been addressed in other ethical frameworks, these issues are contextually dependent (Cooper et al., 2019). Specifically, the scientific research team should consider whether personal data is being processed as part of the project. In particular, the analysis of many citizen science projects revealed a nebulous distinction between what Internet of Things devices that do and do not process personal data. The scientific research team should also consider whether participants may potentially submit sensitive personal data, or whether these data can be inferred about participants. Likewise, whether data has been truly anonymized, or could still be considered personally identifying information, depends on both the data and the environment it has been released into. The scientific research team should ensure data privacy by design, and that the Internet of Things devices used by participants are both privacy-enhancing and secure. This security is particularly important in the context of commercially offered smartphones and wearable devices, where the users may not have control over privacy settings. To this end, a commons of resources for ethics with respect to Internet of Things based citizen science research projects and adequate processes of oversight can be crucial for conducting contextually appropriate studies (Harlow et al., 2020; Jobin et al., 2020).

Another issue that was only briefly addressed in some of our results was the question of differences in privacy law between jurisdictions. In particular, the recent European Union General Data Protection Regulation (GDPR) grants data subjects a number of rights over their personal data. One of these rights is the right to data portability, or the ability to have machine readable data transferred from one device to another. Article 20 applies to data that has been submitted by an individual subject to data subject consent or a contract, and accordingly has a relatively limited operation. Despite the relatively limited circumstances in which it applies, this right may have a direct impact on citizen science with Internet of Things devices (Quinn, 2018). Therefore, researchers should integrate strategies to deal with these concerns in their study protocol.

This review also identifies ethical issues that may sit outside the realm of a specific field of legislative regulation. The lack of regulation for citizen science projects include potential trade-offs between privacy, data quality and open access to data. The ethical issues surrounding data quality are also dependent on the study design, the discipline and devices in question. To resolve data quality issues as part of citizen science research, researchers must consider a number of factors contextually. Specifically, it is necessary to consider the types of data that are being collected and in what context. For certain types of data such as visual data of wildlife, the accuracy of data might be less important than the portability of devices (Verma et al., 2016; Dema et al., 2019). To this end, it is important to customize or design Internet of Things data collection devices that are appropriate for the environment in which they are used. Pejovic and Skarlatidou observe how a number of citizen science projects involving indigenous populations in regional areas required supplying low cost devices for these communities suited for regional research (Pejovic and Skarlatidou, 2020). Likewise, Younis et al. describe how for near field communication (NFC) devices, positioning is vital to ensure the accurate collection of data (Younis et al., 2019).

It is also necessary to consider alternative strategies to raise data quality and representativeness, as well as reduce bias. In particular, algorithmic strategies to reduce bias may include assigning rewards for less popular or more spatially distributed tasks (Acer et al., 2019). Outside of technical strategies, it may be possible to also crowdsource validating data. This process would involve recruiting a separate set of participants whose task it is to guarantee the validity of data collected by another set of participants (Wiggins and He, 2016). Nevertheless, any strategy to reduce bias should be employed contextually, recognizing in some cases respondent bias can offer valuable insights by itself (Havinga et al., 2020). In particular, the studies included in this scoping review demonstrate how Internet of Things devices can help citizen scientists play a more active and personally enriching role than they otherwise would as research subjects. Further, the fact that citizen scientists might have strong personal motivations to participate in research might strengthen the importance of that research. Actively participating research subjects can help generate new forms of social innovation from research through peer production of knowledge (Schäfer and Kieslinger, 2016; Peters and Besley, 2019).

A final issue that is not addressed by any of the studies included are the legal rights that Internet of Things device developers hold (Montori et al., 2018). This issue is related to the types of devices used for research purposes, as defined by the use of terms above. There are a number of possibilities to explain this finding. A first hypothesis is that terms such as “Internet of Things”, “Ubiquitous Computing” and “Pervasive Computing” are academic terms and are not used in a technical context to describe the tools being used. A second one is an inconsistent use of terms across disciplines (Crampton et al., 2016). The third possible explanation is that citizen science research in our sample largely involves smartphones and wearables sold by manufacturers with proprietary clouds, otherwise known as “the intranet of things” (Montori et al., 2018). This third hypothesis is supported by the fact that the majority of the studies (n = 27) included in this scoping review used either apps relying on smartphone sensors or commercially available devices. By contrast, only a minority of studies used custom designed devices, or devices built using microcontrollers such as Raspberry Pi or Arduino circuit boards (Wylie et al., 2014; Black and White, 2016; Verma et al., 2016; Tironi and Valderrama, 2017; Barzyk et al., 2018; Dema et al., 2019; Denefleh et al., 2019).

These commercial devices can be contrasted with custom manufactured open source platforms, which users may require more time to become familiar with (Black and White, 2016; Denefleh et al., 2019). In particular, Theunis et al. note that no device can be used for pervasive effortless data collection due to cost or inherent quality issues (Theunis et al., 2017). Therefore, the use of commercial devices may represent an appropriate compromise between each of these factors. Nevertheless, proprietary Internet of Things and mobile devices may have security vulnerabilities that may not be revealed to the project team (Montori et al., 2018). These vulnerabilities raise specific privacy concerns for data collectors, as well as concerns about the verifiability of any data collected using these platforms (Schmitz et al., 2018). Further, commercial smartphone and wearable developers may have their own intellectual property rights over data uploaded to their platforms. Therefore, it cannot be assumed that all open data (including anonymized data) is prima facie ethical to share and reuse. Instead, the decision to use commercial or open source hardware, as well as any intellectual property concerns, should be determined on a case by case basis.

Conclusion

The increased prominence of citizen science projects has coincided with a proliferation in the number of Internet of Things devices. The portable, low cost and connected nature of these devices has made them ideal for carrying out citizen science research, fostering social innovation. However, the use of these devices also may raise ethical and legal issues. To identify these issues, this scoping review contains an analysis of 34 studies from a variety of fields that employed a variety of different citizen science study designs. Privacy, data quality and intellectual property related concerns were identified as the three main issues communicated by researchers. Building on an analysis of these ethical issues with regard to ethical, legal and social implications, this article identifies recommendations for researchers on how they could ethically integrate participants into citizen science research projects. First, researchers should develop a specific protocol for how to ensure both adequate consent and data protection for non-institutional scientific researchers. This protocol should also allow individuals to exercise their rights under data protection or privacy laws (depending on the jurisdiction). Secondly, researchers should consider the types of data that are being collected using citizen science devices, and what the quality requirements for that data are. Thirdly, where possible researchers should consider how intellectual property rights will be handled, and whether these rights might influence the choice of device. Overall, this analysis of these issues contributes to inform future work on specific ethical issues in citizen science research using Internet of Things devices.

Author Contributions

JS, AJ, and EV contributed to conception of the study. JS and AJ designed and tested the protocol. JS performed the analysis and wrote the first draft of the manuscript. AJ contributed sections to the manuscript. All authors contributed to manuscript revision, read, and approved the submitted version.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

Acer, U. G., Broeck, M. V. D., Forlivesi, C., Heller, F., and Kawsar, F. (2019). Scaling crowdsourcing with mobile workforce. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 3, 1–3532. doi:10.1145/3328906

CrossRef Full Text | Google Scholar

Andersson, M., and Sternberg, H. (2016). Informating transport transparency. 2016 49th Hawaii International Conference on System Sciences, Koloa, HI, United States, January 2016. HICSS). 1841–1850. doi:10.1109/HICSS.2016.234

CrossRef Full Text | Google Scholar

Aoki, P. M., Honicky, R. J., Mainwaring, A., Myers, C., Paulos, E., Subramanian, S., et al. (2009). A vehicle for research: using street sweepers to explore the landscape of environmental community action. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems CHI ’09. Boston, MA, United States, April 2009, Association for Computing Machinery), 375–384. doi:10.1145/1518701.1518762

CrossRef Full Text | Google Scholar

Arksey, H., and O’Malley, L. (2005). Scoping studies: towards a methodological framework. Int. J. Soc. Res. Methodol. 8, 19–32. doi:10.1080/1364557032000119616

CrossRef Full Text | Google Scholar

Auffray, C., Balling, R., Barroso, I., Bencze, L., Benson, M., Bergeron, J., et al. (2016). Making sense of big data in health research: towards an EU action plan. Genome Med. 8, 71. doi:10.1186/s13073-016-0323-y |

PubMed Abstract | CrossRef Full Text | Google Scholar

Barzyk, T., Huang, H., Williams, R., Kaufman, A., and Essoka, J. (2018). Advice and frequently asked questions (FAQs) for citizen-science environmental health assessments. Ijerph 15, 960. doi:10.3390/ijerph15050960

CrossRef Full Text | Google Scholar

Black, I., and White, G. (2016). Citizen science, air quality, and the internet of things. In Internet of things and advanced application in healthcare, 138–169. doi:10.4018/978-1-5225-1820-4.ch005

CrossRef Full Text | Google Scholar

Braz Sousa, L., Fricker, S. R., Doherty, S. S., Webb, C. E., Baldock, K. L., and Williams, C. R. (2020). Citizen science and smartphone e-entomology enables low-cost upscaling of mosquito surveillance. Sci. Total Environ. 704, 135349. doi:10.1016/j.scitotenv.2019.135349 |

PubMed Abstract | CrossRef Full Text | Google Scholar

Budde, M., Öxler, R., Beigl, M., and Holopainen, J. (2016). Sensified gaming: design patterns and game design elements for gameful environmental sensing. Proceedings of the 13th International Conference on Advances in Computer Entertainment Technology ACE ’16, Osaka, Japan, November 2016, Association for Computing Machinery, 1–8. doi:10.1145/3001773.3001832

CrossRef Full Text | Google Scholar

Carton, L., and Ache, P. (2017). Citizen-sensor-networks to confront government decision-makers: two lessons from Netherlands. J. Environ. Manag. 196, 234–251. doi:10.1016/j.jenvman.2017.02.044

CrossRef Full Text | Google Scholar

Cooper, C. (2016). Citizen science: how ordinary people are changing the face of discovery. New York, NY, United States: Abrams.

Google Scholar

Cooper, C., Shanley, L., Scassa, T., and Vayena, E. (2019). Project categories to guide institutional oversight of responsible conduct of scientists leading citizen science in the United States. Cstp 4, 7. doi:10.3929/ethz-b-00033147210.5334/cstp.202

CrossRef Full Text | Google Scholar

Crampton, N. H., Reis, S., and Shachak, A. (2016). Computers in the clinical encounter: a scoping review and thematic analysis. J. Am. Med. Inf. Assoc. 23, 654–665. doi:10.1093/jamia/ocv178

CrossRef Full Text | Google Scholar

Daniels, N. (1996). Justice and justification: reflective equilibrium in theory and practice. Cambridge, United Kingdom: Cambridge University Press.

CrossRef Full Text | Google Scholar

Dema, T., Brereton, M., and Roe, P. (2019). Designing participatory sensing with remote communities to conserve endangered species. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems CHI ’19, Glasgow, Scotland United Kingdom, 2019, Association for Computing Machinery, 1–16. doi:10.1145/3290605.3300894

CrossRef Full Text | Google Scholar

Denefleh, T., Berger, A., Kurze, A., Bischof, A., and Frauenberger, C. (2019). Sensorstation: exploring simple sensor data in the context of a shared apartment. Proceedings of the 2019 on Designing Interactive Systems Conference DIS ’19, San Diego, CA, United States, 2019, Association for Computing Machinery), 683–695. doi:10.1145/3322276.3322309

CrossRef Full Text | Google Scholar

Drosatos, G., Efraimidis, P. S., Athanasiadis, I. N., Stevens, M., and D’Hondt, E. (2014). Privacy-preserving computation of participatory noise maps in the cloud. J. Syst. Software 92, 170–183. doi:10.1016/j.jss.2014.01.035

CrossRef Full Text | Google Scholar

Elwood, S., Goodchild, M. F., and Sui, D. Z. (2012). Researching volunteered geographic information: spatial data, geographic research, and new social practice. Ann. Assoc. Am. Geogr. 102, 571–590. doi:10.1080/00045608.2011.595657

CrossRef Full Text | Google Scholar

English, P. B., Richardson, M. J., and Garzón-Galvis, C. (2018). From crowdsourcing to extreme citizen science: participatory research for environmental health. Annu. Rev. Publ. Health 39, 335–350. doi:10.1146/annurev-publhealth-040617-013702

CrossRef Full Text | Google Scholar

Evans, B. J. (2020). The perils of parity: should citizen science and traditional research follow the same ethical and privacy principles?. J. Law Med. Ethics 48, 74–81. doi:10.1177/1073110520917031

CrossRef Full Text | Google Scholar

Ferster, C., Coops, N., Harshaw, H., Kozak, R., and Meitner, M. (2013). An exploratory assessment of a smartphone application for public participation in forest fuels measurement in the wildland-urban interface. Forests 4, 1199–1219. doi:10.3390/f4041199

CrossRef Full Text | Google Scholar

Guerrero, P., Møller, M. S., Olafsson, A. S., and Snizek, B. (2016). Revealing cultural ecosystem services through instagram images: the potential of social media volunteered geographic information for urban green infrastructure planning and governance. Up 1, 1–17. doi:10.17645/up.v1i2.609

CrossRef Full Text | Google Scholar

Harlow, J., Weibel, N., Al Kotob, R., Chan, V., Bloss, C., Linares-Orozco, R., et al. (2020). Using participatory design to inform the connected and open research ethics (CORE) commons. Sci. Eng. Ethics 26, 183–203. doi:10.1007/s11948-019-00086-3 |

PubMed Abstract | CrossRef Full Text | Google Scholar

Havinga, I., Bogaart, P. W., Hein, L., and Tuia, D. (2020). Defining and spatially modelling cultural ecosystem services using crowdsourced data. Ecosystem Services 43, 101091. doi:10.1016/j.ecoser.2020.101091

CrossRef Full Text | Google Scholar

Havlik, D., Egly, M., Huber, H., Kutschera, P., Falgenhauer, M., and Cizek, M. (2013). “Robust and trusted crowd-sourcing and crowd-tasking in the future internet,” in Environmental software systems. Fostering information sharing IFIP advances in information and communication technology. Editors J. Hřebíček, G. Schimak, M. Kubásek, and A. E. Rizzoli (Berlin, Germany: Springer), 164–176. doi:10.1007/978-3-642-41151-9_16

CrossRef Full Text | Google Scholar

Heiss, R., and Matthes, J. (2017). Citizen science in the social sciences: a call for more evidence. GAIA - Ecological Perspectives for Science and Society. 26, 22–26. doi:10.14512/gaia.26.1.7

CrossRef Full Text | Google Scholar

Irwin, A. (2018). No PhDs needed: how citizen science is transforming research. Nature 562, 480–482. doi:10.1038/d41586-018-07106-5 |

PubMed Abstract | CrossRef Full Text | Google Scholar

Jobin, A., Scheibner, J., and Vayena, E. (2020). Ethics guidelines in citizen science. ETH Zurich 23. doi:10.3929/ethz-b-000428502

CrossRef Full Text | Google Scholar

Katapally, T. R., Bhawra, J., Leatherdale, S. T., Ferguson, L., Longo, J., Rainham, D., et al. (2018). The SMART study, a mobile health and citizen science methodological platform for active living surveillance, integrated knowledge translation, and policy interventions: longitudinal study. JMIR Public Health Surveill 4, e31. doi:10.2196/publichealth.8953 |

PubMed Abstract | CrossRef Full Text | Google Scholar

Komninos, A. (2019). Pro-social behaviour in crowdsourcing systems: experiences from a field deployment for beach monitoring. Int. J. Hum. Comput. Stud. 124, 93–115. doi:10.1016/j.ijhcs.2018.12.001

CrossRef Full Text | Google Scholar

Montori, F., Bedogni, L., and Bononi, L. (2018). A collaborative internet of things architecture for smart cities and environmental monitoring. IEEE Internet Things J. 5, 592–605. doi:10.1109/JIOT.2017.2720855

CrossRef Full Text | Google Scholar

Pejovic, V., and Skarlatidou, A. (2020). Understanding interaction design challenges in mobile extreme citizen science. Int. J. Hum. Comput. Interact. 36, 251–270. doi:10.1080/10447318.2019.1630934

CrossRef Full Text | Google Scholar

Peters, M. A., and Besley, T. (2019). Citizen science and post-normal science in a post-truth era: democratising knowledge; socialising responsibility. Educ. Philos. Theor. 51, 1293–1303. doi:10.1080/00131857.2019.1577036

CrossRef Full Text | Google Scholar

Quinn, P. (2018). Is the GDPR and its right to data portability a major enabler of citizen science?. Glob. Jurist 18, 1. doi:10.1515/gj-2018-0021

CrossRef Full Text | Google Scholar

Rothstein, M. A., Wilbanks, J. T., and Brothers, K. B. (2015). Citizen science on your smartphone: an ELSI research agenda. J. Law Med. Ethics 43, 897–903. doi:10.1111/jlme.12327 |

PubMed Abstract | CrossRef Full Text | Google Scholar

Schäfer, T., and Kieslinger, B. (2016). Supporting emerging forms of citizen science: a plea for diversity, creativity and social innovation. Jcom 15(02), Y02. doi:10.22323/2.15020402

CrossRef Full Text | Google Scholar

Schmitz, H., Howe, C. L., Armstrong, D. G., and Subbian, V. (2018). Leveraging mobile health applications for biomedical research and citizen science: a scoping review. J. Am. Med. Inf. Assoc. 25, 1685–1695. doi:10.1093/jamia/ocy130

CrossRef Full Text | Google Scholar

Seitzinger, P. J., Tataryn, J., Osgood, N., and Waldner, C. (2019a). Foodborne outbreak investigation: effect of recall inaccuracies on food histories. J. Food Protect. 82, 931–939. doi:10.4315/0362-028X.JFP-18-548

CrossRef Full Text | Google Scholar

Seitzinger, P., Osgood, N., Martin, W., Tataryn, J., and Waldner, C. (2019b). Compliance rates, advantages, and drawbacks of a smartphone-based method of collecting food history and foodborne illness data. J. Food Protect. 82, 1061–1070. doi:10.4315/0362-028X.JFP-18-547

CrossRef Full Text | Google Scholar

Shirk, J. L., Ballard, H. L., Wilderman, C. C., Phillips, T., Wiggins, A., Jordan, R., et al. (2012). Public participation in scientific research: a framework for deliberate design. E and S 17, 29–48. doi:10.5751/ES-04705-170229

CrossRef Full Text | Google Scholar

Theunis, J., Peters, J., and Elen, B. (2017). “Participatory air quality monitoring in urban environments: reconciling technological challenges and participation,” in In participatory sensing, Opinions and collective awareness understanding complex systems. V. Loreto, M. Haklay, A. Hotho, V. D. P. Servedio, G. Stumme, J. Theuniset al. (Cham, Switzerland: Springer International Publishing), 255–271. doi:10.1007/978-3-319-25658-0_13

CrossRef Full Text | Google Scholar

Tironi, M., and Valderrama, M. (2017). Unpacking a citizen self-tracking device: smartness and idiocy in the accumulation of cycling mobility data. Environ. Plann. D 36, 294. doi:10.1177/0263775817744781

CrossRef Full Text | Google Scholar

Verma, A., van der Wal, R., and Fischer, A. (2016). Imagining wildlife: new technologies and animal censuses, maps and museums. Geoforum 75, 75–86. doi:10.1016/j.geoforum.2016.07.002

CrossRef Full Text | Google Scholar

Vesnic-Alujevic, L., Breitegger, M., and Guimarães Pereira, Â. (2018). 'Do-It-Yourself' healthcare? Quality of health and healthcare through wearable sensors. Sci. Eng. Ethics 24, 887–904. doi:10.1007/s11948-016-9771-4 |

PubMed Abstract | CrossRef Full Text | Google Scholar

Weir, D., McQuillan, D., and Francis, R. A. (2019). Civilian science: the potential of participatory environmental monitoring in areas affected by armed conflicts. Environ. Monit. Assess. 191, 618. doi:10.1007/s10661-019-7773-9 |

PubMed Abstract | CrossRef Full Text | Google Scholar

Wiggins, A., and He, Y. (2016). Community-based data validation practices in citizen science. Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work and Social Computing CSCW ’16, San Francisco, California, United States, February 2016. Association for Computing Machinery), 1548–1559. doi:10.1145/2818048.2820063

CrossRef Full Text | Google Scholar

Wylie, S. A., Jalbert, K., Dosemagen, S., and Ratto, M. (2014). Institutions for civic technoscience: how critical making is transforming environmental research. Inf. Soc. 30, 116–126. doi:10.1080/01972243.2014.875783

CrossRef Full Text | Google Scholar

Younis, E. M. G., Kanjo, E., and Chamberlain, A. (2019). Designing and evaluating mobile self-reporting techniques: crowdsourcing for citizen science. Personal Ubiquitous Comput. 23, 329–338. doi:10.1007/s00779-019-01207-2

CrossRef Full Text | Google Scholar

Yu, Q., Shi, Y., Tang, H., Yang, P., Xie, A., Liu, B., et al. (2017). eFarm: a tool for better observing agricultural land systems. Sensors 17, 453. doi:10.3390/s17030453

CrossRef Full Text | Google Scholar