From Science to Policy and Society: Enhancing the Effectiveness of Communication

Abstract

Dissemination is now acknowledged as an important component of the research process, in particular for European Union (EU) funded research projects. This article builds on the authors' experience during the EU project DEVOTES (DEVelopment Of innovative Tools for understanding marine biodiversity and assessing good Environmental Status) and aims to assist other scientists to develop a successful dissemination strategy to communicate project achievements. We provide a critical review of the different tools used for outreach to our target audiences, from the academia to the policy makers, and the general public, and try to assess their impact. An effective dissemination strategy and plan should have a clear objective, be designed before the start of the project, identify the target groups and define the methods or tools to be used according to target groups and objectives. The DEVOTES dissemination strategy included two complementary approaches of communication with stakeholders: (i) traditional (e.g., peer reviewed publications, stakeholders workshops, and participation in scientific conferences), and (ii) new (e.g., social networks, smartphone applications) media tools. For each dissemination approach, we defined production targets (e.g., number of articles to be published, individual visitors on the website, etc.) to be achieved by the end of the project, and impact measurements (e.g., citation indices for peer reviewed articles) to monitor the successful implementation of DEVOTES Dissemination. This allowed us to identify which tools had been more (e.g., website) or less useful and relevant (e.g., Facebook) during the project. We conclude that impact measurements cannot be easily identified for all dissemination actions. However, for those that were possible, the DEVOTES dissemination targets were successfully achieved. Overall, the use of the tools and activities outlined in this article, combined with the constant evaluation of the dissemination goals throughout the project duration and the assessment of the effectiveness of the different tools, is essential for the achievement of an effective and timely communication of research results.

Importance of dissemination/communication of science

Common techniques for communication

Science communication has been defined as “the use of appropriate skills, media, activities, and dialogue to produce one or more of the following personal responses to science: Awareness, Enjoyment, Interest, Opinion-forming, and Understanding” (Burns et al., 2003).

Scientists are not only asked to communicate their findings inside and outside academia, but also to build bridges between research and the society at large and, more importantly, to engage the general public, developing a bi-directional and critical dialogue with the different categories of social actors, (i.e., stakeholders).

Dissemination of scientific results to different target groups is increasingly recognized as a responsibility of scientists (Brownell et al., 2013) that needs the support of other professionals, e.g., journalists, artists, Information Technology (IT) specialists and social networks managers (Uyarra and Borja, 2016). Awareness of the need for better science communication has grown enormously over the last 40 years. The communication of science to different target groups, including the society at large, and the transfer of knowledge is now required in research programmes. Science plays a central role in our life, so policy makers and the wide public are not be able to make informed decisions without understanding the scientific basis (Treise and Weigold, 2002; Fischhoff, 2013).

Science is mainly financed through public funds. Worldwide, numerous organizations (e.g., governments, agencies, foundations) and a large diversity of research programmes are in place to fund research and innovation [e.g., Horizon, 2020 European Union (EU) and National Science Foundation (US) programmes]. Both human and economic resources are being used to this end. Therefore, bridging the gap between science and policy through effective dissemination is a must for such funding programmes to be considered as useful and successful. Although some progress that has been made in disseminating health research output to bridge the gap between science and practitioners (Wilson et al., 2010; Neta et al., 2015), this does not apply to most fields of research. Whether research outputs reach the relevant target groups (e.g., society, consumers, specific economic sectors, decision makers, policy makers, etc.) is yet not well-studied, but it is crucial for societies to become more knowledgeable and reach a better capacity to make informed-decisions.

Indeed, until recent times, not much relevance was given to dissemination and a greater focus was placed on ensuring that scientific outputs were reflected in the scientific literature. The potential impact through the development, dissemination and use of project results was often neglected, both in the call for research proposals and the proposals themselves. Many calls for proposals clearly state the need for dissemination activities to increase impact. Science dissemination is now evaluated in research project assessments and constitutes an important criterion to achieve an outstanding and fundable project (Pohl et al., 2010). Furthermore, there is considerable pressure from the funding agencies for scientists to communicate with and to involve society in research through “citizen science.” However, despite its importance, guidance on what it is expected from scientists in terms of dissemination is still weak, and little has been developed as to how the success of any dissemination strategy may be measured.

Taking this into account, the aim of this article is to provide guidance to scientists on planning and implementing an effective dissemination strategy. In order to do so, we first provide a brief overview of the EU approaches to the dissemination of science. We then review the most important dissemination approaches, tools and activities available to a science communicator, and report on their effectiveness and on the difficulties that could be encountered. We illustrate this using the experience gained during the EU-funded project DEVOTES (DEVelopment Of innovative Tools for understanding marine biodiversity and assessing good Environmental Status; http://www.devotes-project.eu). In this project, the consortium prepared a dissemination strategy during the planning phase of the project that aimed at maximizing the impacts of the research. We (the Dissemination Team of the DEVOTES project) have collated a number of theoretically and practically informed frameworks that could be used by other scientists as a guide for planning and accomplishing a fruitful dissemination of their project results and outputs, both at the European and the international level.

The importance of science dissemination for the EU

Over the last decades, the European Commission's economic policy has largely been based on the belief that progress and economic growth are achievable through techno-scientific knowledge and innovation (PotoČnik, 2007). Therefore, if society understands the critical role that science and technology plays, public support should follow naturally. The nature of the science-society relationship has shifted since the 80's, but the idea still lies at the heart of Europe's strategy. Back in the late 1980's, science-society issues were considered a problem that could be solved by increasing classic communication efforts. The paradigm “Public Understanding of Science” (Royal Society, 1985) regarded the communication model as a linear function, where dissemination efforts would fill the knowledge gap and would make citizens supportive of science and technology policies.

The 1990's and EU Framework Programme 5 (FP5) were oriented to “Raising Awareness,” which stressed that researchers should increase their involvement in dissemination activities. Moreover, through the Marie Curie Actions and the launch of gender mainstreaming (European Commission, 2001), more effort was made to attract Early Career Scientists and women into research.

At the beginning of the millennium, the key concepts of “dialogue” and “participation” were introduced, anticipating new ways of governance in science and technology. The EU FP6 funded the “Citizen and Governance in a Knowledge-Based Society” and “Science and Society” calls. The latter was modified to “Science with Society” in FP7, with the aim of improving linkages between science and society. This stressed the idea of considering science and society as a single entity, increasing the role of the wider public and non-research actors in science policy making, and making the results of publicly funded research more accessible (Wilkinson et al., 2016).

The last step in the recent evolution of the European science communication strategy is constructed around “Innovation Union 2020,” where innovation is seen as the key tool for strong and sustainable growth. In this framework, the Responsible Research and Innovation (RRI) concept implies that all societal actors (e.g., researchers, citizens, policy makers, third sector organizations, etc.) work together during the research and innovation process to align its outcomes with the needs, values and expectations of society. One of the key pillars of Horizon 2020 is “tackling societal challenges that are important to all EU citizens and can have a real impact benefitting the citizens.” These benefits include:

• (i) Health, demographic change, and well-being;

• (ii) Food security, sustainable agriculture and forestry, marine, maritime and inland water research, and the Bioeconomy;

• (iii) Secure, clean, and efficient energy;

• (iv) Smart, green, and integrated transport;

• (v) Climate action, environment, resource efficiency, and raw materials;

• (vi) Europe in a changing world—inclusive, innovative and reflective societies;

• (vii) Secure societies—protecting freedom and security of Europe and its citizens.

In summary, the European view on science-society issues has evolved from considering science as a source of rarely questioned knowledge, to a practice deeply intertwined with society (ESF Science and Policy Briefing 50, 2013).

In 2013, the European Commission's launched Horizon 2020 (H2020), a research and innovation programme that will run from 2014 to 2020. H2020 supports scientific research and innovation with an overall budget of approximately €80 billion (European Commission, 2013). The H2020 Communication guidelines (European Commission, 2014) provide a checklist to guide the participants in building a communication strategy specific for their project. This includes guidelines for:

• (i) The good management of resources and people in the dissemination of results, which implies a dedicated work package in the proposal, the preparation of a dissemination plan, the allocation of an adequate budget and the involvement of professionals in the field of science communication;

• (ii) A series of activities to ensure the continuity of the dissemination after the end of the project;

• (iii) Well-defined goals and objectives for the dissemination, with specific deadlines and evaluation criteria to measure its efficiency and impacts;

• (iv) A well-defined audience and specific target groups;

• (v) A distinct communication strategy and dedicated dissemination means for each target group.

Communication tools

There are various approaches to communicate scientific findings, ranging from more formal (e.g., academic activities, lectures, seminars, production of textbooks, SCI publications) to informal activities (e.g., exhibitions, documentaries, media programs, science clubs and societies, educational games, theater performance, open lectures, festivals, magazine articles, and internet-based tools such as websites, blogs, social media, podcasts, newsletters; Burns et al., 2003). Scientific journalism has traditionally been used as the main format for the communication between science and the public, with the aim of filling in the gaps in the knowledge of the society at large (Treise and Weigold, 2002). However, not all topics are equally covered, and around 70% of scientific journalism coverage is on medicine and health. Scientists used to communicate their results in two main ways: (i) publishing in peer-reviewed journals, and (ii) presenting their findings at conferences. Both these methods are mainly directed to other scientists as most of the scientific journals are accessible only through institutional subscriptions, and conferences are mostly attended by other researchers. More recently, scientists have started to use Internet and social media as means to directly communicate. Innovation in new technologies has led to the development of new approaches, which not only encourage the dialogue between scientists and the general public, but also stimulate people to have an active role in science. In this sense, social media has helped science communication to transform itself from a one-way to a two-way system, where users interact directly with the scientist (Figure 1). In addition, citizen science (i.e., the active engagement of general public in scientific research projects, often acting as collectors of data) and crowdfunding (i.e., the request by founders of for-profit, cultural, scientific, and social projects to request funding from many individuals, often in return for future products or equity; Mollick, 2014) are now becoming more and more important in research projects development.

Figure 1

The dissemination experience of DEVOTES

DEVOTES is a EU FP7 collaborative project involving 22 partners distributed across 14 countries in the Atlantic Ocean, and the Baltic, Mediterranean, Black, and Red Seas. DEVOTES was developed with the main objective of improving our understanding of the relationships between anthropogenic pressures, their influence on the climate and their effects on the marine environment. The project was funded for improving and/or enhancing the effectiveness of ecosystem based management (EBM) in order to fully achieve the Good Environmental Status (GES) of European marine waters, in the context of the European Marine Strategy Framework Directive (MSFD; 2008/56/EC). To achieve this goal, DEVOTES developed a wide set of innovative indicators, models and tools to assist in the characterization, quantification and assessment of marine biological diversity, non-indigenous species, food-webs and seafloor integrity status at an European scale.

The communication strategy of DEVOTES was developed during the preparation of the proposal, with the main aim to build a network with the stakeholders and to provide an effective dissemination of the project achievements. The dissemination activities included an interactive communication dialogue with stakeholders, policy makers and society at large, as well as a uni-directional communication of results. In addition to the traditional approach of dissemination, (e.g., publications, presentations in conferences, organization of workshops, documentaries, etc.), DEVOTES made an effort to define the use and development of new tools to actively involve the different target groups, through the development of apps and the use of social media.

All the planned dissemination activities were directed to achieve the main objectives of DEVOTES. These included building knowledge of the functioning of marine ecosystems (i.e., promoting Ocean literacy, see Uyarra and Borja, 2016), and raising the awareness of the implications of human activities on marine ecosystems. Without this solid understanding, the public cannot make informed decisions and respond in an efficient and timely manner to solve environmental issues.

The next two sections will describe the activities carried out during the lifetime of DEVOTES to disseminate results and progress, and will analyze the performance of each tool.

Dissemination approaches

Communication strategy and dissemination plan

Effective communication enhances the impact of a project and the possible uptake of the results. Therefore, the communication strategy of a research project should be discussed in detail and the various phases of the communication strategy should be established during the development of the project proposal. These phases include capturing public interest about the topic, disseminating the project results and outcomes, and finally ensuring and communicating the legacy of the project. The chosen communication approaches should also be established at this stage, as should be the identification of the target audiences.

The different inter-related phases for an effective communication strategy in a research project were taken into account in DEVOTES: the development of a dissemination strategy and plan, and the identification of key reporting elements and of the cross-cutting issues (Figure 2).

Figure 2

2015).

The communication strategy should be developed by a small communication team that includes, at least, the project coordinator, the webmaster, the graphic designer, and one scientist in charge of the dissemination. The inclusion of additional professionals, such as a scientific journalists and artists would be beneficial to this team. In addition, and to ensure that all work carried out within the project has the potential for equal visibility, each work package of the project should nominate their communication officer, who will be the contact point for the communication team.

A time-line must be established for the various phases of communication in line with the timing of deliverables, and considering the necessary time-lag to prepare for the dissemination product linked to the specific deliverable (objective) and target audiences, which should also be defined. Once the communication strategy has been discussed, the communication team should draft a dissemination plan. The dissemination plan is a document that is revised at 6 months intervals throughout the duration of the project. It serves as a guide to the communication team and other project members to outline the actions, product outputs and target audiences to be reached during the project. The lead partner(s) for the different actions are also identified. The dissemination plan is a “living document” that can be revised and adapted to accompany the project development. During the project, the details of the various actions that have been undertaken may be added so that the dissemination plan is slowly transformed into the dissemination report as the project is implemented.

The dissemination plan should be structured to include the following sections, although others may also be necessary: (i) an executive summary; (ii) the target audience(s); (iii) the messages; (iv) the tools and mechanisms; (v) the calendar including the post project legacy; (vi) the assessment and monitoring; (vii) the indicators for the evaluation of the dissemination goals, and (viii) the internal communication. Moreover, a SWOT (Strengths, Weaknesses, Opportunities, and Treats) analysis should be included and revised during the project (Figure 3). The SWOT analysis is a structured planning method that identifies the internal (strengths and weaknesses) and external (opportunities and threats) factors that are helpful or harmful to achieve a specific objective, and can be a useful tool to evaluate the dissemination strategy of a project. The results of the SWOT analysis determine what may assist the dissemination team in achieving its objectives, and in identifying what obstacles must be overcome or minimized to achieve foreseen results. Additionally, annexes can be added in the dissemination plan containing tables with details about the venues, participants, link to the products and other pertinent information. Other annexes may include examples of posters, leaflets, and other materials.

Figure 3

Dissemination actions should be targeted at well-defined audiences. The results of a research project may be of interest to the general public, but also to specialists and high-level policy makers. Different means and media of dissemination, vocabulary, and message are appropriate for each of these categories. This audience needs to be informed about the project, its progress, its results, its outputs and its legacy.

In order to maximize the impact of a research project, it is important to engage with all interested parties and communicate the results of the research. “Interested parties” include a wide variety of stakeholders, as well as the “end-users,” i.e., those who will be able to make use of the findings, outcomes, and products. For the results to be useful, they should be of interest and easily accessible. Ideally, the identified end-users engage with the project at the design stage. Co-design allows end-users to actively participate and communicate their interests, and help the scientists to co-develop the project so as to maximize its uptake and legacy.

Engaging with the stakeholders can be surprisingly difficult, due to insufficient funds to engage them dynamically resulting in “stakeholder fatigue,” because of the multiple requirements both from the project and from other projects on similar topics. There are existing guidelines about stakeholder engagement, such as Durham et al. (2014). For a balanced viewpoint, it is important to engage with different types of stakeholders and to establish a solid discussion with end users and local stakeholders (Saint-Paul and Schneider, 2016).

DEVOTES dissemination strategy

The DEVOTES Dissemination Team developed its communication strategy during the negotiation phase of the grant and requested that each partner nominate a responsible for the dissemination. Dissemination influences the decision-making process, and therefore the first step is to identify the audience, listen to it, identify which decisions are required and therefore what information is necessary (Fischhoff, 2013). The DEVOTES Dissemination Team therefore first focused on building a stakeholder map, identifying the audience and the specific targeted messages, the mechanisms of communication and finally defining a specific timeline for the different activities.

Besides the general public, another six categories of stakeholders were identified as target groups of dissemination, through an analysis of the characteristics of the audience engaged with DEVOTES project: (i) scientists with interest in marine monitoring, biodiversity, and assessment, (ii) higher education institutions, (iii) environmental agencies and/or other institutions operating at the national and regional levels, (iv) decision making authorities, (v) environmental associations, NGOs, fishing, and aquaculture associations, maritime transport associations, port authorities, and (vi) private and industrial stakeholders, including Small and Medium Enterprises (SMEs). The dissemination approach included a strong web presence through a dedicated website, social network accounts, and e-newsletters, participation in conferences and fairs, publication of scientific papers, organization of training activities and networking with other EU funded projects.

The DEVOTES Dissemination Team, with the contribution of all partners, created the database of stakeholders, which now includes more than 1500 contacts in marine environment research and industry. All were contacted early on to introduce them to the project concept through unidirectional communication, emails and the distribution of the electronic newsletter.

Traditional tools

The identification of the audience potentially interested in DEVOTES results and the categorization of the different stakeholder groups were fundamental for the dissemination planning: for each audience cluster identified in the stakeholder map we used dedicated dissemination tools (Figure 4). Statistic information about the use of these tools is discussed in Section Evaluation of the Dissemination Goals of this paper.

Figure 4

The Dissemination Team held regular meetings to revise the plan and adapt it to the progress of the project. This resulted, for example, in a deep revision of the homepage layout and website structure 2 years after the beginning of the project and on the participation in Regional Sea meetings rather than organization of workshops.

The website

Nowadays, the Internet is the primary medium of science communication (Kling and McKim, 2000), and web-based communication is crucial for engaging public audiences with science (Bultitude, 2011). The DEVOTES dissemination strategy included various Internet-based tools the foremost of which was a dedicated website, http://www.devotes-project.eu, used as the main communication channel for the project management, achievements, and progress. A special effort of the Dissemination Team was focused on developing an eye-catching layout and a user-friendly website map. The website, dedicated to all stakeholder categories, was developed by graphic designers, under the supervision of the project coordinator and in accordance with the EU guidelines. The website has been constantly and timely updated with news, promotional material and new project products. The site map included six main sections:

• (i) About the Project, to introduce the project objectives, the work plan, and the partners involved;

• (ii) News and Events, to promote the research progresses, project meetings, and conferences on topics related to DEVOTES and other EU funded projects events;

• (iii) Research Outputs, to promote and provide easy access to scientific publications, reports, and tools developed during the lifetime of the project;

• (iv) Young Scientist Corner, to present early stage career researchers working in DEVOTES [with the series of interviews (“Ph.D. students of the Month”) and to promote training and job opportunities within and outside the project];

• (v) Media Center, to make available the promotional material; and

• (vi) Partners' Area, to facilitate the communication within the consortium.

A full set of informative and promotional material, including factsheets, policy briefs, brochures, and posters, was produced during the lifetime of the project to promote the release of reports, software tools and deliverables. All the promotional products, the website and templates (for presentations, posters, reports, minutes of meetings) were developed using the corporate image of the project, always including the DEVOTES logo and using a consistent color code.

Special attention was dedicated to the early career researchers (ECR), within and outside the project: the Young Scientist Corner included a series of interviews “PhD students of the Month,” as well as announcing job opportunities, post-graduate modules, summer schools, and training activities.

The newsletter and email campaigns

The dissemination campaign of DEVOTES was launched with the publication of press releases in the countries of the members of the consortia. This was followed by an email campaign presenting the project and launching the website to all the potential stakeholders. The mailing campaigns continued with a regular electronic newsletter (approximately every 6 months), brief news (every 3 months), and monthly updates on the project progress. All the issues of the newsletter have been made available for download on the project website and promoted via the project social networks.

To enhance the communication inside the consortium, distribution lists were created at Work Package and Task level, for the General Assembly, the Steering Committee members, and for Advisory Committee members. Moreover, in addition to the Partners' Area of the website, a sharing platform has been included among the e-media tools available for the participant to the project.

Scientific publications

In order to better communicate the scientific results, not only within the scientific community but also to decision and policy makers, all the scientific papers produced in DEVOTES have been made Open Access, either with the gold road, paying the fee for the open access, or with the green road, self-archiving the article. As indicated above, academic institutions subscribe to the different journals, but usually they can only afford the subscription to a small fraction of them. This situation decreases the potential usage and impacts of research, which would be maximized if all research papers were Open Access (Canessa and Zennaro, 2009). Open Access enhances the research cycle, improves the access to international research outputs and the impact of the research. There is a correlation between Open Access publication and citation-count, increasing this from 50 to 250% (Canessa and Zennaro, 2009). Additionally, articles in Open Access are immediately available for free consultation and download and, more importantly, permanently preserved in journals digital archives.

The Dissemination Team created a repository of scientific papers produced during DEVOTES life, named “FP7 EU DEVOTES Community” in Zenodo, the OpenAIRE “orphan repository” available under the link https://zenodo.org/collection/user-devotes-project. With this repository DEVOTES is accomplishing one of the most important objectives of the FP7 Programme, which is the free access to all the research outcomes to scientists and public at large. In addition to Zenodo, the Dissemination Team created a Google Scholar profile for DEVOTES in which all papers are listed, (https://scholar.google.it/citations?user=oSH2JTkAAAAJ&hl=it&oi=pll). This allows scientists to easily obtain information on all the papers published by the project, consult the citations received by each paper, rank them, and obtain the H_index of the project, as an index of the success of the project scientific outcome.

As Open Access publications lead to wider and more efficient dissemination of information, the dissemination strategy of DEVOTES included also the production of an ebook, reporting the scientific results and products developed during the project. The ebook, composed by the articles published in this Research Topic will be freely available for download from the website of the project. Moreover, the ebook will be part of one of the applications for smartphone, which will be available by the end of DEVOTES project (October 2016).

Workshops and participation to conferences

The engagement of stakeholders is crucial to reach the objective of generating improved interfacing mechanisms in the management process, among science, policy, and decision makers and the general public. This can be achieved through targeted workshops, conference sessions, and webinars. Once more, the dissemination has to be tailored to the audience. The scientists working in related fields and projects are more easily reached at special sessions in conferences. Practitioners working at environment agencies, either regional or national are best reached through specially organized workshops, if possible using locally relevant materials as examples. International practitioners, such as the Regional Seas Conventions, European Environment Agency and expert groups (e.g., “Good Environmental Status working group”), are best reached at workshops back-to-back with pre-organized meetings. This both increases the likelihood of participation and reduces travel expenses. It is essential to distribute targeted information that explains the workshop well in advance of the meeting, so that the attendees may register and prolong their stay to participate.

Companies and SMEs are more difficult to contact as a group. Environmental consultancy firms may be in competition with each other, and so reluctant to have a joint meeting, and it may be therefore necessary to have individual or small group meetings. However, it was easier to organize group workshops and meetings for other potential end users, for example aquaculture firms that rely on marine good environmental status.

Documentaries

Films and documentaries are one of the most powerful communication and educational tools (Barnett et al., 2006; Hooper et al., 2011), engaging the public in critical thinking and enhancing public awareness in environmental issues (e.g., climate change, pollution, acidification). The production of documentary films has grown significantly in the past decade, and the distribution of documentaries through the Internet created new opportunities to create societal impact (Karlin and Johnson, 2011). Platforms such as YouTube, iTunes, and Vimeo make online videos easier to be made available, accessed, used, and shared. With the aim of increasing the potential impact of DEVOTES, the dissemination strategy included the preparation of a documentary illustrating the background and the main results of the project. DEVOTES was selected by “Futuris,” the award-winning program of EuroNews on European science, research and innovation, as a successful example of project studying the effects of human activities on marine ecosystems, to raise general interest about the environmental status of European seas. The episode “Improving our understanding of our seas” went on air for 1 week and was then made available on the programme EuroNews YouTube channel. The DEVOTES documentary prepared by the project team will be ready in October 2016. A professional company (partner of the project) worked on the details of the storyboard, collecting videos, interviews and images from the DEVOTES partners. It will be broadcast via Internet-based channels (YouTube, Vimeo), available from the project website and promoted via the project social network accounts. A wide audience will be reached by the use of e-media tools for the promotion of the film to increase the social impact.

Training activities

Training activities and summer schools are an important part of dissemination. They provide for the legacy of a project by disseminating the project results to end users, such as postgraduate students and practitioners. Whereas students enrolled in postgraduate courses may benefit from taught modules, practitioners usually do not have the time or professional freedom to enroll in long-term training courses. Focused and short summer schools therefore provide an important opportunity for practitioners to learn complementary skills. The uptake of scientific results published in scientific papers and text books into curricula usually has a long time lag, sometimes lasting several years. Hence, including the training into postgraduate and summer schools, which can be attended by practitioners, fast-tracks the information to current end-users and those about to enter the job market (postgraduates).

A successful training course should be disseminated to potential end-users in a timely manner. In this way, interested candidates can plan to attend, if they are fully employed, or plan to select the course if they are post-graduates. The information provided should include the necessary context so that the candidate understands what training will be on offer and why they would benefit from attending. The training programme should include the knowledge and skills that will be learned when completed.

In the DEVOTES project, the consortium organized four summer schools to disseminate current “hot topics” addressed throughout the life of the project by the different partners. The topics covered were: genomic tools applied to monitoring; new modeling applied to assess the status of marine systems; innovative, and integrative ecosystem quality assessment tools; and ecosystem services provided by seas. DEVOTES Summer Schools have attracted both early career and senior researchers alike. Keynote talks were given around the specific topics listed above. Unlike the classic symposium format, where attendants are exposed to many but very short presentations, the longer length of the talks in these Summer Schools allowed the speaker to extensively expose different aspects of the subject and disseminate the results of the project in detail.

In addition to the primary dissemination and training tasks, these summer schools had other important objectives: (i) networking with scientists not involved in the project, either as professors or attendees, to bring fresh ideas into the project tasks and deliverables; (ii) give the opportunity to managers, Ph.D. students, Post-Doc, and scientists attending the school to learn about emerging concepts that can be incorporated into their daily research; (iii) disseminate the findings among more ample communities, e.g., through the collaboration with organizations such as EuroMarine, an European marine research network (http://www.euromarinenetwork.eu); and (iv) publish position papers on the topics addressed, which can be a direct (e.g., Borja et al., 2016) or indirect (e.g., Bourlat et al., 2013; Piroddi et al., 2015) result of the school. The Summer Schools have spread the findings of the project to an ample audience, covering more than 30 countries from all continents. A qualitative analysis of the Summer Schools is reported in Section Impact Analysis.

In addition to summer schools, other ways of training have been explored and implemented in DEVOTES. The use of webinars (online live courses) has been used as means to train on specific topics. As indicated above, there is often interest for learning but difficulties in accessing such knowledge. In the case of physical courses, this might be difficult for those working full time or having limited time or economic resources. To overcome such issues, webinars can be a realistic solution. In DEVOTES, webinars have been used to train key stakeholders on the most relevant tool developed under the project. With a total participation of 76 relevant stakeholders, and feedback received, it can be considered a very cost-effective means for communicating and practical training. The webinars are also available on the website of the project, together with short, YouTube training videos, and guidelines.

New tools

Social media

Internet platforms, mobile applications (Apps), and social media have now also become resources to share research progress and to learn. All these tools represent a unique opportunity for scientists to enhance ocean literacy, “understanding of the ocean's influence on you—and your influence on the ocean,” (Carley et al., 2013), allowing citizens to take informed decisions and to be able to participate in public debate about ocean health (Fauville et al., 2014).

Generic and professional social media tools, such as ResearchGate, LinkedIn, Facebook, Twitter, or Instagram have exploded in popularity in the last decade, attracting more and more scientists to using them. As mentioned above, online presence is fundamental for science communication and, together with social media, offers a wide range of benefits for scientists: boost their professional profile, enhance professional network, improve research efficiency and scientific metrics (Bik and Goldstein, 2013; Jucan and Jucan, 2014). Using social networks to promote research results and paper publications has been proved to increase the number of citations of their articles and the H_index (Liang et al., 2014). A strong presence on social media may result in papers having 11 times more possibility to be cited vs. articles lacking of social media presence (Eysenbach, 2011). Additionally, generic social networks offer the opportunity to reach a wide range of people with a more or less developed personal interest in science and to develop that interest (Fauville et al., 2014).

DEVOTES has been present on a few, carefully selected social media tools, both professional and generic, to take advantage of the specific features of each one (pros and cons of the different media tools will be discussed further in Section Comparison of Different Media Tools). The DEVOTES Dissemination Team created an account and a discussion group in LinkedIn, with 206 members, which served as tool to improve sharing knowledge with other scientists and industry professionals in the marine and environment fields, to enhance the ocean literacy among these two target groups. DEVOTES made its social debut early in 2013 (ca. 6 months after the beginning of the project), using the most popular platforms: Facebook, https://www.facebook.com/Devotesproject/), Twitter (@DEVOTESproject), and YouTube. The social media campaign included publishing posts at least three times per week from the project and project coordinator accounts.

To make DEVOTES appealing for the general public and decision makers, the DEVOTES Dissemination Team published posts on the website and social networks on environmental days (e.g., the 22nd March World Water Day, 8th June World Oceans Day), linking the project activities with the topic of each day. For example, on the International Day of Biodiversity (22nd May) we linked its topic “Mainstreaming Biodiversity; Sustaining People and their Livelihoods” with the main message of the DEVOTES Final Conference: “Marine biodiversity is the key to healthy and productive seas.”

Other messages were dedicated to different categories of stakeholders (e.g., environmental agencies, consulting companies) and therefore included more technical aspects, such as the production of the Catalogue of Monitoring Networks and the development of NEAT, the Nested Environmental status Assessment Tool.

Mobile apps

The innovation in mobile computing technologies and their affordability make the learning process possible using mobile applications (“apps” hereafter). Small devices, such as smartphones and tablets, are now part of our daily life, have strong computing power and they are potentially always connected. Applications for smartphones and tablets are considered useful communication tools, which are able to reach out further than our scientific reports and publications do, including society at large (Hsu and Ching, 2013). Therefore, mobile devices represent a great opportunity for education, science communication and ocean literacy. To this end, the DEVOTES dissemination strategy included the development of mobile applications. Two apps already available are “DevoMAP” and “MY-GES.” Another two are planned to be released by October 2016. All apps will be available for iOS and Android devices and downloadable from the project website. DevoMAP and MY-GES aim to disseminate the results from innovative modeling to a wide audience, and to attract the attention of the public, including scientists involved in assessments of GES in European regional seas and those not involved in marine environmental assessments. “DevoMAP” focuses on people directly involved in research and policy, to support the implementation of the MSFD. “MY-GES” targets people interested in our achievements among the general public. By targeting the general public, we aim to make society aware about the Marine Strategy Framework Directive, its implementation and assessments of environmental status. The other two apps will focus on the dissemination of overall project findings: “DevoBook,” as a result of this issue of Frontiers, and “DEVOTES,” an interactive app for the general public, including key questions and findings from all DEVOTES Work Packages and promotional material produced during the project lifetime.

Artistic elements

The use of arts in science communication is still poor but a study, conducted by Curtis et al. (2012), showed that ecologists are willing to use the arts in a scientific forum to promote their results. In particular, they think that the visual (e.g., painting) and performing (e.g., ballets, theater plays) arts can be very useful in communicating scientific information.

In 2015, DEVOTES decided to include a visual artistic element in its dissemination strategy. In collaboration with the EU project CoCoNet (Toward COast to COast NETworks of Marine Protected Areas coupled with sea-based wind energy potential), a calendar was produced to be distributed to the project stakeholders at the end of the year. The topic of the calendar was the MSFD implementation, including an artistic interpretation of the 11 MSFD descriptors of GES, which define how to assess the quality of EU marine systems. Each descriptor was represented in an evocative illustration, associated to each month, and briefly outlined in the explanatory text. December's plate describes an ideal observation system, to monitor environmental quality standards, and integrate the information to assess the status and achieve GES (Figure 5).

Figure 5

The Calendar, distributed to more than 800 relevant stakeholders, was also made available for download from the website, and in only 3 months the page received more than 600 visits.

The importance of networking with other EU projects

These interactions have resulted in undertaking a real inter- and trans-disciplinary research (Lang et al., 2012), allowing DEVOTES to go farther beyond the state of the art. This could not have been possible with the resources of only one project.

Evaluation of the dissemination goals

Impact analysis

The key issue of success of a dissemination tool depends on the ability to supply information and to transfer knowledge to the stakeholders and the potential users (Vermeulen et al., 2009), and then for stakeholders and potential users to use this knowledge. In order to evaluate the success of DEVOTES in terms of public engagement, we present here the quantitative analysis of each dissemination tool discussed above. To assess the performance of the dissemination activities on the web, several analytical tools are being used. All statistical data were regularly analyzed and compared with the impact target identified during the preparation phase of the project (Table 1) in order to measure the success and usefulness of the different tools.

To record the accessibility of DEVOTES website, Advanced Web Statistic 7.0 (AWStats, 2010) is being used to analyze the DEVOTES server log files from October 2012 until 2 years after the end of the project. Here, we present the results from October 2012 to April 2016 (Figure 6). It can be seen that, besides predictable decreases during summer and holiday seasons, use of the website increased until January 2016. Between January and April 2016, a reduction of the DEVOTES social media presence due to other commitments, led to a decreased interest in the website. An average of 2600 visits have been registered per month, with peaks of up to 10,000 hits during the release of the newsletters (e.g., June, September, and November 2013), the annual meetings (e.g., December 2014 and 2015), the revision of the website (March 2015) and peaks in social network activity (e.g., October 2013). A large proportion of the visitors came from Europe, but the website received visitors also from USA, Africa and Asia. Most of them reached the website via direct link, search engine (i.e., Google) and from external pages (i.e., DEVOTES newsletter and LinkedIn).

Figure 6

In order to evaluate the scientific impact of the whole project, two analytical tools were used to monitor the citations: Google Scholar Citations on the Google Scholar DEVOTES profile, and Altmetric, on the Zenodo DEVOTES community. Google Scholar Citations provide the user with several citation metrics. The DEVOTES papers (139, as of 18th August 2016) have a cumulative H_index of 18 and 1083 citations overall. The Altmetric Analytical tool shows the online attention and activity that have been found for each specific article, collecting relevant mentions from social media, newspapers, policy documents, blogs, Wikipedia, and other sources.

The E-media analytical tools and results to evaluate the social media impact of DEVOTES are reported in Table 2, together with the statistics from other “Ocean of Tomorrow” projects started the same year (2012). If we compare the number of social media users, it appears clear that, besides the Facebook page, DEVOTES was able to successfully build its own social community, both in generic (i.e., Twitter) and in professional social media (i.e., LinkedIn).

As the project progressed, there was a positive tendency as more followers (Twitter)/fan(Facebook)/professional-links(LinkedIn) were registered. The traffic on social pages also followed from other dissemination activities, such as the DEVOTES presence in conferences, the organization of summer schools and special sessions, and the participation to global campaigns (i.e., Ocean Sampling Day) and citizen science projects (i.e., My Ocean Sampling Day).

The impact of a successful project dissemination may result in the reassessment and enhancement of the effectiveness of relevant policies, the use of the project results by stakeholders and decision makers, and the creation of business opportunity, as well as s sharing new science-based knowledge.

In order to evaluate the impact of DEVOTES results for policy and decision makers, we monitored the amount of downloads of reports and/or deliverables (Table 3). The number of people visiting and downloading some of the reports and deliverables was very high, going far beyond the amount of persons directly involved in the project (around 200).

In addition to these quantitative evaluations, the DEVOTES Dissemination Team carried out also a qualitative evaluation on the Summer Schools and the internal dissemination activities. Satisfaction surveys conducted after each Summer School indicate that attendants were satisfied with the event. From the 61 participants in the Summer School of 2015 who answered to the satisfaction questionnaire, 67% made at least one contact for future projects and general satisfaction was scored with 8.25/10 (±1.32). However, some of the comments show that attendants were expecting a more interactive format and more opportunities for networking. Therefore, Summer Schools willing to attract students should make an effort to schedule activities with different level of participation.

Comparison of different media tools

The advancements in information and communication technology are leading to a rapid change in the world of science communication, which is now faster and more interactive. The abundance and diversity of online media sources led to an increased amount of content on offer (Porten-Cheé and Eilders, 2015). Scientists should be present in different arenas and make an effort to interact with the general public. DEVOTES took advantages of different new and traditional media tools (Table 4), with the aim of building a “DEVOTES community” which goes beyond the scientific community. If we compare the different dissemination methods used and their performances, it is clear that traditional (e.g., the website) and innovative (e.g., Twitter) tools are strongly related, and that an efficient use of the latter have a positive feedback on the performance of the former. In fact, after our experience in using the different tools during the DEVOTES project, we can rank the different media taking into account their usefulness and cost-benefit: (i) very useful: website, open access publication, sessions at international conferences, stakeholders workshops, Twitter; (ii) useful: summer schools, LinkedIn groups, press releases; (iii) moderately useful: videos, newsletters; and (iv) not very useful: Facebook, smartphone apps.

All the innovative tools should be used as complementary outlet to the traditional tools for the dissemination of new posts from the project website, to share articles, advertise job opportunities, and training events, promote meetings and circulate information about the project progress and results. This should include media that have been shown not to be very useful in the DEVOTES project such as Facebook and mobile apps, reaching audiences familiar with these media. In some cases, the lack of usefulness may be related with the longer time of maturation needed to reach a large audience, such as in the apps. However, not all media tools are necessary: the revision of the dissemination plan and the performance analyses should help to shape the social media strategy, also identifying which tools are redundant (e.g., Facebook and Google+), to avoid overlap. In the case of DEVOTES social media, we decided to focus our attention and efforts on Twitter campaigns, LinkedIn group discussions and website updates, although the Facebook account and the YouTube channel were still active.

Difficulties in engaging the stakeholders

Common difficulties encountered during dissemination to the different target group include sharing information between projects, engagement of local stakeholder, copyright, and open access. Researchers have often participated in previous, related projects but may face some constraints about sharing information. For example, contact details of stakeholders may be protected by privacy laws and therefore the effort of stakeholder mapping may have to be repeated. Conference organizers may also face constraints about distributing the contacts of participants. Another constraint is about data sharing. This may result from a number of issues. Often the data may have been previously collected by a team, of which only one member participates in the new project. This person may therefore not be able to share the data as they are not the sole owner of the data. Another typical example is about data format. Data may exist in a different format, and in the case of historical data, it may only be available in paper reports. The transcribing of such data into digital format can be a very onerous and thankless task. Other examples are obsolete storage such as floppy disks, or storage using obsolete software programmes. Trivial examples include different formats such as using a decimal point vs. a decimal comma or apostrophe. Units may also need to be converted, such as concentration in mass/volume instead of molar concentration.

Copyright and open access of information is another common problem. National or internationally funded research often requires that results be publically available or in “open access” format. While many publishers now offer that option, it comes at a price. The project participants may not have budgeted for such costs. A successful project that may publish about 200 articles may have open access costs of more than 500,000 Euros, a significant proportion of the budget. Making articles freely available without using open access, even for research and educational purposes, may infringe copyright laws.

The engagement of local stakeholders, and crucially of possible end-users, can also be problematic. First it is important to identify these potential stakeholders, and then be able to contact them. Once more, even if one project partner has this information, they may not be able to share it with the other project partners. Once the contact details are known, then the stakeholders are best approached personally, rather than through “mass” email messages. The dissemination team should communicate why the contact is considered to be an important stakeholder. How the stakeholder may participate in the co-design of the project at the onset and the project, how they may participate in the product development phase, and finally how the project information may be of use to the stakeholder, are also relevant points.

Difficulties in engaging the wide public

The health and state of our marine environment and the ecological changes being detected and predicted for the future are a global area of interest. No matter how far we live from the sea, the ocean has a strong influence on Human life, providing food energy, moderating climate, and playing an important role in the economic prosperity of many regions. Yet, the common knowledge and understanding of the oceans is not spread enough among the general public and decision makers.

A large part of the general public still obtains their science news from traditional media, such as television, and print newspapers, but internet-based tools are becoming more widely used among teenagers and young adults. Going online regularly and using Google searches now represent the standard approaches for discovering information about a topic (Bik and Goldstein, 2013). However, people feel overwhelmed by the amount of information available.

Another common problem in disseminating EU research project findings is the translation and cultural adaption of the dissemination tools/mechanisms. Most of the material is produced in English, and only selected products are translated into local languages. Moreover, although people think scientists and policy makers should be engaging in dialogue with the public about science, this is not always translated into a willingness to be personally involved. The general public tend to think that is the role of “experts” and not theirs to advise the governments on science issues. However, people show more interest in research and science when they can be directly involved in the project: citizens are more motivated if they can “actively” contribute to science advancements. If people do not see how they can make the difference or being actively involved, they may lose interest. To this end, we suggest that citizen science activities should be included in research project proposals.

Conclusions

An effective science communication allows people to make sound choices (Fischhoff, 2013) about environmental issues, and help key actors to improve processes and methodologies in marine environment management. From our perspective, the most useful media tools used to disseminate DEVOTES have been the website, the open access publications, sessions at international conferences, stakeholders workshops, and Twitter. Other media could be considered for specific targeted audience.

There are several factors influencing the dissemination of European funded projects, such as the limited project duration (e.g., 2–4 years), which could threaten the dissemination of end products, (see “Threats” reported in Figure 3). This in turn could influence the assessment of the dissemination impact to the stakeholders and the general public. To solve these risks, we suggest to include periodic (at least every year) web-based and physical surveys to monitor the effectiveness of results. Additionally, recent studies reveal that, although having a positive view of science and technology, EU citizens think scientific research is difficult to understand and that scientists should be more effective in communicating scientific results (European Commission, 2007, 2010). Our suggestion is to include (where possible) a citizen science initiative in the communication strategy, in order to actively involve the general public, not only in the collection of data but also in the dissemination process (e.g., increasing the social media audience and presence). In fact, the lack of a citizen science initiative was the factor determining the low success of the DEVOTES Facebook page (see “Weakness” reported in Figure 3).

Therefore, it is fundamental to develop an effective dissemination strategy at the moment of writing a research project proposal, and to perform a constant evaluation of the dissemination results before, during and after the project lifetime, involving all the key actors, advisory board and partners (see “Strengths” reported in Figure 3). To achieve this, the use of different media tools, targeting them to the adequate audience, will ensure the success of the project, by making available all the outcomes and products to the end users.

References

• 1

  AWStats (2010). AWStats Official Website. Available online at: http://www.awstats.org (Accessed June 7, 2016).

  • Google Scholar

• 2

  BarnettM.WagnerH.GatlingA.AndersonJ.HouleM.KafkaA. (2006). The impact of science fiction film on student understanding of science. J. Sci. Technol.15, 179–191. 10.1007/s10956-006-9001-y

  • CrossRef
  • Google Scholar

• 3

  BikH. M.GoldsteinM. C. (2013). An introduction to social media for scientists. PLoS Biol.11:e1001535. 10.1371/journal.pbio.1001535

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 4

  BorjaA.ElliottM.AndersenJ. H.BergT.CarstensenJ.HalpernB. S.et al. (2016). Overview of integrative assessment of marine systems: the ecosystem approach in practice. Front. Mar. Sci.3:20. 10.3389/fmars.2016.00020

  • CrossRef
  • Google Scholar

• 5

  BourlatS. J.BorjaA.GilbertJ.TaylorM. I.DaviesN.WeisbergS. B.et al. (2013). Genomics in marine monitoring: new opportunities for assessing marine health status. Mar. Poll. Bull.74, 19–31. 10.1016/j.marpolbul.2013.05.042

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 6

  BrownellS. E.PriceJ. V.SteinmanL. (2013). Science communication to the general public: why we need to teach undergraduate and graduate students this skill as part of their formal scientific training. J. Undergrad. Neurosci. Educ.12, E6–E10.

  • Pubmed Abstract
  • Google Scholar

• 7

  BultitudeK. (2011). The why and how of science communication, in Science Communication, ed RosulekP. (Pilsen: European Commission), 1–18.

  • Google Scholar

• 8

  BurnsT. W.O'ConnorD. J.StocklmayerS. M. (2003). Science communication: a contemporary definition. Public Underst. Sci.12, 183–202. 10.1177/09636625030122004

  • CrossRef
  • Google Scholar

• 9

  CanessaE.ZennaroM. (2009). Science Dissemination Using Open Access. A Compendium of Selected Literature on Open Access. Trieste: ICTP-SDU.

  • Google Scholar

• 10

  CarleyS.ChenR.HalversenC.JacobsonM.LivingstonC.MatsumotoG.et al. (2013). Ocean Literacy: The Essential Principles and Fundamental Concepts of Ocean Sciences for Learners of All Ages Version 2, A Brochure Resulting from the 2-Week On-Line Workshop on Ocean Literacy through Science Standards. Monterey: National Oceanic and Atmospheric Administration.

  • Google Scholar

• 11

  CurtisD. J.ReidN.BallardG. (2012). Communicating ecology through art: what scientists think. Ecol. Soc.17:3. 10.5751/ES-04670-170203

  • CrossRef
  • Google Scholar

• 12

  DurhamE.BakerH.SmithM.MooreE.MorganV. (2014). The BiodivERsA Stakeholder Engagement Handbook. Paris: BiodivERsA, 108.

  • Google Scholar

• 13

  ESF Science Policy Briefing 50 (2013). Science in Society: Caring for Our Futures in Turbulent Times. Available online at: http://www.esf.org/fileadmin/Public_documents/Publications/spb50_ScienceInSociety.pdf (Accessed June 7, 2016).

  • Google Scholar

• 14

  European Commission (2001) EUR 20022 – Gender in Research. Gender Impact Assessment of the Specific Programmes of the Fifth Framework Programme – An Overview. Available online at: http://ec.europa.eu/research/pdf/gender-in-research-synthesis-report_en.pdf (Accessed August 6 2016).

  • Google Scholar

• 15

  European Commission (2007). Special Eurobarometer. Scientific Research in the Media. Available online at: http://ec.europa.eu/public_opinion/archives/ebs/ebs_282_en.pdf (Accessed June 7, 2016).

  • Google Scholar

• 16

  European Commission (2010). Special Eurobarometer. Science and Technology Report. Available online at: http://ec.europa.eu/public_opinion/archives/ebs/ebs_340_en.pdf (Accessed June 7, 2016).

  • Google Scholar

• 17

  European Commission (2013). Regulation (EU) No. 1291/2013.Brussels.

  • Google Scholar

• 18

  European Commission (2014). Communicating EU Research and Innovation Guidance for Project Participants. Available online at: http://ec.europa.eu/research/participants/data/ref/h2020/other/gm/h2020-guide-comm_en.pdf (Accessed June 7, 2016).

  • Google Scholar

• 19

  EysenbachG. (2011). Can tweets predict citations? Metrics of social impact based on twitter and correlation with traditional metrics of scientific impact. J. Med. Internet Res.13, e123. 10.2196/jmir.2012

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 20

  FauvilleG.DupontS.von ThunS.LundinJ. (2014). Can Facebook be used to increase scientific literacy? A case study of the Monterey Bay Aquarium Research Institute Facebook page and ocean literacy. Comput. Educ. 82, 60–73. 10.1016/j.compedu.2014.11.003

  • CrossRef
  • Google Scholar

• 21

  FischhoffB. (2013). The sciences of science communication.Proc. Natl. Acad. Sci.110(Suppl. 3), 14033–14039. 10.1073/pnas.1213273110

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 22

  HooperR.LilienfeldL.ArrigoJ. (2011). Using documentaries for Earth science education. Eos Trans. AGU92, 361.

  • Google Scholar

• 23

  HsuY. C.ChingY. H. (2013). Mobile app design for teaching and learning: Educators' experiences in an online graduate course. Int. Rev. Res. Open Distributed Learn.14, 117–139.

  • Google Scholar

• 24

  JucanM. S.JucanC. N. (2014). The power of science communication. Procedia Soc. Behav.149, 461–466. 10.1016/j.sbspro.2014.08.288

  • CrossRef
  • Google Scholar

• 25

  KarlinB.JohnsonJ. (2011). Measuring impact: the importance of evaluation for documentary film campaigns.M/C J.14. Available at: http://www.journal.media-culture.org.au/index.php/mcjournal/article/view/444 (Accessed June 7, 2016).

  • Google Scholar

• 26

  KlingR.McKimG. (2000). Not just a matter of time: field differences and the shaping of electronic media in supporting scientific communication. J. Assoc. Inf. Sci. Technol.51, 1306–1320. 10.1002/1097-4571(2000)9999:9999<::aid-asi1047>3.0.co;2-t

  • CrossRef
  • Google Scholar

• 27

  LangD. J.WiekA.BergmannM.StauffacherM.MartensP.MollP.et al. (2012). Transdisciplinary research in sustainability science: practice, principles, and challenges. Sustainability Sci.7, 25–43. 10.1007/s11625-011-0149-x

  • CrossRef
  • Google Scholar

• 28

  LiangX.SuL. Y.-F.YeoS. K.ScheufeleD. A.BrossardD.XenosM.et al. (2014). Building Buzz (Scientists) communicating science in new media environments. J. Mass Commun. Q.91, 772–791. 10.1177/1077699014550092

  • CrossRef
  • Google Scholar

• 29

  MollickE. (2014). The dynamics of crowdfunding: an exploratory study. J. Bus. Venturing29, 1–16. 10.1016/j.jbusvent.2013.06.005

  • CrossRef
  • Google Scholar

• 30

  NetaG.GlasgowR. E.CarpenterC. R.GrimshawJ. M.RabinB. A.FernandezM. E.et al. (2015). A framework for enhancing the value of research for dissemination and implementation. Am. J. Public Health105, 49–57. 10.2105/AJPH.2014.302206

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 31

  PiroddiC.TeixeiraH.LynamC. P.SmithC.AlvarezM. C.MazikK.et al. (2015). Using ecological models to assess ecosystem status in support of the European Marine Strategy Framework Directive. Ecol. Ind.58, 175–191. 10.1016/j.ecolind.2015.05.037

  • CrossRef
  • Google Scholar

• 32

  PohlC.Perrig-ChielloP.ButzB.Hirsch HadornG.JoyeD.LawrenceR.et al. (2010). Questions to Evaluate Inter- and Transdisciplinary Research Proposals. Working paper, Swiss Academies of Arts and Sciences.

  • Google Scholar

• 33

  Porten-CheéP.EildersC. (2015). Spiral of silence online: how online communication affects opinion climate perception and opinion expression regarding the climate change debate. Stud. Commun. Sci.15, 143–150. 10.1016/j.scoms.2015.03.002

  • CrossRef
  • Google Scholar

• 34

  PotoČnikJ. (2007). Let's make science the next headline, in Communicating European Research 2005, ed ClaessensM. (Dordrecht: Springer), 13–17.

  • Google Scholar

• 35

  Royal Society (1985). The Public Understanding of Science. Report of a Royal Society ad hoc Group endorsed by the Council of the Royal Society. Available at: http://royalsociety.org/uploadedFiles/Royal_Society_Content/policy/publications/1985/10700.pdf (Accessed June 7, 2016)

  • Google Scholar

• 36

  Saint-PaulU.SchneiderH. (2016). The need for a holistic approach in mangrove-related fisheries research: a specific review of the German and Brazilian research project MADAM. J. Fish Biol.89, 601–618. 10.1111/jfb.12880

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 37

  TreiseD.WeigoldM. F. (2002). Advancing science communication a survey of science communicators. Sci. Commun.23, 310–322. 10.1177/107554700202300306

  • CrossRef
  • Google Scholar

• 38

  UyarraM. C.BorjaÁ. (2016). Ocean literacy: a ‘new’ socio-ecological concept for a sustainable use of the seas. Mar. Poll. Bull.104, 1–2. 10.1016/j.marpolbul.2016.02.060

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 39

  VermeulenP.BreretonP.LofthouseJ.SmithJ.KehagiaO.KrafftA.et al. (2009). Web-based communication tools in a European research project: the example of the TRACE project. Biotechnol. Agron. Soc.13, 509–520.

  • Google Scholar

• 40

  WilkinsonC.FrankeJ.StroyanJ. (2016). Ex-post evaluation of science in society in FP7. Executive summary.Eur. Union18.

  • Google Scholar

• 41

  WilsonP. M.PetticrewM.CalnanM. W.NazarethI. (2010). Disseminating research findings: what should researchers do? A systematic scoping review of conceptual frameworks. Implement. Sci.5, 1–16. 10.1186/1748-5908-5-91

  • Pubmed Abstract
  • CrossRef
  • Google Scholar