The Role of Stakeholders in Creating Societal Value From Coastal and Ocean Observations

Introduction

Ocean observations, both at and below the sea surface, provide data and information required to underpin assessments, analyses, and predictions of the state of the ocean environment. Ocean observations enable monitoring of ocean climate, provide early warning/tracking of high impact weather and oceanographic events and are used to initialize and verify the performance of forecasting models. The data and information provided by ocean observations, in turn, supports the commercial and industrial sectors (e.g., offshore oil and gas operations, safe and efficient maritime transport, tourism and recreation), and realizes societal benefits such as better management of public health risks and protection of people and property from natural hazards at the coast (Bell et al., 2013; US IOOS, 2018¹). In addition sustained ocean observations, measurements and models provide an important input to weather forecasts and climate projections, delivering socioeconomic benefits far inland (Rayner et al., 2018).

To fully realize the societal and economic benefits of ocean observation there is a requirement for an integrated ocean observing system. The term integrated ocean observing system describes a network of observation platforms and sensors that acquire a huge variety and volume of spatio-temporal data about the ocean environment. Such an integrated observation system includes activities such as data acquisition, transmission, management, and communications, analyses and modeling. These integrated systems contribute to the information-base which enables the benefits from the blue economy to be maximized.

Stakeholder engagement is a key element of the beneficial use of an integrated ocean observing system. A productive and sustainable ocean economy requires strong partnerships. A diversity of stakeholders – managers, decision-makers, users of ocean observing products and services, socioeconomic communities and civil society, and the builders and operators of observing systems, amongst others – contribute to maximize the economic benefit from the blue economy (Malone et al., 2014). A clear understanding of, and mutual agreement on the role and needs of stakeholders is required to bridge the gap between ocean observations, as an activity born out science and engineering, and its benefits to society. Comprehensive stakeholder engagement helps facilitate the identification of gaps in ocean observing, enhancing derived products and services, and ensuring global capacity and capability exists to enable the use of observation systems and science. However, it is also important to recognize that the various stakeholders have different institutional mandates, objectives, operational and strategic priorities, resources and management procedures.

Ocean observing can be a costly endeavor. Whilst there is no globally accepted figure the European Union (2018) have examined the cost of undertaking ocean observing in a number of European countries and further afield. Capital costs alone can be upward of thousands of million euros per country with associated operational costs in the hundreds of millions per year. It is not unreasonable to expect that this level investment produces information that is usable for a variety of applications to meet the needs of communities, and society at large. Unfortunately, there are disconnects between these groups that, more often than not, result in the implementation of monitoring systems without sufficient consideration of the information needs of stakeholders as a group (Christian et al., 2006).

In parallel, the ocean economy is beginning to be well understood. In 2010, ocean industries were estimated to have contributed US$ 1.5 trillion to the global economy (OECD, 2016). The OECD report states that by 2030, many ocean-based industries have the potential to outperform the growth rate of the global economy, both in terms of value-add and employment. The OECD analysis of a “business-as-usual” scenario, predicts that, between 2010 and 2030, the ocean economy could more than double its contribution to global value added, reaching over US$ 3 trillion. It is, therefore, clear that the ocean provides a wealth of economic and social benefits and that these benefits are underpinned by ocean observations, measurements, and forecasts. However, there have been no comprehensive global attempts to value these benefits, although numerous case studies have attempted to quantify components of the benefit accruing from the collection and use of such data. In aggregate, the cost of obtaining and using such data is almost certainly only a small percentage of the value of the benefits derived (Rayner et al., 2018).

Deriving economic benefit from the oceans must be considered in conjunction with the often-concomitant deterioration of ocean health. Plastics pollution and ocean acidification are now joining harmful atmospheric emissions, the spread of invasive species and overfishing as serious societal challenges that must be addressed. Impacts from changing climate including rising sea levels and coastal storms and flooding are also of serious concern. Many observation and monitoring programs inform policies designed to enable protection of the global oceans. These include global policies such as the United Nations (UN) 2030 Agenda, and in particular, Sustainable Development Goal 14 (SDG14) which covers ocean conservation and the sustainable use of ocean, sea and marine resources. Furthermore, ocean observations are important elements of global assessments such as the World Ocean Assessment (WOA) and those undertaken by the Intergovernmental Panel on Climate Change (IPCC).

In order to achieve SDG14, and the wider Sustainable Development Goals (SDGs) it is equally important to recognize societal processes alongside that of contributions from science and technology. The United Nations Framework Convention on Climate Change (UNFCC) and the 2018 Talanoa Dialogue Platform is an example of the greater recognition of the role of humanity in bringing about change. Talanoa, a traditional Fijian word meaning “to talk or speak” is used in the Pacific and has been adopted more widely to describe a process of inclusive, participatory and transparent dialogue, the purpose of which is to share stories, build empathy and to make wise decisions for the collective good. This philosophy should be adopted when considering future stakeholder engagement activities in ocean observation to maximize the contribution to sustainable development for the benefit of humanity.

This paper explores the importance of stakeholder engagement in ocean observation as demonstrated by a number of case studies. It begins by introducing stakeholder engagement within existing coordinating frameworks such as the United States’s Interagency Ocean Observing System (IOOS^®), and the European Ocean Observing System (EOOS) and the grassroots approaches used; where IOOS was primarily established as a community-driven initiative and EOOS seeks to build on the same democratic model. The paper then highlights other initiatives such as public–private partnerships, which are typically established to fulfill a business or societal need, and, introduces the role of the “third” or voluntary sector. This sector typically comprises Non-Governmental Organizations (NGOs) with a broad remit to develop and share knowledge and promote community participation for public benefit. The paper will explore the value stakeholder engagement can bring as well as noting existing shortcomings, such as the lack of geographical range in the cases presented which are based on the knowledge and experience of those contributing. Finally, the paper will make recommendations for the future.

The Role of Stakeholders in Creating Societal Value From Coastal and Ocean Observations

Basic Principles

In the ocean and coastal science community, there has long been an appreciation of the need to create an ocean observation network. However, the importance and urgency of growing the network and engaging a wider range of stakeholders has increased congruently with an increasing understanding and appreciation of ocean issues. A successful ocean observing network includes managers, policy-makers, civil society, general scientists and specialists (including scientists undertaking observations), marine service providers and geo-spatial technology stakeholders. In such a network, each stakeholder has a clear understanding of their unique role and responsibility as well as their related needs. The GEO Blue Planet Initiative is working to build and support such a network. It aims to support the production of relevant, useful and timely data to inform ocean and coastal decision-and policy-making. The Blue Planet initiative acknowledges three pillars for the success of such a network: convergence around common goals, effective communication and the co-production of information and knowledge.

Converging on Common Goals

The SDGs, the Sendai Framework and many others represent global objectives for human development. These internationally negotiated agreements provide clear development goals. These include the sustainable use of our oceans, and security for our coastal populations, among others. As such there is increasing convergence on these common goals which can act as catalysts to assemble human talent, to explore business and technology opportunity, and build collaborations (Hov et al., 2017).

The need for the ocean observation community to respond to global objectives and challenges is increasingly being acknowledged (Guo et al., 2015; Aitsi-Selmi et al., 2016; Anderson et al., 2017; Benedetti-Cecchi et al., 2018). If the coastal and ocean observing community can drive a specific agenda linked to these societal objectives, then the likelihood of a successful network and co-production of information and knowledge can be realized. Mutual recognition of priorities is a strong starting point for collaboration and the formation of partnerships.

Effective Communication

Ocean observing users require data products and information which respond to their needs. These users may be those for whom the data has been specifically collected, or those using data-derived information to inform decisions, e.g., on managing ocean and coastal resources while preventing or mitigating effects on the supporting ecosystems and species therein. These users need to have the information in a way that allows them to make decisions based on scientific evidence but which are presented within a specific social and policy context. In turn engagement of ocean observing users is essential in order to design fit-for-purpose ocean observing systems and networks. Communication is a critical means to enable both the collection of user requirements and the provision of information and products that respond to the user needs. However, targeting communications to such a variety of stakeholders requires a thorough stakeholder analysis and personalized approaches.

The recent inclusion of the importance of ocean observing in several high-level policy statements and agendas (Agenda 2030, the United Nation Ocean Decade for Sustainable Development, G7, etc.), gives an unprecedented gravitas to the ocean observing community in explaining the relevance of their work to policy and society. Scientific evidence is paramount in the development of policies to meet major global challenges. As a result, there is a need to better understand opportunities and constraints to science use to inform policy design and implementation, as there is a need to understand how to make science usable (Dilling and Lemos, 2011; Lemos et al., 2012; Kirchhoff et al., 2013). The usability of science has been widely debated by the climate change science community (van Aalst and Agrawala, 2005; Moss et al., 2013; Kirchhoff et al., 2015). Similarly, the usability of ocean observations is due serious consideration. Traditionally, the scientific community has assumed that if decision-makers are provided with reliable science information, this information will be used to make improved evidence-based decisions (Nisbet and Scheufele, 2009). This assumption is overly simplistic and inaccurate. There remains a compelling need for alternative approaches to the engagement between the producers of scientific information and the target users (practitioners) of this information (Kollmuss and Agyeman, 2002; Nisbet and Scheufele, 2009; Cone et al., 2013). Science–policy interfaces emerge as enablers of a sustained relationship between the ocean observation community and the users of observation data and information.

To adequately inform decision- or policy-making, scientific evidence-based recommendations should also consider the political, cultural and social debate that inevitably and justifiably surrounds these major issues (Horton and Brown, 2018). Ideally, scientific evidence expands alternatives, clarifies choice and enables policy-makers to achieve desired outcomes (McNie, 2007). Therefore, scientific evidence submitted as policy advice should be drawn from natural and physical sciences, and the humanities such as social science, law and economics. This multi-disciplinary approach to delivering scientific advice to policy is still new to the ocean observing community. Communication and capacity building are required within the community, to understand policy drivers and constraints and to learn how to communicate with policy makers in an impactful fashion. The same applies to building effective relationships and mutual trust between the scientific community and industrial users of the ocean information, space and resources. Furthermore, delivering information and services to the industrial end-users, often involves the engagement of intermediary service providers who transfer and customize the ocean observing data into tailored products for end-users.

In summary; effective engagement between sciences and users is inherently tied to effective communication (Vogel et al., 2007). There is increasing support for initiatives that support effective communication by building relationships, trust and dialogue (Nisbet and Scheufele, 2009). Effective communication is also becoming embedded in the core objectives of the global initiatives like GEO Blue Planet or within European associations like EuroGOOS. However, building and maintaining stakeholder interfaces and relationships is time consuming and resource-intensive. While communication has been broadly recognized as a priority, adequate funding for it remains a challenge in the ocean and coastal science community.

Co-production of Information and Knowledge

Once a common objective has been identified, and relationships and mutual understanding have been established, the next step is to co-produce usable information and knowledge. Information produced through collaboration between scientists and practitioners, funders, technology developers, politicians and other users have been shown to produce more usable and concrete outputs (Lemos and Morehouse, 2005; Walter et al., 2007; Roux et al., 2010; Lemos et al., 2012; Kirchhoff et al., 2013; Reed et al., 2014; Howarth and Monasterolo, 2017; Djenontin and Meadow, 2018). Djenontin and Meadow (2018) have recently outlined the elements, principles and processes involved in co-production of information and knowledge based on the growing body of research on this topic (Figure 1).

FIGURE 1

Figure 1. Variables of knowledge co-production work. Source: Djenontin and Meadow (2018).

These variables are not meant to be exhaustive but can be used as a starting point for successful co-production of information and knowledge (Djenontin and Meadow, 2018). It is important that parties embrace the blurring of ‘traditional’ roles of scientists and practitioners (Vogel et al., 2007; Dilling and Lemos, 2011).

The Need for Innovation and Usable Information

There is a need for technological innovation, alongside innovation in designing and implementing policy, and social responses to achieving global sustainability. This is considered one of the grand challenges for Earth System Sciences (Reid et al., 2010). When considering science in service of society: private, public and academic institutions, together with NGOs, need to agree a strategy for translating and transmitting information in the most efficient and effective way (Hov et al., 2017).

Ocean observation is inherently technical and scientific in nature. Accordingly, there are constant technological advances and disruptions, and new industry “players,” both public and private (Denis et al., 2017; Levy et al., 2018). Similarly, there is a large and dynamic network of observation technologies, and institutions and agencies that also benefit from, and drive, advances and disruptions. Examples of disruptive technological advances in the field of ocean observation and more widely include NOAA’s Big Data Project²; EO Data Cubes and Analysis Ready Datasets (Giuliani et al., 2017; Nativi et al., 2017), crowd-sourcing and citizen science (Mazumdar et al., 2017; Brovelli et al., 2018; O’Sullivan et al., 2018) and deep learning and artificial intelligence (Lary et al., 2018). Related to this is the way that Big Data has changed the paradigm when it comes to the availability and exploitation of ocean observing data. Similarly, it challenges the interaction between data providers, service/application providers and users (of different types) (Ceccaroni et al., 2018). These disruptions are not limited to technology and technology partners, but include a change in how observational data are transformed to inform specific societal needs.

The mission and objectives of space and earth observation agencies and institutions are increasingly reflecting the need to improve the distance between observational data and its use by civil society or industry. For example, Space 4.0i, an initiative of the European Space Agency (ESA), combines the global situation of space developments (Space 4.0) with the ‘i’ standing for an ESA-specific interpretation of the tasks. This includes objectives to: (1) innovate – through more disruptive and risk-taking technologies; (2) inform – through the reinforcement of the link with large public and user communities; (3) inspire – through the launch of new initiatives and programs, involving both current and future generations; and (4) interact – through enhanced partnerships with European countries, European institutions, international players and industrial partners. Through this initiative ESA aims to be in a position to drive the realization of a ‘united space in Europe’ for the benefit of its citizens and economy. This approach has been taken a step further by the National Oceanic and Atmospheric Administration (NOAA)’s Big Data Project which has made available a wide range of data from satellites, radars, ships, weather models, and other sources in a cloud environment. This approach mitigates the need to download and store the terabytes of data generated on a daily basis, rather it analyses in the cloud which in turn benefits from high availability, scalability and resilience required by users. NOAA believes this approach will foster innovation and provide value to users.

It is appropriate to acknowledge that the current engagement between the broader EO community (predominantly, space observation) and users (using their broadest conceivable definition) is adding value (Denis et al., 2016; Hossain et al., 2016; Vasko et al., 2017). The commercial Earth Observation data market (in Europe) was estimated at $1.7 billion in 2015 while the value-added service market was estimated to be worth $3.2 billion (Denis et al., 2016).

Embracing Diversity and Interconnectedness

Linking scientific information obtained from observations to users is challenging and complicated. The traditional value chain of data–information–knowledge has been conceived as a linear process. There is, however, a growing body of literature challenging this uni-directional value transfer (Crewe and Young, 2002; Jasanoff, 2003; Nowotny, 2003; Court and Young, 2006; Moll and Zander, 2006; Karl et al., 2007). Climate science and other disciplines of environmental science have embraced an alternative paradigm that views the relationship between science and practice as a complex web of connectivity and engagement across a wide range of stakeholders (Cash et al., 2006; Vogel et al., 2007). Within this web, there are multiple types of institutional arrangements and mechanisms for developing and disseminating scientific information (Dilling and Lemos, 2011). It is proposed that the ocean and coastal science community embrace the view of relationships as ‘spider webs of interactions’ that are ‘composed of nodes and a multitude of ephemeral linkages,’ as described by Vogel et al. (2007).

While the various types of institutions, programs, and groups within this web vary in the degree and mechanism of producing and transferring knowledge, all of these efforts seek to connect with users at different levels (Dilling and Lemos, 2011). It is important for the ocean observation community to recognize and value this web of stakeholders to achieve common global goals.

Case Studies in Stakeholder Engagement

Building a Regional Community-Driven Observing System – The European Ocean Observing System (EOOS)

The international Global Ocean Observing System (GOOS) program is a collection of observing systems, which provide near real-time measurements of the state of the oceans for observing, modeling and analyzing marine and ocean variables and which support operational oceanography worldwide. GOOS is a platform for international cooperation for sustained observations of the oceans, generation of oceanographic products and services and interaction between research, operational, and user communities. The implementation of GOOS activities occurs through programs such as GOOS Regional Alliances (GRA), JCOMM (Joint WMO-IOC Technical Commission for Oceanography and Marine Meteorology) and the IODE (Oceanographic Data and Information Exchange).

EuroGOOS is the principal GRA in Europe and considers the development of services to meet ocean health and climate user needs to ensure complementarity with the three GOOS thematic areas (real time services, ocean health and climate). EuroGOOS, together with the European Marine Board and many other partners, is strongly engaged in building the framework for ocean observing in Europe – The European Ocean Observing System (EOOS)³.

EOOS is a coordinating framework designed to integrate Europe’s ocean observing capacity. This framework supports a systematic and collaborative approach to collecting information on ocean state and variability in the long-term. Observation data, as a product of this effort, can then be used to inform sustainable development, and conservation of the marine environment. EOOS is building on existing initiatives and aims to be inclusive and community-driven, thereby meeting societal needs.

In the decade since OceanObs’09, European ocean observing has evolved considerably (EuroGOOS, 2016a,b, 2018⁴). However, Europe’s capability in ocean observing and monitoring remains largely ad hoc (Figure 2). While there are some long-term monitoring programs in Europe, in many cases, funding is short-term and often only made at national, regional or even institutional level, especially for sustaining infrastructure and monitoring. A European alignment on funding and programming priorities is difficult due to the diversity of priorities in the ocean observing communities, from operational oceanography to wider research, and from environmental assessments to blue economy activities.

FIGURE 2

Figure 2. The existing challenges and untapped potential of fragmented dispersed ocean observing efforts.

Key European science–policy events and foresight initiatives have called for the continent to develop a truly integrated and sustainably funded system of European ocean observing systems. The purpose is to connect the full diversity of European ocean observation and monitoring infrastructures and stakeholders across the ocean observing value chain (EurOCEAN Declarations⁵; EurOCEAN, 2010, 2014; European Marine Board, 2013). EOOS is being developed as a framework to design and implement such integration. This is essential to address the current fragmentation across multiple sectors, regions, and countries, avoiding missed opportunities for collaboration and duplication of effort. Plans are not currently coordinated across different regions, ocean observing platforms and stakeholder communities (e.g., operational services and environmental monitoring). Furthermore, there is a gap in terms of technological and human capacities between the types of ocean variables observed. While observations for the physical variables have attained a high level of quality, data availability and aggregation, biological observations are lagging behind. This is due, to a large extent, to the considerable progress in automatization of physical measurements, as well as historical needs for physical oceanography services for navigation, meteorological services and maritime security. In turn, biological measurements are often collected primarily for scientific needs, rather than to provide practical industry solutions, and therefore rely on project funding. Fisheries management is an exception to this generalization where observations can be used to fulfill a very specific industry need. Traditionally ocean observations have not been well used by the fisheries industry where understanding of the mechanisms involved, the available data, or the large scale correlations are limited. In most cases, statistically significant correlations between population dynamics and population processes break down are yet to be established. This has led to advocating direct monitoring or developing management strategies that are robust to the variation rather than determining the relationships between population dynamics and oceanographic processes (Venkatesan and Sampath, 2017). In addition, biological observations are often complex and harder to automate, and the required human expertise is often lacking (e.g., taxonomy) (Benedetti-Cecchi et al., 2018).

The EOOS process requires openness and collaboration among the variety of ocean observing communities to help build a common strategic vision and a framework for Europe (EuroGOOS-European Marine Board, 2016). This has been driven in its initial stages through a collaboration between the European Global Ocean Observing System, EuroGOOS⁶, and the European Marine Board⁷, designed to stimulate the transition of EOOS from a concept into a tangible initiative.

By 2030, the EOOS framework will help make ocean observation a public utility in Europe. It will do this by strengthening coordination, strategy and sustainability in ocean observation. EOOS will be achieved with an operational implementation cycle that connects Europe’s ocean observing communities, and offers regular opportunities for stakeholder input to evaluate, co-design and fund capability. Putting the needs of users at its center, EOOS will promote European leadership and innovation delivering crucial data to drive environmental policy, ocean governance, sustainable blue economy, and serve society (Figure 3) (European Marine Board-EuroGOOS, 2018). Strengthened and streamlined coordination of the European ocean observing capability will allow enhanced contribution to international frameworks and efforts, e.g., the UN Agenda 2030⁸ and the United Nations Decade of Ocean Science for Sustainable Development (2021–2030)⁹. In addition to the broad requirements for ocean observing systems as previously described, drivers specific to Europe include the Marine Strategy Framework Directive¹⁰, the Common Fisheries Policy¹¹, and the Maritime Spatial Planning Directive¹². A more detailed list of known and emerging drivers requiring systematic ocean observing are listed in the EOOS Consultation document (EuroGOOS-European Marine Board, 2016)¹³ and EOOS Strategy 2018–2022 (European Marine Board-EuroGOOS, 2018).

FIGURE 3

Figure 3. The added value of a coordinated EOOS framework.

The EOOS framework is open and inclusive adding value to existing efforts across three focus areas:

(1) Better Coordinated and Sustained in situ Ocean Observing

EOOS will connect stakeholders across the ocean observing community with a focus on in situ observations, linked to remote sensing and modeling and to ensure full integration and responding to user needs.

(2) Ocean Variables Relevant to Society

EOOS will serve as a European focal point for systematic, long-term observation and monitoring as a forum to discuss, coordinate and implement international standards (e.g., Essential Ocean Variables and Essential Biodiversity Variables) and define European priorities for wider ocean variables. EOOS will promote innovative, adaptable ocean observing that can respond to evolving user needs, apply emerging technology and help invest in observations and Big Data initiatives.

(3) Integrated Ecosystem Approach

EOOS will promote multi-platform, integrated and thematic observing, which is crucial for sustainable management of ocean activities, and to assess ecosystem health and functioning and the interfaces with climate and the wider earth system.

Importantly, EOOS will build on strengthening existing capabilities, enhancing coordination at pan-European, regional and local scales, while bringing about an integrated European capacity for a global good. Regular stakeholder consultations and events are organized to collect feedback on the EOOS development from a wide range of ocean observing implementers, funders and networks. Events taking place between 2015 and 2018 have informed the EOOS Strategy and Implementation Plan 2018–2022 (EuroGOOS-European Marine Board, 2018; European Marine Board-EuroGOOS, 2018).

EOOS will also connect ocean observing users, system implementers and funders across multiple geographical scales from national to regional and sea-basin scale. This requires buy-in and support from both the bottom-up community of infrastructure owners and data providers and top-down institutions (European Union and national competent authorities) that can provide political endorsement and, potentially, resources. The entire value chain should be considered from observations to information, products and services, including satellite and in situ observations, and data assimilation into models to produce products and services such as forecasts. In Europe, the Copernicus Program¹⁴ (satellite) and Copernicus Marine Service¹⁵ (ocean services) are key initiatives for the operational ocean observation value chain, whilst the European Marine Observation and Data Network (EMODnet) provides a central open access gateway to wider ocean observing and monitoring data and data products¹⁶.

Early action for EOOS is to map existing infrastructures and stakeholders including the main ocean observing users (e.g., academia, public authorities, industries, policymakers). This mapping will be repeated and updated on a regular basis, most likely as a joint effort across key stakeholder organizations, and will inform EOOS stakeholder engagement.

EOOS promotes responsible research and innovation¹⁷ by engaging stakeholders throughout the co-define strategy and planning process. This is done through a regular implementation feedback cycle. Users and data providers are routinely consulted to evaluate and update the user requirements. They contribute their knowledge of, and expertise in the latest scientific and technological advancements, as well as consider evolving policy drivers.

Stakeholder Interaction and Dialogue

EOOS implementation is progressively achieved through dialogue amongst the communities, users, and funders. Each milestone in the EOOS development is followed by a stakeholder engagement loop (Table 1).

TABLE 1

Table 1. Milestones in the EOOS development and stakeholder engagement.

EOOS will help funders (at national, regional, and European levels) to meet with implementers and users to exchange and develop a common understanding of the full European capability and the benefits of cooperation. This will allow funders to critically assess the real gaps in the system together with a business case for recommended upgrades to the system. The intention is that this will lead to economic efficiency, increased cost-benefit, and greater societal impact.

Strengthening and Diversifying Partnerships

By strengthening and expanding partnerships, EOOS will link closer with fisheries and environmental monitoring initiatives, as well as efforts outside of the operational Essential Ocean Variables (EOVs) and across Earth Observation sectors. EOOS will assist with regional and sub-regional alignment through existing initiatives such as the GRAs, e.g., EuroGOOS (and associated Regional Operational Oceanographic Systems), European Regional Sea Conventions, Regional Fisheries Management Organizations (RFMOs), etc. EOOS enhancements will also be advised to the GEO Blue Planet initiative to ensure complementarity in interactions with users.

The EOOS Implementation Plan 2018–2022 outlines six thematic areas: mapping and stakeholder engagement, policy context and foresight, implementation, funding, communications, and governance. For each thematic area, the plan proposes concrete activities. The plan also includes early actions where tasks have already been started by the community and action is ongoing or imminent.

In order to build a successful, long-term framework, the full cycle of EOOS implementation needs to be considered. Figure 4 outlines key steps in the EOOS cycle, including the first cycle and how it could link to subsequent cycles. Stakeholder engagement, consultation and co-design are essential throughout the full cycle. For example, future ocean observing and monitoring requirements need to be co-defined through consultation with a broad range of users and wider stakeholders from academia to industry. And co-creation of observing system design would also need to bring in program managers and funders to ensure this is both fit-for-use and has a realistic business plan and financial sustainability. It is also noted that all stages would require stakeholders working across multiple sectors but also across geographical scales will be involved, spanning global to European and from Regional to National, including joint programming.

FIGURE 4

Figure 4. Key steps in the EOOS implementation cycle showing the first cycle and then following cycles. Each cycle is proposed to last 5 years. Cross-cutting and intrinsic to each step in the EOOS cycle is stakeholder engagement, dialogue and co-design; communication and societal engagement; and governance and coordination.

Communication Activities

EOOS outreach activities have been actively performed by EuroGOOS and the European Marine Board (EMB), including promotion at relevant conferences and events and the EOOS website. The EOOS communication strategy is being developed to encompass the discourse of several EOOS-related strategies (e.g., GOOS 2030 strategy and the UN Ocean Decade for Sustainable Development priorities), and set out a clear communications plan for the coming years.

The United States Interagency Ocean Observation Committee (IOOC) – Fulfilling National Requirements

In the United States, coordination among federal agencies, departments, and offices enables initiatives and activities that would not be possible by a single agency. By establishing relationships between regional, national, and global ocean observations, the United States Interagency Ocean Observing Committee (IOOC) enables powerful new approaches to scientific research and maritime operations. Ultimately, the human and organizational partnerships serve as the connective tissue between the diffuse system elements. Experts have long contemplated how best to design a governance structure that optimizes funding, technology transfer, and data integration. This effort culminated in the early 2000s with the release of the IOOS Implementation Plan. This plan formed 11 regional associations spanning the United States coasts, Great Lakes, and Caribbean (Figure 5) directed by the National Oceanic and Atmospheric Administration (NOAA). The IOOC, created by the executive branch of government and mandated by Congress, expands this network to strengthen partnerships across the federal government and provide strategic guidance. While NOAA provides the essential framework for core elements of the system, the IOOC represents the interests of the federal government and its vast cache of ocean infrastructure and programs – with the explicit objective for determining how to best utilize these resources for bolstering the IOOS enterprise.

FIGURE 5

Figure 5. The 11 regional associations spanning the United States coasts, Great Lakes, and Caribbean as established in the Integrated Ocean Observing System (IOOS^®) implementation plan.

Over the past decade, the IOOC has refined its model beyond immediate agency commitments and missions in order to develop consensus strategies and to lay the groundwork for future ocean priorities. IOOC members and staff connect programmatic initiatives to executive requirements, legislative directives, and community recommendations. The IOOC Co-Chairs and member agencies play a pivotal role in executing the ocean observing initiatives and can leverage greater attention to their particular agency-based goals. The IOOC works closely with other Interagency Working Groups (IWGs) in other thematic areas including Ocean Partnerships, Facilities and Infrastructure, Ocean and Coastal Mapping, Ocean Acidification, and others. The IOOC is chartered by the White House Office of Science and Technology Policy (OSTP) Subcommittee on Ocean Science and Technology (SOST) and legislated by the Integrated Coastal and Ocean Observing System Act of 2009.

The IOOC is broadly focused on federal capacity-building by strengthening: interagency collaboration; community engagement; leadership opportunities; evaluation of effectiveness; and, program and organizational development.

The IOOC responds to the needs of ocean and coastal communities by harnessing the knowledge of multiple-agency representatives. This allows for increased understanding and visibility to address important scientific and technological challenges. One primary tool the IOOC uses for rapidly meeting both community needs and government mandates is through commissioning task teams of federal subject matter experts. IOOC task teams are comprised of three or more agencies and required to develop a budget and timeline for deliverables. This advance planning enables groups to successfully accomplish a set of objectives in a timely manner with staff and resources provided by the IOOC. For example, the previous suite of core ocean biological data variables was identified as outdated. An IOOC-commissioned task team addressed the issue by developing a federal survey, convening an expert workshop, and producing a series of reports published through White House Office of Science and Technology Policy. The result has been a greater focus on federal agency approaches to biological data collection and management in the oceans. It has also increased interaction regionally, nationally, and globally – along with positive reinforcement of the agencies participating on the team.

The capacity of the IOOC and its staff to support interagency projects is unprecedented among committees of this kind and permits more efficient workflows. IOOC Co-Chairs have a unique opportunity to guide these efforts working closely with IOOC members and stakeholders. IOOC members can lead task teams, developing greater visibility for their own agencies and helping to shape plans for improved ocean observations and data integration. This model for improving operational capability by connecting observing system agencies, institutions and other high-level authorities with user needs can be applied beyond the borders of the United States and provides a demonstration of how the approach of Vogel et al. (2017) can work in practice.

Project Azul: A Public–Private Partnership for Ocean Observation

Public investment in research and development in Brazil is not common and long-term commitment for funding research is often difficult. The discovery of vast reservoirs of pre-salt oil in ultra-deep Brazilian waters and the technological challenges associated with its exploitation have presented an opportunity for change. The availability of these resources has raised the interest of private companies in the regional dynamics of the oceans. This interest is also partly fueled by the National Petroleum Agency, which encourages oil companies to invest 1% of their exploration budget in research and development projects. Project Azul was established as a result of this scheme. This initiative is focused on the Santos Basin region, where the majority of pre-salt oil reservoirs in the South-eastern region of Brazil are located (Figure 6). The project started in 2012, fully financed by Shell Brazil, a private oil and gas operator.

FIGURE 6

Figure 6. Santos Basin region (red line) and the pre-salt reservoirs.

The main objective of the project is the development of an ocean observing system for the Santos Basin region (dos Santos et al., 2015). The project is designed as a partnership between the Laboratory of Computational Methods in Engineering of the Federal University of Rio de Janeiro, and Prooceano, an ocean technology company headquartered in Rio de Janeiro. The university was responsible for numerical modeling and data assimilation while Prooceano was responsible for the observations and data analysis (Project Azul Dataset, 2018). During the pilot phase which took place from 2012 to 2016, 60 surface drifters, 36 Lagrangian floats, and 5 underwater gliders were deployed. These datasets were used to produce a consistent dynamic representation, and to increase the oceanographic knowledge of the Santos Basin region. Temperature and salinity profiles collected by gliders and floats (along with remote sensing data) were assimilated by a regional ocean model and evaluated against drift trajectories and others non-assimilated data sets (Fragoso et al., 2016). The assimilation of data resulted in significant improvements in the representation of important mesoscale features of the basin.

One of the important outcomes of Project Azul was the investment in professional and academic training in operational oceanography. During the pilot phase of the project, peer-reviewed articles, book chapters, doctoral theses, master’s dissertations and undergraduate monographs were produced from data and information generated during the project. Other important outputs from the project are still under development but will permit a more accurate understanding of the ocean dynamics in the south-eastern Brazilian region. Professional and academic capability in operational ocean modeling and data assimilation was also developed through establishing cooperative relationships with international universities and other scientific institutions.

Project Azul also made a significant contribution to the wider acquisition of oceanographic data in Brazilian waters such as the hydrographic datasets collected with the use of gliders. The use of gliders and the other Lagrangian sensors permitted a continuous and systematic data acquisition that is of fundamental importance to achieving a consistent understanding of the time-variability of local ocean dynamics (Figure 7). This kind of effort was unprecedented in Brazilian oceanography mainly because of the costs involved.

FIGURE 7

Figure 7. The glider used in project AZUL, the position of a monitored section lower right side and a temperature vertical section generated by the acquired data.

Another significant contribution of the project is that all the data acquired during the project is freely available to the scientific community. This has greatly advanced the state of knowledge on ocean dynamics and ocean sciences more broadly in Brazilian waters. All of the information generated is updated in near real-time on the project website. The website displays each sensor trajectory and the latest vertical profiles and geostrophic sections computed from the acquired data. Regional ocean model forecasts are also made available.

Project Azul II

The second phase of project Azul begun at the end of 2017 and will continue until 2021. The second phase of the project aims to expand the ocean data acquisition with the addition of an autonomous surface vehicle and two instrumented anchor lines. The new sensors will allow an even more comprehensive observation of the ocean dynamics and will include both meteorological and wave data. Other important developments include the implementation of an operational wave forecast model; an increase in the regional ocean model horizontal space resolution and improvements related to the data assimilation system. Furthermore, other important developments include optimizing the use of autonomous vehicles such as making changes to the predefined gliders routes in order to better inform the hydrodynamic ocean models.

Project Azul proves the feasibility of a public–private partnership for systematic long-term oceanic observation with tangible benefits to all parties involved. It is important to emphasize that Project Azul has improved the oceanographic knowledge of Santos Basin region and it is recommended that subsequent initiatives follow the proven partnership model of the project. There are benefits to applying the model, not only to the whole Brazilian ocean margin but also in other developing countries where financial constraints may be limiting progress in ocean sciences. However, the principals of private–public partnerships can support advances in ocean observing independent of financial constraints and provide a case study of what can be achieved through the personalized stakeholder engagement and dialogue promoted earlier in this paper.

Partnerships Between Academic Institutions and the Private Sector – The Coastal Data Information Program (CDIP)

There is a need to establish how academic institutions and United States federally/government funded ocean observing and science programs can better foster private partnerships and to understand how these partnerships can fulfill a specific need for industry, to transition research to operations, and to ensure returns from the research investment. The Coastal Data Information Program (CDIP), of the Scripps Institution of Oceanography, University of California, San Diego, has identified some guidelines which have evolved over time to support this type of partnership.

Since the CDIP’s inception in 1975, more than 275 wave observation stations have been deployed around the coastal United States, Caribbean and Pacific Islands (Thomas, 2018). The United States Army Corps of Engineers (USACE) and the State of California have been the primary funders of these stations. However, significant public–private partnerships have developed between the State of California, several federal agencies and with industry.

The Port of Long Beach

A key example of the CDIP’s public–private partnership is that with the Port of Long Beach, California. The port at Long Beach handles US$ 180 billion in trade annually, with 175 shipping lines connecting to 217 national and international seaports. Presently, 50% of California’s oil comes in through the port which has a containment capacity of 3 days. One of the challenges for the port is how to address the transiting of the larger vessels. Andeavor (previously Tesoro and now Marathon Petroleum) has been leading the “Under Keel Clearance” (UKC) project at Long Beach in order to address this issue. At present, the channel depth is 85 ft. Prior to the start of the UKC project, the deepest draft allowed in the port was 65 ft. This conforms to the port mandate of a 10% under keel clearance. To improve operational efficiency, Andeavor’s goal was to establish the ability to transit the vessels safely into port with a 69-ft draft allowance. The tides and wave action play a significant role in these channel transits. For every degree that an 1,100 ft crude oil tanker pitches, 9.6 ft of draft is lost. For this reason, Andeavor and several other oil companies transfer cargo to smaller vessels in order to enter port facilities – a process known as lightering. Since the lightering process is costly and potentially hazardous for human safety and the environment, Andeavor paved the way for eliminating offshore lightering. However, in order to do so there is a reliance on the timely delivery of data for their operations, knowledge of the metrics for data uptime, and access to high quality and responsive personnel.

In collaboration with Jacobson Pilots, the Marine Exchange of Southern California, the California Oil Spill and Response, the National Oceanic and Atmospheric Association (NOAA), the USACE and CDIP, Andeavor embarked on the UKC project in 2014. They contracted a Dutch firm, Protide, to analyze and display, in real-time, the parameters required to transit these 1,100-ft tankers into the port. The parameters required to calculate the UKC are tides, high-resolution bathymetry, wave observations and wave forecast models. NOAA Center for Operational Oceanographic Products and Services (CO-OPS)¹⁸ provides the tides, NOAA Coast Survey¹⁹ provided the bathymetry, and NOAA National Centers for Environmental Prediction (NCEP)²⁰ provides the real-time wave forecast model, the Nearshore Wave Prediction System (NWPS), and WaveWatch III (WW III). CDIP is providing the wave buoy observations and a short-term wave forecast, which is a 1-h nowcast of the wave model.

Observations and Nowcast Model

Since the pitching of the vessel is caused by waves on the stern of the vessel, it is essential to have access to accurate wave data. Three CDIP buoys in the San Pedro Bight support this project. With central-infrastructure support from the USACE, the San Pedro Buoy has been supported from the California Department of Boating and Waterways since 1981. It is deployed in the separation zone of the westerly shipping lane. The San Pedro South Buoy, with support from Andeavor, has been deployed since 2014, and is located in the separation zone of the southbound shipping lane. The Long Beach Channel Buoy was deployed in 2015 with initial support from the NOAA Integrated Ocean Observing System (IOOS^®) and additionally Andeavor. These three buoys serve as a real-time update and validation for the nowcast and forecast wave models (Figure 8).

FIGURE 8

Figure 8. The observations from the Long Beach Channel buoy are often higher than the WW III forecast model due to the reflection off the Long Beach breakwater and the local offshore bathymetry.

As mentioned, both NOAA and CDIP are providing wave models for the UKC project. The NOAA WW III and NWPS model suite consists of global and regional nested grids. These wind-driven forecast models are invaluable for vessel scheduling of the port transits. CDIP provides a buoy-driven wave model that assimilates the suite of CDIP high-resolution, offshore buoy data from the Southern California Bight. CDIP is providing a one to 3-h swell forecast which is highly accurate since it is based on the buoy observations themselves. This short-term forecast is essential for making any last-minute correction to the transiting plans (Figure 9).

FIGURE 9

Figure 9. A web display at the Marine Exchange of Southern California, San Pedro updates every 30 min (http://cdip.ucsd.edu/custom_pages/marine_exchange/). The three graphs highlight the differences between the buoy observations (blue), WW III (red) and the NWPS (green).

Harnessing User Engagement to Develop Guidelines

Following engagement and discussion with the user a set of guidelines was developed based on finding solutions to the problems and challenges faced by a specific user. In the case of the CDIP these were:

Offer a reliable, robust and high-quality instrument that is designed for a specific need: CDIP deploys the Datawell wave buoys whose wave motion sensor is based on a stabilized platform, accelerometers, and magnetic compass. These buoys have an accuracy of 0.5% of the measured value and a resolution of 1.4° for wave direction. The buoys are known for being robust and reliable. CDIP calculates the data return for each buoy-year. In 2017, the return for San Pedro was 99.95%, San Pedro South 97.49% and the Long Beach Channel Buoy 99.99%.

Provide data analysis which is based upon comprehensive, automated quality control routines that include human decision-making as appropriate: Rigorous quality controls are implemented at several stages in the processing. On-line documentation describes the quality control measures that are incorporated into CDIP’s basic data handling programs, outlining the methodology for data checks and editing. The Quality Control of Real-Time Data (QARTOD) describes the appropriate quality control measures²¹. There are very specific quality control checks that are unique to the Datawell buoy, such as the check factor and the vertical and horizontal ratio of the orbital motion. This is a good indicator of the bio-fouling or damage to the accelerometer (see footnote 20) (Figure 10).

FIGURE 10

Figure 10. The check factor indicated biofouling on the Long Beach Channel Buoy. As a result, divers cleaned the buoy and mooring line, returning the check factor to a normal signal.

Provide reliable dissemination of the complete data set (time series, spectral, parameter), both real-time and historic: CDIP receives the complete xyz surface displacements at 1.28 Hz and spectral files from the buoy every 30 min. After the quality control process, the data are disseminated and archived at CDIP in Network Common Data Form (NetCDF) format, complete with International Standards Organization (ISO) and climate forecast (cf)-compliant metadata. All of the data and statistical products are available on the CDIP website. These data are also disseminated to the National Data Buoy Center for transmission on the Global Telecommunication Service and National Weather Service marine broadcasts. For the Port of Long Beach project, CDIP serves the latest NetCDF files via the Thematic Real-time Environmental Distributed Data Services (THREDDS) to Charta/Protide in Rotterdam where they are ingested into the UKC analysis.

Automate a suite of diagnostic information in order to minimize downtime: CDIP has a suite of diagnostic applications in place. These applications monitor the complete end-to-end system, including system hardware, data transmission via iridium, analysis, data dissemination and offsite positioning.²²

Provide access to a human response to user requests in a timely manner: If these data are used for critical operations such as shipping and port navigation, or health and safety concerns, personnel should be available 24/7. The IT and Management of CDIP are on “pager duty,” which consists of a 1-month rotation of 24/7 monitoring. If there is an issue with the buoy, i.e., it has not updated in over 2 h or if the buoy is offsite, the team will be emailed and a message sent to the person on duty. Depending on the issue, the appropriate personnel will be notified to correct the problem.

These guidelines demonstrate a potential path for transitioning research to operations based on a specific user case.

Other Considerations in Stakeholder Engagement

Using Community Groups and Non-governmental Organizations to Support Broader Stakeholder Engagement

Non-governmental organizations (NGOs) are widely recognized as key players within areas including economic development, human rights, humanitarian action and environment protection. NGOs are best-known for two different, but often interrelated, types of activity – the delivery of services to people in need, and the organization of policy advocacy, and public campaigns. The World Bank (Malena, 1995) defines NGOs as “private organizations that pursue activities to relieve suffering, promote the interests of the poor, protect the environment, provide basic social services or undertake community development” and states that NGOs contribute by introducing innovative approaches and promoting community participation. The World Bank also state that NGOs can facilitate greater awareness of diverse stakeholder views something which should be embraced by the ocean observing community.

Learned societies and professional bodies undertake community development for a specific sector of the community by affording professionals in the field they represent opportunities for career development and recognition of competency but in general do not fit the typical description of an NGO. They are membership organizations who bring together like-minded individuals. Most learned societies are non-profit organizations who strive to create, curate and disseminate knowledge and information- their activities typically include holding regular conferences for the presentation and discussion of new research results and publishing or sponsoring academic journals in their discipline. Examples within the ocean observing field include the Society for Underwater Technology (SUT), the Marine Technology Society (MTS) and The Oceanography Society (TOS) amongst others. Some also act as professional bodies, for example the Institute of Marine Engineering, Science and Technology (IMarEST), setting standards of professional competence and ethics and awarding appropriate qualified individuals certificates of registration. The role of a professional body, therefore is to ensure that employers, government and wider society can have confidence in the knowledge, experience and commitment of professionally registered individuals. This may be crucial when dealing with a diverse range of stakeholders where mutual trust has been identified as being a key component of successful engagement. Professional Bodies in particular strive to achieve technical excellence through members’ expertise and provide impartial and independent to advice to, for example, policy-makers. Professional bodies constantly monitor policy relevant to their profession, seeking to anticipate unintended outcomes not appreciated during the policy formation and constantly ready to offer advice or make representations to help avoid poor decisions (Chartered Institute of Building [CIOB], 2016).

Like civil society groups and trade associations, learned societies and professional bodies have large networks – either through their own membership bases or by linkages with others with similar missions and visions. They can use their networks to advocate for and promote ocean observing but can play a much wider role as a whole – by running events on topical research issues, providing networking and career development support, training students, work with schools and universities to encourage young people to study and take up careers in oceanography, liaising with industry, and engaging with the media and the general public. One such example is the Oceans of Knowledge conference series organized by the Operational Oceanography Special Interest Group (OOSIG) of the IMarEST. The most recent event was held in collaboration with the industry liaison group of the Partnership for Observation of the Global Oceans (POGO) and discussed how ocean observations improve ocean, weather and climate prediction enabling better informed business decisions at sea, on land and in the air. Stakeholders have discussed the role that ocean observations and derived information play in supporting a wide range of industries. These stakeholders have included the shipping industry (route planning and search and rescue), oil spill response and offshore energy through to non-marine sectors such as insurance and re-insurance, retail and logistics and aviation.

There are a number of examples which can be used to demonstrate the potential that NGOs could have in engaging communities to make contributions to ocean observing should the right policies, incentives and education and engagement mechanisms be put in place. NGOs such as those which represent communities of individuals such as recreational fishers, surfers and sailors or who represent trades such as shipping, offshore oil and gas and renewables, aquaculture and fisheries have the potential to bring together groups to significantly enhance ocean observing.

Brewin et al. (2016) investigated the possibility of using recreational surfers (citizen science) as platforms for monitoring environmental indicators in the coastal zone to enhance sampling coverage required for better coastal management. The study involved a recreational surfer using a GPS device and a temperature sensor for a period of 1 year and then comparing the SST data collected by the surfer with data collected from a nearby oceanographic station (L4) and satellite observations in order to assess the accuracy. The conclusion was that high-quality data on SST in the coastal environment could be obtained using surfers. Additionally, the individual was provided details on their surfing performance using the data acquired from the GPS data. This may help motivate data collection by surfers. The study further concluded that 40 million independent measurements on environmental indicators per year around the United Kingdom coastline could be made by the surfing community. Surfer magazine²³ reports that a host of surf-related non-profit organizations have emerged that are taking the initial strides to enact positive social, environmental, and global change. Additionally, surfers are playing an important role in environmental governance through programs such as the World Surfing Reserves (Salamone, 2017). As such the potential for surfing NGOs to contribute to ocean initiatives is significant.

Ships of opportunity provide another means to improve coverage of ocean observations. Bucklin et al. (2001) and Melvin et al. (2016) highlighted the potential for the use of fishing vessels to undertake biological and environmental sampling. Commercial ships have a presence on the high seas second to none and offer society a feasible and cost-effective opportunity to contribute to solving observational deficiencies (SCOR, 2011). The Ship of Opportunity Program (SOOP) is an effort by the international community to address both scientific and operational goals for building a sustained ocean observing system. A number of partnerships such as OceanScope and more recently the World Ocean Council’s Smart Ocean: Smart Industries have proposed formal partnerships with the maritime industries (commercial vessel owners and operators as well as others) to “enable systematic and sustained observation of the structure and dynamics of the ocean water column so that physical, chemical, and biological processes can be studied simultaneously across all the inter-connected ocean basins” (World Ocean Council, 2018)²⁴.

The Barcelona World Ocean Race²⁵ is another example of how working in partnership can support ocean observations. The Fundació Navegació Oceànica Barcelona (Barcelona Foundation for Ocean Sailing – FNOB) committed itself to working with the IOC of UNESCO in collaboration with other scientific research institutions, to protect the environment, in particular the ocean. To achieve this, the event provided a significant and valuable platform to contribute in assisting scientific research in the world’s oceans. Amongst other scientific projects each of the teams in the Barcelona World Race in 2015 deployed an Argo float to facilitate the study of the structure and dynamics of the water masses of the oceans.

Other Stakeholder Groups – A Case Study in Aviation

The case studies so far have largely focused on stakeholders that have a common interest – that they are direct beneficiaries of ocean observing products and services or are the providers of those products and services. Typically these stakeholders are easy to identify. However, there are other beneficiaries that may not recognize that they are stakeholders or who have not been identified as such. In the ocean observing community this might be non-marine sectors such as aviation or retail who benefit from weather forecasts that are improved by knowledge of the ocean state.

Ocean forecasts have been used for many years for operational planning in the maritime industries such as shipping, offshore oil and gas and offshore renewable energy generation. Decisions made during the phases of design, installation and operations are supported by ocean information from models (often incorporating marine observations through assimilation) as well as ocean observations themselves, often alongside weather products.

The importance of the ocean in the weather and climate system is increasingly being recognized and operational systems are now moving toward coupled prediction lead to improvements on seasonal to climate timescales but also to short-range forecasts. These improvements are being driven by the needs of stakeholders outside of the marine industries. Engagement of non-marine industries in understanding the benefits of supporting improved ocean observation and forecasting is particularly challenging but is essentially in ensuring the development of products and services are fit-for-purpose for beneficiaries.

One of the biggest users of meteorological services at the United Kingdom based Met Office is the aviation industry. As one of two World Area Forecast Centres for aviation, the Met Office provides global forecasts of winds and hazards for en-route civil aviation. The Met Office also provides forecasts for civil aviation for the United Kingdom, including forecasting for airports, leisure pilots (such as hot air balloonists), and United Kingdom helicopter operations. All of these services will be improved to an extent by the use of coupled prediction systems.

One specific case is that of the presence of fog at airports can which lead to delays and cancellations and in turn economic penalties and reputational damage to airlines (Roquelaure and Bergot, 2009). Fog has been shown to be particularly sensitive to air–sea interactions (Fallmann et al., 2019) and as such forecasts of fog can only be improved by ocean observations and derived models. San Francisco International Airport (SFO) provides a real case of where an improved knowledge of fog and low cloud would be advantageous. The airport has two closely spaced parallel runways. Both runways can only be used at the same time when the approach is completely clear of any cloud. Maximum flow rates change from 35 aircraft landing per hour when there is cloud, to up to 54 per hour once the clear conditions are reached. Therefore an accurate forecast of cloud clearance can be of considerable benefit for planning for both Air Traffic Control and the airlines coming into SFO. Low cloud over the ocean off the San Francisco coast comes in over the airport on an almost daily basis through the summer months. Forecasting the exact clearance time of this cloud over the runway and approach zone poses a challenge as conditions can vary considerably day-by-day and most weather models struggle to capture some of the finer detail in this area. The Met Office have recently taken part in an experiment running a high resolution (333 m) model over the San Francisco area with some encouraging results in terms of improved forecasts of cloud clearance. It is believed that running a high resolution coupled model over the shallow water of the bay would improve forecasts further due to representing the diurnal cycle of sea surface temperature (SST).

A further consideration is that of helicopter operations. Improvements in short term forecasts for convective events and lightning strikes, are valuable for identifying potential hazards to helicopter operations. On January 19th 1995, a lightning strike caused the ditching of an AS332 Super Puma helicopter into the North Sea, fortunately with no loss of life. Surface observations and radio-soundings indicated that convective cells with anvils circulated northward in a well-organized flow with some showers occurring on that day. These weather conditions are not unusual in the winter in the North Sea, with convective events typically associated with cold-air outbreaks. These conditions have resulted in a number of helicopters being struck by lightning (Broc et al., 2005). As well as the risk to life, lightning strikes to helicopters are expensive due to direct repair costs and subsequent loss of business while the helicopter is taken out of service. Improved forecasts of winter lightning risk and convection in the North Sea will improve the safety and efficiency of operations.

There are also benefits over to the aviation sector over longer timescales. For example, coupled models have been shown to improve medium-range forecasts of storm tracks, which in turn leads to improved forecasts of strong winds. Strong winds at airports can limit airport capacity, which is especially significant at busy airports such as London Heathrow which operates at 98% capacity. Consequently any disruption to capacity at Heathrow has a significant impact in terms of delays and short-notice cancellations of flights. Improved forecasts of strong winds at longer lead times enable improved planning and consequently airports can reduce disruption to passengers (for example, by proactively canceling flights in advance and rebooking passengers onto alternative flights). Finally, improved ocean forecasting has significant impacts on the quality of longer-range forecasts (monthly to seasonal) which enable the industry to better plan for events such as colder or warmer than average winters.

Other Stakeholder Groups – Indigenous Communities and Vulnerable People

A number of examples are available which demonstrate the benefit ocean observations can bring to support indigenous communities and which highlight the importance of conversations with those communities to establish what is required and the best method of delivery. The GEO Blue Planet initiative²⁶ describes the case of the European Union-funded MESA project (Monitoring for Environment and Security in Africa) which provides services to support the fisheries sector in West Africa. This consists of daily forecasts of ocean conditions sent via SMS to small-scale fishermen, as well as maps of potential fishing zones (PFZ) and daily bulletins on fishing vessel activities sent to fisheries managers. The service enables fishermen to increase their efficiency, reduce their costs and avoid venturing out to sea when the conditions are too dangerous.

The importance of the engagement of Indigenous people was recognized more than 10 years ago by the Sustained Arctic Observing Networks (SAON) Initiating Group. Here, Indigenous people raised the need to define their role in Arctic observing, including the role of traditional and local knowledge, the differences and similarities between knowledge systems, and restrictions on personal data and community based monitoring (SAON Initiating Group, 2008). In 2018, a statement made at the Arctic Observing Summit²⁷ further recognized that Arctic Indigenous Peoples have acquired a dynamic knowledge system, allowing for a broader understanding across biological, physical, social, and spiritual domains. It went on to say that to fully benefit decision making at all scales, this knowledge and Indigenous societal priorities need to play a central role in the development of future Arctic Observing Systems.

There are further considerations to be considered with vulnerable communities. When it comes to natural disasters such as hurricanes or floods it is often assumed that everyone is exposed to the same risk and as such it might be assumed that the impact to men or women is equal. However, statistics demonstrate that women and children are 14 times more likely to die than men during a disaster (Araujo et al., 2007). This vulnerability is further enhanced in societies where there are larger social inequalities where the socioeconomic positions of women can make them more vulnerable to disasters. This was evident in the 2004 Asian Tsunami where three times as many women as men died (Oxfam, 2005). The reasons varied but according to the report from Oxfam (2005) among the common factors included: that many men were out fishing or away from home, so had more opportunity to flee the tsunami while in India women traditionally wait on the beaches to unload the fish from the boats so were in an extremely vulnerable location. In general, men could run faster to escape the water and those caught in the sea used their greater strength to survive by clinging on to debris while in Sri Lanka the evidence was that women were simply not able to swim. In Aceh, the Indonesian province that bore the brunt of the disaster, many men had simply moved away to find work. Women, in contrast, were at home, and efforts to save their children slowed their flight.

However, the report from Oxfam (2005) suggests that rather than perceiving women as ‘vulnerable victims’ their specific perspectives and capacities should be used to develop the best way to responding to disasters and preventing such disasters in the future and this can be equally applied to their engagement with the development of ocean observing systems. By engaging women and representatives of vulnerable communities in the planning stages of ocean observing systems their knowledge and social practices could be used to better focus the observations and advise on the best dissemination of the information acquired to ensure the information required is delivered on time and in a format that can be easily understood and disseminated. As Aguilar (2008) state “Women’s high level of risk awareness, social networking practices, extensive knowledge of their communities, task in managing natural environmental resources and caring abilities makes of them important players of effective risk assessment, early warning, disaster response and recovery actions.” The Hyogo Framework for Action (HFA) in 2005 and its successor the Sendai Framework makes suggestions for how women should be included in preparing for natural disasters and these. The UNESCO program on disaster preparedness and prevention emphasizes the needs and roles of women in building a culture of disaster resilience which can also be applied to ocean observing. The World Meteorological Organization (WMO) has begun to take steps to address gender issues across weather and climate services that could and should be replicated by the ocean observing community²⁸.

Gender Diversity in the Ocean Observing Community

While section “Other Stakeholder Groups – Indigenous Communities and Vulnerable People” primarily relates to communities in the developing world and gender inequality when it comes to natural disasters there is an additional problem that exists across the globe. This is that there is a lack of diversity in those recruited and retained in roles across science, technology, engineering and mathematics (STEM) and this includes ocean observing. Firstly, diversity is crucial to develop the intellectual capital. For example, when any staff, researchers of faculty members leave an organization the knowledge base is eroded but this is prevalent in women leaving a company to have children, often returning to a lesser role. Secondly, a diverse mix of people lends itself to increased creativity and innovation- different ideas and different thinking. Thirdly, analogous to a company maintaining a competitive edge by having a better understanding of all potential customers and markets and their requirements this can be applied to users of ocean observation. For example, women have a unique approach to the application of science and its value in improving the quality of life. Finally, there are wider benefits where mixing diversity can lead to new priorities, perspectives and questions and ultimate effect the future direction of ocean observing.

Discussion and Recommendations

A Focus on Services

There is a growing industry that is converging on the concept of climate services (Brooks, 2013; Brasseur and Gallardo, 2016; Le Cozannet et al., 2017; Hewitt and Golding, 2018). Climate services focus on process, partners, users and stakeholders by pushing the transformation of science to benefit society. The concept is well-accepted to the point of the creation of a global framework and platforms (Global Framework for Climate Services²⁹, Climate Services Partnership³⁰); symposia series (International Conference on Climate Services³¹), climate service research institutions (German Climate Service Centre³²; Ouranos³³), numerous climate service funding instruments (ERA4CS³⁴) and a journal (Climate Services, Elsevier³⁵).

It is reasonable to argue that by successfully defining a “climate services” concept, the science and scientific infrastructure supporting the conversion of climate science to policy enjoys a high-level of prominence. There is in fact many publications and literature describing the development of climate services to support public and private sector users and stakeholders (Guido et al., 2016; van den Hurk et al., 2016; Vogel et al., 2017). The coastal and ocean observing community has all the component elements to create a concept for a “services-focused science community” in the same way that climate services are now promoted. The intentional definition of an ocean observation services industry may be a powerful mechanism to enable the engagement between ocean observations and society. Networks and boundary agents such as GEO³⁶ and the GEO Blue Planet initiative³⁷ and GOOS can play a leadership role in this regard.

A New Science-Society Norm for Coastal and Ocean Observations

The science and technology basis for coastal and marine observations is dynamic and fast moving. Equally so, the society and how it responds to external stimuli is constantly changing. Within this context, there are basic principles to guide the creation of societal value from ocean observation products to ensure they remain relevant and useful to coastal and ocean communities and industries. Ocean and coastal information producers and practitioners should replace the view of a linear unidirectional value chain with that of a complex web of interactions and the role of transdisciplinary networks (Max-Neef, 2005; Binder et al., 2015; Polk, 2015). Sustainable development needs and the ocean observing fitness for purpose should be placed at the core. New technologies should be embraced and promoted within a new paradigm on open, user-friendly data access can enable society to engage as users and stakeholders within knowledge networks (Overpeck et al., 2011; Smith and Doldirina, 2016). The EOOS framework which is already stimulating partnerships in scientific and technological ocean observation foresight (e.g., Benedetti-Cecchi et al., 2018) demonstrates this value and shows that cross-disciplinary stakeholder dialogue and partnerships are crucial to take stock of current and emerging developments to ensure the current and future observing system is fit-for-purpose. The perception, trust and acceptance of science by society is not static and new technologies and governance mechanisms will require an equally novel approach to continue mutual engagement.

Converging on Common Objectives

As previously discussed the science community and society needs to converge around challenges or common objectives (Curley, 2016) and this is no different for the ocean observing community. Currently, there are particularly strong and globally acceptable expressions of common objectives for humanity. The SDGs, the Paris Agreement and many others provide the basis for the co-design and co-development of solutions for sustainability. It is important to recognize that science and the evaluation of scientific evidence cannot be divorced from the political, cultural and social debate that inevitably and justifiably surrounds these major issues (Cutcher-Gershenfeld et al., 2017; Horton and Brown, 2018). It is vital to recognize the importance of effective communication and invest in engagement and communication activities to develop capacity of all those within the network (Hossain et al., 2016).

Communicating Effectively

Effective communication is the cornerstone of building trust, partnerships and cross-disciplinary engagement needed to address the SDGs. Deepening stakeholder engagement and building advocacy and visibility is crucial for stakeholder and wider societal and policy buy-in, engagement and investment in ocean sustainability and responsible blue economy.

The relevance of ocean observation to society can be promoted through ocean literacy activities. An ocean-literate person understands the importance of the ocean to humankind; can communicate about the ocean in a meaningful way; and is able to make informed and responsible decisions regarding the ocean and its resources (Tuddenham et al., 2013³⁸). Ocean literacy is also an imperative in establishing science–policy and public–private interfaces. Ocean literacy enables sharing understanding of basic concepts and facts about the ocean, but also developing common values and building personal emotional links to the ocean, which are paramount for responsible research and innovation (RRI) as well as policy and decision-making.

Citizen science is a growing area where society can be engaged in scientific data collection and, in many cases data analysis, while at the same time becoming more ocean literate (Garcia-Soto et al., 2017; Monestiez et al., 2017; Haklay et al., 2018; Visbeck, 2018).

Effective communication in ocean observing involves a broad variety of stakeholders at various scales, across disciplines and spanning many governance levels. Targeting communications to such a variety of stakeholders requires a thorough stakeholder analysis and personalized approaches. Furthermore, the recent inclusion of the importance of ocean observing in several high-level policy statements and agendas (Agenda 2030, the UN Ocean Decade for Sustainable Development, G7, etc.), gives an unprecedented gravitas to the ocean observing community in explaining the relevance of their work to policy and society.

Communication should be embedded in the core objectives of the many ocean observing initiatives and recognized as a priority and funded accordingly.

Making Better Use of the Case Study Approach

The value of the case study approach is well-recognized in the fields of business, law and policy where qualitative research methods are often used to examine real-life situations and set future directions. Case studies could provide a useful way to engage a much wider stakeholder community in ocean observing systems by presenting the data in very publicly accessible ways to enable the reader to apply the experience in his or her own real-life situation. The case study approach has the advantage of conveying a message to a wide range of stakeholders; either making scientists aware of potential new applications of their research; enabling users to understand the benefits of observations and information or to demonstrate the benefits to policy-makers in a real-life context.

To ensure that case studies are of value and not perceived as biased and non-scientific there is a need to develop a systematic approach to gathering information. Whilst the case studies presented here mostly provide anecdotal evidence as the importance of stakeholder engagement and partnerships in ocean observing only the case from Long Beach enumerates the monetary benefit which reinforces the views of Rayner et al. (2018) that there remains no comprehensive global attempt to value the social and economic benefits derived from ocean observations. It is therefore, recommended that a standard methodology is developed akin to the OECD (2016) protocol for measuring the ocean economy and adopted by the ocean observation community to build a consistent global view.

The key elements of the methodology are that it should create a consistent format for each user story that specifically describes the problem faced by the user; the commercial value of the problem faced by the user and/or cost expectations of the user; and, the delivery requirements. An important stakeholder will be the intermediary service providers (those organizations that take raw data and turn it into useable information for the user). The services may utilize new data services or be built on core services provided to support the ocean economy. In addition to a consistent methodology a common portal for best practice should be used – for example the central repository of best practice materials hosted through existing ocean best practice initiatives.

Better Understanding User Requirements

In addition to development of a methodology for building case studies and user stories, there is a need for sustained dialogue with users (intermediary users and end-users, as well as users from different domains and disciplines). What is essential that users have confidence in the observations, and consider that information provided is a valuable asset to their operations or policy- and decision-making. This could be achieved by addressing Service Level Agreements to provide the user with confidence in the reliable delivery of information, ensuring appropriate delivery methods are available to the user(s) demonstrating the veracity of the analytical approach using historical data prior to implementation in an operational environment and the creation of appropriate development, test and operational environments.

Asking users what they do (and how they do it), what decisions they make and what information they use to make decisions is in many cases more important than asking them about data and information needs. Producers and intermediary and end-users should work together to co-design and co-produce information and knowledge in an iterative and collaborative manner. It is important that all stakeholders involved in this process take ownership and accountability for the outcomes (Pagano and Volpin, 2001; Corringham et al., 2008).

Capacity Development and the Exchange of Knowledge

All the stakeholder case studies touch on the vital role that human capacity development plays in a successful integrated ocean observing system. This was particularly evident during Project Azul where one of the most important outcomes was the investment made in professional and academic training in operational oceanography. The lack of trained personnel is well recognized as a major obstacle to the development of ocean observing systems (Malone et al., 2014). A survey conducted by the Intergovernmental Oceanographic Commission of the United Nations Educational, Scientific and Cultural Organization (IOC of UNESCO, 2013) highlighted concerns related to capacity development in ocean observing³⁹ Some of these concerns included: an overdependence on project-oriented, short-term international support; training programs being mostly one-off with unsure long-term benefits; inadequate infrastructure inhibiting training programs, where they are most needed; lack of ship-based training programs (data collection/data handling); lack of a critical mass of ocean experts (both scientific and technical) and few mechanisms or incentives to retain built capacity and available experts; and, competition from non-marine sectors for new entrants and talent. The survey further demonstrated that problems are exacerbated in Small Island Development States (SIDS) and in particular in the Pacific Islands. The vastness of the ocean space and the remoteness of outer islands creates unique problems including a lack of vessels, which in turns hinders access to outer islands and create barriers to communication.

The UNDP (2009) highlights that there are four core principles that need to be addressed with regards to capacity development. These are: institutional arrangements; leadership; knowledge; and, accountability. Knowledge has traditionally been fostered at the individual level, mostly through education. But it can also be created and shared within an organization, such as through on-the-job training or even outside a formal organizational setting through general life experience, and supported through an enabling environment of effective educational systems and policies (UNDP, 2009).

Learned societies and professional bodies are uniquely placed to contribute to capacity development in ocean observing. Whilst often working in partnership, they are largely independent from government funding agencies, universities or science-based industries with their one of their primary objectives to provide opportunities for the exchange of ideas and practices to build capacity. Most are organized organically with regionally and specialist sub-networks with ever growing international memberships which allow capacity development based on international standards and ethical practices but cognizant of local needs. The independence of these bodies enables capacity development in an integrated way from working with early education providers to advising on policy at governmental and intergovernmental level.

Related to capacity it is essential that a gender perspective be integrated into all policies, plans and decision-making processes, including those related to risk assessment, early warning, information management, education and training (UNISDR, 2005, p. 4), capacity development, in general public awareness products, and in advocacy for ocean observing systems and their benefits. More gender-balanced representation within international and regional networks of experts should also be sought and the capacities of professional and other NGOs should be used as an enabler to do so.

Capacity development efforts in operational oceanography, including training programs, need to take a long-term perspective- and take advantage of existing regional network. Incentives for continuous updating of professional knowledge and sharing of expertise should be considered. Stakeholder engagement needs to include all those who stand to benefit from the enhanced capacity to ensure ownership and commitment to the process, and personally invested in its success.

Finally, hand in hand with capacity development is knowledge transfer. Knowledge transfer is essential to achieve effective dialogue and information exchange across the science–policy–society interface. As described in the EOOS case study, the use of knowledge brokers for ocean observation will be considered as a mechanism for stakeholder engagement and knowledge exchange and could be something that could be implemented wider across ocean observing systems.

Governance Design

The case studies presented all work under different governance models. It is important to consider governance in the optimization of the existing capacities and their usefulness, for design of new fit-for-purpose systems, as well as for engaging cross-disciplinary stakeholders in co-creation or co-financing. Governance structures will be varied depending on the needs and the starting point in each of the systems, system of systems, or framework design, however, a set of elements should be considered in all of them. Those elements consist (but aren’t restricted to): stakeholder engagement, requirement gathering and feedback loop mechanisms; operational and organizational support to the processes; advisory and decision-making capacities; user interface. Effective communication, as previously described, should be a cross-cutting liaising element within the governance and vis-à-vis external stakeholders.

Conclusion

Stakeholder engagement and communication are critical to achieve sustained funding and truly reap the benefits of integration and joint prioritization in the field of ocean observation. Stakeholders play a fundamental role in building, evolving and sustaining integrated ocean observing systems. Stakeholders and users should be identified for each stage of the ocean observing framework design. Once stakeholders have been identified a process of selecting the most appropriate mechanisms for dialogue, buy-in, and co-design across stakeholder groups and communities should be developed. The evaluation and measurement of success is an essential component of any ocean observing system framework and may include a range of performance metrics, including cost-benefit analyses, stress tests and system experiments, pilot actions, as well as stakeholder consultation and engagement mechanisms. Ultimately, by engaging all stakeholders and creating successful partnerships there will be improved economic, societal and environment benefits realized from sustained integrated ocean observing systems.

Author Contributions

This is a community white paper so is built out of a number of separate contributions. BM was white paper coordinator and lead author. BM, LC, LA, JH, NR, and JT were lead authors of community contributions. All others are contributing authors to each separate contribution.

Conflict of Interest Statement

FS and JP were employed by Prooceano. JF was employed by Shell Brazil. GJ is an independent consultant. ZR was employed by Vattenfall Wind Power Ltd.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

Special thanks are given to Dr. Helen Wells of the Met Office, UK for her significant contribution on the case study related to aviation.

Footnotes

1. ^https://cdn.ioos.noaa.gov/media/2018/02/US-IOOS-Enterprise-Strategic-Plan_v101_secure.pdf
2. ^https://www.noaa.gov/big-data-project
3. ^www.eoos-ocean.eu
4. ^http://eurogoos.eu/publications/
5. ^http://www.euroceanconferences.eu/
6. ^http://eurogoos.eu/
7. ^www.marineboard.eu
8. ^https://sustainabledevelopment.un.org/
9. ^https://en.unesco.org/ocean-decade
10. ^http://ec.europa.eu/environment/marine/eu-coast-and-marine-policy/marine-strategy-framework-directive/index_en.htm
11. ^https://ec.europa.eu/fisheries/cfp_en
12. ^https://ec.europa.eu/maritimeaffairs/policy/maritime_spatial_planning_en
13. ^http://www.eoos-ocean.eu/promotional-materials/
14. ^http://www.copernicus.eu/
15. ^http://marine.copernicus.eu/
16. ^www.emodnet.eu
17. ^https://www.marinaproject.eu/
18. ^https://tidesandcurrents.noaa.gov/
19. ^https://www.nauticalcharts.noaa.gov/
20. ^http://www.ncep.noaa.gov/
21. ^https://ioos.noaa.gov/project/qartod/
22. ^http://cdip.ucsd.edu/diag
23. ^https://www.surfer.com/features/10-nonprofits-worth-your-love/
24. ^https://www.oceancouncil.org/global-issues-platforms/program-focus/smart-oceans-smart-industries/
25. ^https://www.barcelonaworldrace.org
26. ^https://geoblueplanet.org/blue-planet-activities/stakeholder-engagement-wg/
27. ^http://www.arcticobservingsummit.org/sites/arcticobservingsummit.org/files/AOS_Statement_Aug24_clean.pdf
28. ^https://ane4bf-datap1.s3-eu-west-1.amazonaws.com/wmocms/s3fs-public/GAP_Draft.pdf?VDGolo0GoiMq9aT5FAHzO2uHJdKJTqmZ
29. ^http://www.wmo.int/gfcs/
30. ^http://www.climate-services.org/
31. ^http://www.climate-services.org/iccs/
32. ^http://www.gerics.de/
33. ^https://www.ouranos.ca/en/
34. ^http://www.jpi-climate.eu/ERA4CS/
35. ^https://www.journals.elsevier.com/climate-services
36. ^http://www.earthobservations.org/index2.php
37. ^https://geoblueplanet.org/
38. ^http://oceanliteracy.wp2.coexploration.org/
39. ^http://unesdoc.unesco.org/images/0022/002268/226864e.pdf

References

Aguilar, L. (2008). Training Manual on Gender and Climate Change. San José, Costa Rica: IUCN, UNDP, GGCA. Available at: https://portals.iucn.org/union/sites/union/files/doc/training_manual_on_gender_and_climate_change (accessed March 13, 2019).

Google Scholar

Aitsi-Selmi, A., Blanchard, K., and Murray, V. (2016). Ensuring science is useful, usable and used in global disaster risk reduction and sustainable development: a view through the Sendai framework lens. Palgrave Commun. 2:16016. doi: 10.1057/palcomms.2016.16

CrossRef Full Text | Google Scholar

Anderson, K., Ryan, B., Sonntag, W., Kavvada, A., and Friedl, L. (2017). Earth observation in service of the 2030 agenda for sustainable development. Geo Spat. Inf. Sci. 20, 77–96. doi: 10.1080/10095020.2017.1333230

CrossRef Full Text | Google Scholar

Araujo, A., Quesada-Aguilar, A., Aguilar, L., and Pearl, R. (2007). Gender Equality and Adaptation. Fact sheet, Women’s Environment and Development Organization (WEDO) and the International Union for Conservation of Nature (IUCN). Available at: http://drr.upeace.org/english/documents/References/Topic%208-A%20Gender%20Approach%20to%20DRR%20and%20Peace%20Making//Araujo.%20Gender%20equality (accessed 13th March 2019).

Bell, M. J., Guymer, T. H., Turton, J. D., MacKenzie, B. A., Rogers, R., and Hall, S. P. (2013). Setting the course for UK operational oceanography. J. Operat. Oceanogr. 6, 1–15. doi: 10.1080/1755876X.2013.11020146

CrossRef Full Text | Google Scholar

Benedetti-Cecchi, L., Crowe, T., Boehme, L., Boero, F., Christensen, A., Gremare, A., et al. (2018). “Strengthening Europe’s Capability in Biological Ocean Observations,” in Future Science Brief 3 of the European Marine Board, eds P. A Muniz, P. Kellett, K. Larkin, and J. J. Heymans (Ostend: European Ocean Observing System).

Google Scholar

Binder, C. R., Absenger-Helmli, I., and Schilling, T. (2015). The reality of transdisciplinarity: a framework-based self-reflection from science and practice leaders. Sustain. Sci. 10, 545–562. doi: 10.1007/s11625-015-0328-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Brasseur, G. P., and Gallardo, L. (2016). Climate services: lessons learned and future prospects. Earth Future 4, 79–81. doi: 10.1002/2015EF000338

CrossRef Full Text | Google Scholar

Brewin, R. J. W., de Mora, L., Jackson, T., Brewin, T. G., and Shutler, J. (2016). Correction: on the potential of surfers to monitor environmental indicators in the coastal zone. PLoS One 11:e0162591. doi: 10.1371/journal.pone.0162591

PubMed Abstract | CrossRef Full Text | Google Scholar

Broc, A., Delannoy, A., Montreuil, E., Lalande, P., and Laroche, P. (2005). Lightning strike to helicopters during winter thunderstorms over north sea. Aerosp. Sci. Technol. 9, 686–691. doi: 10.1016/J.AST.2005.07.001

CrossRef Full Text | Google Scholar

Brooks, M. S. (2013). Accelerating innovation in climate services: the 3 E’s for climate service providers. Bull. Am. Meteorol. Soc. 94, 807–819. doi: 10.1175/BAMS-D-12-00087.1

CrossRef Full Text | Google Scholar

Brovelli, M. A., Minghini, M., and Zamboni, G. (2018). “New Generation Platforms for Exploration of Crowdsourced Geo-Data,” in Earth Observation Open Science and Innovation, eds P.-P. Mathieu and C. Aubrecht (Cham: Springer International Publishing), 219–243. doi: 10.1007/978-3-319-65633-5-9

CrossRef Full Text | Google Scholar

Bucklin, A., Goudey, C., Wiebe, P. H., and Williams, W. G. (2001). FleetLink: collection and telemetry of ocean and weather data from commercial fishing vessels. Sea Technol. 42, 27–31.

Google Scholar

Cash, D. W., Adger, W. N., Berkes, F., Garden, P., Lebel, L., Olsson, P., et al. (2006). Scale and cross-scale dynamics: governance and information in a multilevel world. Ecol. Soc. 11:12. doi: 10.5751/ES-01759-110208

CrossRef Full Text | Google Scholar

Ceccaroni, L., Velickovski, F., Blaas, M., Wernand, M. R., Blauw, A., and Subirats, L. (2018). “Artificial Intelligence and Earth Observation to Explore Water Quality in the Wadden Sea,” in Earth Observation Open Science and Innovation, eds P.-P. Mathieu and C. Aubrecht (Cham: Springer International Publishing), 311–320. doi: 10.1007/978-3-319-65633-5-18

CrossRef Full Text | Google Scholar

Chartered Institute of Building [CIOB] (2016). Understanding the Value of Professions and Professional Bodies. Available at: https://www.ciob.org/sites/default/files/CIOB%20research%20-%20Professions%20Report.pdf [accessed October 22, 2018].

Christian, R. R., DiGiacomo, P. M., Malone, T., and Talaue-McManus, L. (2006). Opportunities and challenges of establishing coastal observing systems. Estuaries Coast 29, 871–875. doi: 10.1007/BF02786538

CrossRef Full Text | Google Scholar

Cone, J., Rowe, S., Borberg, J., Stancioff, E., Doore, B., and Grant, K. (2013). Reframing engagement methods for climate change adaptation. Coast. Manag. 41, 345–360. doi: 10.1080/08920753.2013.803926

CrossRef Full Text | Google Scholar

Corringham, T., Westerling, A., and Morehouse, B. J. (2008). Exploring use of climate information in wildland fire management: a decision calendar study. J. For. 106, 71–77. doi: 10.1093/jof/106.2.71

CrossRef Full Text | Google Scholar

Court, J., and Young, J. (2006). Bridging research and policy in international development: an analytical and practical framework. Dev. Pract. 16, 85–90. doi: 10.1080/09614520500450925

CrossRef Full Text | Google Scholar

Crewe, E., and Young, J. (2002). Bridging Research and Policy: Context, Evidence and Links. Working Paper 173- Results of ODI Research Presented in Preliminary Form for Discussion and Critical Comment. Available at: https://www.files.ethz.ch/isn/100449/wp173.pdf (accessed March 13, 2019).

Curley, M. (2016). Twelve principles for open innovation 2.0. Nature 533, 314–316. doi: 10.1038/533314a

PubMed Abstract | CrossRef Full Text | Google Scholar

Cutcher-Gershenfeld, J., Baker, K. S., Berente, N., Flint, C., Gershenfeld, G., Grant, B., et al. (2017). Five ways consortia can catalyse open science. Nature 543, 615–617. doi: 10.1038/543615a

PubMed Abstract | CrossRef Full Text | Google Scholar

Denis, G., Claverie, A., Pasco, X., Darnis, J.-P. P., de Maupeou, B., Lafaye, M., et al. (2017). Towards disruptions in earth observation? new earth observation systems and markets evolution: possible scenarios and impacts. Acta Astronaut. 137, 415–433. doi: 10.1016/j.actaastro.2017.04.034

CrossRef Full Text | Google Scholar

Denis, G., de Boissezon, H., Hosford, S., Pasco, X., Montfort, B., and Ranera, F. (2016). The evolution of earth observation satellites in europe and its impact on the performance of emergency response services. Acta Astronaut. 127, 619–633. doi: 10.1016/j.actaastro.2016.06.012

CrossRef Full Text | Google Scholar

Dilling, L., and Lemos, M. C. (2011). Creating usable science: opportunities and constraints for climate knowledge use and their implications for science policy. Glob. Environ. Chang. 21, 680–689. doi: 10.1016/j.gloenvcha.2010.11.006

CrossRef Full Text | Google Scholar

Djenontin, I. N. S., and Meadow, A. M. (2018). The art of co-production of knowledge in environmental sciences and management: lessons from international practice. Environ. Manage. 61, 885–903. doi: 10.1007/s00267-018-1028-3

PubMed Abstract | CrossRef Full Text | Google Scholar

dos Santos, F. A., de Oliveira, A. L. S. C., Soares, F. L. M., Carvalho, G. V., São Tiago, P. M., Santos, N. G., et al. (2015). “Projeto Azul: operational oceanography in an active oil and gas area southeastern Brazil,” in Coastal Ocean Observing Systems, 1st Edn, eds Y. Liu, H. Kerkering, and R. H. Weisberg (London: Elsevier), 99–116. doi: 10.1016/B978-0-12-802022-7.00020-1

CrossRef Full Text | Google Scholar

EurOCEAN (2010). “Grand challenges for marine research in the next decade,” in Proceedings of the Conference Report and Ostend Declaration, eds N. McDonough and J.-B. Calewaert (Oostende: Flanders Marine Institute (VLIZ)). Available at: http://www.marineboard.eu/eurocean

EurOCEAN (2014). “Connecting science, policy and people,” in Proceedings of the Conference Report and Rome Declaration, eds K. Larkin, N. Wouters, A.-T. Caetano and N. McDonough (Rome: National Research Council). Available at: http://www.marineboard.eu/eurocean

Google Scholar

EuroGOOS (2016a). “European operational oceanography: delivering services for blue growth and ecosystem-based management”, in EuroGOOS Policy Brief, eds J. She, D. Eparkhina, and G. Nolan (Brussels: EuroGOOS). Available at: http://eurogoos.eu/download/publications/EuroGOOS-Policy-Brief-2016.pdf

EuroGOOS (2016b). “Operational Oceanography for Sustainable Blue Growth,” in Proceedings of the Seventh EuroGOOS International Conference, 28–30 October 2014, eds E. Buch, Y. Antoniou, D. Eparkhina, and G. Nolan (Lisbon: EuroGOOS). Available at: http://eurogoos.eu/download/publications/EuroGOOS-2014-Conference-Proceedings.pdf

Google Scholar

EuroGOOS (2018). “Operational Oceanography serving Sustainable Marine Development,” in Proceedings of the Eight EuroGOOS International Conference. 3–5 October 2017, eds E. Buch, V. Fernández, D. Eparkhina, P. Gorringe, and G. Nolan (Bergen: EuroGOOS). Available at: http://eurogoos.eu/download/publications/EuroGOOS-2017-Conference-Proceedings.pdf

EuroGOOS-European Marine Board (2016). Towards an end-to-end, Integrated and Sustained Ocean Observing System for Europe. Brussels: EuroGOOS-European Marine Board. Available at: http://www.eoos-ocean.eu/materials/

EuroGOOS-European Marine Board (2018). EOOS Implementation Plan 2018-2022. Available at: http://www.eoos-ocean.eu/materials/

European Marine Board (2013). Navigating the Future IV. Position Paper 20 of the European Marine Board. Ostend: European Marine Board.

European Marine Board-EuroGOOS (2018). EOOS Strategy 2018–2022. Available at: http://www.eoos-ocean.eu/materials/

European Union (2018). Study on Costs, Benefits and Nature of an Extended European Ocean Observing System. Brussels: European Union.

Fallmann, J., Lewis, H., Castillo Sanchez, J., and Lock, A. (2019). Impact of high-resolution ocean-atmosphere coupling on fog formation over the North Sea. Q. J. the R. Meteorol. Soc. doi: 10.1002/qj.3488

CrossRef Full Text | Google Scholar

Fragoso, M. R., Carvalho, G. V., Soares, F. L. M., Faller, D. G., Assad, L. P. F., Toste, R., et al. (2016). A 4D-variational ocean data assimilation application for Santos Basin, Brazil. Ocean Dyn. 66, 419–434. doi: 10.1007/s10236-016-0931-5

CrossRef Full Text | Google Scholar

Garcia-Soto, C., van der Meeren, G. I., Busch, J. A., Delany, J., Domegan, C., Dubsky, K., et al. (2017). Earth Observations in Support of the 2030 Agenda for Sustainable Development. Japan Aerospace Exploration Agency (JAXA) on Behalf of GEO. Available at: https://www.earthobservations.org/documents/publications/201703_geo_eo_for_2030_agenda.pdf doi: 10.1007/s10236-016-0931-5

CrossRef Full Text

Giuliani, G., Chatenoux, B., De Bono, A., Rodila, D., Richard, J.-P., Allenbach, K., et al. (2017). Building an earth observations data cube: lessons learned from the swiss data cube (SDC) on generating analysis ready data (ARD). Big Earth Data 1, 100–117. doi: 10.1080/20964471.2017.1398903

CrossRef Full Text | Google Scholar

Guido, Z., Rountree, V., Greene, C., Gerlak, A., and Trotman, A. (2016). Connecting climate information producers and users: boundary organization, knowledge networks, and information brokers at caribbean climate outlook forums. Weather Clim. Soc. 8, 285–298. doi: 10.1175/WCAS-D-15-0076.1

CrossRef Full Text | Google Scholar

Guo, H. D., Zhang, L., and Zhu, L. W. (2015). Earth observation big data for climate change research. Adv. Clim. Chang. Res. 6, 108–117. doi: 10.1016/j.accre.2015.09.007

CrossRef Full Text | Google Scholar

Haklay, M., Mazumdar, S., and Wardlaw, J. (2018). “Citizen Science for Observing and Understanding the Earth,” in Earth Observation Open Science and Innovation, eds P.-P. Mathieu and C. Aubrecht (Cham: Springer International Publishing), 69–88. doi: 10.1007/978-3-319-65633-5-4

PubMed Abstract | CrossRef Full Text | Google Scholar

Hewitt, C., and Golding, N. (2018). Development and pull-through of climate science to services in China. Adv. Atmos. Sci. 35, 905–908. doi: 10.1007/s00376-018-7255-y

CrossRef Full Text | Google Scholar

Horton, P., and Brown, G. W. (2018). Integrating evidence, politics and society: a methodology for the science–policy interface. Palgrave Commun. 4:42. doi: 10.1057/s41599-018-0099-3

CrossRef Full Text | Google Scholar

Hossain, F., Serrat-Capdevila, A., Granger, S., Thomas, A., Saah, D., Ganz, D., et al. (2016). A global capacity building vision for societal applications of earth observing systems and data: key questions and recommendations. Bull. Am. Meteorol. Soc. 97, 1295–1299. doi: 10.1175/BAMS-D-15-00198.1

CrossRef Full Text | Google Scholar

Hov,Ø, Terblanche, D., Carmichael, G., Jones, S., Ruti, P. M., and Tarasova, O. (2017). Five priorities for weather and climate research. Nature 552, 168–170. doi: 10.1038/d41586-017-08463-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Howarth, C., and Monasterolo, I. (2017). Opportunities for knowledge co-production across the energy-food-water nexus: making interdisciplinary approaches work for better climate decision making. Environ. Sci. Policy 75, 103–110. doi: 10.1016/j.envsci.2017.05.019

CrossRef Full Text | Google Scholar

IOC of UNESCO (2013). INF-1313. Baseline Study for an Assessment of National Capacities and Needs in Marine Research, Observation and Data/Information Management. Available at: http://iocunesco.org/index.php?option=com_oe&task=viewDocumentRecord&docID=11244D

Jasanoff, S. (2003). Technologies of humility: citizen participation in governing science. Minerva 41, 223–244. doi: 10.1023/A:1025557512320

CrossRef Full Text | Google Scholar

Karl, H. A., Susskind, L. E., and Wallace, K. H. (2007). A dialogue, not a diatribe: effective integration of science and policy through joint fact finding. Environment 49, 20–34. doi: 10.3200/ENVT.49.1.20-34

CrossRef Full Text | Google Scholar

Kirchhoff, C. J., Carmen Lemos, M., and Dessai, S. (2013). Actionable knowledge for environmental decision making: broadening the usability of climate science. Annu. Rev. Environ. Resour. 38, 393–414. doi: 10.1146/annurev-environ-022112-112828

CrossRef Full Text | Google Scholar

Kirchhoff, C. J., Lemos, M. C., and Kalafatis, S. (2015). Creating synergy with boundary chains: can they improve usability of climate information? Clim. Risk Manag. 9, 77–85. doi: 10.1016/j.crm.2015.05.002

CrossRef Full Text | Google Scholar

Kollmuss, A., and Agyeman, J. (2002). Mind the gap: why do people behave environmentally and what are the barriers to pro-environmental behaviour. Environ. Educ. Res. 8, 239–260. doi: 10.1080/1350462022014540

CrossRef Full Text | Google Scholar

Lary, D. J., Zewdie, G. K., Liu, X., Wu, D., Levetin, E., Allee, R. J., et al. (2018). “Machine Learning Applications for Earth Observation,” in Earth Observation Open Science and Innovation, eds P. P. Mathieu and C. Aubrecht (Cham: Springer), doi: 10.1007/978-3-319-65633-5-8

CrossRef Full Text | Google Scholar

Le Cozannet, G., Nicholls, R. J., Id, J. H., Id, W. V. S., Id, K. L. M., Wal, R. S. W., et al. (2017). Sea Level change and coastal climate services: the way forward. J. Mar. Sci. Eng. 5, 1–28. doi: 10.3390/jmse5040049

CrossRef Full Text | Google Scholar

Lemos, M. C., Kirchhoff, C. J., and Ramprasad, V. (2012). Narrowing the climate information usability gap. Nat. Clim. Chang. 2, 789–794. doi: 10.1038/nclimate1614

CrossRef Full Text | Google Scholar

Lemos, M. C., and Morehouse, B. J. (2005). The co-production of science and policy in integrated climate assessments. Glob. Environ. Chang. 15, 57–68. doi: 10.1016/j.gloenvcha.2004.09.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Levy, G., Vignudelli, S., and Gower, J. (2018). Enabling earth observations in support of global, coastal, ocean, and climate change research and monitoring. Int. J. Remote Sens. 39, 4287–4292. doi: 10.1080/01431161.2018.1464101

PubMed Abstract | CrossRef Full Text | Google Scholar

Malena, C. (1995). Working with NGOs : a Practical Guide to Operational Collaboration Between the World Bank and Nongovernmental Organizations (English). Washington,DC: World Bank.

Google Scholar

Malone, T. C., DiGiacomo, P. M., Gonçalves, E., Knap, A. H., Talaue-McManus, L., de Mora, S., et al. (2014). Enhancing the global ocean observing system to meet evidence based needs for the ecosystem-based management of coastal ecosystem services. Nat. Resour. Forum 38, 168–181. doi: 10.1111/1477-8947.12045

CrossRef Full Text | Google Scholar

Max-Neef, M. A. (2005). Foundations of transdisciplinarity. Ecol. Econ. 53, 5–16. doi: 10.1016/j.ecolecon.2005.01.014

PubMed Abstract | CrossRef Full Text | Google Scholar

Mazumdar, S., Wrigley, S., and Ciravegna, F. (2017). Citizen science and crowdsourcing for earth observations: an analysis of stakeholder opinions on the present and future. Remote Sens. 9:87. doi: 10.3390/rs9010087

CrossRef Full Text | Google Scholar

McNie, E. C. (2007). Reconciling the supply of scientific information with user demands: an analysis of the problem and review of the literature. Environ. Sci. Policy 10, 17–38. doi: 10.1016/j.envsci.2006.10.004

CrossRef Full Text | Google Scholar

Melvin, G. D., Gerlotto, F., Lang, C., and Trillo, P. (2016). Fishing vessels as scientific platforms: an introduction. Fisheries Res. 178, 1–3. doi: 10.1016/j.fishres.2016.02.022

CrossRef Full Text | Google Scholar

Moll, P., and Zander, U. (2006). Managing the Interface– From Knowledge to Action in Global Change and Sustainability Science. Munich: Oekom Verlag.

Google Scholar

Monestiez, P., Seys, J., Wêsławski, J. M., and Zielinski, O. (2017). Advancing Citizen Science for Coastal and Ocean Research. Available at: http://www.marineboard.eu/file/633/download?token=sMDNN4Xb [accessed October 17, 2018].

Moss, R. H., Meehl, G. A., Lemos, M. C., Smith, J. B., Arnold, J. R., Arnott, J. C., et al. (2013). Hell and high water: practice-relevant adaptation science. Science 342, 696–698. doi: 10.1126/science.1239569

PubMed Abstract | CrossRef Full Text | Google Scholar

Nativi, S., Mazzetti, P., and Craglia, M. (2017). A view-based model of data-cube to support big earth data systems interoperability. Big Earth Data 1, 1–25. doi: 10.1080/20964471.2017.1404232

CrossRef Full Text | Google Scholar

Nisbet, M. C., and Scheufele, D. A. (2009). What’s next for science communication? promising directions and lingering distractions. Am. J. Bot. 96, 1767–1778. doi: 10.3732/ajb.0900041

PubMed Abstract | CrossRef Full Text | Google Scholar

Nowotny, H. (2003). Democratising expertise and socially robust knowledge. Sci. Public Policy 30, 151–156. doi: 10.3152/147154303781780461

CrossRef Full Text | Google Scholar

OECD (2016). The Ocean Economy in 2030. Paris: OECD Publishing, doi: 10.1787/9789264251724-en

CrossRef Full Text

O’Sullivan, C., Wise, N., and Mathieu, P.-P. (2018). “The Changing Landscape of Geospatial Information Markets,” in Earth Observation Open Science and Innovation, eds P.-P. Mathieu and C. Aubrecht (Cham: Springer International Publishing), 3–23. doi: 10.1007/978-3-319-65633-5_1

CrossRef Full Text | Google Scholar

Overpeck, J. T., Meehl, G. A., Bony, S., and Easterling, D. R. (2011). Climate data challenges in the 21st century. Science 331:700. doi: 10.1007/978-0-387-76495-5

CrossRef Full Text | Google Scholar

Oxfam (2005). The Tsunami’s Impact on Women. Available at: https://policy-practice.oxfam.org.uk/publications/the-tsunamis-impact-on-women-115038.

Google Scholar

Pagano, M., and Volpin, P. (2001). The Political Economy of Corporate Governance. CEPR Discussion Paper no. 2682.

Google Scholar

Polk, M. (2015). Transdisciplinary co-production: designing and testing a transdisciplinary research framework for societal problem solving. Futures 65, 110–122. doi: 10.1016/j.futures.2014.11.001

CrossRef Full Text | Google Scholar

Project Azul Dataset (2018). Available at: www.projetoazul.eco.br (accessed November 7, 2018).

Rayner, R., Gouldman, C., and Willis, Z. (2018). The Ocean Enterprise - understanding and quantifying business activity in support of observing, measuring and forecasting the oceans. J. Operat. Oceanogr.

Reed, M. S., Stringer, L. C., Fazey, I., Evely, A. C., and Kruijsen, J. H. J. (2014). Five principles for the practice of knowledge exchange in environmental management. J. Environ. Manage. 146, 337–345. doi: 10.1016/j.jenvman.2014.07.021

PubMed Abstract | CrossRef Full Text | Google Scholar

Reid, W. V., Chen, D., Goldfarb, L., Hackmann, H., Lee, Y. T., Mokhele, K., et al. (2010). Earth system science for global sustainability: grand challenges. Science 330, 916–917. doi: 10.1126/science.1196263

PubMed Abstract | CrossRef Full Text | Google Scholar

Roquelaure, S., and Bergot, T. (2009). Contributions from a local ensemble prediction system (LEPS) for improving fog and low cloud forecasts at airports. Weather Forecast. 24, 39–52. doi: 10.1175/2008WAF2222124.1

CrossRef Full Text | Google Scholar

Roux, D. J., Stirzaker, R. J., Breen, C. M., Lefroy, E. C., and Cresswell, H. P. (2010). Framework for participative reflection on the accomplishment of transdisciplinary research programs. Environ. Sci. Policy 13, 733–741. doi: 10.1016/j.envsci.2010.08.002

CrossRef Full Text | Google Scholar

Salamone, K. E. (2017). Surfing Towards Marine Conservation? An Examination of World Surfing Reserves as Marine Conservation Practice. Ph.D. thesis, San Diego State University, San Diego.

Google Scholar

SAON Initiating Group (2008). Observing the Arctic. Report of the Sustained Arctic Observing Network (SAON) Initiating Group. Available at: https://www.arcticobserving.org/images/pdf/Background/saon_report_final_web.pdf

SCOR (2011). OceanScope- A Proposed Partnership between the Maritime Industries and the Ocean Observing Community to Monitor the Global Ocean Water Column. Report of SCOR/IAPSO Working Group 133. Available at: http://www.scor-int.org/Publications/OceanScope_Final_report.pdf (accessed March 13, 2019).

Google Scholar

Smith, L. J., and Doldirina, C. (2016). Remote sensing: a case for moving space data towards the public good. Space Policy 37, 162–170. doi: 10.1016/j.spacepol.2016.11.006

CrossRef Full Text | Google Scholar

Thomas, J. (2018). Physical and Meteorological data from the Scripps Institution of Oceanography (SIO) Coastal Data Information Program (CDIP) stations from 2018-01-01 to 2018-07-31 (NCEI Accession 0175198). Version 7.7. Boulder, CO: NOAA National Centers for Environmental Information.

Tuddenham, et al. (2013). Ocean Literacy – The Essential Principles and Fundamental Concepts of Ocean Sciences for Learners of All Ages. Available at: http://www.coexploration.org/oceanliteracy/documents/OceanLitChart.pdf (accessed March 13, 2019).

Google Scholar

UNDP (2009). Capacity Development – A UNDP Primer. Available at: http://www.undp.org/content/dam/aplaws/publication/en/publications/capacity-development/capacity-development-a-undp-primer/CDG_PrimerReport_final_web.pdf (accessed March 13, 2019).

Google Scholar

UNISDR (2005). Hyogo Framework for Action 2005–2015: Building the Resilience of Nations and Communities to Disasters. Available at: https://www.unisdr.org/files/1037_hyogoframeworkforactionenglish.pdf (accessed March 13, 2019).

Google Scholar

van Aalst, M., and Agrawala, S. (2005). Bridging the Gap Between Climate Change and Development, in Bridge Over Troubled Waters: Linking Climate Change and Development. Washington: OECD Publishing, 133–146. doi: 10.1787/9789264012769-7-en

CrossRef Full Text

van den Hurk, B. J. J. M., Bouwer, L. M., Buontempo, C., Döscher, R., Ercin, E., Hananel, C., et al. (2016). Improving predictions and management of hydrological extremes through climate services. Clim. Serv. 1, 6–11. doi: 10.1016/j.cliser.2016.01.001

CrossRef Full Text

Vasko, C. A., Adriaensen, M., Bretel, A., Duvaux-Bechon, I., and Giannopapa, C. G. (2017). Space assets, technology and services in support of energy policy. Acta Astronaut. 138, 295–300. doi: 10.1016/j.actaastro.2017.06.005

CrossRef Full Text | Google Scholar

Venkatesan, R., and Sampath, V. (2017). Linking ocean observation, and fisheries - relevance to deep ocean living resources. Fish Ocean Opj. 5:555660. doi: 10.19080/OFOAJ.2017.05.555660

CrossRef Full Text

Visbeck, M. (2018). Ocean science research is key for a sustainable future. Nat. Commun. 9, 1–4. doi: 10.1038/s41467-018-03158-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Vogel, C., Moser, S. C., Kasperson, R. E., and Dabelko, G. D. (2007). Linking vulnerability, adaptation, and resilience science to practice: pathways, players, and partnerships. Glob. Environ. Chang. 17, 349–364. doi: 10.1016/j.gloenvcha.2007.05.002

CrossRef Full Text

Vogel, J., Letson, D., and Herrick, C. (2017). A framework for climate services evaluation and its application to the caribbean agrometeorological initiative. Clim. Serv. 6, 65–76. doi: 10.1016/j.cliser.2017.07.003

CrossRef Full Text | Google Scholar

Walter, A. I., Helgenberger, S., Wiek, A., and Scholz, R. W. (2007). Measuring societal effects of transdisciplinary research projects: design and application of an evaluation method. Eval. Prog. Plann. 30, 325–338. doi: 10.1016/j.evalprogplan.2007.08.002

PubMed Abstract | CrossRef Full Text | Google Scholar