Stakeholders’ feedback integration in spatial planning using Importance Performance Analysis: the case of the Provincial Strategic Plan of the Autonomous Province of Bolzano-South Tyrol (IT)

Abstract

Importance Performance Analysis (IPA) is a research method employed to comprehend and analyze stakeholder feedback on particular issues. A substantial body of research has demonstrated the efficacy of this methodology as a quantitative approach. To date, there has been a paucity of research using IPA as a methodology for integrating stakeholder feedback into the development of strategic spatial plans. The integration of stakeholder feedback is crucial for the development of inclusive, well-informed, and responsive decision-making processes. The objective of this paper is to contribute to the existing body of knowledge on the utilization of the IPA method in the context of spatial planning. This research presented in this paper employs the Provincial Strategic Plan of the Autonomous Province of Bolzano-Bozen as a case study. In particular, the IPA method was employed to analyze the level of importance and performance of the objectives set out in the Provincial Strategic Plan. Importance refers to the importance of the specific objectives for the future development of the area, while performance refers to how well the specific objectives were formulated. The IPA method has been demonstrated to be an effective assessment instrument. The findings demonstrate that the IPA approach enabled the effective gathering of stakeholders’ feedback in a systematic manner, identifying pivotal issues and potential shortcomings, and providing policy makers with structured recommendations on how to reformulate strategies and plans.

1 Introduction

The effective integration of stakeholder feedback into decision-making processes is a fundamental aspect of successful strategic planning (Innes and Booher, 2004; Healey, 1997a,b). This integration encompasses inputs, preferences, perceptions, opinions, and perspectives from individuals and organizations that are actively involved in shaping the future of a specific place, such as a region, province, municipality, or area. Stakeholders possess considerable political and/or economic influence over resources due to historical dependencies, institutional mandates, and economic interests. Consequently, they can provide feedback in the planning process on proposed objectives, actions, and measures, as well as on key issues and potential gaps. It is therefore crucial to integrate stakeholder feedback, as it provides decision-makers with invaluable insights that are vital for enhancing policy effectiveness and promoting inclusive, diverse, and well-informed decision-making (Reed et al., 2018). Moreover, it is crucial for the efficacy of a democratic system. The integration of stakeholders in spatial planning serves to reinforce the legitimacy and credibility of strategic planning by demonstrating transparency, responsiveness, and accountability to those affected by the outcomes of the plan (Bryson, 2004; Healey, 1998; Cheng and Li, 2000).

The concept of stakeholder engagement/involvement has been extensively researched and promoted at the policy level (Glasson and Marshall, 2007; Morphet, 2005; Faludi, 2000; Albrechts, 2004, 2006; Albrechts and Balducci, 2013; Camagni, 2017; Kwatra et al., 2021). A paucity of attention has been devoted to the methods and practices for the systematic analysis and effective integration of stakeholder feedback collected during the spatial planning engagement process. The integration of stakeholder feedback can be a complex and challenging process due to the volume of information that must be systematized and the diverse, and at times conflicting, needs and interests of the stakeholders involved. The most frequently employed assessment techniques include spatial multi-criteria analysis, Pareto analysis, discriminant analysis, group discussion, and the Delphi study. Among other techniques, importance-performance analysis (IPA) is a popular research method used to test and determine the importance and performance (or satisfaction) of a given item. It is also used to understand customer satisfaction and to formulate improvement strategies for products and services (Martilla and James, 1977). IPA was initially developed for use in marketing research, tourism management, and service quality assessment (Lai and Hitchcock, 2015; Cheng and Li, 2000; Ennew et al., 1993; Hollenhorst et al., 1992; Boley et al., 2017). It was subsequently applied to the analysis of stakeholders’ feedback on specific issues in other fields, including waste management (Bui et al., 2020; Tseng, 2011). IPA has also been used in relation to the value of ecosystem services, particularly to assess users’ perspectives on cultural ecosystem services in urban contexts, for multifunctional landscape optimization in urban parks and in the management of both natural and built environments, as well as supporting evaluations of users’ experiences with ecosystem services (Gai et al., 2022).

To the best of our knowledge, no research has been conducted, nor has an IPA application been performed, in the field of strategic spatial planning. The objective of this paper is to address the identified gap in the literature and implement the IPA method in spatial planning by analysing and integrating stakeholder feedbacks collected during a stakeholder consultation process. This is demonstrated by the presentation of an empirical case study based on an applied research activity that Eurac conducted in 2023. The empirical case is the Provincial Strategic Plan (PSP) developed by the Autonomous Province of Bolzano-Bozen (PAB). In particular, the IPA method is employed to examine the feedback provided by stakeholders concerning the level of importance and performance of the objectives set forth in the Provincial Strategic Plan (PSP). The term “importance” is used to denote the degree to which the specific objectives of the PSP are deemed crucial for the future development of the area in question. The term “performance,” on the other hand, pertains to the extent to which the specific objectives of the PSP are formulated in a satisfactory manner.

This paper employs the IPA to illustrate that the method is an efficacious assessment technique for policymakers in the field of planning who seek to incorporate stakeholder feedback into their decision-making process. The objective of this paper is to demonstrate that the IPA enables policymakers to consider the diverse perspectives, needs, and preferences of stakeholders; and that it is a systematic and data-driven tool that facilitates the analysis and integration of stakeholder feedback into the strategic development process. Furthermore, the objective is to demonstrate that policymakers will be able to identify those requiring improvement and immediate action.

The paper is structured as follows: after a brief introduction, Section 2 presents the relevant concepts within a conceptual framework; Section 3 describes the role of science in policy-making; Section 4 is the methodological chapter; and Section 5 describes the main quantitative and qualitative results obtained. Section 6 discusses the results. The paper concludes with a discussion and conclusion section.

2 Conceptual background

This section introduces some key concepts, including strategic spatial planning, stakeholder involvement, and IPA (Figure 1). The conceptual background facilitates comprehension of the process used to integrate stakeholder feedback in spatial planning via IPA. Furthermore, the section examines the conceptual and epistemological aspects of IPA.

Figure 1

2.1 Strategic spatial planning

The overarching objective of strategic spatial planning is to devise and implement strategies and plans to manage spatial change (Healey, 1997a; Camagni, 2017) and address the increasingly complex planning challenges associated with climate change, biodiversity loss, and balanced territorial development. Strategic spatial planning is a social process (Healey et al., 2006) through which a range of actors from diverse institutional settings come together to develop strategies and projects for the management of spatial changes (Kunzmann, 2013). It necessitates collaboration across disciplines, sectors, and levels of government, involving a multitude of actors with diverse and occasionally conflicting interests and objectives in the decision-making process (Albrechts, 2004, 2006, 2015; Tanese et al., 2006; Hersperger et al., 2019).

2.1.1 Strategic spatial planning phases

The strategic planning process is divided into two principal phases, as delineated by Hersperger et al. (2019): plan-making and plan-implementation. The strategic spatial plan-making phase entails the formulation of visions, the delineation of goals, objectives, and priorities, and the establishment of decision frameworks. This phase is undertaken with the objective of representing, managing, and influencing urban transformations. Plan implementation is the process by which plans are transformed into urban developments, facilitating the realization of intended changes while preventing those that are contrary to the desired outcome. The spatial planning process is not linear; rather, it is open to recursive and iterative phases where feedback frequently results in the review and revision of earlier decisions. This cyclical nature allows for continuous improvement and adaptation to changing circumstances (UNESCAP, 2024).

2.1.2 Stakeholder involvement process in spatial planning

The involvement of stakeholders in spatial planning is of critical importance, as it ensures the inclusion of diverse perspectives, reflects the needs and aspirations of the community, and increases the likelihood of support and effective implementation. The degree of stakeholder involvement varies considerably depending on the number of factors, including the identity of the stakeholders, the roles they assume, and the way they participate (Pongponrat et al., 2006). Much of the literature suggests that participatory models can be ordered according to a single dimension, that is, from less to more or from lower to higher (Bobbio, 2019). The International Association for Public Participation (IAP2, 2018) has developed a participation spectrum which describes five levels of participation: inform, consult, involve, collaborate and empower. According to Bobbio (2019) these one-dimensional models highlight an important point: the involvement can be more or less intense, that is, more or less influential. However, the intensity of involvement and the weight of influence are not the only sensitive dimensions. No single measure of effectiveness can be proposed. It is not possible to order all participatory arrangements in a single ranking. If the design of public participation processes can embed different aims and values, the challenge is not to distinguish between weak and strong involvement or between good and bad ones, but rather to understand the different design choices that can be made, the problems that can be tackled, the values to be pursued and/or neglected, the trade-offs that can be hypothesized among them and the results that can be attained.

2.2 The conceptual and epistemological foundations of importance-performance analysis

The method was initially employed by Martilla and James in 1977 and has since become a pervasive research technique for elucidating and analyzing stakeholder perceptions on specific issues, or to evaluate and prioritize various attributes of a product, service, or experience based on their perceived importance and performance on those attributes (Wyród-Wróbel and Biesok, 2017). By collecting data using mainly surveys, IPA works by plotting these attributes on a two-dimensional grid, thereby enabling organizations to identify areas requiring improvement, strengths to be maintained, and aspects of low priority.

IPA has been regarded as an easy, simple, and effective assessment technique (Phan et al., 2022; Deng, 2007; O’Neill and Palmer, 2004). An assessment method is considered simple, easy, and effective if it: (a) is well-suited to addressing the problem; (b) produces reliable and valid results and yields precise and accurate data that truly reflect the phenomena under study; (c) provides findings that can be translated into action or used to solve real-world problems and have practical implications; (d) facilitates the generation of novel insights, addressing deficiencies in existing knowledge or offering a novel perspective on a previously understood phenomenon.

In consideration of the statements and the conceptual and epistemological nature of IPA, the following connotations are ascribed to it. IPA is a structured tool that can be readily employed to assess the significance and performance of a given topic or issue. This is achieved by investigating how well an attribute aligns with the expectations of a target group (Martilla and James, 1977). IPA provides findings that can be translated into action or used to solve real-world problems. The results help to prioritize areas that need improvement and allocate resources where they are most needed (Oh, 2001). The visual representation of IPA results serves to highlight critical areas for action, thereby enabling planners to make informed decisions and implement effective strategies (Azzopardi and Nash, 2013). By employing a structured data collection method, IPA provides reliable and valid insights into stakeholder perceptions, thereby enhancing the credibility and reliability of the findings (Matzler et al., 2004). It elicits robust, stakeholder-driven insights, thereby enhancing the credibility and reliability of findings (Sever, 2015). Finally, IPA facilitate the generation of novel insights. The findings derived from IPA are described as actionable and capable of informing policy, solving real-world problems, and having practical implications. On the other side IPA is not able to reduce the complexity of high-level conflicts that may exist among different stakeholders or between stakeholders and local administration.

2.3 Role of scientists in policymaking

The role of science in policymaking, defined as the science-policy interface and evidence-based policy making, serves as a conduit for transforming scientific knowledge into actionable policies, bridging the gap between science and policy domains (Strydom et al., 2010; Fazey et al., 2018). The science-policy interface relies on a science-based methodology rooted in empirical evidence, expert knowledge, and stakeholder engagement to identify policy needs, synthetize knowledge, and co-develop solutions that are socially, economically, and politically feasible (Castella et al., 2014). In this practice, science is responsible in knowledge synthesis and communication to the policy side, while policymakers are responsible for designing evidence-based policies and inclusive decision-making.

It is key to consider the potential risks associated with the science-policy interface, including the credibility of the scientific community, the legitimacy of practitioners, and the salience of policymakers (Opdam et al., 2013; Castella et al., 2014). Regarding the field of science, a significant potential risk is the possibility of scientific findings becoming enmeshed in the political and policy implementation processes, thereby compromising the integrity of the scientific method. There is a risk of undue influence from vested interests, which may result in the cherry-picking of evidence or the manipulation of research findings to align with specific agendas. Furthermore, the intrinsic complexity of numerous policy matters can result in oversimplification or misrepresentation of scientific evidence, which may ultimately lead to misguided policies or ineffective interventions (Watson Robert, 2005; Wynanda et al., 2014). To mitigate these risks, it is essential that there be transparency, integrity, and ongoing dialog between scientists, policymakers, and stakeholders. Such dialog ensures the responsible and effective use of science to address pressing societal challenges, thereby fostering trust and cooperation among all involved parties. In navigating the complexities of the science-policy interface, this collaborative approach enables the development of evidence-based policies that are robust, inclusive, and responsive to the needs of society.

3 Methodology

3.1 Research context

The Autonomous Province of Bolzano-Bozen, commonly known as South Tyrol, is Italy’s northernmost province with a population of 536,933 and an area of 7398.38 km². Located entirely in the core region of the Alps, the territory of South Tyrol is predominantly mountainous, to the extent that it is estimated that little more than 5% of the province’s administrative area is suitable for permanent settlements (ASTAT, 2013) (Figure 2). This makes land a particularly scarce and precious resource, and the sustainable management of spatial development an issue of paramount importance on the political agenda in South Tyrol. The Provincial Strategic Plan (PSP) is the responsibility of the Provincial Spatial Planning and Cartography Office of the PAB and serves as the primary strategic spatial planning tool for the provincial government, providing long-term guidance in managing current and emerging territorial changes. Its overall objective is to guide these changes toward a sustainable future, thereby enhancing the quality of life for present and future generations.

Figure 2

The PSP builds on and implements the new “Territory and Landscape Law” (Legge Provinciale, 2018), which proposes medium-and long-term objectives that are legally binding and serve as a guiding compass for all subsequent planning instruments. The PSP is organized around six cross-cutting themes which are listed in Table 1. The six cross-cutting themes were identified based on the following criteria: (a) relevance to the future spatial development of the territory; (b) alignment with existing strategies at provincial, national and European level; and (c) capacity to implement the objectives of the Provincial Law “Territory and Landscape.” The PSP contains 17 general and 50 specific objectives and over 136 actions to achieve them, each with a defined implementation timetable. These objectives are predominantly cross-sectoral, while the actions also extend to sectors beyond spatial planning and fall within the competence of other departments and public institutions. The development of the PSP can be described as an iterative process, unfolding over time through recursive phases, involving external academics, sectoral experts, various stakeholders and collaboration between different departments of the Province to ensure the transversal and intersectoral nature of the issues addressed. The process of drafting the plan started in 2021 and ended in 2023, when the PSP was presented and submitted to the Provincial Council for evaluation. The procedure for approving the plan is expected to start in mid-2024.

3.2 Positionality of Eurac researchers

The research presented in this paper is based on applied research (Greenwood and Levin, 2007). The Institute for Regional Development of Eurac Research was commissioned by the Autonomous Province of Bolzano-Bozen to provide scientific support for the elaboration of the Provincial Strategic Plan. This paper presents a description of the activities undertaken by Eurac researchers in relation to the stakeholder consultation process and the subsequent analysis of the stakeholder feedback. The activities were conducted over the course of 1 year, from 2022 to 2023.

A working group comprising officials from the Provincial Spatial Planning and Cartography Office and a group of Eurac researchers with expertise in a range of fields and disciplines was established (PSP Working Group). The PSP Working Group proceeded to develop the cross-cutting themes, identify the general and specific objectives of the PSP (described in the next chapter), and select the stakeholders to be consulted. The decision-making process entailed iterative interactions between the various provincial departments involved in the development of the PSP and the PSP working group.

The Eurac research team was constituted of a core group of experts in regional development, participatory processes, and spatial planning, complemented by an extended team comprising specialists in fields such as mobility, housing, energy, climate change, biodiversity, and others. This was done to ensure interdisciplinary collaboration. Decision-making was a collective process, with responsibility distributed among the Eurac team. Each decision was discussed and then taken collegially. The researchers’ own backgrounds, beliefs, and values influenced the selection of the cross-cutting teams and the identification of the objectives of the PSP. Researcher could add suggest relevant topics. However, they were unable to assume a position on matters where the Provincial Office made an independent decision, such as the identification of stakeholders to be involved in the decision-making process and the type of stakeholder involvement to be carried out.

Regarding ethical considerations, Eurac researchers were transparent and explicit with stakeholders regarding the role of the researchers, the rationale for stakeholder involvement, and the intended use of data throughout the process. The researchers obtained informed consent and ensured confidentiality. A conflict of interest was identified with one researcher, who was affiliated with an environmental association that participated in the survey. To avoid any bias, she opted not to contribute to the completion of the questionnaire. Additionally, Eurac Research was among the selected stakeholders. However, as we elaborated the strategic objective of the PSP, we abstained from participating in the online questionnaire. It is of the utmost importance to gain an understanding of the positionality of the researcher, the potential for bias, and the degree of objectivity to maintain transparency, credibility, and ethical standards in research (Fenge et al., 2019; Alvesson and Sköldberg, 2000).

In terms of the power dynamic between researchers, policymakers, and stakeholders, which is an inherent aspect of the science-policy interface, the Eurac researchers held a position of authority throughout the process. This included selecting the methods, designing the survey, collecting the data, and interpreting the results. Moreover, Eurac researchers proffer recommendations that are instrumental in guiding policymakers’ decision-making processes. Notwithstanding the circumstances, the researchers succeeded in establishing a relationship of trust with the provincial authorities, thereby creating a reciprocal dynamic in which the contributions of both parties were duly recognized and acknowledged. As the stakeholders were not involved in a genuinely open and participatory process, but were only asked to provide inputs, the researchers were unable to establish a fully collaborative relationship with them.

3.3 Research design

This section aims to delineate the methodology employed to address the research question (Gideon, 2012) (Figure 3). A case study approach is used. The Provincial Strategic Plan (PSP) has been selected as it is a case study that the authors know very well, and it represents a modern approach to urban planning. The case study can help validate existing methods for stakeholder integration and inspire new application. By examining the Bolzano strategic plan, researchers can contribute to the academic discourse and give insights that can be applied to other urban contexts facing similar challenges. Stakeholder feedback was collected via an online questionnaire and subsequently subjected to quantitative analysis using the IPA approach. IPA was performed to assessed stakeholder responses on the importance and performance levels of specific objectives of the PSP. A further IPA analysis was carried out to check whether there was a difference of opinion between stakeholders on the specific objectives of the PSP. Finally, a qualitative analysis of stakeholders’ feedback, collected through the open-ended questions of the questionnaire, was conducted using thematic analysis. While the quantitative analysis assesses the extent to which the specific objectives align with stakeholders’ expectations, the qualitative analysis facilitates the categorization of stakeholder feedback. Ultimately, the findings were synthesized into a report containing recommendations for the Province on how to integrate stakeholder feedback into the final draft of the PSP.

Figure 3

3.3.1 Data collection: Web-based questionnaire

The data set comprising stakeholder feedback to be analyzed using IPA was collected using an anonymous web-based questionnaire (Fowler, 2014). The survey was conducted using SurveyMonkey software. The web-based questionnaire was developed for the purpose of collecting stakeholder opinions regarding the level of performance (formulation) and importance (relevance for future development) of the specific objectives of the PSP. The online questionnaire was selected for its capacity to rapidly and economically gather a substantial quantity of both quantitative and qualitative data, which can then be subjected to statistical analysis. The questionnaire was designed and structured in a manner that facilitated the application of the IPA. The questionnaire comprised 50 specific objectives, which were rated on a Likert scale. Stakeholders were requested to select a minimum of two cross-cutting issues (as detailed in section 3) for which they wished to evaluate the formulation of specific objectives. For each selected cross-cutting issue, respondents were invited to provide comments and suggest potential reformulations for objectives that were rated as unsatisfactory. Alternatively, they could propose new objectives if they felt that existing ones were lacking. This was done through open-ended questions. Furthermore, the questionnaire requested information regarding the name and type of organization represented, the sector and specific area of activity, as well as data on the role and demographic details of the organization’s representative (gender, age group, educational background). To guarantee the inclusion of only fully completed questionnaires in the data analysis, the questionnaire was designed with mandatory responses for each question. To avoid any potential falsification of basic data, the questionnaire included the option of selecting “Do not know” for the evaluation of specific objectives and “Prefer not to answer” for demographic data. Prior to the initial data collection phase, the questionnaire was pre-tested to ensure that respondents could complete it without any complications. It was open from 3 April to 21 April 2023.

3.3.2 Data analysis: IPA analysis of stakeholder opinions

The IPA was used to quantitatively assess stakeholder responses on the level of importance and performance of specific objectives of the PSP. To measure importance and performance, respondents rated each objective of the PSP using a Likert scale ranging from 1 (not at all) to 4 (very much). The mean of the Likert scale, which is 2.5 and was used to assess importance and performance, was considered the threshold. The responses for both importance and performance are then plotted on a four-quadrant grid. Quadrant A, “Focus Here,” is the most critical area of the Importance/Performance matrix, as it indicates specific objectives that are rated as highly important but poorly performed in terms of formulation. Quadrant B “Keep up the good work” indicates objectives that are rated as very important and high performance. Quadrant C “Low Priority” indicates objectives that are both unimportant and unsatisfactory. Quadrant D, “Irrelevant Benefits,” indicates objectives that are not considered very important but of high performance (see Figure 4).

Figure 4

3.3.3 Data analysis: IPA analysis of segmented stakeholders

Questions that are phrased in a positive way or that seem to suggest a desired response may lead respondents to answer more positively. To avoid potential bias related to the possibility of receiving high ratings for all specific objectives, the sample was segmented into “Primary Stakeholders” in specific issues (PS) and “Other Stakeholders” (OS), and a separate IPA was conducted to check whether there were significant differences between these two groups of stakeholders. PS are those stakeholders who, by virtue of their domain and the nature of their activities, are closely related to the issue under analysis. OS are those stakeholders that are not directly related to the issue. For example, organizations working in the environmental field are considered PS for the climate change issue; organizations working in the commercial field are considered OS for the same issue; organizations working in the research and planning field are considered PS for all issues. With this segmentation (Mimbs et al., 2020), a separate IPA was conducted to determine whether there were significant differences between primary stakeholders (PS) in specific issues and other stakeholders (OS). The purpose of this analysis is to see if these two different groups had different opinions on the specific objective of the PSP. In other words, to assess the differences between stakeholders characterized by a “strong link” or “weak link” to the issue, and to evaluate specific intervention proposals in case of significant differences.

3.3.4 Data analysis: thematic analysis

Stakeholder responses to the questionnaire were qualitatively analyzed using thematic analysis (Morin et al., 2021). This approach was instrumental in examining the explicit information provided by stakeholders and distilling stakeholder opinions. Using an inductive approach, stakeholder comments were grouped into major categories. This type of analysis proved useful in identifying which specific objectives of the PSP needed to be reformulated and which specific objectives needed to be added.

3.4 Sampling

A total of 75 stakeholders were identified and contacted, of whom 13 were umbrella organizations, defined as entities that coordinate groups of smaller organizations with common themes and interests. To guarantee that no organization expressing interest in participating in the survey would be excluded, interested stakeholders were invited to participate to the survey by completing an online “membership form,” which was promoted on the provincial website and local press channels. Table 2 illustrates the profile of the stakeholders who responded to the online questionnaire, while Table 3 depicts the categorization of stakeholders (i.e., those engaged with specific themes and other stakeholders) based on their relations with cross-cutting teams. In total, 155 online questionnaires were collected, of which 71 were fully completed. Of the 71 complete responses obtained, only 63 passed the accuracy and quality check and were deemed suitable for inclusion in the analysis.

4 Results

The responses to the 63 valid questionnaires were quantitatively analyzed using IPA and segmented analysis. The responses to the questionnaire were also qualitative analyzed using thematic analysis, aiming to provide the province with concrete suggestions for reformulating the PSP’s specific objectives. The following paragraph presents the quantitative IPA results, as well as the qualitative results collected in the form of suggestions, and finally the results of the segmented analysis.

4.1 Quantitative IPA results

For the criterion “importance,” the specific objectives of Theme A “Climate change” were rated as the most important (3.54), followed by those of Theme D “Mobility, accessibility, digitalisation,” Theme F “Health, social inclusion, participation,” Theme C “Innovation, attractiveness, competitiveness,” Theme E “Quality of place, culture, quality of life” and Theme B “Landscape, ecosystems, biodiversity” (3.33). Regarding the criterion “performance,” theme D “Mobility Accessibility Digitalisation” (3.25), theme F “Health Social Inclusivity Participation” and theme E “Settlement Quality Culture Quality of Life” show the highest performance rates in formulating objectives, followed by Themes B “Landscape Ecosystems Biodiversity” (2.95), Theme A “Climate change” and Theme C “Innovation Attractiveness Competitiveness” (2.90). For each cross-cutting theme, Table 4 provides the number of respondents (N), the mean scores for importance and performance, and the standard deviation.

In terms of the average of each specific objective of all the cross-cutting themes, they were evaluated favorably (Table 5). According to the importance criterion, ratings range from “quite” (3) to “very” (4), whereas according to the performance criterion, ratings range from “little” (2) to “quite” (3). Most specific objectives are in quadrant B (highly important and highly performative, i.e., well formulated) and in quadrant A (highly important and not performative, i.e., poorly formulated). There are very few in quadrant C (low important; low performative) or quadrant D (low important; highly performative). A total of 17 specific objectives have a performance threshold of 2.5 and an importance rating ranging from “quite” (3) to “very” (4), which means they are highly important and should be retained, but they need to be reformulated (fall into quadrant B). In the plan, both specific objectives in quadrants A and B must be included, but the first need not be revised, whereas the second does (see Figure 5).

Figure 5

4.2 Qualitative IPA results

A total of 273 qualitative suggestions were collected through open-ended questions in the questionnaire. The suggestions were qualitatively analyzed through thematic analysis and classified into three major categories: (a) general comments, (b) reformulation of specific objectives, and (c) introduction of new specific objectives. Regarding general comments, 17 were received that could be applied to all objectives. In these comments, it was highlighted that the objectives should be expressed in a direct and active way, that subordinate clauses should be reduced, and that technical language should be avoided. Further recommendations included the avoidance of footnotes, the incorporation of a glossary to explain the terms used, the reformulation of abstract objectives to make them more concrete, and the provision of examples for clarity. Regarding suggestions for the reformulation of the current specific objectives, 207 comments were received. The suggestions were classified into the following categories: rewording sentences, requests for clarification of terminology within the specific objectives, suggestions concerning the content or ambitiousness of the specific objectives, and recommendations for concrete measures to implement the specific objectives. A total of 49 suggestions were received regarding new specific objectives. The suggestions were divided into two categories: recommendations for addressing issues that were deemed insufficiently or not at all addressed in the specific objectives under evaluation and proposals for concrete measures to implement a new specific objective. The comments were used to provide recommendations to the Autonomous Province of Bolzano on the reformulation of specific objectives of the Plan and/or the addition of new ones. Table 6 provides three illustrative examples of objectives that were reformulated based on the aforementioned comments.

4.3 IPA results of primary stakeholders in specific themes (PS) and other stakeholders (OS)

Respondents rated most of the specific objectives as “quite” or “very” important. As expected, this can be attributed to the fact that most of the specific objectives were framed in a positive manner, which leads respondents to give a higher rating owing to social desirability bias and the tendency to agree with positively framed statements. To minimize the potential for bias, a new IPA was conducted on a segmented sample. For each specific objective the mean values of importance and performance were recalculated distinguishing between stakeholders in specific themes (PS) and other stakeholders (OS), to determine whether there was notable discrepancy in opinion between two groups of stakeholders. The focus was on those specific objectives where the absolute difference between the values provided by the two groups of stakeholders exceeded the threshold of 0.40 for at least one of the two criteria (importance and performance). A total of 27 specific objectives out of 50 were selected for further analysis. A detailed examination of each of these was conducted and specific recommendations were provided where deemed necessary. To illustrate, regarding Theme A, “Climate Change,” three out of eight specific objectives exhibited a difference exceeding the threshold (Table 7). In the case of two specific objectives (A1.4 and A2.1), the stakeholders in specific themes ascribed a greater importance value than the other stakeholders. It is therefore recommended that these two objectives should not be eliminated, even if the value assigned by the entire sample is low. Conversely, the PS assigned a lower performance score to the PS for the specific objective A1.5 than the OS. As a result, it is strongly recommended that this objective be reformulated.

5 Discussion

5.1 IPA as an effective assessment technique to elicit stakeholder feedback in spatial planning

Strategic planning is defined by its continuous nature, whereby local actors are engaged in a process of listening and consultation throughout the planning process. This engagement extends beyond the drafting of the plan and is instead integrated into all phases of the planning process, from the diagnostic phase, through the selection of objectives and the definition of intervention strategies, to implementation and the post-implementation phase of evaluating the results (Mela, 2009). Our research shows that IPA has proven to be an excellent method to integrate stakeholder feedback in spatial planning. The practical applicability and capacity of IPA to generate actionable insights make it a valuable tool in spatial planning.

Enhance inclusive decision-making: IPA assures that spatial planning is founded upon substantial, stakeholder-driven insights, thereby ensuring more effective and responsive sustainable development. This may serve to reinforce the democratic process, counteract the polarization of opinions, prevent conflicts, reduce inequalities and social tensions, and enhance the sense of responsibility of the community and its sense of belonging to the development process, as stated by Palmia (2023). By explicitly delineating areas of significant importance and suboptimal performance, our findings have demonstrated that IPA facilitated a data-driven decision-making process. IPA facilitated the Province’s ability to define objectives in a logical, goal-oriented manner, thereby ensuring that planning initiatives were aligned with the actual needs and preferences of stakeholders. As a result of IPA’s inclusive approach, which involved gathering input from a diverse range of stakeholders, including individuals and organizations, multiple perspectives were considered, leading to a more comprehensive understanding of the issues at stake and aligning with the principles of inclusive planning (Baker et al., 2010).

Provides findings that can be translated into action: IPA facilitated the explicit and traceable identification of areas that were performing well and were already satisfactory, as well as those that require improvement, from the perspective of the stakeholders. This proved particularly useful in the context of the Provincial Spatial Plan, whereby the most favorably rated objectives could be distinguished from those that required correction. In accordance with existing literature, IPA helped prioritize areas of intervention and objectives. This prioritization enables decision-makers to concentrate on the most critical and pertinent matters, thereby guaranteeing that decision-makers direct their attention to critical and relevant issues that align with the priorities of the stakeholders.

Produces reliable and valid results: IPA provides a clear and structured framework for stakeholders to express their views, which making it easier for researchers to organize, analyse, and interpret the data.

5.2 Limitation on the use of IPA in spatial planning

Our research has demonstrated that IPA has several limitations when applied to strategic spatial planning. IPA application in strategic spatial planning is limited by issues of subjectivity, complexity, dynamics, quantification, oversimplification, implementation barriers, and conflict resolution. Planners should be aware of these limitations and consider supplementary methods to ensure a comprehensive and balanced approach to spatial planning.

Subjectivity of responses: Subjectivity is a significant factor that must be considered when analyzing data with IPA. IPA relies heavily on stakeholder feedback, which is often highly subjective. The veracity of the results obtained using the IPA method is contingent upon the precision of the responses provided by the stakeholders in question. The importance and performance of a given subject may be perceived differently by different stakeholders based on their personal interests. This subjectivity can lead to biases in the data, potentially skewing the results and complicating the analysis. In our research the specific objectives to be evaluated via IPA were clearly delineated; it was not possible for stakeholders to reformulate the specific objectives of the PSP to avoid any potential bias in the responses. The subjective nature of responses and the variability in stakeholder perceptions is discusses in Martilla and James (1977).

Diverse Stakeholder Interests: In spatial planning, stakeholders often include businesses, residents, and environmental groups, each with distinct and sometimes conflicting priorities. IPA does not effectively reconcile these differences, leading to challenges in achieving a consensus on planning priorities. When stakeholders have different or conflicting priorities, IPA does not provide mechanisms for resolving these conflicts. To ascertain whether differences emerge among different types of stakeholders, in our research we conducted a segmented analysis. Our research shows the importance of integrating quantitative data with qualitative approaches, that might be useful to solve potential conflicts. The challenges in balancing diverse stakeholder interest in spatial planning is confirmed by Reed et al. (2006).

Complexity of spatial factors vs. simplistic analysis: IPA provides a relatively simple two-dimensional analysis that may not fully capture the complexity of spatial planning issues. Spatial planning involves a wide array of factors such as land use, transportation, environmental impact, and social equity. IPA might oversimplify these complex and interrelated elements, overlooking nuances and interdependencies between topics. The straightforward nature of IPA results can occasionally result in oversimplification by policymakers. In many instances, as evidenced also in our research, additional qualitative research is necessary to gain insight into the underlying factors and to consider the broader context in a more comprehensive manner. The complex and dynamic nature of spatial planning and the need for comprehensive approaches that go beyond simplistic analyses is deeply analyzed in Healey (2006), Albrechts (2004), and Churchman (1967). Healey (2006) discusses the complex and dynamic nature of spatial planning.

Scalability and resource: The implementation of IPA in large-scale spatial planning projects can presents several challenges as the collection, aggregation, and interpretation of data from a large and diverse set of stakeholders can become a complex and unwieldy process. Our research demonstrates that obtaining feedback from stakeholders and analyze them soundly was a time-consuming process. The process requires a considerable investment of time, effort, and potentially financial resources to conduct surveys and data analysis. Many stakeholders abandoned the questionnaire halfway through. Innes and Booher (2010) discuss the barriers to be implementing planning strategies and the importance of collaborative approaches to resolve conflicts.

5.3 Research-policy-practice interface

The collaboration between Eurac Research and the provincial officers was based on empirically validated methodologies, which enabled the development of robust data collection and analysis tools, as well as the formulation of recommendations based on analysis techniques. The research findings indicate that the contributions of stakeholders, in conjunction with those of the researchers, were instrumental throughout the process, resulting in a comprehensive reformulation of the objectives of the Provincial Strategic Plan (PSP). The revised objectives of the plan now reflect the demands and perspectives of a broad public, transcending the narrow confines of administrators or planners. This inclusivity ensures that the plan addresses the needs of a wider community and promotes greater public engagement, thereby enhancing its cross-sectoral nature (Albrechts, 2004). In line with recent academic literature, our insights underscore the crucial role that research plays in supporting evidence-based decision-making, stakeholder engagement and strategic planning processes at various levels of governance. Our research demonstrated the significant contribution of scientific inquiry to the planning process and stakeholder consultation. In this context, researchers act as knowledge brokers, facilitating the translation of scientific findings into actionable policy recommendations (Wynanda et al., 2014). Furthermore, collaborative knowledge production contributes to the robustness and relevance of policy and planning outcomes (Pütz and Brassel, 2021).

6 Conclusion

The objective of this research was to conduct an importance-performance analysis (IPA) in order to investigate the opinions of stakeholders regarding the level of importance and performance of the objectives set out in the Provincial Strategic Plan (PSP) of the Autonomous Province of Bolzano-Bozen (PAB). Most of the specific objectives were considered very important. Among them, the majority were rated as very satisfactory, i.e., well formulated; some were rated as very important but with low satisfaction, i.e., not well formulated. A total of 273 qualitative suggestions were collected and used to reformulate the specific objectives of the Plan, including concerns, priorities, proposals for new objectives, and general inputs. A segmented analysis was also conducted to ascertain whether there were any notable discrepancies between the evaluations of specific objectives by the two groups of stakeholders. However, no significant differences were observed. The study has demonstrated that IPA is also an effective assessment technique and highly valuable approach in the context of spatial planning, particularly for policymakers and researchers engaged in the nexus between research and policy formulation. In general, the IPA method has been demonstrated to be an effective tool for the collection and integration of diverse stakeholder feedback and perceptions in a pragmatic and systematic manner. The IPA method was found to be an effective tool for analyzing stakeholder input and prioritizing issues according to stakeholder relevance. It provided opportunities for local actors and organizations to express their perspectives on current and future development efforts. Furthermore, it provided the province with easily digestible information on areas where more work was needed and others where the formulation of objectives was already very satisfactory.

References

• 1

  AlbrechtsL. (2004). Strategic (spatial) planning reexamined. Environ. Plann. B Design31, 743–758. doi: 10.1068/b3065

  • CrossRef
  • Google Scholar

• 2

  AlbrechtsL. (2006). Shifts in strategic spatial planning? Some evidence from Europe and Australia. Environ. Planning A38, 1149–1170. doi: 10.1068/a37304

  • CrossRef
  • Google Scholar

• 3

  AlbrechtsL. (2015). Ingredients for a more radical strategic spatial planning. Environ. Plann. B Design42, 510–525. doi: 10.1068/b130104p

  • CrossRef
  • Google Scholar

• 4

  AlbrechtsL.BalducciA. (2013). Practicing strategic planning: in search of critical features to explain the strategic character of plans. disP Plan. Rev.49, 16–27. doi: 10.1080/02513625.2013.859001

  • CrossRef
  • Google Scholar

• 5

  AlvessonM.SköldbergK. (2000). Reflexive methodology: New vistas for qualitative research. London: Sage.

  • Google Scholar

• 6

  ASTAT (2013). Dauersiedlungsgebiet in Südtirol 2012. Bozen: Italien.

  • Google Scholar

• 7

  AzzopardiE.NashR. (2013). A critical evaluation of importance-performance analysis. Tour. Manag.35, 222–233. doi: 10.1016/j.tourman.2012.07.007

  • CrossRef
  • Google Scholar

• 8

  BakerM.HincksS.SherriffG. (2010). Getting involved in plan making: participation and stakeholder involvement in local and regional spatial strategies in England. Environ. Plann. C Gov. Policy.28, 574–594. doi: 10.1068/c0972

  • CrossRef
  • Google Scholar

• 9

  BobbioL. (2019). Designing effective public participation. Polic. Soc.38, 41–57. doi: 10.1080/14494035.2018.1511193

  • CrossRef
  • Google Scholar

• 10

  BoleyB. B.McGeheeN. G.HammettA. T. (2017). Importance-performance analysis (IPA) of sustainable tourism initiatives: the resident perspective. Tour. Manag.2017, 66–77. doi: 10.1016/j.tourman.2016.10.002

  • CrossRef
  • Google Scholar

• 11

  BrysonJ. (2004). What to do when stakeholders matter: a guide to stakeholder identification and analysis techniques. Public Manag. Rev.6, 21–53. doi: 10.1080/14719030410001675722

  • CrossRef
  • Google Scholar

• 12

  BuiT.-D.TsaiF. M.TsengM.-L.WuK.-J.ChiuA. S. F. (2020). Effective municipal solid waste management capability under uncertainty in Vietnam: utilizing economic efficiency and technology to foster social mobilization and environmental integrity. J. Clean. Prod.259:120981. doi: 10.1016/j.jclepro.2020.120981

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 13

  CamagniR. (2017). “Integrated spatial planning: why and how?” in Seminal studies in regional and urban economics. ed. CapelloR. (Berlin: Springer), 391–398.

  • Google Scholar

• 14

  CastellaJ. C.BourgoinJ.LestrelinG.BouahomB. (2014). A model of the science–practice–policy interface in participatory land-use planning: lessons from Laos. Landsc. Ecol.29, 1095–1107. doi: 10.1007/s10980-014-0043-x

  • CrossRef
  • Google Scholar

• 15

  ChengE. W.LiH. (2000). Importance-performance analysis: a case study of the tourist satisfaction with Hong Kong. Leis. Sci.22:7464. doi: 10.1080/01490400050177464

  • CrossRef
  • Google Scholar

• 16

  ChurchmanC. W. (1967). Wicked problems. Manag. Sci.14, B141–B142. doi: 10.1007/978-1-4419-1153-7_200919

  • CrossRef
  • Google Scholar

• 17

  DengW. (2007). Using a revised importance–performance analysis approach: the case of Taiwanese hot springs tourism. Tour. Manag.28, 1274–1284. doi: 10.1016/j.tourman.2006.07.010

  • CrossRef
  • Google Scholar

• 18

  EnnewC. T.ReedG. V.BinksM. R. (1993). Importance-performance analysis and the measurement of service quality. Eur. J. Mark.27, 59–70. doi: 10.1108/03090569310026402

  • CrossRef
  • Google Scholar

• 19

  FaludiA. (2000). The performance of spatial planning. Planning Prac. Res.15, 299–318. doi: 10.1080/713691907

  • CrossRef
  • Google Scholar

• 20

  FazeyJ.SchäpkeN.CanigliaG.PattersonJ.HultmanJ.Van MierloB.et al. (2018). Ten essentials for action-oriented and second order energy transitions, transformations and climate change research. Energ. Res. Soc. Sci.40, 54–70. doi: 10.1016/j.erss.2017.11.026

  • CrossRef
  • Google Scholar

• 21

  FengeL. A.OakleyL.BethanT.BeerS. (2019). The impact of sensitive research on the researcher: preparedness and positionality. Int J Qual Methods18:3161. doi: 10.1177/1609406919893161

  • CrossRef
  • Google Scholar

• 22

  FowlerF. J. (2014). Survey research methods, survey research is designed to generate statistical data about a target audience. London, Sage.

  • Google Scholar

• 23

  GaiS.FuJ.RongX.DaiL. (2022). Importance–performance analysis and improvement of an urban park’s cultural ecosystem services based on users’ perspectives: a Beijing case study. J. Asian Archit. Building Eng.22, 726–739. doi: 10.1080/13467581.2022.2049800

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 24

  GideonL. (2012). Handbook of survey methodology for the social sciences. New York, NY: Springer.

  • Google Scholar

• 25

  GlassonJ.MarshallT. (2007). Regional planning. London: Routledge.

  • Google Scholar

• 26

  GreenwoodD. J.LevinM. (2007). Introduction to action research: Social research for social change. London: Sage Publications.

  • Google Scholar

• 27

  HealeyP. (1997a). Making strategic spatial plans. 1st Edn. London: Routledge.

  • Google Scholar

• 28

  HealeyP. (1997b). Collaborative planning: Shaping places in fragmented societies. London: Red Globe Press.

  • Google Scholar

• 29

  HealeyP. (1998). Building institutional capacity through collaborative approaches to urban planning. Environ. Plann. Econ. Space30, 1531–1546. doi: 10.1068/a301531

  • CrossRef
  • Google Scholar

• 30

  HealeyP. (2006). Collaborative planning: Shaping places in fragmented societies. Palgrave London: Macmillan Press.

  • Google Scholar

• 31

  HealeyP.KhakeeA.MotteA.NeedhamB. (2006). Making strategic spatial plans: Innovation 608 in Europe. New York, NY: Taylor & Francis.

  • Google Scholar

• 32

  HerspergerA.GrădinaruS.OliveiraE.PagliarinS.PalkaG. (2019). Understanding strategic spatial planning to effectively guide development of urban regions. Cities94, 96–105. doi: 10.1016/j.cities.2019.05.032

  • CrossRef
  • Google Scholar

• 33

  HollenhorstS.OlsonD.FortneyR. (1992). Use of importance–performance analysis to evaluate state park cabins: the case of the West Virginia state park system. J. Park. Recreat. Adm.10, 1–11.

  • Google Scholar

• 34

  IAP2. (2018). IAP2’s public participation spectrum. Available at: https://iap2.org.au/wp-content/uploads/2020/01/2018_IAP2_Spectrum.pdf (Accessed September 09, 2024).

  • Google Scholar

• 35

  InnesJ. E.BooherD. E. (2004). Reframing public participation: strategies for the 21st century. Plan. Theory Pract.5, 419–436. doi: 10.1080/1464935042000293170

  • CrossRef
  • Google Scholar

• 36

  InnesJ. E.BooherD. E. (2010). Planning with complexity: An introduction to collaborative rationality for public policy. London: Routledge.

  • Google Scholar

• 37

  KunzmannK. (2013). Strategic planning: a chance for spatial innovation and creativity. disP-the 618. Plan. Rev.49, 28–31. doi: 10.1080/02513625.2013.859003

  • CrossRef
  • Google Scholar

• 38

  KwatraS.KumarA.SharmaS.SharmaP. (2021). Stakeholder participation in prioritizing sustainability issues at regional level using analytic hierarchy process (AHP) technique: a case study of Goa, India. Environ. Sust. Indic.11:100116. doi: 10.1016/j.indic.2021.100116

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 39

  LaiI. K. W.HitchcockM. (2015). Importance–performance analysis in tourism: a framework for researchers. Tour. Manag.48, 242–267. doi: 10.1016/j.tourman.2014.11.008

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 40

  Legge Provinciale (2018). Legge provinciale 10 luglio 2018, n. 912 (provinz.bz.it). Province of Bolzano, Bolzano.

  • Google Scholar

• 41

  MartillaJ. A.JamesJ. C. (1977). Importance-performance analysis. J. Mark.41, 77–79. doi: 10.2307/1250495

  • CrossRef
  • Google Scholar

• 42

  MatzlerK.BailomF.HinterhuberH. H.RenzlB.PichlerJ. (2004). The asymmetric relationship between attribute-level performance and overall customer satisfaction: a reconsideration of the importance-performance analysis. Ind. Mark. Manag.33, 271–277. doi: 10.1016/S0019-8501(03)00055-5

  • CrossRef
  • Google Scholar

• 43

  MelaA. (2009). Pianificazione strategica e partecipazione. Sociol. Urbana e Rurale.89, 147–169. doi: 10.3280/SUR2009-089005

  • CrossRef
  • Google Scholar

• 44

  MimbsB. P.BoleyB. B.WoosnamK. M.GreenG. T. (2020). Importance-performance analysis of residents’ and tourists’ preferences for water-based recreation in the southeastern United States. J. Outdoor Recreat. Tour.31:100324. doi: 10.1016/j.jort.2020.100324

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 45

  MorinJ.OlssonC.AtikcanE. (2021). Thematic analysis. Research Methods in the Social Sciences: An A-Z of key concepts. Oxford University Press.

  • Google Scholar

• 46

  MorphetJ. (2005). A meta-narrative of planning reform. Town Plan. Rev.76, 4–9.

  • Google Scholar

• 47

  O’NeillM. A.PalmerA. (2004). Importance-performance analysis: a useful tool for directing continuous quality improvement in higher education. Qual. Assur. Educ.12, 39–52. doi: 10.1108/09684880410517423

  • CrossRef
  • Google Scholar

• 48

  OhH. (2001). Revisiting importance-performance analysis. Tour. Manag.22, 617–627. doi: 10.1016/S0261-5177(01)00036-X

  • CrossRef
  • Google Scholar

• 49

  OpdamP.NassauerJ. I.WangZ.AlbertC.BentrupG.McAlpineC.et al. (2013). Science for action at the local landscape scale. Landsc. Ecol.28, 1439–1445. doi: 10.1007/s10980-013-9925-6

  • CrossRef
  • Google Scholar

• 50

  PalmiaF. (2023). Enabling meaningful public participation in spatial planning processes. Nairobi: UN-Habitat.

  • Google Scholar

• 51

  PhanT. T. T.NguyenV. V.NguyenH. T. T.LeeC.-H. (2022). Integrating citizens’ importance-performance aspects into sustainable plastic waste Management in Danang, Vietnam. Sustainability14:10324. doi: 10.3390/su141610324

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 52

  PongponratK.PongquanS.ShivakotiG. (2006). Initiation of participatory tourism planning by local government: a case study of community cleaning day, Koh Samui, Surat Thani Province, southern Thailand. J. Hospitality Tour.4, 65–90.

  • Google Scholar

• 53

  PützM.BrasselR. (2021). “Knowledge exchange at science-policy inter-faces in the fields of spatial planning, land use and soil management: a Swiss case study” in Sustainable land management in aEuropean context. Human-environment interactions. eds. WeithT.BarkmannT.GaaschN.RoggaS.StraußC.ZscheischlerJ., vol. 8 (Cham: Springer).

  • Google Scholar

• 54

  ReedM. S.FraserE. D. G.DougillA. J. (2006). An adaptive learning process for developing and applying sustainability indicators with local communities. Ecol. Econ.59, 406–418. doi: 10.1016/j.ecolecon.2005.11.008

  • CrossRef
  • Google Scholar

• 55

  ReedM. S.VellaS.ChalliesE.de VenteJ.FrewerL.Hohenwallner-RiesD.et al. (2018). A theory of participation: what makes stakeholder and public engagement in environmental management work?Restor. Ecol.26:12541. doi: 10.1111/rec.12541

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 56

  SeverI. (2015). Importance-performance analysis: a valid management tool?Tour. Manag.48, 43–53. doi: 10.1016/j.tourman.2014.10.022

  • CrossRef
  • Google Scholar

• 57

  StrydomW. F.FunkeN.NienaberS.NortjeK.SteynM. (2010). Evidence-based policymaking: a review. S. Afr. J. Sci.106, 5–6. doi: 10.4102/sajs.v106i5/6.249

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 58

  TaneseA.Di FilippoE.RennieR. (2006). La pianificazione strategica per lo sviluppo dei territori: Analisi e strumenti per l’innovazione. Soveria Mannelli: Rubettino Editore Srl.

  • Google Scholar

• 59

  TsengM.-L. (2011). Importance–performance analysis of municipal solid waste management in uncertainty. Environ. Monitor. Assessment172, 171–187. doi: 10.1007/s10661-010-1325-7

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 60

  UNESCAP. (2024). Training reference material—Effective stakeholder engagement for the 2030 agenda. Available at: https://www.unescap.org/sites/default/files/Final.Effective%20Stakeholder%20Engagement%20for%20the%202030%20Agenda%20rev.pdf (Accessed February 20, 2024).

  • Google Scholar

• 61

  Watson RobertT. (2005). Turning science into policy: challenges and experiences from the science–policy interface. Phil. Trans. R. Soc. B360, 471–477. doi: 10.1098/rstb.2004.1601

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 62

  WynandaV. E. W.DriessenP. P. J.RunhaarH. (2014). Towards productive science-policy interfaces: a research agenda. JEAPM16:1450007. doi: 10.1142/S1464333214500070

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 63

  Wyród-WróbelJ.BiesokG. (2017). Decision making on various approaches to importance-performance analysis (IPA). Eur. J. Bus. Sci. Technol.3, 123–131. doi: 10.11118/ejobsat.v3i2.82

  • CrossRef
  • Google Scholar