Measuring the multidimensional reputation of a medicines regulatory agency: development and validation of a public-oriented scale

Abstract

Objective:

The reputation of public agencies, encompassing the dimensions of performance, morality, procedure, and technical competence, is fundamental to understanding their behavior. However, standardized, individual-level measures of reputation suitable for surveys targeting the general public are lacking. This study aims to develop and validate a survey instrument for the general public to measure the multi-dimensional reputation of public agencies, with a focus on South Korea’s medicines regulatory agency.

Methods:

Survey items were developed based on previous literature, refined through expert consultation, and validated through a population survey. The validation study involved 1,000 participants from the public, selected using a quota sampling method stratified by age, sex, and region, according to the South Korean census. Validity was assessed through exploratory factor analysis and hypothesis testing, while reliability was evaluated using internal consistency.

Results:

Exploratory factor analysis identified a three-dimensional structure of reputation, encompassing performance, procedure, and technical competence, while morality was not distinctly identified as a separate dimension. Construct validity, including convergent and discriminant validity, was confirmed. The internal consistency of the three dimensions was acceptable, with Cronbach’s alpha coefficients ranging from 0.87 to 0.91. The overall reputation of the medicines regulatory agency was measured at 72 out of 100. The specific dimension scores were as follows: 74 for technical competence, 71 for performance, and 70 for procedure.

Conclusion:

The agency should recognize the multidimensional nature of reputation and foster an environment that enables the public to observe and evaluate these dimensions. Reputation management strategies should emphasize not only technical expertise but also performance and procedural aspects to ensure a well-rounded reputation.

Introduction

Reputation refers to the image of an organization as perceived by diverse audiences. It is defined as “symbolic beliefs about an organization—its capacities, intentions, history, mission—and these images are embedded in a network of multiple audiences (1).” Reputation carries different meanings in the private and public sectors (2, 3). In the private sector, reputation focuses on performance aspects such as competitive advantage or profitability (4). In contrast, reputation in the public sector extends beyond performance to include procedural and moral dimensions (5).

A distinct characteristic of bureaucratic reputation is its multidimensionality (5, 6). This reflects the ambiguity of public organizational goals and the challenges these organizations face (7, 8). Public organizations must address a wide range of interests, which are critical in shaping reputation. While classical literature on bureaucracy highlights expertise, derived from information asymmetry, as a source of bureaucratic power (9, 10), bureaucratic reputation is not confined to expertise alone. Carpenter categorizes reputation as extending beyond expertise to include performative, technical, legal-procedural, and moral dimensions (1). Performative reputation is shaped by perceptions of an organization’s decision-making and effectiveness in achieving objectives, while technical reputation concerns its scientific, methodological, and analytical capacities. Legal-procedural reputation is based on adherence to accepted rules and norms, and moral reputation stems from value-driven and ethical behaviors that generate emotive judgments from the public.

Organizational reputation has emerged as a significant topic in bureaucratic studies over the past decade (11). Building a strong reputation is a critical element of regulatory power and plays a key role in understanding the functions of public administration. Carpenter emphasized that reputation “shapes the power of government organizations, and more broadly, the powers of the state (1). Bureaucratic reputation also serves as an established framework for studying public sector organizations and is utilized as an essential factor in understanding regulation and the behavior of regulators. Numerous empirical studies have demonstrated that reputation is a crucial determinant of bureaucratic behavior (12–14).

Thus, reputation is a key concept in the study of public sector organizations; however, its conceptual ambiguity and multidimensionality present significant challenges for accurate measurement (6, 15). In the private sector, where reputation often focuses on performance metrics such as profitability, a variety of measurement tools are available (16). However, in the public sector, tools for measuring reputation remain limited. Recently, Lee and Van Ryzin proposed an indicator to measure the reputation of public agencies from the perspective of citizens (17). Nonetheless, this indicator has a limitation in that it does not fully capture the multidimensionality of reputation. Building on this, Overman et al. developed a tool to measure the multidimensional reputation of public agencies by incorporating the perspectives of various stakeholders (6).

Several issues related to medicines regulation have emerged as critical concerns (18, 19). Ensuring expedited approvals while safeguarding patient safety are key objectives of medicines regulatory agencies (20–22). To achieve these conflicting goals, agencies have implemented expedited approval programs (23–25) and, more recently, have actively incorporated real-world data/evidence into regulatory decision-making processes (26–29). The successful implementation of these initiatives is closely linked to the reputation of the organization. This study aims to develop and validate a survey instrument for measuring the multi-dimensional reputation of public agencies from the perspective of the general public, with a focus on South Korea’s medicines regulatory agency.

Methods

We developed the survey items based on a literature review, refined them using expert feedback, and validated the instrument through a population survey.

Literature review

Following Carpenter’s four-dimensional conceptualization of reputation (1), we generated an initial pool of survey items through a comprehensive review of the literature on bureaucratic reputation. Studies focusing on aspects of corporate reputation, such as prices, profits, and investments, were excluded. This process identified two key papers (6, 17), leading to a total of 40 survey items.

Interviews with experts

The preliminary questionnaire was assessed for face validity through semi-structured, face-to-face interviews. We evaluated the relevance, comprehensibility, acceptability, and feasibility of the questionnaire using cognitive interview techniques, including think-aloud and probing methods (30, 31). Three experts participated in the assessment: two academics specializing in regulatory science and public health—one of whom had experience working at the Ministry of Food and Drug Safety (MFDS) for one and a half years—and an employee of a pharmaceutical company with over a decade of experience in regulatory affairs. The interviews, conducted in January 2024, each lasted approximately 1 h. The insights gained from these interviews were instrumental in finalizing the questionnaire. Three laypersons evaluated the appropriateness and clarity of the survey items that were finalized based on expert interview results.

Population survey

Participants and sample size

The study population consisted of South Korean public aged 19 years or older. The sample size (n = 1,000) was determined using a 95% confidence interval, a 3.1% margin of error, and a standard deviation of 0.5%. A quota sampling method was employed, stratifying the sample by sex, age, and region according to South Korean census data, and the required number of completed surveys for each quota was obtained. The survey was conducted by the agency Realmeter between April 20 and May 6, 2024. Invitations were randomly sent to the pre-registered individuals at the agency’s mobile phones until the target number of completed surveys was reached. Participants were informed of the study’s objectives in the invitations, and those who consented were directed to an encrypted website to complete the survey. Upon completion, participants received a voucher valued at US $4. The study protocol was reviewed and approved by the Institutional Review Board of Hanyang University (HYUIRB-202403-028).

Questionnaires and scoring

The final questionnaire comprised four dimensions, each with 10 survey items, addressing performance, morality, procedure, and technical competence. Responses were recorded on a 5-point Likert scale, ranging from 1 (strongly disagree) to 5 (strongly agree), with an additional option of 6 (do not know). Since this study aims to measure the reputation of regulatory agencies among the general public, option 6 was included to ensure the suitability of the survey items for lay respondents. Scores were calculated as the mean of the summed scores for each dimension, excluding responses marked as 6. In addition to the 40 items related to reputation, six other variables were included to test construct validity: budget, autonomy, performance, favorability, political orientation, and socio-economic status. These variables were also measured using a 5-point Likert scale, with an additional option of 6 (do not know).

Exploratory factor analysis

Item analysis

We analyzed the distribution, kurtosis, and skewness of responses for each item. Items that were deemed challenging for the public to answer were excluded. Items with “do not know” (answer 6) responses exceeding 7% or “neutral” (answer 3) responses exceeding 35% were considered unacceptable and were subsequently removed.

Exploratory factor analysis

Following item exclusion, exploratory factor analysis (EFA) was conducted to confirm the underlying factor structure of bureaucratic reputation. Sampling adequacy was assessed using Bartlett’s Test of Sphericity and the Kaiser-Meyer-Olkin (KMO) test to determine whether the data were suitable for factor analysis (32). Parallel analysis was employed to determine the number of factors to retain, keeping only those with eigenvalues greater than 1.0 (33). EFA was performed using varimax rotation and the maximum likelihood method. A stepwise approach was applied to enhance factor loadings and minimize cross-loadings, with factor loadings above 0.5 deemed satisfactory.

Construct validity

For construct validity, correlations between reputation and six additional variables (budget, autonomy, performance, favorability, political orientation, and socio-economic status) were assessed. We hypothesized strong relationships between reputation and variables such as budget, autonomy, performance, and favorability (convergent validity), and weak correlations with political orientation and socio-economic status (discriminant validity). Spearman’s rank correlation coefficient were calculated, and the strength of correlations was assessed using Cohen’s criteria: strong (r > 0.5), moderate (0.3 < r ≤ 0.5), and weak (r ≤ 0.3) (34).

Reliability

Cronbach’s alpha was used to assess the internal consistency of each factor, with values greater than 0.7 considered acceptable (34).

Results

Refined items

Following the interviews, most items were deemed suitable for the South Korean context. However, some items within dimension A (performance) were revised. Experts emphasized the importance of consistency, predictability, and responsiveness in MFDS decisions, recommending the inclusion of questions addressing these aspects. Based on their input, questions A2, A3, A4, and A7 were revised. Three laypersons indicated that the finalized survey items were clear and appropriate.

Item analysis

Supplementary Table 1 presents the characteristics of the study participants. Of the 9,874 invitations sent, 1,000 individuals completed the survey, resulting in a response rate of 10.12%. Table 1 presents the distribution of responses to the 40 items in the final version of the questionnaire. After item analysis, 5 items from dimension A: performance (A1, A3, A4, A7, A10), 6 items from dimension B: morality (B2, B5, B6, B7, B9, B10), 6 items from dimension C: procedure (C2, C3, C7, C8, C9, C10), and 2 items from dimension D: technical competence (D4, D5) were removed. The remaining 21 items were deemed acceptable for exploratory factor analysis.

Exploratory factor analysis

EFA was conducted on a sample of 752 participants, excluding 248 participants who selected “do not know” (answer 6) for the remaining 21 items. The initial test for sampling adequacy indicated that the sample was suitable for factor analysis (Bartlett’s Test of Sphericity: p < 0.0001; Kaiser-Meyer-Olkin (KMO) test = 0.98). Parallel analysis revealed a three-factor structure.

Table 2 presents the results of the EFA, including factor loadings greater than 0.5. Through a stepwise approach, seven items (A5, B3, B4, B8, D2, D6, and D8) were removed. The analysis identified three dimensions: A: performance, D: technical competence, and C: procedure. Out of four items initially expected to load onto the B: morality dimension, three were excluded, and one was reassigned to the dimension C: procedure during the stepwise approach to finalizing the model. The three dimensions partially reflect Carpenter’s four-dimensional conceptualization of reputation. The dimensions accounted for 22, 21, and 21% of the total variance, respectively. Cronbach’s alpha values for technical procedure, performance, and competence were 0.88, 0.87, and 0.91, respectively.

Supplementary Table 2 presents the distribution of responses for additional survey items included in the questionnaire to assess construct validity. Table 3 presents the correlation coefficients between the three dimensions and integrated reputation, as well as other variables, including budget, autonomy, performance, favorability, political orientation, and socio-economic status. Performance and favorability showed strong associations with the reputation dimensions, with correlation coefficients ranging from 0.67 to 0.78, demonstrating convergent validity. In contrast, political orientation and socio-economic status exhibited weak associations with the reputation dimensions, with correlation coefficients ranging from −0.11 to 0.11, indicating discriminant validity. Autonomy and budget showed moderate associations with the reputation dimensions, with correlation coefficients ranging from 0.45 to 0.52.

Figure 1 illustrates the distribution of reputation scores across three distinct dimensions—Performance, Procedure, and Technical Competence—as well as the integrated reputation score. Each histogram displays the frequency of scores within a range of 0 to 100. The integrated reputation scores reveal a bell-shaped distribution centered around 72, reflecting an overall positive yet diverse evaluation of reputation. The scores for each dimension were as follows: technical competence scored 74, performance 71, and procedure 70.

Figure 1

Discussion

This study, building on Carpenter’s framework (1), developed a tool for the general public to measure the “reputation” of a medicines regulatory agency, categorized into four dimensions: performance, morality, procedure, and technical competence. The survey items were refined through interview with experts, item analysis, and factor analysis, and various tests confirmed that the tool demonstrated construct validity and reliability.

Interesting findings

This study revealed several interesting findings. First, from the public’s perspective, the technical dimension is relatively easier to evaluate, whereas other dimensions, such as performance, morality, and procedure, are more difficult to assess. Second, the factor analysis identified three key dimensions of reputation: performance, technical competence, and procedure. Meanwhile, survey items related to morality were excluded during the analysis process, with one item being integrated into the procedure dimension. Third, using the proposed tool, the reputation of a medicines regulatory agency was measured at 72 out of 100. The specific dimension scores were as follows: 74 for technical competence, 71 for performance, and 70 for procedure.

Developing a reputation measurement tool for the public

The specialized nature of regulatory agency tasks raises questions about whether the general public is adequately equipped to evaluate such institutions (35). Unlike industry professionals or regulated entities, the public often lacks direct interactions with regulatory agencies, making assessments more challenging (13). The study findings highlight this difficulty: 19 out of 40 proposed survey items were excluded during item analysis, particularly those outside the technical dimension. This outcome suggests that the public, especially those without insider knowledge, struggles to evaluate procedural and moral aspects of medicines regulation, likely due to the abstract or highly technical nature of these concepts.

Audiences play a critical role in measuring reputation. Organizations operate within broad and diverse audience networks, including regulatees, industry professionals, and the general public, each of whom perceives regulatory performance differently (15). The evaluator’s proximity to the organization significantly influences both the feasibility and outcomes of reputation assessments. Those with direct regulatory experience may assess agencies based on compliance and efficiency, whereas the general public may rely on indirect perceptions shaped by media coverage or publicized policy decisions.

Public evaluations remain crucial as medicines regulation ultimately serves society, making the public broader stakeholders (36). The legitimacy of bureaucratic organizations hinges on public support, which is essential for policy credibility. Restricting evaluators to direct stakeholders alone may lead to risks such as regulatory capture or public alienation due to excessive regulatory specialization (37). Despite certain challenges, public evaluations of regulatory agencies remain both justified and meaningful.

Dimensions of reputation

The public identified technical competence, performance, and procedure as distinct dimensions but did not recognize morality as a separate one. Technical competence was perceived as a clear and distinct dimension, aligning with the understanding that regulatory agencies are tasked with approving drugs based on scientific evidence (38, 39). Moreover, the public differentiated between the technical processes involved in drug approval and the outcomes of those processes, treating them as separate dimensions. However, morality was not distinctly identified as an independent dimension; rather, some of its aspects were integrated into the procedural dimension. This phenomenon may be attributed to the overlap among dimensions and the inherent distance between the evaluator and the evaluated.

The first reason is the overlap between dimensions of reputation (40). The dimensions of reputation are interrelated, and some may overlap with each other (15). During the operationalization of conceptual components, overlaps between certain dimensions may be further strengthened. In fact, a study by Overman et al., which measured the reputation of regulatory agencies, found that the reputation of the European Chemical Agency, as perceived by various stakeholders, consisted of the dimensions of performance, procedure, and morality, excluding technical competence (6).

The second factor is the distance between the evaluator and the evaluated (13). Minimal distance is necessary to assess potentially overlapping dimensions of reputation (5). Stakeholders closely connected to an organization can more clearly identify different dimensions. However, this study evaluated regulatory agencies from the public’s perspective. The relatively large distance between evaluators and the evaluated may have led to the omission of certain dimensions. Notably, this distance does not exert a one-way influence. Public perceptions of regulatory agencies differ from those of stakeholders, often emphasizing different aspects.

Practical implications for medicines regulatory agencies

The findings of this study provide practical implications for medicines regulatory agencies. First, regulatory agencies should clearly define the dimensions of reputation and establish conditions that enable the public to observe and evaluate these dimensions. Enhancing transparency in decision-making processes and presenting regulatory outcomes in a format that is easily accessible to the public are crucial. In this regard, regulatory agencies in countries with well-established regulatory systems publish Public Assessment Reports (PARs) to promote transparency and facilitate public understanding (41–43). These strategies can clarify the role of regulatory agencies and foster greater public recognition of the value and importance of public organizations.

Second, regulatory agencies should reassess their strategies for improving reputation. While the agency currently focuses on enhancing technical competence and expertise, this study suggest that the public values not only technical competence but also performance and procedural aspects as key components of reputation. The agency should take proactive measures to strengthen its reputation across all these dimensions. Due to the nature of regulatory agencies, moving beyond conventional bureaucratic practices is essential. Instead, adopting flexible policy operations and establishing collaborative frameworks with stakeholders are critical to achieving a more comprehensive and balanced reputation.

One way to implement such a flexible and collaborative approach is by leveraging international frameworks such as the WHO Global Benchmarking Tool (GBT) (44). GBT provides a structured methodology for assessing regulatory maturity, ensuring that agencies meet global standards in governance, technical competence, and procedural efficiency (45). By systematically evaluating key regulatory functions, GBT helps agencies identify and address gaps that could undermine public trust (45). Adopting GBT can serve as a concrete strategy for improving regulatory reputation.

A key advantage of GBT is its role in facilitating regulatory reliance, enabling agencies to build credibility by aligning with internationally recognized standards. Agencies that achieve Maturity Level 3 (ML3) or higher in GBT assessments are acknowledged as competent regulatory bodies, reducing redundant assessments and expediting decision-making processes (46). This shift not only enhances technical capacity but also reinforces procedural transparency (47)—both of which, as this study highlights, are crucial for public reputation. By integrating GBT into strategic planning, regulatory agencies can go beyond technical improvements and establish a more holistic and credible reputation.

Study limitations

This study has several limitations. First, as it employed a survey-based approach, this study cannot rule out the possibility of selection bias, meaning that survey respondents may have been skewed toward groups with specific characteristics. Second, as the survey was conducted using a self-administered format, some items may have been difficult for respondents to understand, potentially leading to variations in how the survey questions were interpreted. Third, to validate the survey instrument, the study focused on a single agency. Consequently, the findings may have limited generalizability to other types of regulatory agencies. Fourth, perceptions of regulatory agencies are shaped by the experiences and expectations of their audiences. As such, even for agencies of similar types, reputation assessments may not be universally applicable due to variations in national contexts. Fifth, public evaluations of regulatory agencies may be shaped by specific events. Although no major regulatory issues related to the agency occurred before or after this study was conducted, the potential impact of such events should be considered when interpreting the findings.

Conclusion

This study developed and validated a survey instrument to measure the multidimensional reputation of public agencies, with a focus on South Korea’s medicines regulatory agency. The findings revealed that the agency’s reputation was composed of three dimensions: technical competence, performance, and procedure. The theoretical dimension of morality was only partially represented, being integrated into the procedure dimension. The agency should recognize the multidimensional nature of reputation and foster an environment that enables the public to observe and evaluate these dimensions. Reputation management strategies should emphasize not only technical expertise but also performance and procedural aspects to ensure a well-rounded reputation.

References

• 1.

  Carpenter D . Reputation and power: organizational image and pharmaceutical regulation at the FDA. New Jersey: Princeton University Press (2014).

  • Google Scholar

• 2.

  Rainey HG Bozeman B . Comparing public and private organizations: empirical research and the power of the a priori. J Public Adm Res Theory. (2000) 10:447–70. doi: 10.1093/oxfordjournals.jpart.a024276

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 3.

  Barnett ML Jermier JM Lafferty BA . Corporate reputation: the definitional landscape. Corp Reput Rev. (2006) 9:26–38. doi: 10.1057/palgrave.crr.1550012

  • CrossRef
  • Google Scholar

• 4.

  Wæraas A Byrkjeflot H . Public sector organizations and reputation management: five problems. Int Public Manag J. (2012) 15:186–206. doi: 10.1080/10967494.2012.702590

  • CrossRef
  • Google Scholar

• 5.

  Maor M . Theorizing bureaucratic reputation In: Organizational reputation in the public sector. London: Routledge (2014). 17–36.

  • Google Scholar

• 6.

  Overman S Busuioc M Wood M . A multidimensional reputation barometer for public agencies: a validated instrument. Public Adm Rev. (2020) 80:415–25. doi: 10.1111/puar.13158

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 7.

  Rainey HG Jung CS . A conceptual framework for analysis of goal ambiguity in public organizations. J Public Adm Res Theory. (2015) 25:71–99. doi: 10.1093/jopart/muu040

  • CrossRef
  • Google Scholar

• 8.

  Chun YH Rainey HG . Goal ambiguity and organizational performance in US federal agencies. J Public Adm Res Theory. (2005) 15:529–57. doi: 10.1093/jopart/mui030

  • CrossRef
  • Google Scholar

• 9.

  Weber M . The theory of social and economic organization. New York: Free Press (1964).

  • Google Scholar

• 10.

  Simon HA . Administrative behavior: A study of decision-making processes in administrative organizationsNew York: Free Press (1997).

  • Google Scholar

• 11.

  Bustos EO . Organizational reputation in the public administration: a systematic literature review. Public Adm Rev. (2021) 81:731–51. doi: 10.1111/puar.13363

  • CrossRef
  • Google Scholar

• 12.

  Moynihan DP . Extra-network organizational reputation and blame avoidance in networks: the hurricane Katrina example. Governance. (2012) 25:567–88. doi: 10.1111/j.1468-0491.2012.01593.x

  • CrossRef
  • Google Scholar

• 13.

  Busuioc EM Lodge M . The reputational basis of public accountability. Governance. (2016) 29:247–63. doi: 10.1111/gove.12161

  • CrossRef
  • Google Scholar

• 14.

  Maor M . Organizational reputation and jurisdictional claims: the case of the US Food and Drug Administration. Governance. (2010) 23:133–59. doi: 10.1111/j.1468-0491.2009.01470.x

  • CrossRef
  • Google Scholar

• 15.

  Wæraas A Maor M . Organizational reputation in the public sector. London: Routledge (2014).

  • Google Scholar

• 16.

  Hall R . A framework linking intangible resources and capabiliites to sustainable competitive advantage. Strateg Manag J. (1993) 14:607–18. doi: 10.1002/smj.4250140804

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 17.

  Lee D Van Ryzin GG . Measuring bureaucratic reputation: scale development and validation. Governance. (2019) 32:177–92. doi: 10.1111/gove.12371

  • CrossRef
  • Google Scholar

• 18.

  Van Norman GA . Drugs, devices, and the FDA: part 1: an overview of approval processes for drugs. JACC. (2016) 1:170–9. doi: 10.1016/j.jacbts.2016.03.002

  • CrossRef
  • Google Scholar

• 19.

  Eichler H-G Bloechl-Daum B Brasseur D Breckenridge A Leufkens H Raine J et al . The risks of risk aversion in drug regulation. Nat Rev Drug Discov. (2013) 12:907–16. doi: 10.1038/nrd4129

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 20.

  Downing NS Zhang AD Ross JS . Regulatory review of new therapeutic agents—FDA versus EMA, 2011–2015. N Engl J Med. (2017) 376:1386–7. doi: 10.1056/NEJMc1700103

  • CrossRef
  • Google Scholar

• 21.

  Fountzilas E Said R Tsimberidou AM . Expanded access to investigational drugs: balancing patient safety with potential therapeutic benefits. Expert Opin Investig Drugs. (2018) 27:155–62. doi: 10.1080/13543784.2018.1430137

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 22.

  Zhang X Peck CC Wang Y Szucs TD Sun W Bai X et al . Strength of clinical evidence supporting the United States Food and Drug Administration accelerated approvals from 2015 to 2022. BMC Med. (2024) 22:587. doi: 10.1186/s12916-024-03800-6

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 23.

  Hwang TJ Ross JS Vokinger KN Kesselheim AS . Association between FDA and EMA expedited approval programs and therapeutic value of new medicines: retrospective cohort study. BMJ. (2020):371. doi: 10.1136/bmj.m3434

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 24.

  Hoekman J Boon W . Changing standards for drug approval: a longitudinal analysis of conditional marketing authorisation in the European Union. Soc Sci Med. (2019) 222:76–83. doi: 10.1016/j.socscimed.2018.12.025

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 25.

  Kepplinger EE . FDA's expedited approval mechanisms for new drug products. Biotechnolo. Law Rep. (2015) 34:15–37. doi: 10.1089/blr.2015.9999

  • CrossRef
  • Google Scholar

• 26.

  Burns L Roux NL Kalesnik-Orszulak R Christian J Hukkelhoven M Rockhold F et al . Real-world evidence for regulatory decision-making: guidance from around the world. Clin Ther. (2022) 44:420–37. doi: 10.1016/j.clinthera.2022.01.012

  • CrossRef
  • Google Scholar

• 27.

  Bakker E Plueschke K Jonker CJ Kurz X Starokozhko V Mol PGM . Contribution of real-world evidence in European medicines Agency's regulatory decision making. Clin. Pharmacol. Ther. (2023) 113:135–51. doi: 10.1002/cpt.2766

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 28.

  Mohetaer M Tuersun A Li P Wang S Zhang X Chen Y . Drug review and approval policies based on real-world evidence in China and the United States: a comparative study. Clin Ther. (2024) 46:1059–68. doi: 10.1016/j.clinthera.2024.09.009

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 29.

  Wessel D Pogrebnyakov N . Using social media as a source of real-world data for pharmaceutical drug development and regulatory decision making. Drug Saf. (2024) 47:495–511. doi: 10.1007/s40264-024-01409-5

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 30.

  Collins D . Pretesting survey instruments: an overview of cognitive methods. Qual Life Res. (2003) 12:229–38. doi: 10.1023/A:1023254226592

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 31.

  Wolcott MD Lobczowski NG . Using cognitive interviews and think-aloud protocols to understand thought processes. Curr Pharm Teach Learn. (2021) 13:181–8. doi: 10.1016/j.cptl.2020.09.005

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 32.

  Thompson B . Exploratory and confirmatory factor analysis: understanding concepts and applications. Washington, DC: American Psychological Association (2004).

  • Google Scholar

• 33.

  Schmitt TA . Current methodological considerations in exploratory and confirmatory factor analysis. J Psychoeduc Assess. (2011) 29:304–21. doi: 10.1177/0734282911406653

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 34.

  de Vet HCW Terwee CB Mokkink LB Knol DL . Measurement in medicine: a practical guideCambridge University Press (2011).

  • Google Scholar

• 35.

  Martin GP . Representativeness, legitimacy and power in public involvement in health-service management. Soc Sci Med. (2008) 67:1757–65. doi: 10.1016/j.socscimed.2008.09.024

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 36.

  Eichler H-G Pignatti F Flamion B Leufkens H Breckenridge A . Balancing early market access to new drugs with the need for benefit/risk data: a mounting dilemma. Nat Rev Drug Discov. (2008) 7:818–26. doi: 10.1038/nrd2664

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 37.

  Son K-B Park S . Perceptions of regulatory decision-making for new drugs from the viewpoints of the manufacturers in South Korea. Front Med. (2022) 9:869262. doi: 10.3389/fmed.2022.869262

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 38.

  Woodcock J . Evidence vs. access: can twenty-first-century drug regulation refine the tradeoffs?Clin Pharmacol Ther. (2012) 91:378–80. doi: 10.1038/clpt.2011.337

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 39.

  Eichler H-G Abadie E Raine JM Salmonson T . Safe drugs and the cost of good intentions. N Engl J Med. (2009) 360:1378–80. doi: 10.1056/NEJMp0900092

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 40.

  Carpenter DP Krause GA . Reputation and public administration. Public Adm Rev. (2012) 72:26–32. doi: 10.1111/j.1540-6210.2011.02506.x

  • CrossRef
  • Google Scholar

• 41.

  Keyter A Salek S Banoo S Walker S . Can standardisation of the public assessment report improve benefit-risk communication?Front Pharmacol. (2020) 11:855. doi: 10.3389/fphar.2020.00855

  • CrossRef
  • Google Scholar

• 42.

  CIRS . R&D briefing 94, the value of reference agency assessment reports in enabling regulatory reliance. London: Centre for Innovation in Regulatory Science (2024).

  • Google Scholar

• 43.

  IFPMA Assessment Reports . A tool for reliance frequently asked question. Geneva: International Federation of Pharmaceutical Manufacturers and Associations (2020).

  • Google Scholar

• 44.

  Guzman J O"Connell E Kikule K Hafner T . The WHO global benchmarking tool: a game changer for strengthening national regulatory capacity. BMJ Glob Health. (2020) 5:e003181. doi: 10.1136/bmjgh-2020-003181

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 45.

  World Health Organization . Global benchmarking tool (GBT) for evaluation of National Regulatory Systems of medical products, manual for benchmarking and formulation of institutional development plans. Geneva: World Health Organization (2023).

  • Google Scholar

• 46.

  World Health Organization . List of National Regulatory Authorities (NRAs) operating at maturity level 3 (ML3) and maturity level 4 (ML4). (2025). Available online at: https://www.who.int/publications/m/item/list-of-nras-operating-at-ml3-and-ml4. (Accessed March 3, 2025).

  • Google Scholar

• 47.

  Shi J Chen X Hu H Ung COL . Benchmarking drug regulatory systems for capacity building: an integrative review of tools, practice, and recommendations. Int J Health Policy Manag. (2023) 12:8100. doi: 10.34172/ijhpm.2023.8100

  • CrossRef
  • Google Scholar