Improving Interactions Between Health Technology Assessment Bodies and Regulatory Agencies: A Systematic Review and Cross-Sectional Survey on Processes, Progress, Outcomes, and Challenges

Introduction

The transition of a product from benchside to clinical use involves several stages and engagements with different stakeholders (1). The first interaction is often with regulators who provide marketing authorization following satisfactory review of the product's risk–benefit profile (i.e., evaluation of safety, efficacy, and quality). Here, emphasis is usually placed on evidence generated from well-controlled studies (those with high internal validity), such as randomized controlled trials (RCTs) (2). Moreover, relative efficacy against an active comparator is often not a requirement, and placebo comparators are considered to provide simpler statistical and clinical interpretation (3, 4).

Once a product has gained marketing authorization, market access is further dictated by the particular healthcare system's financing mechanisms (5). Usually, payers relying on the assistance of health technology assessment (HTA) bodies decide whether to reimburse a product based on its relative value under current clinical practice scenarios (6). The value assessments usually focus on relative performance (such as relative safety, relative effectiveness, and cost effectiveness) of a technology against currently available clinical options. Unlike regulators who may accept short-term or surrogate outcomes, payers usually prefer long-term clinical outcomes (3, 7). Moreover, their assessment may involve a broader perspective such as the consideration of the potential social, legal, ethical, and political impacts of adopting the new technology (8, 9).

Given the differences in decision mandates of HTA/reimbursement bodies and regulatory agencies, their activities have been distinct from each other (Table 1). However, there are growing interests in aligning the activities of these institutions (12, 13). The interests in improving harmonization/alignment stem from criticisms that the current “silo-based model” is ill-equipped to drive innovation, that it hinders the rapid adoption of evolving clinical evidence, and that it delays timely patient access to life-saving technologies (3, 13). As an example, among all new medicines approved by the European Medicines Agency (EMA) between March 2000 and March 2018, only 56% were recommended by the United Kingdom (UK)'s National Institute for Health and Care Excellence (NICE) for reimbursement (14). Moreover, less than half of new cancer medications assessed during 2013–2017 across 20 countries received positive reimbursement recommendations (15). While market dynamics (such as the availability of cheaper or clinically superior therapeutic options) could influence these trends, it is anticipated that greater interaction/alignment between different stakeholders could improve efficiency in the drug development processes and increase the availability and access to innovative therapies to improve patient outcomes (16).

TABLE 1

Table 1. Characteristics of different agencies (2, 10, 11).

To date, limited reviews have examined the experiences across different markets regarding harmonization/interaction initiatives between HTA/reimbursement bodies and regulatory agencies or their impacts and challenges (3). Such an exercise is needed to improve understanding of the current landscape, identify learning opportunities, and develop insight into areas requiring improvement for effective harmonization/alignment. Thus, in the present study, we aimed to provide a synthesis of the literature regarding opportunities and outcomes of interaction/harmonization initiatives between HTA bodies and regulatory agencies. The systematic literature review was supplemented by a cross-sectional survey among European HTA bodies and regulatory agencies to provide further insight into current trends.

Methods

Literature Search Strategy

A systematic review of the literature was performed by searching MEDLINE, EMBASE, and the International Pharmaceutical Abstracts database from their inception up to October 21, 2019. We used two sequential search strategies to identify relevant information. In the first search strategy (Strategy A), we looked for papers related to either HTA (using the terms “health technology assessment” or “cost effectiveness” or “economic evaluation” or “economic analysis” or “cost–benefit analysis” or “cost-utility analysis”) or regulatory [using the terms “drug approval” or “pharmaceutical regulation” or “drug legislation” or “pharmaceutical administration” or “European medicines agency (EMA)” or “food and drug administration (FDA)”] decision-making processes (Supplementary Table 1). In the second strategy (Search B), we combined the keywords in Search A with “synergy” or “collaboration” or “alignment” or “partnership” “harmonization” or “scientific advice” or “parallel consultation” (Supplementary Table 2). Citations from different databases were combined in Endnote X9 (Clarivate analytics^®, USA), and duplicates were removed. Subsequently, the titles and abstracts were screened, and those deemed likely to be eligible were subjected to full text assessment. Once the relevant articles were selected, additional articles were identified by exploring their bibliographies. We also searched for gray literature (working papers, commissioned reports, policy documents, etc.) via Google scholar and several national and multinational institutional websites including those of the EMA, US FDA, and the European Network for Health Technology Assessment (EUnetHTA) (Supplementary Tables 3–5). We started with general searches on websites and reviewed the hits to identify relevant materials. We then followed up to search their references and ascertain specific case examples.

Study Inclusion Criteria

Only studies published in the English language at the time of this review were included. Moreover, to be eligible for selection, a report had to describe in whole or in part assessments on harmonization/interactions between HTA bodies and regulatory agencies, focusing on mechanisms, implementation models, outcomes, or challenges. Our review centered mainly on pharmaceuticals, although in some respects, insights from medical devices were considered. Furthermore, the review focused on reports from Europe, North America, Australia/New Zealand, and Asia. However, with the exception of Europe in which we decided a priori to include all relevant data regardless of country, for all others, only information from high-income countries (HICs) were targeted. An HIC was defined as per the criteria used by the World Bank to include any country with a gross national income per capita of US$12,376 or more in 2019 (17). All articles were screened by one author (ROA). However, a second author (MLDB) provided a rapid perusal of the appropriateness of inclusion of documents in the final report.

Online Survey

An online cross-sectional survey among European HTA bodies and regulatory agencies was conducted using LimeSurvey^® Pi (v3.1.4). The same set of questions seeking insight into HTA–regulatory agencies interactions were sent to both agencies (Supplementary Table 6). The survey was conducted between January and April 2020.

Analytical Approach

We undertook a narrative synthesis that was largely inductive in nature, that is, centered on themes that were described or highlighted in detail in the literature (18). However, we also applied a deductive method by synthesizing specific information such as those related to alignment of evidentiary requirements, stakeholder involvement and perspectives, and program implementation challenges and successes. The key papers selected were those that had information to help develop the main themes for the report. However, references were also made to other papers outside those informing the themes to allow for broader contextualization. Information originating from the survey was used to corroborate the findings from the systematic review.

Results

Literature Search Results

The bibliographic search identified 30,110 citations from which 4,354 were duplicates. Of the remaining 25,755 unique articles, 25,651 were excluded as unrelated based on titles and abstracts and 104 articles were selected for full-text evaluation. Of the 104 articles undergoing full-text assessment, 16 were retained and 88 articles were excluded. An additional 6 articles were identified by reference screening, and 38 more resources were retrieved via the gray literature search particularly from institutional websites. Figure 1 summarizes the flowchart of the reports' screening steps. The description of the key reports included in this review are provided in Supplementary Table 7.

FIGURE 1

Figure 1. Schematic process of the review.

Survey Response

The online survey received responses from 22 HTA bodies and 6 regulatory agencies. Response rates were 18% (6/34) and 61% (22/36) for regulatory agencies and HTA bodies. The HTA bodies and regulatory agencies were from 21 and 5 different EU countries, respectively. Of the regulatory agencies, one was from Western Europe, two from Northern Europe, two from Central and Eastern Europe, and a representative from the EMA. Among the HTA bodies, nine were from Western Europe, seven from Central and Eastern Europe, four from Northern Europe, and two from Southern Europe. In total, 21 of the 32 (66%) surveyed countries (27 EU, 4 EFTA, and UK) were covered, with some variation across regions (33% Southern, 63% Northern, 70% Western, and 83% Central/Eastern Europe).

What Is HTA/Regulatory Harmonization—Is It Necessary?

Harmonization is broadly considered to encompass the streamlining of regulatory and reimbursement processes (19). It is also deemed process oriented and centered on reducing the time between regulatory and reimbursement decisions and minimizing duplication of work (10, 20). Such an approach is viewed to have potential positive implications for the healthcare system in terms of improving patient care, innovation, and system sustainability (3, 7, 16). Regardless, there are divergent views as to whether harmonization between HTA and regulatory agencies is needed and if at all desirable. Proponents of harmonization initiatives posit that it presents an opportunity to develop economies of scale particularly with respect to evidentiary requirements and/or alignment of a product's lifecycle (11, 21). Critics, on the other hand, have highlighted that such mechanisms may have some unintended adverse impacts. In particular, separate regulatory and reimbursement functions are seen to allow health technologies to undergo robust quality assurance processes while being available on a free market (3). Thus, harmonization is recognized by some to potentially trigger overregulation that hinders the abilities for markets to function thereby leading to market failures (3). Moreover, cross-border harmonization mechanisms are also viewed by some as having the potential to diminish local decision-making power that could lead to the adoption of methods and standards that may not be well suited to the local context (22).

Types of Harmonization/Interactions

From the survey, 67% (4/6) of regulatory agencies reported that they have an established formal link for interacting with HTA bodies while the remaining 33% (2/6) indicated that their engagement is informal. Among the 22 HTA bodies that responded to the survey, half (11/22) indicated having a formal link of collaborating with regulatory agencies. Ten out of 22 indicated that their engagement with regulators is only informal or sporadic, whereas one HTA body reported no interactions at all.

The interactions between regulatory and HTA agencies can be viewed across all the three phases of the product life cycle: (a) the premarketing phase, (b) the phase of actual market entry, and (c) the postlaunch phase (19). While this distinction is useful, a continuous link between the different phases is assumed. Moreover, current harmonization models or approaches can broadly be considered along the broader spectrum of evidentiary needs and those focusing on specific processes and timeframes (Figure 2). In the subsequent sections, we will discuss key issues identified from the literature and survey pertaining to harmonization mechanisms with an emphasis on alignment of evidentiary requirements, tripartite dialogues, parallel submissions (reviews), adaptive licensing pathways (3), and postmarketing collaborations and highlight their implementation challenges and successes.

FIGURE 2

Figure 2. Conceptual display of the key avenues for health technology assessment (HTA)–regulatory harmonization of which alignment of evidentiary requirement is a central theme.

Alignment of Evidentiary Requirements

One overarching theme that transcended across the literature and survey is the need to align evidentiary requirements for HTA bodies and regulatory agencies (2, 7, 10, 21, 23). In general, while there are distinct data needs for HTA bodies and regulatory agencies, there is considerable scope to minimize the gap in their evidentiary requirements through improved alignment (24, 25). In particular, subject to demographic, epidemiological, and other factors, clinical data are generally regarded as transferable across geographical and social boundaries (22). In this context, most discussion have centered on “safety” and “clinical outcomes,” as these requirements are common to both regulators and HTA bodies (2). The discussions in the literature have given a significant boost to comparative-effectiveness research (CER) and relative efficacy (26–28). Relative efficacy is considered as the degree to which an intervention does more good than harm, under ideal circumstances, compared with one or more alternatives in achieving the desired results (26, 29).

The information obtained from relative efficacy studies has the potential to meet the evidentiary requirements of both HTA and regulatory agencies (2, 30). For example, relative efficacy studies can provide the comparative clinical data necessary to support health economic modeling or cost-effectiveness analysis. Nonetheless, while it is generally viewed that relative efficacy of a health technology will be consistent across different settings, very few studies have examined that assumption (31). In the RE-LY trial, for example, the relative efficacy of dabigatran—a new oral direct thrombin inhibitor—varied between countries even under RCT conditions, depending on the efficiency of warfarin management (32). Thus, relative efficacy can differ between different settings (countries) when healthcare practice varies. This raises additional challenge of the acceptability of evidence generated from relative efficacy studies by HTA bodies given their preference for evidence derived in their local clinical context, that is, real world settings (31, 33). Moreover, to fully meet the needs of both regulators and HTA bodies, a number of methodological issues need to be addressed. These include study design (e.g., will consideration be given to indirect comparison, or will head-to-head clinical trials between the new product and its comparator be required?) (3), endpoints (e.g., will HTA bodies consider surrogate endpoints or will they accept only clinical endpoints?), comparator (e.g., will it be the standard of care or any suitable therapeutic alternative?) (25), and target patient population (as relative efficacy varies across patient subpopulations, what will be the optimal patient population?) (2, 34).

There is increasing recognition that CER (which is very similar to relative effectiveness) can be integrated into the existing two-stage assessment framework of regulatory and HTA agencies (2, 3). However, historically, the adoption of active-comparator relative efficacy studies to support regulatory approval has been slow, as enabling laws such as the US Food Drug and Cosmetic Act of 1938, as subsequently amended in 1962, do not require assessment of comparative effectiveness (35). Regardless, in the USA, recent developments such as the establishment of the Patient-Centered Outcomes Research Institute, as part of the Affordable Care Act, has embodied a need for CER (36, 37). The most prominent drivers of CER appear to be “cost pressure” or a “search for value” (38).

From the survey, one respondent indicated that, in 2018, a proposal from their government suggested that the regulatory agency should make the evaluations on relative efficacy proactively to inform the reimbursement decisions by the national HTA body. It was noted that several objections were raised to this proposal and that this suggestion has not yet become a formal regulated duty of the regulatory agency. Regardless, the summaries of new drugs published nationally by the regulatory agency have since then been slightly modified to include more information on relative efficacy.

In general, a standard methodology for compiling comparative data is yet to be developed (39). Thus, there exist ambiguity as to how CER will be appropriately designed to reflect regulators, HTA bodies (or payers), patients, and clinicians' perspectives. As opined by Woodcock (36) from a regulator's perspective, “the tolerance for (and recognition of the probability of) error is probably the greatest divide separating the CER enterprise and the current framework for medical product regulation.”

Global initiatives such as the Green Park Collaborative is exploring the scientific feasibility of developing methodological guidance relative to evidence generation that meets the needs of different stakeholders (40). In Europe, following the publication of the conclusions of the Pharmaceutical forum in 2008, the European commission gave the EMA the political mandate to interact with HTA bodies with the aim to improve the availability and best use of data relevant to HTA (41). The primary objective of this joint project of regulators and HTA bodies was to examine how information in the European Public Assessment Reports (EPARs) can contribute to relative effectiveness assessment by EU Member States' HTA organizations (EUnetHTA). The collaboration between the EMA and EUnetHTA on EPARs started in February 2010 and lasted for more than 2 years. An evaluation of the EPAR pilot program suggested that it provided a platform for discussions about better exchange of data and information (42). It was further noted that the parallel review of EPARs has been helpful to the individual organizations to not only critically review the end-product “assessment report” employing a predefined methodology but also mutually identify areas for future improvement (42). Consequently, the EMA's Road Map to 2015 identified the need for further improvement of EPARs given their use for HTAs (12). A joint EMA-EUnetHTA 3-year workplan 2013–2015 was further instituted to build on this work (43). A report of this 3-year work plan was published in March 2016, which suggested that it facilitated the identification of areas for possible synergies as well as helping to improve understanding of the differences between individual agencies' procedures (44).

Overall, there is some recognition that the gap in evidentiary requirements between regulatory and HTA agencies has narrowed over the past few years (25, 45, 46). For example, Dekker et al. (46), recently examined the similarities and differences in evidentiary requirements of regulatory agencies and HTA bodies (in particular, NICE) with respect to Alzheimer's disease approved. They found a large overlap in the inclusion of phase III RCTs in regulatory and HTA assessments, although the focus on specific outcomes differed slightly (46). Moreover, a 2016 survey revealed close alignment of the perspectives of HTA bodies and regulators on several evidentiary blocks (Figure 3) including the use of patient reported outcomes (PROs), whereas disagreements in areas such as the inclusion of secondary efficacy parameters were documented (25).

FIGURE 3

Figure 3. Perspective of health technology assessment (HTA) assessors and regulators regarding areas where alignment in evidentiary requirement could occur*. (A) Acceptable primary end point. (B) Inclusion of active comparator arm in the trial. (C) Use of patient reported outcomes. (D) Use of health-related quality of life measures. (E) Choice and use of surrogate measures. (F) Criteria considered in choice of comparator: therapeutic. (G) Use of subgroup analyses. (H) Inclusion and choice of secondary efficacy parameters. (I) Definition of unmet medical need. (J) Use of biomarkers to monitor patient outcomes. HTA, health technology assessment. *Graph produced by author using data from Wang et al. (25) in which a questionnaire-based survey was conducted among regulators (n = 7) and HTA agencies (n = 8) between August and September 2016.

Tripartite Early Dialogues

In several markets, systems exist for pharmaceutical manufacturers to seek advice from regulators during the design of their clinical development programs (47, 48). Although the advice provided through these so-called scientific advice procedures is usually not legally binding (48), adherence to the recommendations can minimize the risk that regulators will later raise objections during assessment of the corresponding marketing authorization applications. For example, an analysis by Hofer et al. revealed that from 2008 to 2012, 85% of applications that received and followed early scientific advice by the EMA were ultimately granted marketing authorization compared to only 41% that did not (49). The concept of early dialogue with HTA bodies is relatively new (50). Regardless, this form of engagement offers manufacturers an opportunity to obtain early insight regarding the evidence needs (e.g., safety, efficacy/effectiveness, cost effectiveness, budgetary impact) and how this should be communicated to reduce bottlenecks during product launch. From the perspectives of the regulatory and HTA agencies, early engagement with developers has the potential to improve the efficiency of the decision-making process (11, 51).

Harmonizing this process via tripartite “early dialogue” meetings consisting of regulators, manufacturers, and HTA bodies can increase collaboration and improve understanding among the different parties. Regulatory agencies and HTA bodies can offer joint (parallel) advice (in areas such as defining unmet medical need, analysis methodology, acceptable primary endpoints, etc.) (25) and discuss divergent data needs with the aim of minimizing discrepancies and identifying trade-offs, whereas manufacturers can have a single forum to discuss any potential claims or concerns (52). The opportunity to incorporate patients' and clinicians' perspectives in these discussions could further enrich the data needs (11, 53).

Tripartite advisory models have been implemented in different jurisdictions (Table 2). In 2010, the EMA commenced a pilot on Parallel Scientific Advice (PSA) together with HTA bodies (44). Moreover, in May 2014, the EMA released a “Best Practice Guidance for Pilot EMA-HTA Parallel Scientific Advice Procedures” for public consultation (63), and the EMA-HTA PSA was formalized in 2015. Some of the issues specified in the guidance document were the following: (1) all medicinal products are eligible irrespective of their eligibility for the central procedure; (2) it is the applicant's choice which HTA bodies could participate (usually the number of HTA bodies participating should not exceed 5); (3) the invited HTA bodies are not obliged to participate; (4) a common briefing document is used; (5) advice is not legally binding (however, when regulators give scientific advice, it is based on the current state of the art in medicines development). While they recognize that due to evolving scientific knowledge, an alternative approach to that advice may be appropriate; where companies choose not to apply the advice, they are requested to justify clearly their position in any subsequent marketing authorization application) (52); (6) the process is confidential; and (7) the Administrative work is done by the EMA (63). As of December 2015, the total number of completed procedures for the EMA-HTA scientific advice was 63 (44). An analysis of 43 PSA procedures showed that the HTA bodies with highest representation were NICE (involved in 90% of all PSA procedures), followed by the German Federal Joint Committee (65%), Italian Medicines Agency (45%), Dental and Pharmaceutical Benefits Agency (Sweden) (35%), National Authority for Health (France) (19%), Main Association of Austrian Social Security Institutions (10%), Catalan Agency for Health Quality and Assessment (10%), and National Institute for Sickness and Invalidity Insurance (Belgium) (3%) (45).

TABLE 2

Table 2. Overview of early tripartite dialogues and parallel submission interactions.

An analysis of the first 11 EMA-HTA PSAs also showed that the majority of the questions posed by developers were in relation to the design of clinical studies such as endpoint and comparators (Supplementary Figure 1) (64), a trend that is expected. Moreover, analysis of 518 answers provided by regulators and HTA assessors in 31 PSAs conducted during 2010–2015 revealed that full agreements, partial agreements, and disagreements were reached in 61, 23, and 16% of responses, respectively (45). In particular, the occurrence of divergence in recommendations provided were seen to be lowest for the study patient population and highest regarding selection of comparator (45). Where divergence in recommendations have occurred, notable cases of successful compromises in product development have also been reported from parallel EMA-HTA PSAs. For example, in one instance, a company preparing to launch a novel therapy for chronic obstructive pulmonary disease (COPD) proposed utilizing a licensed comparator in its pivotal trial. The EMA sided with this proposal; however, an HTA representative who was present requested a different comparator not licensed for use, yet routinely used. The solution was to introduce a new arm of the pivotal study to include both comparators, meeting the recommendations from both advisors (65). In another case, a pharmaceutical company had developed a novel therapy as a first-in-class treatment for a rare oncological disease. Since no other product had previously been licensed for this indication, the company proposed standard of care as its comparator, and the EMA agreed. However, HTA bodies present requested the use of an off-label active comparator and the pharmaceutical company settled on this pathway (53).

When Tafuri and colleagues (66) analyzed the uptake of the comparator recommendations at the time of 31 PSAs (during 2010–2015) within the actual development, they found that manufacturers implemented comparators to address both the needs of regulators and of at least one HTA body in 12 out of 21 studies (almost 60%). Studies in which manufacturers followed the regulators' and >50% of the HTA bodies' advice were 8/21 (38%), while those following exclusively the regulatory advice were 7/21 (about 30%). Only in two studies did the manufacturer implement recommendations, neither from the regulators nor from the HTA advice. Moreover, it was found that changes were never implemented solely based on the HTA advice. For the primary endpoint in all included studies (23 out of 23) manufacturers implemented both the requests of the regulators and at least one HTA body. In 15 studies out of those 23, the manufacturer complied with the advice of both the regulators and >50% of the HTA bodies. These data suggest, to some extent, that manufacturers seem to be more inclined to satisfy the regulatory advice (66). In 2017, the EMA-HTA PSA was replaced with the EMA-HTA Parallel Consultation (PC) process, the key update being the incorporation of the European Network for Health Technology Assessment (EUnetHTA) and Early Dialogue Working Party (EDWP), although all other aspects of the PSA remained largely unchanged (23).

Overall, regulators and HTA bodies have expressed positive views about tripartite dialogues (11, 51, 52). Moreover, pharmaceutical manufacturers have identified several benefits with the process including reducing development program risk and creating common multistakeholder understanding of unmet medical need and acceptability criteria for innovative study design approaches (53). Nonetheless, the value of any dialogue is dependent on the stability of the advice or when it is provided. Thus, to derive maximum benefit, planning is critical. For example, if advice is sought too early, issues that may arise after the clinical trials have commenced may not be addressed or it would be costly to revise and collect new information (65). On the other hand, if sought too late, there may be insufficient time to complete the clinical trials before the target market date. For a potential product that is not a strong candidate for an early access pathway, it is suggested that the ideal time for a sponsor to initiate the dialogue is during phase II evaluation, or just after achieving proof of concept, and at least 6 months prior to planned phase III initiation (53). Regardless, earlier consideration of strategic advice in the phase I setting may be necessary to discuss assumptions and concepts for potential accelerated development opportunities before data generation. Overall, increased confidence in the PSA dialogue process is likely to be achieved via provision of formal (e.g., written) feedback (3). However, projects have tended to adopt different approaches. For example, whereas NICE-MHRA PSA Program (57) and the Tapestry Network pilots (60) provide formal written postconsultation reports, in the case of the Swedish authorities, the responsibility for documenting any discussions lies with the applicants themselves (3). It is also useful for manufacturers pursuing tripartite dialogue to recognize that any advice provided is contextualized within existing knowledge, and this may evolve as scientific and clinical understanding progresses.

Despite the potential benefits of early tripartite dialogues, some perceived disadvantages of the process include the fact that there are no formal mechanisms for addressing divergence between the parties. Moreover, the desire to achieve one consolidated position from HTA bodies and regulatory agencies has been opposed in some settings, as this is viewed to limit the rights of individual agencies to develop and express their own independent views (54). Another potential setback with tripartite meetings is that they may present additional financial hurdles for pharmaceutical companies, as these dialogues are usually provided as a fee-for-service (23, 57). Regulatory capture could also be seen to present a conflict of interest as early interactions can imply that regulatory and HTA bodies are potentially engaged in codevelopment of medicines (65). Furthermore, there are concerns that the role of HTA agencies as final gatekeepers may be compromised through early involvement with developers. However, according to McAuslane et al., if the current challenges with early dialogue are properly addressed, it “is the process that will likely provide the greatest return on investment of time and effort to identify, develop, review, and recommend important new medicines, especially those that address an unmet medical need” (67).

Parallel Submission (Review)

Parallel submissions seek to reduce the time between regulatory and reimbursement decisions by aligning their review processes. In Australia, as part of a Memorandum of Understanding, a parallel process of regulatory [Therapeutic Goods Administration (TGA)] and reimbursement [through Pharmaceutical Benefits Advisory Committee (PBAC)] submission has been implemented since 2011 (Table 2) (68). While a product must still be listed on the Australian Register of Therapeutic Goods before it is listed on the Pharmaceutical Benefits Scheme (PBS), the Australian Drug Evaluation Committee (ADEC) and (PBAC) can now receive submissions for a product in parallel. A PBS listing cannot occur prior to the product being listed on the Australian Register of Therapeutic Goods for the relevant indication. Moreover, if the final TGA approval for a product is received before a PBAC recommendation has been made, the PBAC secretariat will check that any proposed PBS listing is fully consistent with the final TGA registration (55). If there are any discrepancies, the PBAC reconsiders its evaluation. By 2012, five products (linagliptin, testosterone solution, ivabradine, mycophenalate sodium, and rifaximin) had their PBAC decisions deferred until ADEC recommendations were made (69). Analysis of regulatory and HTA review found that when TGA took a longer than average time to review products, those products typically received a negative recommendation from PBAC, although it was unclear whether similar issues were raised by both agencies (70). The usefulness of parallel submission has been highlighted in one case study involving pembrolizumab—a medicine used to treat melanoma that has spread or cannot be removed by surgery (advanced melanoma) or to prevent postsurgery relapse. The manufacturer put the medicine through parallel review. This resulted in the listing of pembrolizumab on the PBS, only 4.5 months after TGA approval (71). Ordinarily, when undertaken sequentially, the median time between a positive TGA recommendation and PBS listing has been found to exceed 30 months (69). An analysis by the Center for Innovation in Regulatory Science (CIRS) on the appraisals of 38 New Active Substances (NASs) introduced between 2014 and 2018 suggested that the TGA/PBAC parallel process may have been instrumental in the shorter time between regulatory approval to HTA decision in Australia (median = 17 days) compared to Canada (median = 161 days), France (median = 202 days), England (median = 311 days), Germany (median = 123 days), Scotland (median = 173 days), and Sweden (median = 158 days) (56).

In Canada, a manufacturer can submit for a Canadian Agency for Drugs and Technologies in Health (CADTH) Common Drug Review before a Health Canada Notice of Compliance (NOC) is issued. For the Health Canada/CADTH parallel review process, the submission to CADTH can occur up to 180 days before the date of anticipated NOC from Health Canada (72). This accelerates the process since the Canadian Drug Expert Committee can release its reimbursement recommendation to CDR immediately following the regulatory decision. A review of 56 NASs appraised by CADTH from 2014 to 2016 showed that parallel review reduced the time from regulatory approval to HTA recommendation. The median time from Health Canada approval to CADTH recommendation was 158 days for drugs undergoing parallel review (n = 22) compared to 377 days for drugs undergoing sequential review (n = 34) (73).

In 2010, the US Food and Drug Administration (FDA) and Centers for Medicare and Medicaid Services (CMS) announced the FDA-CMS Parallel Review pilot program for medical devices (61). Through the program, manufacturers can request initiation of a CMS national coverage determination (NCD) while the product is still under FDA review. After 5 years, the program's impact was deemed to have been minimal, and interest among manufacturers remained low (74). In particular, only one device (Exact Sciences' Cologuard test, a multitarget stool DNA test developed for noninvasive screening for colorectal cancer) was approved through the process. The time from premarket approval submission to the final NCD was 489 days compared to an average of 612 days for other NCDs issued in 2013 (75). While this suggested that parallel-review process could shorten the expected interval from regulatory approval to coverage determination, the experience with Cologuard was only a single case. The only other product known to have undergone the pilot parallel review was Medtronic's Symplicity renal denervation system. However, the process was not completed as the device's phase III trial (SYMPLICITY HTN-3) failed to meet its primary efficacy endpoint (76). In 2016, the FDA-CMS Parallel Review was fully implemented and extended indefinitely (62). Since then, increased interest in the parallel review program among developers has been reported, and in 2017, Foundation Medicine's FoundationOne CDx next generation sequencing (NGS)-based test was approved under the scheme (77).

In May 2019, the Medicines Evaluation Board (MEB) and the Netherlands Healthcare Institute (ZIN) launched their pilot “Parallel Procedures MEB-ZIN.” The stated objective is to “shorten the time from registration to reimbursement of a medicine” (59). The “Parallel Procedures MEB-ZIN” will commence in mid-2020. However, by March 2020, two manufacturers [Insmed BV for their amikacin liposomal inhalation suspension (ALIS) (Arikayce^®) and Novo Nordisk BV for their oral dosage form of Semaglutide (Rybelsus^®)] had registered their products to undergo the parallel review process (59).

Overall, one major challenge with parallel review is that if a product fails to obtain regulatory approval, it renders the work of HTA bodies redundant and a waste of time and resources. In Australia, if regulatory approval is not granted for a product that goes through parallel review, the sponsor company is made to pay a cost-recovery fee to compensate for the resource used for HTA evaluation (70). This may not be ideal for developers. Thus, to minimize such an occurrence, it may be useful that the HTA's initial review centers on the less resource-intensive components so that the time between assessments remains shortened while at the same time not committing too many resources in the event of negative regulatory outcome. Moreover, the institution of a mechanism to select technologies that are most likely to secure regulatory approval to undergo parallel review might also be essential. Regardless, without the requisite data to meet the evidentiary needs of regulators and HTA bodies, parallel submission may not always lead to earlier market access, as an unfavorable review outcome could still occur (3). Hence, a strategy of combining alignment of evidentiary data needs and early dialogue may be necessary to ensure that trials are designed in such a way that the data necessary to meet the needs of both agencies are collected. One such initiative is the EXCITE Programme by The MaRS Excellence in Clinical Innovation and Technology Evaluation, which brings together a broad spectrum of research under one harmonized platform based on relationships brokered with academic health research facilities across the Ontario province in Canada. Through EXCITE, medical devices undergo a combination of clinical testing and HTA review in order to obtain the evidence needed for both federal licensing and provincial health system adoption (78).

Adaptive Licensing Pathways

Since the emergence of drug regulation, approval mechanisms have been challenged by the need to achieve a balance between ensuring timely access for patients without compromising safety (79, 80). The traditional regulatory paradigm is characterized by well-defined structures and rigid processes that require several years of research, development, and authorization for a medicine to reach the market (81). However, this mechanism is criticized as outdated and that it ignores the complexities of health technologies, as well as the diversity in population features and disease progression (3). Hence, there has been a push for a shift from the traditional approach, which relies on extensive testing and the marketing authorization for large groups of patients (with a single decision point focus) to a procedure that employs periodic or staged assessment and reassessment using an evolving evidence base (81, 82). There have been a number of proposals (Supplementary Table 8) advocating for planned adaptive approaches to drug licensing using terms such as “staggered entry,” “adaptive approval,” and “progressive authorization.” However, much of the conceptual framework of adaptive licensing emanated from the New Drug Development Paradigms (NEWDIGS) collaboration that started in 2010 as an initiative of the Massachusetts Institute of Technology (MIT) and was hosted by the MIT Center for Biomedical Innovation (81). The concept has, however, been renamed to “adaptive pathways (APs)” to better reflect a focus on the development and managed introduction of medicines rather than a new way of regulating and authorizing medicines (83). Moreover, over the past years, regulators in different jurisdictions have had several mechanisms in place to facilitate an earlier access of new promising medicines especially in areas of high unmet medical needs and for orphan diseases (84). In Europe, the EMA early access and registration tools include Priority Medicine (PRIME) (85), conditional marketing, and authorization and approval under exceptional circumstances (86), as well as compassionate use exist (87). Hence, the concept of AP is not necessarily a new licensing pathway but a way of getting clinical data in order to design a smart development program to meet the evidentiary needs (82, 83).

In 2014, the EMA launched the AP pilot program inviting participation from companies that had candidate products that were early in clinical development (88). The AP is based on key principles such as the need for early dialogue (collaboration) with multiple stakeholders (regulators, HTA agencies, and patient and healthcare professional representatives) to identify a subset of patients expected to present a favorable benefit–risk profile as well as significant emphasis on the use of real-world data to supplement clinical trial data. Other criteria for the selection of products for the APs pilot was an iterative development plan. The iterative development plan can follow two registration scenarios: (1) starting with a marketing authorization for a well-defined subpopulation and expanding the population (“widening of the indication” scenario) or (2) obtaining a Conditional Marketing Authorization, whether based on surrogate endpoints or not and conducting confirmatory studies afterwards (“prospectively planned reduction of uncertainty” scenario) (Supplementary Figure 2) (89). In the AP pilot project, EMA received 62 applications out of which seven progressed to a formal scientific advice (one) or parallel regulatory-HTA scientific advice (six). The reasons for non-acceptance into the pilot included (1) development programs that did not have scope for expansion and iteration, (2) proposals for areas without unmet need, and (3) late stage development programs (where no changes to the plan could be effected) (88).

In general, the “safe harbor” environment of APs is intended to foster an increasing willingness to share information, data, and expertise, thereby improving collaboration between the different agencies (83, 89). For example, actual and modeled clinical development and licensing programs of three case studies as part of the Janus initiative concluded that the adaptive licensing approach increases stakeholder commitment (90). Regardless, to achieve the intended acceleration in patients access, the APs need to identify ways to reduce the time lag between marketing authorization and reimbursement. This further reinforces the importance of early dialogue with HTA bodies and regulatory agencies as well as alignment of evidentiary needs (53). To facilitate such interactions, the Innovative Medicines Initiative (IMI) through the ADAPT-SMART initiative assembled together stakeholders to develop better ways to achieve APs (91).

To ensure efficient adoption of APs, appropriate legal structures need to be in place (80). For example, a report by Oye et al. indicated that attorneys from the US FDA, EMA, and the Singapore Health Sciences Agency found that existing statutes in their jurisdictions provided authority for adaptive licensing, although gaps were noted in the Canadian legislation (92). Moreover, the success of APs requires a more “system-wide” approach including, for example, the willingness of patients to participate in clinical research to evaluate benefit/risk and determine if new medicines were effective and how different stakeholder perspectives are reflected in the decision-making process (90). As highlighted by Schulthess et al. (93), the bigger challenge facing the adoption of APs borders on how to incorporate different decision-making processes into AP methodologies “to ensure the appropriate balance is struck between earlier access to new medicines, a given regulator's willingness to facilitate that to occur, a healthcare provider's willingness to accept more focused data before prescribing a new medicine, as well as the provider's correlating willingness to restrict their off-label prescribing practices and to participate in real-world clinical research to progressively reduce uncertainties, a given payer's willingness to purchase such medicines, and having strong multifaceted postauthorization systems in place to facilitate all of this in as safe and dependable a manner as possible.”

Post-authorization Data/Evidence Generation

At the time of marketing authorization, the available information relating to a medicine may not yet be sufficient to fully assess the benefit/risk profile to the desired degree of certainty (94, 95). Therefore, regulatory agencies may require the generation of additional data, for example, in the form of clinical studies after authorization. Nonetheless, there is considerable opportunity for HTA bodies and regulatory agencies to collaborate toward providing guidance on the design of postapproval studies that can fulfill both of their needs (19, 25). This would be necessary to avoid developers' duplication of efforts in the postlaunch evidence generation phase, for example, with respect to the planning and execution of postauthorization efficacy studies (PAESs) and postauthorization safety studies (PASS) (44). In this context, a collaboration between EMA and EUnetHTA on postauthorization data collection commenced in March 2011. The discussions began from collaboration on two projects: European Network of Centers for Pharmacoepidemiology and Pharmacovigilance (ENCePP), under the leadership of EMA, and the EVIDENT database—a database containing evidence information on new technologies. Through this collaboration, reports on the feasibility of conducting postapproval studies in Europe were produced as well as guidelines on the necessary methodological standards to execute such studies (96). In 2016, the EMA instituted the Scientific Advice Working Party (SAWP)/The Pharmacovigilance Risk Assessment Committee (PRAC) joint scientific advice for the PASS/PAES studies (97).

There are additional opportunities for HTA bodies and regulatory agencies engagement around optimization of real-world data (RWD) generation such as the use of patient registries (98) or the opportunity to share periodic benefit–risk assessment reports and therapeutic value reassessments. This would include alignment on key areas such as outlining the definition of data to be collected (i.e., minimum dataset) in registries (19). In this context, the EMA launched an initiative with EUnetHTA representation in 2015 to facilitate the establishment of patient registries as well as introducing and supporting a systematic and standardized approach to their contribution to serve regulatory and HTA needs (99). There is also opportunity for collaboration regarding the use of additional sources for the collection of RWD, such as data derived from electronic patient records. Within this space, the IMI GetReal project (2013–2016) was a multistakeholder initiative, involving regulators, HTA agencies, patient organizations, academics, and industry that sought to propose and create tools to support new robust methods of RWE synthesis for use in medicine development and decision-making throughout the product cycle including the initial regulatory and postapproval phases. While this project has ended, the work continues via the IMI GetReal initiative launched in 2018 (100).

Barriers and Challenges to Harmonization

Despite the increasing interest and potential for synergies between HTA bodies and regulatory agencies, several barriers, hurdles, and challenges were identified via the survey and systematic review. Some survey respondents indicated that the intensity of cooperation is low, as there is no institutional framework for cooperation and it all comes down to individual initiatives. Moreover, there is a need to build trust and understanding between all relevant stakeholders including HTA bodies, regulatory agencies, payers, manufactures, clinicians, and patients through effective communication and transparency (11, 51). In particular, mechanisms for continuous open dialogue are important to foster the development of stronger relationships and minimize misconceptions. For example, some manufacturers have expressed misgivings about harmonization initiatives for fear that HTA bodies/payers might be able to influence market authorization decisions and vice versa (51). Improved knowledge of each other's functions, roles, and remits may also reduce misunderstandings that may lead to unintended policy consequences that can create misalignment (101).

Furthermore, greater understanding of each agency's remits and processes provides a medium for conveying realistic expectations about the extent of coordination and agreement that is achievable. An HTA body responded in the survey that it is sometimes “difficult to separate evidence review from policy issues and financial consideration” when engaging with regulators. There are also practical differences in areas such as evidentiary requirements that need to be acknowledged. For example, HTA bodies may be hesitant to accept trials using placebo control when an active comparator exists, although this may be acceptable to regulators (2, 3, 25). On the other hand, it is conceivable that payers and HTA bodies may agree to some of the clinical outcomes specified by regulators while also highlighting the need for additional data such as those related to quality-of-life and long-term effects (46). These potential differences need to be anticipated so that efforts can be channeled into areas with greatest potential for harmonization. Furthermore, it is also important to establish leadership and clearly defined roles and responsibilities to minimize a culture of blame if the intended outcomes of harmonization are not attained (101). For example, regarding the use of CER and relative efficacy studies, it is critical to outline how this should be implemented, who will be responsible, how will such research be funded (and by whom), and how will the data generated be disseminated. It must be anticipated that allocation of responsibilities can be a sensitive area, as different agencies may seek to fiercely guard their existing responsibilities and operations (54, 101).

Harmonization is also likely to introduce changes to the current regulatory and reimbursement pathways that would only be possible with the implementation of new supportive structures and, if required, legislations (80). For example, in some cases, legislative amendments may be needed since the roles of regulators are specifically defined in law or by their governments and HTA bodies may or may not be fully established in law. Indeed, one survey respondent indicated that within their jurisdiction, “HTA assessment is not mandatory and regulators are not obligate to cooperate with us.” In general, there are also concerns that regulatory structures need revamping, as they were instituted when no formal reimbursement mechanisms were in place (14). Existing laws may hinder cooperation between the respective agencies, as one survey respondent indicated that in their country, “social laws does not provide many opportunities to do so.” Moreover, the sharing of confidential information between different stakeholders may be limited by law. One respondent indicated that “although there are potential synergies there is limited scope for joint operations. Horizon scanning was explored but it was not possible for the regulatory agency to share this information.” There are also concerns among companies about how proprietary information may be handled by the different agencies given their unique working styles (101). In particular, regulators are usually bound by strict confidentiality codes, whereas payers have varied frameworks that center around transparency in decision-making (6). Thus, to support harmonization initiatives, a system is needed that offers interagency exchange of information to allow each to learn how the other is using available data to inform decision-making. For example, a central component of APs is the need for continuous data collection. In this context, information related to appropriate usage (compliance) as well as those related to effectiveness and safety collected by payers may be relevant to regulators to inform periodic regulatory assessment (3). To facilitate the sharing of data, some pilot programs have been tested as part of the EMA's “Road Map to 2015” (12). An optional information sharing process has also been established to permit Health Canada and CADTH to exchange information regarding a drug under review, for submissions filed with CADTH on a pre-NOC basis. Sponsors are encouraged to agree to this data sharing mechanism to facilitate the parallel review process (102). The FDA-CMS Parallel Review also has an inbuilt mechanism for cross-sharing relevant data (61).

Another challenge to broader harmonization is the availability of resources required for regular cooperation between the agencies. For example, one survey respondent indicated that “we are not in the same city, so face-to-face meetings need resources,” and others hinted that resources for joint training is limited. In this context, several initiatives have relied on user fees (e.g., scientific advice), and alternative funding models may be needed to ensure their sustainability (23, 57, 102). Moreover, a balance needs to be struck in the pursuit of national/regional collaborations as opposed to international harmonization initiatives. This is because while initiatives with an international scope (e.g., Green Park Collaborative) may offer high value to industry because their outputs could apply across multiple markets, and may reduce duplication of similar efforts in multiple jurisdictions, they are likely to face jurisdictional challenges as well as issues related to context-specific disparities (e.g., differences in standard of care and relevant comparators, economic and political priorities, and healthcare delivery systems).

Discussion

HTA–Regulatory Harmonization: Current Progress and Challenges

Improvements in health technology raise hopes for better patient outcomes and a more efficient delivery of health care. However, the processes of diffusion and implementation of new health technologies require a series of steps and engagement with different stakeholders, and many healthcare systems continue to struggle with finding ways to ensuring prompt access to safe and efficacious healthcare products for their patients. A large part of this challenge has been ascribed to the two sequential processes of regulatory and reimbursement decision-making, which is deemed to be ill-suited to facilitate timely, well-informed patient access, stimulate drug development, and simultaneously ensure routine collection and evaluation of all relevant information on benefits and risks (83). Against this background, the alignment of regulatory and HTA processes has been proposed as a means to remedy the situation toward increasing the effectiveness of decision-making, mitigating the disconnect between different agencies and their stakeholders, as well as promoting public trust in the review processes.

In this systematic review and cross-sectional survey HTA bodies, we found that there has been progress over time in narrowing the gap in evidentiary requirements between HTA bodies and regulatory agencies. Different models and approaches aimed at fostering closer interactions between agencies were also identified. The initiatives described often require organizations to work outside of their traditional remits, to engage with different stakeholders, and, in some instances, to modify their processes. Regardless, the expected level of change necessary to adopt different models or the practicalities (time and resources needed) of their implementation vary. Broadly, we have considered five key areas: early tripartite dialogues, alignment of evidentiary needs, parallel submissions (reviews), adaptive licensing pathways, and post-marketing data generation. However, these mechanisms must feedback into each other, and they should not be viewed as mutually exclusive.

Concerns over potential merger of regulatory and reimbursement functions to have adverse impacts on quality assurance processes as well as distorting the free market dynamics may explain some of the hesitance to greater collaboration between these agencies (22). Despite this, there is considerable scope to develop economies of scale particularly with respect to evidence generation. Harmonization/alignment of evidentiary requirements is likely to be feasible and most effective when targeted at common requirements that are assessed by both HTA agencies and regulators (i.e., clinical outcomes). However, greater friction is expected in areas that are more specific to each agency (e.g., economic, budgetary impact). Thus, in promoting harmonization, it is important to include outcome measures within trials that satisfy both agencies' needs. For example, regulators must ensure acceptable choice of surrogate endpoints, particularly those that have demonstrated good correlation with hard outcomes (such as mortality). Since, most HTA–regulator discussions are related to specific trial design and the selection of appropriate comparators and outcomes, a focus on product-level harmonization appears appealing to foster closer evidentiary alignment. Regardless, emphasis on initiatives addressing the evidentiary expectations of HTA bodies and regulatory agencies at the level of therapeutic areas may offer greater return at maximizing the efficient use of scarce organizational resources and for generating outputs that are of wider importance. Furthermore, significant efficiencies for both companies and the reviewing agencies may be gained via development of condition specific as well as general methodological guidance at the international level. Overall, future discussions/initiatives around alignment of evidentiary requirements for HTA bodies and regulatory agencies must recognize that integration of certain scientific elements are possible and appropriate, but others may not be suitable for alignment and that forcing such alignment could be defeatist.

Models such as APs, although still in early development and largely applied to pharmaceuticals, have potential to be adopted for medical devices as well. For example, the continued reassessment characteristics of adaptive licensing would thus align well in assessing the impact of incremental innovation and real-life usage of the medical device. Some results of parallel submission programs suggest that such mechanisms may reduce time between regulatory approval and reimbursement decisions. Moreover, some analyses have revealed the positive effect of tripartite dialogues on clinical development programs (66), although mechanisms for addressing divergence need to be clarified. Moreover, the long-term impacts of such measures are yet to be evaluated. The increasing desire for the use of real-world evidence to supplement RCT data also provides further opportunities for increased alignment between HTA and regulatory agencies throughout the product cycle, but methods and standards require further refinement with time.

A number of practical enablers, challenges, and barriers were also identified in the literature and survey, which require attention to improve harmonization between HTA and regulatory agencies. In particular, the need to build trust and buy-in from all stakeholders is important. Moreover, greater understanding of each institution's processes as well as objective characterization of each agencies' needs, responsibilities, and resources is important to building collaboration. Regardless, mechanisms need to be better developed on how to secure the exchange of confidential data between different agencies. Further consideration of resources available should inform models and approaches that are pursued. It is expected that each approach to harmonization will provide different benefits and challenges. Thus, in deciding on a harmonization approach, reflection on local contextual factors including healthcare system, political factors, and resource availability are important. While a desire for international harmonization initiatives is understandable due to their potential to generate outputs that could apply across different markets, their inherent limitation such as likelihood of lacking context-specific focus need to be recognized (22).

Study Limitations

This study has several limitations. First, while inferences were made to specific cases, the evidence synthesis was largely qualitative and descriptive in nature, thereby amenable to interpretation bias. The included reports were mainly reviewed by a single author given the available resources and time constraints. Given the very broad nature of the topic (which resulted in substantial search hits, the majority of which were excluded), it is possible that some reports may have been missed. However, we do not think that further major themes outside of what has been discussed would have emerged. In particular, most of the themes identified from the literature were further highlighted in the survey. A number of deliberations such as PSAs are usually treated as confidential (23, 57). Thus, the information extracted from published literature may not be entirely reflective of the processes and outcomes. Thus, to gain further insight, a symposium organized by the HTx consortium (https://www.htx-h2020.eu/) has been planned for October 2020 to engage both HTA bodies and regulatory agencies on the matter of harmonization/alignment. For example, the views of different agencies on whether alignment is a more achievable goal than complete harmonization will be solicited at the forum. Furthermore, as the systematic review was limited to articles published in the English language, this may limit the generalizability of the findings. In the survey, the response rate from the regulatory agencies was low (<20%). However, the final number of included regulatory agencies (n = 6) is comparable to that from a 2016 survey by Wang et al. (n = 7) (25). Lastly, the survey respondents were mainly from Europe, and their experiences may not be generalizable to other jurisdictions.

Conclusions

There has been progress over time in narrowing the gap in evidentiary requirements for HTA and regulatory agencies. In many European countries as well as in US, Canada, and Australia, a formal link of collaboration between HTA bodies and regulatory agencies has been instituted. To date, several mechanisms such as early tripartite dialogues, parallel submission (review), adaptive pathways to licensing, and postauthorization data generation have been explored as avenues for improving collaboration. A number of pilot initiatives have also shown positive effects of these models to reduce the time between regulatory and HTA decisions, which may translate into faster patients' access to life-saving therapies. Thus, future approaches aimed at improving harmonization/interaction between HTA bodies and regulatory agencies should build on these existing models/mechanisms while examining their long-term impacts. Several barriers including legal, organizational, and resource-related factors were also identified, and these need to be addressed to achieve greater alignment in the current regulatory and reimbursement landscape.

Author Contributions

RO-A and MLDB contributed to study conception, analysis, and manuscript preparation. CEH contributed to manuscript preparation and revision for intellectual content. All authors read and approved the final manuscript version prior to submission.

Funding

This manuscript emanated from research undertaken within the HTx project. The HTx project was funded by the European Union's Horizon 2020 research and innovation program under grant agreement no. 825162.

Conflict of Interest

RO-A, CEH, and MLDB are employees of the Copenhagen Centre for Regulatory Sciences (CORS). CORS is a cross-faculty university anchored institution involving various public (Danish Medicines Agency, Copenhagen University) and private stakeholders (Novo Nordisk, Lundbeck, Ferring Pharmaceuticals, LEO Pharma) as well as patient organizations (Rare Diseases Denmark). The center is purely devoted to the scientific aspects of the regulatory field and with a patient-oriented focus and the research is not company-specific product or directly company related. In the last 5 years, CORS has received funding from Novo Nordisk, Lundbeck, Ferring Pharmaceuticals, and LEO pharma for projects not related to this study.

Acknowledgments

The study authors are grateful to the members of the HTx consortium for their comments on an earlier draft of this manuscript. We also thank Milou Hogervorst and Ayla Lokhorst for their support with organizing the online survey.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fmed.2020.582634/full#supplementary-material

References

1. Clark D, Dean G, Bolton S, Beeson B. Bench to bedside: the technology adoption pathway in healthcare. Health Technol. (2019) 10:537–45. doi: 10.1007/s12553-019-00370-z

CrossRef Full Text | Google Scholar

2. Eichler HG, Bloechl-Daum B, Abadie E, Barnett D, Konig F, Pearson S. Relative efficacy of drugs: an emerging issue between regulatory agencies and third-party payers. Nat Rev Drug Discov. (2010) 9:277–91. doi: 10.1038/nrd3079

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Tsoi B, Masucci L, Campbell K, Drummond M, O'Reilly D, Goeree R. Harmonization of reimbursement and regulatory approval processes: a systematic review of international experiences. Expert Rev Pharmacoecon Outcomes Res. (2013) 13:497–511. doi: 10.1586/14737167.2013.814962

CrossRef Full Text | Google Scholar

4. Katz R. FDA: evidentiary standards for drug development and approval. NeuroRx. (2004) 1:307–16. doi: 10.1602/neurorx.1.3.307

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Panteli D, Edwards S. Ensuring Access to Medicines: How to Stimulate Innovation to Meet Patients' Needs? (2018). Available online at: http://www.euro.who.int/__data/assets/pdf_file/0009/379701/PB29_AUSTRIA_web_13082018.pdf (accessed March 30, 2020).

PubMed Abstract | Google Scholar

6. Goodman C. HTA 101: Introduction to Health Technology Assessment. Bethesda, MD: National Library of Medicine (US) (2014).

7. Angelis A, Lange A, Kanavos P. Using health technology assessment to assess the value of new medicines: results of a systematic review and expert consultation across eight European countries. Eur J Health Econ. (2018) 19:123–52. doi: 10.1007/s10198-017-0871-0

CrossRef Full Text | Google Scholar

8. Iskrov G, Dermendzhiev S, Miteva-Katrandzhieva T, Stefanov R. Health economic data in reimbursement of new medical technologies: importance of the socio-economic burden as a decision-making criterion. Front Pharmacol. (2016) 7:252. doi: 10.3389/fphar.2016.00252

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Panzitta M, Bruno G, Giovagnoli S, Mendicino FR, Ricci M. Drug delivery system innovation and health technology assessment: upgrading from clinical to technological assessment. Int J Pharm. (2015) 495:1005–18. doi: 10.1016/j.ijpharm.2015.09.026

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Liberti L, Pichler F, Walker S. Preparing for regulator review and reimbursement decisions—a case for cooperation between regulatory authorities, sponsors and health technology assessment agencies. Pharm Med. (2009) 23:263–7. doi: 10.1007/BF03256778

CrossRef Full Text | Google Scholar

11. Henshall C, Mardhani-Bayne L, Fronsdal KB, Klemp M. Interactions between health technology assessment, coverage, and regulatory processes: emerging issues, goals, and opportunities. Int J Technol Assess Health Care. (2011) 27:253–60. doi: 10.1017/S0266462311000262

CrossRef Full Text | Google Scholar

12. European Medicines Agency. Road Map to 2015: The European Medicines Agency's Contribution to Science, Medicines and Health. London: EMA (2011).

13. Mossialos E, Mrazek M, Walley T. Regulating Pharmaceuticals in Europe: Striving for Efficiency, Equity and Quality. Copenhagen: European Observatory on Health Systems and Polices (2004).

Google Scholar

14. Grignolo A, Siu A. Improving drug development and patient access with the right people, processes, and culture: what needs to happen right now to bring better medicines to the patients who need them. Ther Innov Regul Sci. (2019) 53:398–402. doi: 10.1177/2168479018814224

PubMed Abstract | CrossRef Full Text | Google Scholar

15. IQVIA Institute for Human Data Science. Global Oncology Trends 2018: Innovation, Expansion and Disruption. (2018). Available online at: www.iqvia.com/institute/reports/global-oncology-trends-2018 (accessed January 20, 2020).

16. OECD. New Health Technologies: Managing Access, Value and Sustainbility. Paris: OECD Publishing (2017).

Google Scholar

17. The World Bank. New Country Classifications by Income Level: 2019–2020. (2019). Available online at: https://blogs.worldbank.org/opendata/new-country-classifications-income-level-2019-2020 (accessed January 02, 2020).

18. Bradley EH, Curry LA, Devers KJ. Qualitative data analysis for health services research: developing taxonomy, themes, and theory. Health Serv Res. (2007) 42:1758–72. doi: 10.1111/j.1475-6773.2006.00684.x

PubMed Abstract | CrossRef Full Text | Google Scholar

19. European Commission. HTA Network Reflection Paper on “Synergies Between Regulatory and HTA Issues on Pharmaceuticals”. (2016). Available online at: https://ec.europa.eu/health/sites/health/files/technology_assessment/docs/ev_20161110_co06_en.pdf (accessed March 30, 2020), European Commission.

20. Enzmann H. New trends and challenges in the European regulation of innovative medicines. Regul Toxicol Pharmacol. (2016) 80:314–20. doi: 10.1016/j.yrtph.2016.05.033

CrossRef Full Text | Google Scholar

21. Office of Health Economics (OHE). Workshop on HTA and Regulatory Review. (2010). Available online at: https://www.ohe.org/news/workshop-hta-and-regulatory-review (accessed January 15 2020).

22. Hutton J, Trueman P, Facey K. Harmonization of evidence requirements for health technology assessment in reimbursement decision making. Int J Technol Assess Health Care. (2008) 24:511–7. doi: 10.1017/S0266462308080677

PubMed Abstract | CrossRef Full Text | Google Scholar

23. European Medicines Agency. Parallel Consultation With Regulators and Health Technology Assessment Bodies. (2019). Available online at: https://www.ema.europa.eu/en/human-regulatory/research-development/scientific-advice-protocol-assistance/parallel-consultation-regulators-health-technology-assessment-bodies (accessed January 15 2020).

24. Balaisyte L, Joos A, Hiligsmann M. Early dialogue in Europe: perspectives on value, challenges, and continuing evolution. Int J Technol Assess Health Care. (2018) 34:514–8. doi: 10.1017/S0266462318000545

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Wang T, McAuslane N, Liberti L, Leufkens H, Hovels A. Building synergy between regulatory and HTA agencies beyond processes and procedures-can we effectively align the evidentiary requirements? A survey of stakeholder perceptions. Value Health. (2018) 21:707–14. doi: 10.1016/j.jval.2017.11.003

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Messner DA, Towse A, Mohr P, Garau M. The future of comparative effectiveness and relative efficacy of drugs: an international perspective. J Comp Eff Res. (2015) 4:419–27. doi: 10.2217/cer.15.8

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Institute of Medicine Committee on Comparative Effectiveness Research Prioritization. Initial National Priorities for Comparative Effectiveness Research. Washington, DC, USA: The National Academies Press (2009).

Google Scholar

28. Jonsson B. Relative effectiveness and the European pharmaceutical market. Eur J Health Econ. (2011) 12:97–102. doi: 10.1007/s10198-011-0297-z

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Eichler HG, Abadie E, Breckenridge A, Flamion B, Gustafsson LL, Leufkens H, et al. Bridging the efficacy-effectiveness gap: a regulator's perspective on addressing variability of drug response. Nat Rev Drug Discov. (2011) 10:495–506. doi: 10.1038/nrd3501

CrossRef Full Text | Google Scholar

30. Wildson T. A Comparative Analysis of the Role and Impact of Health Technology Assessment. London: Charles River Associates (2011).

Google Scholar

31. Towse A, Jonsson B, McGrath C, Mason A, Puig-Peiro R, Mestre-Ferrandiz J, et al. Understanding variations in relative effectiveness: a health production approach. Int J Technol Assess Health Care. (2015) 31:363–70. doi: 10.1017/S0266462315000719

CrossRef Full Text | Google Scholar

32. Connolly SJ, Ezekowitz MD, Yusuf S, Eikelboom J, Oldgren J, Parekh A, et al. Dabigatran versus warfarin in patients with atrial fibrillation. N Engl J Med. (2009) 361:1139–51. doi: 10.1056/NEJMoa0905561

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Garber AM, Sox HC. The role of costs in comparative effectiveness research. Health Aff. (2010) 29:1805–11. doi: 10.1377/hlthaff.2010.0647

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Myshko D. Addressing Reimbursement During Development. (2012). Available online at: https://www.pharmavoice.com/article/2012-05-addressing-reimbursement-during-development/ (accessed January 12 2020).

35. Miller HI. A proposal for FDA reform. Nat Rev Drug Discov. (2002) 1:642–8. doi: 10.1038/nrd874

CrossRef Full Text | Google Scholar

36. Woodcock J. Comparative effectiveness research and the regulation of drugs, biologics and devices. J Comp Eff Res. (2013) 2:95–7. doi: 10.2217/cer.13.9

CrossRef Full Text | Google Scholar

37. Patient Protection and Affordable Care Act. Public Law 111–148. Available online at: www.gpo.gov/fdsys/pkg/BILLS-111hr3590enr/pdf/BILLS-111hr3590enr.pdf (accessed January 08, 2020).

38. Messner DA, Mohr P, Towse A. Futurescapes: evidence expectations in the USA for comparative effectiveness research for drugs in 2020. J Comp Eff Res. (2015) 4:385–400. doi: 10.2217/cer.15.6

CrossRef Full Text | Google Scholar

39. Blackstone EA, Fuhr JP Jr, Ziernicki D. Will comparative effectiveness research finally succeed? Biotechnol Healthc. (2012) 9:22–6.

PubMed Abstract | Google Scholar

40. CTMP. Green Park Collaborative. (2020). Available online at: http://www.cmtpnet.org/green-park-collaborative/ (accessed January 12, 2020).

41. European Commission. High level Pharmaceutical Forum 2005–2008 Conclusions and Recommendations. (2008). Available online at: https://op.europa.eu/en/publication-detail/-/publication/4fddf639-47cc-4f90-9964-142757d2515a (accessed February 03, 2020).

42. Berntgen M, Gourvil A, Pavlovic M, Goettsch W, Eichler HG, Kristensen FB. Improving the contribution of regulatory assessment reports to health technology assessments—a collaboration between the European Medicines Agency and the European network for Health Technology Assessment. Value Health. (2014) 17:634–41. doi: 10.1016/j.jval.2014.04.006

CrossRef Full Text | Google Scholar

43. European Medicines Agency. EMA-EUnetHTA Three-Year Work Plan. (2013). Available online at: https://www.ema.europa.eu/en/documents/other/european-medicines-agency-eunethta-three-year-work-plan_en.pdf (accessed January 09, 2020).

44. European Medicines Agency. Report on the Implementation of the EMA-EUnetHTA Three-Year Work Plan 2012–2015. (2016). Available online at: https://www.ema.europa.eu/en/documents/report/report-implementation-european-medicines-agency-european-network-health-technology-assessment_en.pdf (accessed January 23, 2020).

45. Tafuri G, Pagnini M, Moseley J, Massari M, Petavy F, Behring A, et al. How aligned are the perspectives of EU regulators and HTA bodies? A comparative analysis of regulatory-HTA parallel scientific advice. Br J Clin Pharmacol. (2016) 82:965–73. doi: 10.1111/bcp.13023

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Dekker M, Bouvy JC, O'Rourke D, Thompson R, Makady A, Jonsson P, et al. Alignment of European regulatory and health technology assessments: a review of licensed products for alzheimer's disease. Front Med. (2019) 6:73. doi: 10.3389/fmed.2019.00073

PubMed Abstract | CrossRef Full Text | Google Scholar

47. U.S. Department of Health and Human Services Food and Drug Administration. Best Practices for Communication Between IND Sponsors and FDA During Drug Development Guidance for Industry and Review Staff Good Review Practice. (2017). Available online at: https://www.fda.gov/media/94850/download (accessed March 25, 2019).

48. European Medicines Agency. European Medicines Agency Guidance for Applicants Seeking Scientific Advice and Protocol Assistance. London: EMA (2017).

49. Hofer MP, Jakobsson C, Zafiropoulos N, Vamvakas S, Vetter T, Regnstrom J, et al. Regulatory watch: impact of scientific advice from the European Medicines Agency. Nat Rev Drug Discov. (2015) 14:302–3. doi: 10.1038/nrd4621

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Backhouse ME, Wonder M, Hornby E, Kilburg A, Drummond M, Mayer FK. Early dialogue between the developers of new technologies and pricing and reimbursement agencies: a pilot study. Value Health. (2011) 14:608–15. doi: 10.1016/j.jval.2010.11.011

CrossRef Full Text | Google Scholar

51. Fronsdal K, Pichler F, Mardhani-Bayne L, Henshall C, Røttingen JA, Mørland B, et al. Interaction initiatives between regulatory, health technology assessment and coverage bodies, and industry. Int J Technol Assess Health Care. (2012) 28:374–81. doi: 10.1017/S0266462312000591

CrossRef Full Text | Google Scholar

52. European Medicines Agency. EMA-HTA Workshop Bringing Together Stakeholders for Early Dialogue in Medicines Development: Report From the Public Workshop Hosted by the European Medicines Agency (EMA) in London on 26 November 2013. London: EMA (2013).

53. Khan S, Carter M. European medicines agency-health technology assessment parallel consultation platform: an industry perspective. Clin Pharmacol Ther. (2019) 105:822–5. doi: 10.1002/cpt.1337

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Wonder M, Backhouse ME, Hornby E. Early scientific advice obtained simultaneously from regulators and payers: findings from a pilot study in Australia. Value Health. (2013) 16:1067–73. doi: 10.1016/j.jval.2013.07.007

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Department of Health. TGA and PBAC Parallel Process and Requirements. (2018). Available online at: http://www.pbs.gov.au/info/publication/factsheets/shared/tga-pbac-parallel-process (accessed March 25, 2020).

56. Centre for Innovation in Regulatory Science (CIRS). Review of HTA outcomes and timelines in Australia, Canada and Europe 2014–2015. London: CRIS (2017).

57. NICE. Scientific Advice. (2020). Available online at: https://www.nice.org.uk/about/what-we-do/life-sciences/scientific-advice (accessed January 03, 2020).

Google Scholar

58. Swedish Medical Products Agency. Scientific Advice. (2019). Available online at: https://www.lakemedelsverket.se/en/permission-approval-and-control/advisory/scientific-advice (accessed January, 15, 2020).

Google Scholar

59. Netherlands Healthcare Institute. Pilot 'Parallel Procedures MEB-ZIN'. (2020). Available online at: https://www.zorginstituutnederland.nl/werkagenda/parallelle-procedures-cbg-zin (accessed April 25, 2020).

60. Trapestry Network. Pilots of Multi-Country, Multi-Stakeholder Consultations in Drug Development: From Proof of Concept to Tangible Benefits. (2012). Available online at: https://www.tapestrynetworks.com/sites/default/files/publication_pdf/Pilots-of-multi-stakeholder-consultations-in-drug-development-6-June-2012.pdf (accessed February 05, 2020).

61. Federal Register. Parallel Review of Medical Products. (2010). Available online at: https://www.federalregister.gov/documents/2010/09/17/2010-23252/parallel-review-of-medical-products (accessed February 12, 2020).

62. Federal Register. Program for Parallel Review of Medical Devices. (2016). Available online at: https://www.federalregister.gov/documents/2016/10/24/2016-25659/program-for-parallel-review-of-medical-devices (accessed January 11, 2020).

Google Scholar

63. European Medicines Agency. Best Practice Guidance for Pilot EMA HTA Parallel Scientific Advice procedures. (2014). Available online at: https://www.ema.europa.eu/en/documents/regulatory-procedural-guideline/draft-best-practice-guidance-pilot-european-medicines-agency-health-technology-assessment-parallel_en.pdf (accessed January 15, 2020).

64. Tschank D. The Impact of the EMA-EUnetHTA Collaboration on Drug Development: der Mathematisch-Naturwissenschaftlichen Fakultät, Friedrich-Wilhelms-Universität Bonn (2014).

65. Bramley T, Spackman E, Hebborn A, Ho LK, Parcher B, McLaughlin T. To seek or not to seek parallel European medicine agency and health technology assessment scientific advice? Value & Outcomes Spotlight. (2017) 12–15.

66. Tafuri G, Lucas I, Estevao S, Moseley J, d'Andon A, Bruehl H, et al. The impact of parallel regulatory-health technology assessment scientific advice on clinical development. Assessing the uptake of regulatory and health technology assessment recommendations. Br J Clin Pharmacol. (2018) 84:1013–9. doi: 10.1111/bcp.13524

CrossRef Full Text | Google Scholar

67. McAuslane N, Liberti L, Connelly P. The confluence of accelerated regulatory and health technology assessment access pathways. Clin Pharmacol Ther. (2019) 105:935–42. doi: 10.1002/cpt.1315

PubMed Abstract | CrossRef Full Text | Google Scholar

68. Australian Government. Memorandum of Understanding. (2010). Available online at: http://www.pbs.gov.au/industry/useful-resources/memorandum/MOU-MA-Sep2010.pdf (accessed January 12, 2020).

Google Scholar

69. Pearce A, van Gool K, Haywood P, Haas M. Delays in access to affordable medicines: putting policy into perspective. Aust Health Rev. (2012) 36:412–8. doi: 10.1071/AH11110

PubMed Abstract | CrossRef Full Text | Google Scholar

70. Centre for Innovation in Regulatory Science (CIRS). Review of HTA outcomes and timelines in Australia, Canada and Europe 2014–2018. London: CRIS (2019).

Google Scholar

71. GlaxoSmithKline Australia Pty Ltd, ViiV Healthcare Pty Ltd. The Pharmaceutical Benefitis Scheme in Australia: An Explainer on System Components. (2018). Available online at: https://au.gsk.com/media/421635/gsk-viiv-the-pbs-in-australia-feb-2018.pdf (accessed January 21, 2020).

72. CADTH. CADTH Common Drug Review Will Accept Submissions Up to Six Months Pre-Notice of Compliance (NOC). (2018). Available online at: https://cadth.ca/news/cadth-common-drug-review-will-accept-submissions-six-months-pre-notice-compliance-noc (accessed January 15, 2020).

73. Liberti L. Will Parallel Regulatory and HTA Review Processes Reduce Access Time for Patients in Canada? (2018). Available online at: https://www.cadth.ca/sites/default/files/symp-2018/presentations/april16-2018/Concurrent-Session-B1-Lawrence-Liberti.pdf (accessed January 12, 2020).

74. Podemska-Mikluch M. FDA-CMS Parallel Review: A Failed Attempt at Spurring Innovation. Arlington, VA: Mercatus Center at George Mason University (2016).

Google Scholar

75. Ridge JR, Statz S. Exact Sciences' experience with the FDA and CMS parallel review program. Expert Rev Mol Diagn. (2015) 15:1117–24. doi: 10.1586/14737159.2015.1069184

PubMed Abstract | CrossRef Full Text | Google Scholar

76. Bhatt DL, Kandzari DE, O'Neill WW, D'Agostino R, Flack JM, Katzen BT, et al. A controlled trial of renal denervation for resistant hypertension. N Engl J Med. (2014) 370:1393–401. doi: 10.1056/NEJMoa1402670

CrossRef Full Text | Google Scholar

77. Mezher M. FDA, CMS: Second Parallel Review Decision Ever for NGS Test. (2017). Available online at: https://www.raps.org/regulatory-focus%E2%84%A2/news-articles/2017/12/fda,-cms-second-parallel-review-decision-ever-for-ngs-test (accessed January 23, 2020).

78. MaRS Discovery District. Why the Medical Device Industry is Shifting Focus to Early Health Technology Assessment. (2020). Available online at: https://www.marsdd.com/news/why-the-medical-device-industry-is-shifting-focus-to-early-health-technology-assessment/ (accessed January 15, 2020).

79. Bouvy JC, Jonsson P, Longson C, Crabb N, Garner S. Health technology assessment in the context of adaptive pathways for medicines in Europe: challenges and opportunities. Clin Pharmacol Ther. (2016) 100:594–7. doi: 10.1002/cpt.448

CrossRef Full Text | Google Scholar

80. Brownsword R, Scotford E, Yeung K. The Oxford Handbook of Law, Regulation, and Technology. Oxford, UK: Oxford University Press (2017).

Google Scholar

81. Eichler HG, Oye K, Baird LG, Abadie E, Brown J, Drum CL, et al. Adaptive licensing: taking the next step in the evolution of drug approval. Clin Pharmacol Ther. (2012) 91:426–37. doi: 10.1038/clpt.2011.345

PubMed Abstract | CrossRef Full Text | Google Scholar

82. Nicotera G, Sferrazza G, Serafino A, Pierimarchi P. The iterative development of medicines through the European medicine agency's adaptive pathway approach. Front Med. (2019) 6:148. doi: 10.3389/fmed.2019.00148

PubMed Abstract | CrossRef Full Text | Google Scholar

83. Eichler HG, Baird LG, Barker R, Bloechl-Daum B, Børlum-Kristensen F, Brown J, et al. From adaptive licensing to adaptive pathways: delivering a flexible life-span approach to bring new drugs to patients. Clin Pharmacol Ther. (2015) 97:234–46. doi: 10.1002/cpt.59

PubMed Abstract | CrossRef Full Text | Google Scholar

84. Stein D, Soni M. Early Access Programs Opportunities and Challenges for Real-World Data Collection. (2018). Available online at: https://www.evidera.com/wp-content/uploads/2018/05/Early-Access-Programs-Real-World-Data.pdf (accessed January 12, 2020).

Google Scholar

85. European Medicines Agency. PRIME: Priority Medicines. (2020). Available online at: https://www.ema.europa.eu/en/human-regulatory/research-development/prime-priority-medicines (accessed March 22, 2020).

86. European Medicines Agency. Conditional Marketing Authorisation. (2020). Available online at: https://www.ema.europa.eu/en/human-regulatory/marketing-authorisation/conditional-marketing-authorisation (accessed April 02, 2020).

Google Scholar

87. European Medicines Agency. Compassionate Use. (2020). Available online at: https://www.ema.europa.eu/en/human-regulatory/research-development/compassionate-use (accessed January 12, 2020).

Google Scholar

88. European Medicines Agency. Final Report on the Adaptive Pathways Pilot. (2020). Available online at: https://www.ema.europa.eu/en/documents/report/final-report-adaptive-pathways-pilot_en.pdf (accessed February 16, 2020).

Google Scholar

89. National Institute for Public Health and the Environment. The Adaptive Pathways Process: Benefits and Potential Hurdles From a Dutch Perspective. Bilthoven: RIVM (2016).

90. Baird LG, Trusheim MR, Eichler HG, Berndt ER, Hirsch G. Comparison of stakeholder metrics for traditional and adaptive development and licensing approaches to drug development. Ther Innov Regul Sci. (2013) 47:474–83. doi: 10.1177/2168479013487355

PubMed Abstract | CrossRef Full Text | Google Scholar

91. Innovative Medicines Initiative. ADAPTSMART. (2020). http://adaptsmart.eu/adapt-smart-kick-off-meeting-the-innovative-medicines-initiatives-adapt-smart-laying-the-foundations-and-building-consensus-to-make-mapps-work-for-all/ (accessed March 15, 2020).

92. Oye K, Baird LG, Chia A, Hocking S, Hutt PB, Lee D, et al. Legal foundations of adaptive licensing. Clin Pharmacol Ther. (2013) 94:309–11. doi: 10.1038/clpt.2013.95

PubMed Abstract | CrossRef Full Text | Google Scholar

93. Schulthess D, Baird LG, Trusheim M, Unger TF, Lumpkin M, Hoos A, et al. Medicines adaptive pathways to patients (MAPPS): a story of international collaboration leading to implementation. Ther Innov Regul Sci. (2016) 50:347–54. doi: 10.1177/2168479015618697

PubMed Abstract | CrossRef Full Text | Google Scholar

94. European Medicines Agency. European Medicines Agency Post-Authorisation Procedural Advice for Users of the Centralised Procedure. (2019). Available online at: https://www.ema.europa.eu/en/documents/regulatory-procedural-guideline/european-medicines-agency-post-authorisation-procedural-advice-users-centralised-procedure_en.pdf (accessed March 12, 2020).

95. Food and Drug Administration. Postmarketing Studies and Clinical Trials—Implementation of section 505(o)(3) of the Federal Court Food, Drug and Cosmetic Act 2011. (2011). Available online at: http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM172001.pdf (accessed March 13, 2020).

96. European Network of Centres for Pharmacoepidemiology and Pharmacovigilance. ENCePP Guide on Methodological Standards in Pharmacoepidemiology. (2018). Available online at: http://www.encepp.eu/standards_and_guidances/methodologicalGuide.shtml (accessed January 12, 2020).

Google Scholar

97. Tavridou A, Sarinic V. PRAC Interaction with SAWP PASS Pilot and Other Consultations. (2016). Available online at: https://www.ema.europa.eu/en/documents/presentation/presentation-prac-interaction-sawp-anna-tavridou_en.pdf (accessed April 25, 2020).

98. McGettigan P, Olmo CA, Plueschke K, Castillon M, Zondag DN, Bahri P, et al. Patient registries: an underused resource for medicines evaluation: operational proposals for increasing the use of patient registries in regulatory assessments. Drug Saf . (2019) 42:1343–51. doi: 10.1007/s40264-019-00848-9

CrossRef Full Text | Google Scholar

99. European Medicines Agency. Patient Registries. (2018). Available online at: https://www.ema.europa.eu/en/human-regulatory/post-authorisation/patient-registries (accessed January 15, 2020).

Google Scholar

100. Get Real. Overall Objectives. (2020). Available online at: https://www.imi-getreal.eu/About-GetReal/Overall-objectives (accessed January 16, 2020).

101. Tsoi B, O'Reilly D, Masucci L, Drummond M, Goeree R. Harmonization of HTA—based reimbursement and regulatory approval activities: a qualitative study. J Popul Ther Clin Pharmacol. (2015) 22:e78–89.

Google Scholar

102. CADTH. Procedures for the CADTH Common Drug Review and Interim Plasma Protein Product Review. (2020). Available online at: https://cadth.ca/sites/default/files/cdr/process/Procedure_and_Guidelines_for_CADTH_CDR.pdf (accessed March 25, 2020).