Abstract
Background:
Multicriteria Decision Analysis (MCDA), a formal decision support framework, has been growing in popularity recently in the field of health care. MCDA can support pricing and reimbursement decisions on the macro level, which is of great importance especially in countries with more limited resources.
Objectives:
The aim of this systematic review was to facilitate the development of future MCDA frameworks, by proposing a set of criteria focusing on the purchasing decisions of single-source innovative pharmaceuticals in upper middle-income countries.
Methods:
A systematic literature review was conducted on the decision criteria included in value frameworks (VFs) or MCDA tools. Scopus, Medline, databases of universities, websites of Health Technology Assessment Agencies, and other relevant organizations were included in the search. Double title-abstract screening and double full-text review were conducted, and all extracted data were double-checked. A team of researchers performed the merging and selection process of the extracted criteria.
Results:
A total of 1,878 articles entered the title and abstract screening. From these, 341 were eligible to the full-text review, and 36 were included in the final data extraction phase. From these articles 394 criteria were extracted in total. After deduplication and clustering, 26 different criteria were identified. After the merging and selection process, a set of 16 general criteria was proposed.
Conclusion:
Based on the results of the systematic literature review, a pool of 16 criteria was selected. This can serve as a starting point for constructing MCDA frameworks in upper middle-income countries after careful adaptation to the local context.
Introduction
The Need for Complex Value Assessment in HTA
Resource allocation in health care is a complex process, economic constraints in all countries necessitate a rational priority setting and a transparent decision-making framework (Baltussen and Niessen, 2006). This need called Health Technology Assessment (HTA) into being, a multidisciplinary field of policy analysis using explicit analytical frameworks, with the main purpose to inform technology-related policymaking in health care (Facey et al., 2006). Essentially, HTA aims to provide the best available information regarding various health technologies in a variety of settings (Velasco-Garrido and Busse, 2005; Grutters et al., 2011).
Another key aspect of health care decision making, in line with the multidisciplinary nature of HTA, is that it is not based on a single criterion, but on a set of different criteria, like efficacy or effectiveness, safety, cost-effectiveness, equity or fairness, etc. (Guindo et al., 2012) that are often in conflict with one another (Keeney and Raiffa, 1993). Lack of awareness of the value elements considered across the decision-making continuum as well as lack of transparency in decision making can potentially create tension amongst stakeholders (Tanios et al., 2013). As a result, more and more organizations are proposing and implementing a “value assessment framework” (or “value framework” (VF) for short) for aiding health care decisions with a clear list of criteria that has to be taken into account (Neumann et al., 2018). It has also been a generally accepted principle that health care decision‐making should rely on strong scientific evidence (Rudmik and Drummond, 2013), which called the need for a scientifically justified method to evaluate more than one aspect of the decision-making problem at once. When criteria of a value framework are weighted as opposed to each other and given a specific measurement method (so-called scoring function), of which a summary of scores can be calculated, the framework is called a Multi-Criteria Decision Analysis (MCDA) or a Multiple-Criteria Decision Making (MCDM) tool.
What Is MCDA?
MCDA is the collective term for those formal decision support approaches, that take multiple criteria into account in an explicit way (Belton and Stewart, 2002). MCDA has already been used in various fields of science, and it has been gaining popularity recently in the field of health care decision making as well (Marsh et al., 2014; Antioch et al., 2017). Key elements of the MCDA structure are: 1) clearly defined and well-structured criteria to assess the alternatives that are being evaluated in the particular decision setting, 2) weights to express the differences in relative importance of various criteria, and 3) the aforementioned scoring functions for converting performance measurements of the analyzed alternatives into scores that can be aggregated to inform the decision-makers on the overall performance of the alternatives (Thokala et al., 2016).
Potential of MCDA Use in Health Care
MCDA can be used on all levels of decision making in health care. At micro-level decision making when choosing the treatment to be administered to a certain patient, MCDA can be especially helpful for involving the patient’s voice into his/her own treatment decision (Marsh et al., 2017) through shared decision making [e.g. American Society of Clinical Oncology (ASCO) framework (Cherny et al., 2018)]. The mezzo, clinical guideline or hospital-level decision making can also be guided by MCDAs, for example during the development of the hospital formulary list or a new clinical guideline. Last but not least MCDA can be of great help at macro level in structuring risk-benefit assessment to support market authorization decisions (Angelis and Phillips, 2020) or informing reimbursement decisions for different health technologies, health care programs or health priorities (e.g. EVIDEM [Evidence and Value: Impact on Decision Making) framework (Goetghebeur et al., 2008); The Advance Value Framework (Angelis and Kanavos, 2017)].
In general, an MCDA tool can have two potential roles in reimbursement decisions. If an explicit threshold of MCDA final scores is used as a hard decision rule, MCDAs can be used as independent decision-making tools. For example, if the total attainable score of the MCDA is 100 points, a threshold at 60 points can be set above which all assessed health technologies can receive reimbursed status. The other general approach is to use the MCDA framework in a supportive role, without being the single instrument that guides the final decision. This way, MCDA frameworks can be used as part of deliberate processes (Baltussen et al., 2017), or as a way of conducting so-called augmented cost-effectiveness analyses (Garrison et al., 2018).
The International Society of Pharmacoeconomics and Outcomes Research (ISPOR) had a task force dedicated to the topic of MCDA and published two comprehensive reports (Marsh et al., 2016; Thokala et al., 2016) guiding the development of such tools. Wide MCDA-related research has been conducted both regarding the theoretical side, like reviewing the various approaches adopted in health care MCDAs (Marsh et al., 2014), and the methodological side, for example on the selection of the optimal criteria weighting methods (Németh et al., 2019). Some examples have also been published on the practical issues of using MCDA in real-life decision settings, from applying the EVIDEM framework in Lombardy, Italy (Radaelli et al., 2014), through assessing the value of hospital technologies in Hungary (Endrei et al., 2014) or improving the current procurement framework for off-patent pharmaceuticals in Indonesia (Inotai et al., 2018a). Above all, the usefulness of incorporating MCDA in HTA to support transparent and systematic appraisal of health care interventions has been demonstrated in a proof-of-concept study (Goetghebeur et al., 2012). Also, it has been shown that MCDA integrated into HTA allows a consistent approach for appraising health care interventions, by promoting systematic consideration of all decision criteria and the underlying evidence (Tony et al., 2011).
HTA in Upper Middle-Income Countries
The level of HTA used in reimbursement decisions is uneven throughout countries, often being less developed in upper-middle-income countries compared to high-income ones. Nonetheless, countries with limited resources can spend a lower amount of monetary resources on health care compared to high-income countries, making their allocative decisions more influential for the whole system (Kaló et al., 2016). This situation is often troubled with a worse general health status of the population (Boncz et al., 2014), and insufficient capacity of HTA expertise (Kaló et al., 2013), therefore the need for and the challenges of implementing HTA for health care technologies are even greater in these countries (Sorenson et al., 2009; Inotai et al., 2012). Application of an MCDA tool adapted to local circumstances can be a good option for overcoming these challenges for upper-middle-income countries, especially for innovative, usually expensive pharmaceuticals.
In this study, we aimed to facilitate the development of such transparent decision-making processes. The main goal of our research was to provide a wide set of criteria based on a systematic literature review to assist the development of future MCDA tools in upper-middle-income countries for purchasing decisions of single-source innovative pharmaceuticals. Authors believe that the proposed criteria set can be a potential starting point for national adaptation in individual countries, which necessitates conclusion on how many criteria are applied at the national level and how the weight of each criterion is determined to reflect priorities of national health policies.
Methods
Sources of Information
A systematic literature review was conducted to identify value framework and MCDA articles listing criteria relevant for reimbursement level decision making of pharmaceuticals. The systematic literature review covered Scopus and Medline (via PubMed) databases. It was extended with a targeted literature review on specific databases of universities (Centre for Reviews and Dissemination (CRD) database; University of York, England; University of British Columbia, Canada) and websites of Health Technology Assessment Agencies (Independent Institute for Quality and Efficiency in Health Care (IQWIG), Germany; Agency for Health Quality and Assessment of Catalonia (AQUAS), Spain; Swedish Agency For Health Technology Assessment and Assessment Of Social Services (SBU), Sweden; National Institute for Health and Care Excellence (NICE), England; Scottish Medicines Consortium (SMC), Scotland; French National Authority for Health (HAS), France; Canadian Agency for Drugs and Technologies in Health (CADTH), Canada). Additionally, the following organizations’ websites were searched for eligible VFs and MCDAs: World Health Organisation (WHO) Health Evidence Network database; Institute for Clinical and Economic Review (ICER); European Union Projects (The Seventh Framework Programme (FP7), Horizon 2020 Programme, Innovative Medicines Initiative Joint Action Programme 2 (IMI2)); European Network for Health Technology Assessment (EUnetHTA); Health Technology Assessment International (HTAi); Innovation and Value Initiative (IVI); International Society for Pharmacoeconomics and Outcomes Research (ISPOR); European Patients’ Forum (EPF); European Federation of Pharmaceutical Industries and Associations (EFPIA); Pharmaceutical Research and Manufacturers of America (PhRMA); and the Society for Medical Decision Making. Registries of systematic literature reviews were also searched through PROSPERO and Cochrane Reviews databases.
Search Limitations and Terms
The literature search was performed on the 28th of March 2019 for all databases. All articles on MCDA tools or VFs relevant for reimbursement level decision making on pharmaceuticals, listing a clear set of criteria and published since January 2013 were included. A restriction of publication date was used as the field of value frameworks has evolved remarkably in the past 5 years, and we wished to focus on more recent approaches, to capture relevant value elements in the present and near future. The search was also limited to English-language papers.
The first part of the search term was a combination of domains relevant to MCDA or VFs, in addition to the second term related to health care or pharmaceuticals. Synonyms and MeSH Terms of these two domains were used to conduct a comprehensive search in each database. The detailed search syntax is included in Supplementary Figure 1.
Screening Phase
After deduplication, title and abstract screening and full-text screening were performed by two independent researchers. Disagreements between researchers were resolved by a principal researcher.
During the title and abstract screening phase, articles were excluded based on the following hierarchy: 1) Irrelevant title and no English abstract; 2) Not related to MCDA/VF OR MCDA/VF in other fields than human health care; 3) MCDA/VF in other fields than pharmaceuticals. Articles not fitting any of these exclusion criteria were eligible for full-text screening.
During full-text screening, articles were excluded if they fit in any of the following exclusion criteria: 1) Full text not found; 2) Abstract or commentary; 3) Duplicate; 4) Not English; 5) Non-pharmaceutical focus; 6) Not listing a clear set of criteria (e.g. purely methodological); 7) MCDA/VF supporting patient-level or shared decision making; 8) MCDA/VF supporting clinical guideline level/hospital level decision making; 9) Not original paper (e.g. framework already included in the review without major changes in criteria; literature review) or 10) Other specified reasons. Articles that did not fit any of the previous exclusion criteria were included in the data extraction phase. Other systematic literature reviews with a similar scope to this study were excluded by exclusion criterion 6, but were kept in a separate folder for snowball search.
Data Extraction Phase
To systematically assess the findings, a standardized data extraction form was developed. The data extraction table and categorization principles for criteria were revised after a pilot data extraction phase. Qualitative research synthesis was used for presenting information, per the type of data. All criteria were extracted and clustered into seven predefined categories (disease-related; treatment-related; economic; societal; uniqueness and complexity; patient experience; other). All criteria related to the severity of disease, size of affected population, public health priorities or unmet need were considered disease-related. Treatment-related criteria were categorized as relating efficacy and/or safety of the treatment, or the strength of evidence of those. Criteria were considered as economic if they related to either the cost-effectiveness, budget impact, sustainability or potential for use outside of the reimbursed indication of the treatment. Additionally, organizational aspects of the assessed technology and the local investments of manufacturers were also considered as economic criteria. Societal criteria included equity aspects, prevention, productivity, caregiver burden, and legal aspects. Specific criteria for innovative technologies were categorized as uniqueness and complexity, including the innovative profile of the treatment and the complexity of manufacturing. Those criteria directly affecting patients’ experience other than those traditionally assessed in HTA (e.g. efficacy and safety) were categorized as patient experience. These criteria included patient convenience, adherence, and persistence, as well as value-added services.
The following information was extracted from each included study hence framework: 1) first author and year of publication; 2) country/region of (proposed) implementation; 3) organization developing the framework; 4) objective of the framework; 5) scope of use (e.g. general on all health technologies, oncologic drugs, orphan drugs); 6) name of criteria listed; 7) definition of each criterion, if available; 8) categorization of each criterion by the data extractor (into one of the 7 predefined categories); 9) scoring function of each criterion if available (which determines how a technology will be evaluated according to a particular criterion).
All extracted data were double-checked by a researcher other than the data extractor. Snowball search was performed to identify further relevant studies in included articles and in systematic literature reviews identified throughout the screening phases. When more than one article discussed the same framework, only the most recent article was included in the data extraction. Although, different adaptations of the same framework were included if criteria varied significantly.
Quality Assessment
Identified frameworks were not assessed for quality, as the aim of the research was to collect potential value elements from recently published frameworks not to evaluate the current practice of the field.
Merging and Selection Process of Criteria
When the data extraction phase was over, the merging and selection process of extracted criteria has started. First, criteria were deduplicated that of synonyms and substantially similar concepts. Then a multi-stakeholder team of experts went through all remaining criteria by the predefined categories in three rounds of workshops. Stakeholders involved were academics (2), HTA doers (2) and patient representatives (2) (Table 1).
Table 1
| ACADEMIA | A professor from an academic HTA Center with primer research focus on HTA methodologies |
| An associate professor from an academic HTA Center with primer research focus on HTA methodologies | |
| HTA DOER | A former HTA doer with 5 years of working experience at a national HTA body as a senior health economist |
| An HTA doer, who coordinates large scale evidence synthesis projects | |
| PATIENT REPRESENTATIVE | An oncology patient with thorough experience in validating patient reported outcome instruments |
| A rare disease patient with a master’s degree in health economics and several years of experience in patient advocacy at national and international level |
Detailed profiles stakeholders involved in the selection and merging process of criteria (n = 6).
The merging and selection process was conducted in compliance with the ISPOR MCDA Emerging Good Practices Task Force report’s principles: completeness, non-redundancy, nonoverlaps, and preference independence (Marsh et al., 2016).
Results
Results of the Systematic Literature Review
After the deduplication of results, a total of 1824 articles were identified through literature searches. The targeted literature search and the snowball method detected an additional 54 new records. Altogether, titles and abstracts of 1,878 records were screened from which 341 were eligible for the full-text screening phase. In total, 35 articles were considered to be eligible for the qualitative synthesis and were included in the data extraction phase (Figure 1).
Figure 1
A total of 37 different frameworks—MCDAs and VFs—were identified in the 35 articles. Based on the current World Bank Country and Lending rules for the 2020 fiscal year (World Bank Country and Lending Groups Country Classification, 2020), 9 were categorized as developed specifically for an upper-middle-income country. All others were meant to be implemented in a high-income country or were not specified other than continent or region (e.g. Europe, Latin-America, Central-Eastern Europe). Thirteen of the 37 had a general focus on all health technologies, 9 on medicines in general, while 6, 5 and 4 frameworks focused on orphan drugs, oncologic drugs, and off-patent drugs, respectively. The number of criteria listed ranged between 5 and 28, with an average of 11.3. About half of the frameworks (n=18) had explicit definitions of criteria listed, others published only the names of criteria. More than a third of the frameworks (n=15) had scoring functions, of which two had the same general scoring function proposed for all their criteria (e.g. 4-point scale (0–3)). About a third of the frameworks (n=14) had neither definitions nor scoring functions published for their criteria. Regarding the categories of criteria, all of them contained at least one treatment-related criterion. Economic and disease-related criteria were the second and third most prevalent, 86% (n = 32) and 81% (n = 30) of frameworks contained at least one of them, respectively. Fifty-nine percent (n=22) of value frameworks contained criteria categorized as societal, whereas 38% (n = 14) and 32% (n = 12) listed at least one criterion concerning the patient experience and the uniqueness and complexity of the treatment, respectively (Table 2).
Table 2
| Author, Year | Country | Scope | Criteria | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Number of criteria | Definitions | Scoring functions | Disease-related | Treatment-related | Economic | Societal | Uniqueness & complexity | Patient experience | |||
| Angelis and Kanavos, 2017 | Europe | Medicines | 28 | yes | no | x | x | x | x | x | x |
| Annemans et al., 2017 | Europe | Orphan drugs | 17 | yes | no | x | x | x | x | x | |
| MSKCC, 2015 | USA | Oncologic drugs | 9 | yes | yes | x | x | x | x | ||
| Brixner et al., 2017 | Emerging countries | Off-patent drugs | 14 | no | no | x | x | ||||
| CADTH, 2016 | CAN | Oncologic drugs | 8 | yes | no | x | x | x | x | ||
| Cherny et al., 2015 | Europe | Oncologic drugs | 6 | no | no | x | |||||
| Dankó and Molnár, 2017 | Middle-income countries - CEE | General | 5 | yes | no | x | x | x | |||
| Dionne et al., 2015 | CAN | Medicines | 10 | yes | no | x | x | x | x | x | x |
| Doyle et al., 2019 | USA | Oncologic drugs | 5 | no | no | x | x | ||||
| Drake et al., 2017 | Latin-America | General | 17 | no | no | x | x | x | x | ||
| MoCA, 2014 | Europe | Orphan drugs | 6 | yes | yes | x | x | ||||
| Gilabert-Perramon et al., 2017 | ESP - Catalonia | Orphan drugs | 27 | no | no | x | x | x | x | ||
| Henshall and Schuller, 2013 | Global | General | 18 | no | no | x | x | x | x | x | x |
| Howard et al., 2018 | AUS | General | 6 | yes | yes | x | x | x | x | x | x |
| Hu et al., 2015 | CHN | Off-patent drugs | 10 | no | no | x | x | x | x | ||
| ICER, 2017 | USA | General | 13 | yes | yes* | x | x | x | x | x | |
| Inotai et al., 2018 | IDN | Off-patent drugs | 9 | yes | yes | x | x | x | x | ||
| Iskrov et al., 2013 | BUL | Medicines | 18 | no | yes | x | x | x | x | x | |
| Iskrov et al., 2016 | BUL | Orphan drugs | 11 | yes | yes | x | x | x | x | ||
| Iskrov and Stefanov, 2016 | BUL | Medicines | 15 | no | no | x | x | x | x | ||
| Jaramillo et al., 2016 | COL | General | 15 | yes | yes* | x | x | x | x | ||
| Kolasa, 2014 | POL | General | 9 | no | no | x | x | x | x | ||
| Kolasa et al., 2016 | POL | Orphan drugs | 10 | yes | yes | x | x | x | x | ||
| Kristensen et al., 2017 | Europe | General | 9 | no | no | x | x | x | x | x | |
| Kwon et al., 2017 | KOR | Oncologic drugs | 8 | yes | yes | x | x | x | x | x | |
| Lakdawalla et al., 2018 | USA | General | 12 | yes | no | x | x | x | x | x | x |
| Oortwijn et al., 2017 | Global | General | 7 | no | yes | x | x | ||||
| Radaelli et al., 2014 | ITA - Lombardia | General | 8 | no | no | x | x | x | x | ||
| Radu et al., 2016 | ROU | Medicines | 6 | yes | yes | x | x | ||||
| Ramli et al., 2013 | MYS | Medicines | 12 | no | no | x | x | x | |||
| Sussex et al., 2013 | Europe | Orphan drugs | 8 | no | no | x | x | x | x | ||
| Toumi and Rémuzat, 2017 | Europe | Value added medicines | 19 | yes | no | x | x | x | x | x | x |
| Williams et al., 2014 | UK | Medicines | 10 | no | yes | x | x | x | |||
| Williams et al., 2014 | ESP | Medicines | 10 | no | yes | x | x | x | |||
| Williams et al., 2014 | GER | Medicines | 9 | no | yes | x | x | x | |||
| Youngkong, 2014 | THA | General | 9 | yes | no | x | x | x | x | ||
| Zah et al., 2015 | Global | General | 6 | no | no | x | x | x | x | ||
Summary table on the key results of the systematic literature review.
Abbreviations of upper-middle-income countries highlighted in bold.
*General, not criterion-specific scoring function.
AUS, Australia; AUT, Austria; BEL, Belgium; BUL, Bulgaria; CAN, Canada; CEE, Central and Eastern Europe; CHN, China; COL, Colombia; ESP, Spain; FRA, France; GER, Germany; GRE, Greece; IDN, Indonesia; ISR, Israel; ITA, Italy; KOR, Republic of Korea; MSKCC, Memorial Sloan Kettering Cancer Center; MYS, Malaysia; NED, the Netherlands; NOR, Norway; POL, Poland; POR, Portugal; ROU, Romania; SWE, Sweden; THA, Thailand; UK, United Kingdom; USA, United States of America.
Results of the Merging and Selection Process of Criteria
In total 419 criteria were extracted. After deduplication and clustering of similar criteria by each category, 26 criteria remained (Figure 2). These were the following: 1) disease-related criteria: severity of disease; size of affected population; unmet need; public health priority; 2) treatment-related criteria: efficacy; safety; strength of evidence; 3) economic criteria: cost; cost-effectiveness; budget impact; capacity; supply track record; potential use outside of the reimbursed indication(s); Local investment; 4) societal criteria: equity; prevention; productivity; caregiver burden; 5) uniqueness and complexity of treatment criteria: innovative profile of the treatment; manufacturing complexity; number of indications; spill-over effect; formulation; 6) patient experience criteria: patient convenience; patient adherence; value-added services.
Figure 2
Throughout the expert workshops, these 26 criteria were further merged to reduce overlaps and minimize the number of criteria proposed as possible. Criterions “efficacy” and “safety” were merged into a new criterion named “health gain.” The criterion called “size of affected population” was merged into criterion “equity,” thereby rare disease patients got listed as one aspect of the neglected populations. Meanwhile, the geographical access aspect of criterion “equity” (e.g. access for patients living in rural areas) was merged into criterion “availability and accessibility of alternative therapies.” Criterion “prevention” was merged into criterion “public health priority” as a potential focus of local public health policies, and it is highlighted as an example in the description of the new criterion. Likewise, the criterion concerning “caregiver burden” was merged into criterion “productivity,” where thereby the productivity loss of households, including both patients and caregivers, is taken into account. Also, the criterion “patient convenience” was merged into criterion “patient adherence and persistence” to reduce overlaps of capturing both effects and potential causes. Criterion “cost” was excluded because overlaps with criterion “cost-effectiveness” and criterion “capacity” was excluded because it is more health system-related than treatment-related. Criterion “number of indications,” “spill-over effect,” and “formulation” were excluded based on expert suggestions, to keep the initial set of criteria as curtail and condensed as possible. The expert workshops also explored potential incentives for the use of each criterion and proposed descriptions and examples of scoring functions. The final set of 16 criteria proposed for local adaptation is shown in Table 3.
Table 3
| Criterion | Incentive | Proposed scoring function | |
|---|---|---|---|
| DISEASE-RELATED | SEVERITY OF DISEASE Prognosis and lifetime burden of illness | To reflect societies’ positive discrimination towards therapies for more severe diseases. | Chronic life-threatening (100%); acute life-threatening (80%); chronic with severe invalidity (60%); acute with severe invalidity (40%); other chronic diseases (20%); other acute diseases (0%) |
| AVAILABILITY AND ACCESSIBILITY OF ALTERNATIVE THERAPIES Unmet medical need | To provide incentives for developing drugs targeting patients with real unmet need (i.e., e-health for non-city residents) | Patients have no current treatment (100%); patients have limited access to current treatment (50%); patients have access to effective and safe therapy (0%) | |
| PUBLIC HEALTH PRIORITY Local health system priorities, common goals, and specific interests | To incentivize therapies in diseases with high prevalence and significant public health burden (i.e., medical prevention) | Yes (100%); no (0%) | |
| TREATMENT-RELATED | HEALTH GAIN Health benefits of using the pharmaceutical including improved quality of life, survival, clinical surrogate endpoints, and/or safety | To incentivize the development of drugs with an added benefit, either through better efficacy and/or better safety. | Therapeutic advancement compared to the standard of care (based on ASMR*): major (100%); important (75%); moderate (50%); minor (25%); no therapeutic advancement (0%) |
| STRENGTH OF EVIDENCE Robustness of supporting clinical evidence, expert consensus, availability of evaluations from other countries | To incentivize the investment of manufacturers to improve the evidence base of new technologies both in clinical trials and in real world. | Evidence synthesis of RCTs plus real world data (100%); evidence synthesis of RCTs (80%); cohort studies (60%); case control studies (40%); case series/reports (20%); no evidence available (0%) | |
| ECONOMIC | COST-EFFECTIVENESS Incremental cost-effectiveness ratio (ICER) compared to willingness to pay threshold | To incentivize the value for money aspect. | Dominant therapy (100%); below 3× GDP/capita (50%); Between 3× GDP/capita and 6× GDP/capita (50%–0% linear function); above 6x GDP/capita (0%); No data about cost-effectiveness (0%); EXCLUSION CRITERIA—the technology is dominated. |
| PROPORTION OF THE DRUG BUDGET Total financial burden on the relevant health care budget(s) | To investigate the affordability of the medical technology. | How much is the proportion of the drug in the country’s annual drug budget? Saving money (100%); between 0% and 1% (100%–0% linear function); above 1% (0%) | |
| POTENTIAL FOR USE OUTSIDE THE REIMBURSED INDICATION Potential for off-label use of the drug, as a burden on the Payer | To disincentivize the utilization of the technology outside the reimbursed indication(s). | No potential for off-label used based on objective parameters or company guarantee (100%); Some potential for off-label use, patient selection is not based on an objective parameter (50%); Frequent off-label use predicted (0%) | |
| SUPPLY TRACK RECORD Sustainability of manufacturer business practices and supply track record assuring continuity | To incentivize supply reliability. | Manufacturer is financially capable and willing to guarantee supply (100%); No precedence of supply problems in the last 5 years (80%); Single precedence of supply problems in the last 5 years (50%); minor and fairly frequent problems in the last 5 years (20%); major and multiple problems in the last 5 years (0%) | |
| LOCAL INVESTMENT Potential for the manufacturer to make local investment, supporting the local economy | To incentivize manufacturers to invest into the local economy. | The manufacturer has significant local investment in the country (100%); The manufacturer has moderate local investment in the country (67%); The manufacturer has minor local investment in the country (33%); The manufacturer has no local investment in the country (0%) | |
| SOCIETAL | EQUITY Reducing health disparities | To incentivize medical therapies to neglected populations (pediatric patients, rare diseases, pregnancy). | Ultra-orphan disease (100%); indication for paediatric, rare, pregnant, or psychiatric patients (75%); none of the above (0%) |
| PRODUCTIVITY Financial protection of households through improved productivity, reduced caregiver burden, or avoided patient cost (reduction in the medical costs of the patients) | To incentivize therapies improving the productivity of patients and/or caregivers. | Improvement in the financial status of the household: Major (100%); minor (50%); no improvement (0%); no data (0%) | |
| UNIQUENESS & COMPLEXITY | INNOVATIVE PROFILE OF TREATMENT First in class therapies | To incentivize companies to develop novel technologies instead of me-too products. | Is it a first in class therapy? Yes (100%); no (0%) |
| MANUFACTURING COMPLEXITY The complexity of the manufacturing technology | To appreciate efforts of manufacturers to develop innovative products requiring complex manufacturing processes. | Manufacturing technology complexity: expensive biotechnological processes (100%); complex synthetic path consisting of at least three independent chemical transformation; (50%); manufacturing requires the use of separation techniques for most intermediates (50%); anything else (0%) | |
| PATIENT EXPERIENCE | PATIENT ADHERENCE AND PERSISTENCE Potential improvements in adherence and persistence of patients taking the pharmaceutical | To incentivize potential improvements in the adherence and/or persistence of patients using the medical technology due to improved patient convenience, tolerability, etc. | Treatment improves adherence and/or persistence: Significantly (100%); moderately (50%); no potential improvement (0%) |
| VALUE ADDED SERVICES TO PATIENTS Additional services provided by the manufacturer to patients | To incentivize efforts of manufacturers to improve patient experience through transparently provided added value services (i.e., mobile application, disease awareness, patient education) | Company provides multiple value-added services, e.g., disease awareness and education (100%); company provides one single value-added service (50%); company does not provide value-added services (0%) |
The proposed set of criteria with descriptions, potential incentives for inclusion and proposed scoring functions.
*ASMR is a five-level scale used by the French National Authority for Health (HAS) used to evaluate the incremental or added value of the product.
Discussion
To the best of our knowledge, this is the first publication supporting the creation of MCDA for the reimbursement of innovative pharmaceuticals in upper-middle-income countries. Similar publications identified throughout our systematic literature review were a systematic review on the use of methodological frameworks to set health care priorities in low-income and lower middle-income countries (Wiseman et al., 2016), MCDA tools specific to one certain upper-middle-income country with either a specific or a general scope (Iskrov et al., 2013; Ramli et al., 2013; Youngkong, 2014; Hu et al., 2015; Iskrov and Stefanov, 2016; Jaramillo et al., 2016; Radu et al., 2016) and an MCDA tool for emerging countries focusing on the evaluation of off-patent pharmaceuticals (Brixner et al., 2017).
The proposed set of 16 criteria provides a comprehensive initial framework that can be adapted to upper-middle-income countries’ circumstances based on local priorities. As the literature review indicated, there is increasing attention to criteria concerning patient experience. To emphasize its significance, two independent criteria are proposed to capture this emerging aspect. Criteria concerning the uniqueness and complexity of the therapy were included as important aspects of innovative medicines, especially in the light of recently launched genetic therapies.
The merging and selection process aimed to minimize the number and magnitude of overlaps in the framework to prevent double-counting. However, some overlap is inevitable in any MCDA. For example, a potential remaining overlap might be between criteria “patient adherence and persistence” and “value added services”. Even though value-added services can potentially improve patient adherence and persistence, the importance of these services should be emphasized.
The core of an MCDA decision-supporting tool for repeated use is in its availability of increasing the transparency of decision making. When MCDAs are published, but without the detailed definitions and scoring functions of the criteria they may fail to fulfill this purpose. With almost half of the identified frameworks not publishing either definitions or scoring functions, we thought it was an absolute must to propose one for each criterion, even if they can be further modified during local adoption. This approach hopefully further supports the interpretation thereby the implementation of our work.
Previous Systematic Literature Reviews With a Similar Scope
Throughout the literature review, nine previous systematic literature reviews of MCDAs and VFs were identified. The most recent by Kolasa et al. present the then-current state of knowledge concerning the MCDA utilization in the value assessment of drug therapies. They identified 18 MCDAs in April 2017 concerning pricing and reimbursement level decision making of pharmaceuticals, similarly to the present review. Kolasa and colleagues revealed several shortcomings in the implementation of MCDA to health care decision–making, like the lack of consistency of approaches chosen toward the type and the number of stakeholders involved in the MCDA experiments. Similarly to our findings, they also noted that none of the reviewed studies have reported how MCDA results impacted the real-life settings, and that examples of MCDA’s successful launch in a real-life decision-making process are genuinely scarce (Kolasa et al., 2018). Another recent systematic review by Frazão and colleagues aimed to analyze and synthesize articles found in the literature on applying MCDA to health care while assessing general issues and methodological aspects. They analyzed the trends in MCDA publications in addition to the types of sources and methods used to develop particular parts of the MCDAs. Based on their review, they identified a growing trend in the application of MCDA in the field of health, with great emphasis from the year 2014 (Frazão et al., 2018). This finding is in line with our approach of restricting the timeframe of our research to 2013 and beyond. Another systematic review aimed to summarize the evaluation criteria of orphan medicines with a regional scope, particularly for Central and Eastern European countries (Zelei et al., 2016). The authors concluded that due to external price referencing of pharmaceuticals, the relative budget impact of orphan drugs is expected to be higher in CEE than in Western European (WE) countries unless accessibility of patients remains more limited in poorer European regions. This observation may apply for other innovative medicines and upper middle-income countries too, further strengthening the need for locally adopted MCDA tools.
Recommendations for Adapting the Study Results
The maximum number of criteria in an MCDA is recommended to set around 10, otherwise too many criteria might mean too little weight for each criterion (Inotai et al., 2018b). Hence, we suggest selecting approximately 10 of our 16 proposed criteria during the adaptation period, based on local priorities (Figure 2). Health care system structure, decision-making framework, the health status of the population, financial backgrounds, and several other key factors can differ significantly across countries, making transferability a key issue in health care decision-making (Goeree et al., 2011). Just like reimbursement recommendations of HTA bodies cannot be used for decision making in other countries without considering transferability (Kaló et al., 2012), the same applies for constructing MCDAs. It has been strongly recommended that MCDA frameworks should only contain locally relevant criteria, taking into close consideration local feasibility, as local experts are always more familiar with specific problems, health care priorities and processes of the particular country than international experts. Furthermore, the development of a new MCDA should be based on the decision-making criteria that are currently used in particular countries (Inotai et al., 2018b). Some steps of MCDA development identified by the second ISPOR Task Force report (Marsh et al., 2016) can be conducted by a small number of experts and policymakers (e.g. defining the decision problem), while others (e.g., selecting and structuring criteria) necessitate a wider, multi-stakeholder approach. Incorporating the preferences of various stakeholders can be conducted at specifically designed workshops (Inotai et al., 2018b), though the results of previously conducted desk research can be a good starting point in all cases. Our research aims to provide a background for such future implementations.
Potential Limitations of MCDA Use
Like all methodologies, MCDA has potential limitations. A scoping review of different priority setting approaches found that while the intention of developing such tools is for them to eventually be used to guide routine policymaking, not many have been integrated into routine practice. Cited limitations included the technical complexity of the approaches and resource demand (Kapiriri and Razavi, 2017). The lack of and need for publishing the real-word experience of using MCDAs has been highlighted by other researchers (Kapiriri and Razavi, 2017; Kolasa et al., 2018) and it is in line with our concerns as well. Such publicly available information can be crucial in supporting the implementation of MCDA in more countries and should be highly encouraged to publish. Another example of potential limitations of MCDA that has been discussed arises when incorporating cost-based criteria. Critics believe that the preferences of technology providers or users may not be the appropriate basis for evaluating the efficiency of technologies as the benefit that may be forgone is more prevalent in their thinking than the willingness to pay (WTP) of third-party payers. And even if the exercise aims to find the WTP with broad and imprecise definitions of criteria, its appropriateness can be questionable (Marsh et al., 2018). Another critique of the stakeholder involvement in the development of MCDAs is that when reviewed, there were no common patterns in the types and the size of the study population of involved stakeholders (Kolasa et al., 2018). Key issues of real-world feasibility may also emerge from the inappropriate selection of an analysis method, wrong interpretation of results, and using software solutions that are not widely available, as the results of the criteria ranking can depend on the various MCDA models that are being used (Kujawski, 2003). This is part of the reason why MCDA and its development method should be simple and easy to understand (Inotai et al., 2018b), and the trade-off between scientific accuracy and resource intensity and general participant burden should be taken into account in case of choosing between criteria weighting methods (Németh et al., 2019). To interpret the results of an MCDA evaluation easier, Angelis and Kanavos suggested a framework in which the aggregate value score (that does not include cost-related criteria) produced by the MCDA process would be the benefit component. The so-called incremental cost value ratio (ICVR), the purchasing cost per incremental MCDA value unit gained could be used similarly as the incremental cost-effectiveness ratio (ICER) is applied in several countries (Angelis and Kanavos, 2016).
Limitations of the Study
Limitations of the present research include, that only published sets of criteria were taken into account. Therefore, the criteria being used in confidential decision-making frameworks could not be incorporated into the analysis. In addition, the categorization of criteria was conducted based on personal judgments of researchers. We tried to minimize the personal variability by 1) creating an objective categorization guideline during the pilot data extraction phase and 2) by categorizing all criteria by two independent researchers and resolving conflicts by a principal researcher. A general limitation was, that a significant amount of frameworks (n=14) had nether descriptions, nor scoring functions published for their criteria listed as value elements. Therefore, the precise interpretation of certain criteria gathered from the literature was challenging and forms a source of uncertainty. Finally, as MCDA frameworks should always be adapted to the local environment and the particular decision problem, it is possible that some special cases may require additional specific criteria. Therefore, our set of 16 criteria should be used as a starting point only.
Future research can focus on MCDAs developed for other fields of health care and other decision problems. Since the theoretical side of MCDA has been relatively well-researched during the last few decades, more emphasis could be put on the practical side of MCDA implementation, based on real-life examples.
Conclusion
Based on the results of the systematic literature review, we established a pool of 16 criteria that can serve as a basis for constructing MCDAs to inform reimbursement decision making of innovative pharmaceuticals, especially in upper middle-income countries. The feasibility of implementation and adaptation to the local context are key aspects that should be taken into account in all cases. When adapting to the circumstances of a certain upper-middle-income country, we suggest setting up a group of local stakeholders to select and weigh not more than 10 of the 16 criteria based on local priorities. Well-defined descriptions and scoring functions of the criteria are inevitable for objective and transparent decision making. Further research is needed on the real-life examples of MCDAs as literature on the experiences after MCDA implementation is scarce and incomplete.
Funding
The study was funded by Novartis International AG.
Statements
Author contributions
Conception and design: ZK, BN, IJ, HT, JS. Analysis and interpretation of data: ZK, BN, IJ, BE, HT, MK. Drafting of the manuscript: IJ, BN, BE. Critical revision of the manuscript for important intellectual content: ZK, MK, HK, SA. Supervision: JS, ZK, SA.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
This work was supported in part by a contract with Novartis International AG. The funding agreement ensured the authors’ independence in designing the study, interpreting the data, writing, and publishing the report. The content of the paper and the conclusions are those of each author and cannot be understood as reflecting those of the organization that employs them.
Supplementary material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphar.2020.01203/full#supplementary-material
References
1
AngelisA.KanavosP. (2016). Value-based assessment of new medical technologies: towards a robust methodological framework for the application of multiple criteria decision analysis in the context of health technology assessment. Pharmacoeconomics34 (5), 435–446. doi: 10.1007/s40273-015-0370-z
2
AngelisA.KanavosP. (2017). Multiple criteria decision analysis (MCDA) for evaluating new medicines in health technology assessment and beyond: the Advance Value Framework. Soc. Sci. Med.188, 137–156. doi: 10.1016/j.socscimed.2017.06.024
3
AngelisA.PhillipsL. D. (2020). Advancing structured decision-making in drug regulation at the FDA and EMA. Br. J. Clin. Pharmacol. 2020, 1–11. doi: 10.1111/bcp.14425
4
AnnemansL.AyméS.Le CamY.FaceyK.GuntherP.NicodE.et al. (2017). Recommendations from the European working Group for Value Assessment and Funding Processes in rare diseases (ORPH-VAL). Orphanet. J. Rare Dis.12 (1), 50. doi: 10.1186/s13023-017-0601-9
5
AntiochK. M.DrummondM. F.NiessenL. W.VondelingH. (2017). International lessons in new methods for grading and integrating cost effectiveness evidence into clinical practice guidelines. Cost Effect. Res. Allocation15 (1), 1. doi: 10.1186/s12962-017-0063-x
6
BaltussenR.NiessenL. (2006). Priority setting of health interventions: the need for multi-criteria decision analysis. Cost Effect. Resource Allocation4 (1), 14. doi: 10.1186/1478-7547-4-14
7
BaltussenR.JansenM. P. M.BijlmakersL.GruttersJ.KluytmansA.ReuzelR. P.et al. (2017). Value assessment frameworks for HTA agencies: the organization of evidence-informed deliberative processes. Value in Health20 (2), 256–260. doi: 10.1016/j.jval.2016.11.019
8
BeltonV.StewartT. (2002). Multiple criteria decision analysis: an integrated approach (Norwell, Massachussets, US; Dordrecht, The Netherlands: Kluwer Academic Publishers).
9
BonczI.VajdaR.ÁgostonI.EndreiD.SebestyénA. (2014). Changes in the health status of the population of Central and Eastern European countries between 1990 and 2010. Eur. J. Health Econ.15 (1), 137–141. doi: 10.1007/s10198-014-0602-8
10
BrixnerD.ManiadakisN.KalóZ.HuS.ShenJ.WijayaK. (2017). Applying multi-criteria decision analysis (MCDA) simple scoring as an evidence-based HTA methodology for evaluating off-patent pharmaceuticals (OPPs) in emerging markets. Value Health Reg. Issues13, 1–6. doi: 10.1016/j.vhri.2017.02.001
11
Canadian Agency for Drugs Technologies in Health (CADTH) (2016). pCODR Expert Review Committee Deliberative Framework 16 March 2019) (CADTH). Available from: https://www.cadth.ca/sites/default/files/pcodr/The%20pCODR%20Expert%20Review%20Committee%20%28pERC%29/pcodr_perc_deliberative_frame.pdf
12
ChernyN. I.SullivanR.DafniU.KerstJ. M.SobreroA.ZielinskiC.et al. (2015). A standardised, generic, validated approach to stratify the magnitude of clinical benefit that can be anticipated from anti-cancer therapies: the European Society for Medical Oncology Magnitude of Clinical Benefit Scale (ESMO-MCBS). Ann. Oncol.26 (8), 1547–1573. doi: 10.1093/annonc/mdv249
13
ChernyN. I.de VriesE. G.DafniU.Garrett-MayerE.McKerninS. E.PiccartM.et al. (2018). Comparative assessment of clinical benefit using the ESMO-magnitude of clinical benefit scale version 1.1 and the ASCO value framework net health benefit score. J. Clin. Oncol.37 (4), 336–349. doi: 10.1200/JCO.18.00729
14
DankóD.MolnárM. P. (2017). Balanced assessment systems revisited. J. Market Access Health Policy5 (1), 1355190. doi: 10.1080/20016689.2017.1355190
15
DionneF.MittonC.DempsterB.LyndL. D. (2015). Developing a multi-criteria approach for drug reimbursement decision making: an initial step forward. J. Popul. Ther. Clin. Pharmacol.22 (1), e68–e77.
16
DoyleJ. J.HawrylukE.NiemiraJ.WoodB. (2019). Evidence-based valuation in oncology: lessons learned from a case study. Ther. Innovation Regul. Sci.53 (3), 403–411. doi: 10.1177/2168479018786701
17
DrakeJ. I.de HartJ. C. T.MonleónC.ToroW.ValentimJ. (2017). Utilization of multiple-criteria decision analysis (MCDA) to support healthcare decision-making FIFARMA, 2016. J. Market Access Health Policy5 (1), 1360545. doi: 10.1080/20016689.2017.1360545
18
EndreiD.MolicsB.ÁgostonI. (2014). Multicriteria decision analysis in the reimbursement of new medical technologies: real-world experiences from Hungary. Value Health17 (4), 487–489. doi: 10.1016/j.jval.2014.01.011
19
FaceyK.TopferL.-A.ChanL. (2006). Health technology assessment (HTA) glossary (Sweden: INAHTA Secretariat, c/o SBU Stockholm).
20
FrazãoT. D.CamiloD. G.CabralE. L.SouzaR. P. (2018). Multicriteria decision analysis (MCDA) in health care: a systematic review of the main characteristics and methodological steps. BMC Med. Inf. Decision making18 (1), 90. doi: 10.1186/s12911-018-0663-1
21
GarrisonL. P.Jr.NeumannP. J.WillkeR. J.BasuA.DanzonP. M.DoshiJ. A.et al. (2018). A health economics approach to US value assessment frameworks—summary and recommendations of the ISPOR Special Task Force Report [7]. Value Health21 (2), 161–165. doi: 10.1016/j.jval.2017.12.009
22
Gilabert-PerramonA.Torrent-FarnellJ.CatalanA.PratA.FontanetM.Puig-PeiróR.et al. (2017). Drug evaluation and decision making in Catalonia: development and validation of a methodological framework based on multi-criteria decision analysis (MCDA) for orphan drugs. Int. J. Technol. Assess. Health Care33 (1), 111–120. doi: 10.1017/S0266462317000149
23
GoereeR.HeJ.O’ReillyD.TarrideJ.-E.XieF.LimM.et al. (2011). Transferability of health technology assessments and economic evaluations: a systematic review of approaches for assessment and application. Clinico. Econ. Outcomes Res.: CEOR3, 89. doi: 10.2147/CEOR.S14404
24
GoetghebeurM. M.WagnerM.KhouryH.LevittR. J.EricksonL. J.RindressD. (2008). Evidence and Value: Impact on DEcisionMaking–the EVIDEM framework and potential applications. BMC Health Serv. Res.8 (1), 270. doi: 10.1186/1472-6963-8-270
25
GoetghebeurM. M.WagnerM.KhouryH.LevittR. J.EricksonL. J.RindressD. (2012). Bridging health technology assessment (HTA) and efficient health care decision making with multicriteria decision analysis (MCDA) applying the EVIDEM framework to medicines appraisal. Med. Decision making32 (2), 376–388. doi: 10.1177/0272989X11416870
26
GruttersJ. P.SeferinaS. C.Tjan-HeijnenV. C.van KampenR. J.GoettschW. G.JooreM. A. (2011). Bridging trial and decision: a checklist to frame health technology assessments for resource allocation decisions. Value Health14 (5), 777–784. doi: 10.1016/j.jval.2011.01.005
27
GuindoL. A.WagnerM.BaltussenR.RindressD.van TilJ.KindP.et al. (2012). From efficacy to equity: Literature review of decision criteria for resource allocation and healthcare decisionmaking. Cost Effect. Resource Allocation10 (1), 9. doi: 10.1186/1478-7547-10-9
28
HenshallC.SchullerT. (2013). Health technology assessment, value-based decision making, and innovation. Int. J. Technol. Assess. Health Care29 (4), 353–359. doi: 10.1017/S0266462313000378
29
HowardS.ScottI. A.JuH.McQueenL.ScuffhamP. A. (2018). Multicriteria decision analysis (MCDA) for health technology assessment: the Queensland Health experience. Aust. Health Rev. 43 (5), 591–599. doi: 10.1071/AH18042
30
HuS.ZhangY.HeJ.DuL.XuM.XieC.et al. (2015). A case study of pharmaceutical pricing in China: setting the price for off-patent originators. Appl. Health Econ. Health Policy13 (1), 13–20. doi: 10.1007/s40258-014-0150-5
31
InotaiA.PékliM.JónaG.NagyO.RemákE.KalóZ. (2012). Attempt to increase the transparency of fourth hurdle implementation in Central-Eastern European middle income countries: publication of the critical appraisal methodology. BMC Health Serv. Res.12 (1), 332. doi: 10.1186/1472-6963-12-332
32
InotaiA.BrixnerD.ManiadakisN.DwiprahastoI.KristinE.PrabowoA.et al. (2018a). Development of multi-criteria decision analysis (MCDA) framework for off-patent pharmaceuticals–an application on improving tender decision making in Indonesia. BMC Health Serv. Res.18 (1), 1003. doi: 10.1186/s12913-018-3805-3
33
InotaiA.NguyenH. T.HidayatB.NurgozhinT.KietP. H. T.CampbellJ. D.et al. (2018b). Guidance toward the implementation of multicriteria decision analysis framework in developing countries. Expert Rev. Pharmaco. Outcomes Res.18 (6), 585–592. doi: 10.1080/14737167.2018.1508345
34
Institute for Clinical Economic Review (ICER) (2017). Overview of the ICER value assessment framework and update for 2017–2019 14 March 2019). Available from: https://icer-review.org/wp-content/uploads/2018/03/ICER-value-assessment-framework-update-FINAL-062217.pdf
35
IskrovG.StefanovR. (2016). Criteria for drug reimbursement decision-making: an emerging public health challenge in Bulgaria. Balkan Med. J.33 (1), 27–35. doi: 10.5152/balkanmedj.2015.15185
36
IskrovG. G.RaychevaR. D.StefanovR. S. (2013). Insight into reimbursement decision-making criteria in Bulgaria: implications for orphan drugs. Folia Med.55 (3–4), 80–86. doi: 10.2478/folmed-2013-0032
37
IskrovG.Miteva-KatrandzhievaT.StefanovR. (2016). Multi-criteria decision analysis for assessment and appraisal of orphan drugs. Front. Public Health4, 214. doi: 10.3389/fpubh.2016.00214
38
JaramilloH. E. C.GoetghebeurM.Moreno-MattarO. (2016). Testing multi-criteria decision analysis for more transparent resource-allocation decision making in Colombia. Int. J. Technol. Assess. Health Care32 (4), 307–314. doi: 10.1017/S0266462316000350
39
KalóZ.LandaK.DoležalT.VokóZ. (2012). Transferability of National Institute for Health and Clinical Excellence recommendations for pharmaceutical therapies in oncology to Central-Eastern European countries. Eur. J. Cancer Care21 (4), 442–449. doi: 10.1111/j.1365-2354.2012.01351.x
40
KalóZ.BodrogiJ.BonczI.DózsaC.JónaG.KöviR.et al. (2013). Capacity building for HTA implementation in middle-income countries: the case of Hungary. Value Health Reg. Issues2 (2), 264–266. doi: 10.1016/j.vhri.2013.06.002
41
KalóZ.GheorgheA.HuicM.CsanádiM.KristensenF. B. (2016). HTA implementation roadmap in Central and Eastern European countries. Health Econ.25, 179–192. doi: 10.1002/hec.3298
42
KapiririL.RazaviD. (2017). How have systematic priority setting approaches influenced policy making? A synthesis of the current literature. Health Policy121 (9), 937–946. doi: 10.1016/j.healthpol.2017.07.003
43
KeeneyR. L.RaiffaH. (1993). Decisions with multiple objectives: preferences and value trade-offs (Cambridge, United Kingdom: Cambridge University Press).
44
KolasaK.ZwolinskiK. M.KaloZ.HermanowskiT. (2016). Potential impact of the implementation of multiple-criteria decision analysis (MCDA) on the Polish pricing and reimbursement process of orphan drugs. Orphanet. J. Rare Dis.11 (1), 23. doi: 10.1186/s13023-016-0388-0
45
KolasaK.ZahV.KowalczykM. (2018). How can multi criteria decision analysis support value assessment of pharmaceuticals?-findings from a systematic literature review. Expert Rev. Pharmaco. Outcomes Res.18 (4), 379–391. doi: 10.1080/14737167.2018.1467759
46
KolasaK. (2014). Are pricing and reimbursement decision-making criteria aligned with public preferences regarding allocation principles in the Polish healthcare sector? Expert Rev. Pharmaco. Outcomes Res.14 (5), 751–762. doi: 10.1586/14737167.2014.940903
47
KristensenF. B.LampeK.WildC.CerboM.GoettschW.BeclaL. (2017). The HTA Core Model®—10 years of developing an international framework to share multidimensional value assessment. Value Health20 (2), 244–250. doi: 10.1016/j.jval.2016.12.010
48
KujawskiE. (Ed.) (2003). “4.7. 3 Multi-Criteria Decision Analysis: Limitations, Pitfalls, and Practical Difficulties,” in INCOSE International Symposium. Vol. 13 no. 1. (Washington, DC: Wiley Online Library), 1169–1176. doi: 10.1002/j.2334-5837.2003.tb02692.x
49
KwonS.-H.ParkS.-K.ByunJ.-H.LeeE.-K. (2017). Eliciting societal preferences of reimbursement decision criteria for anti cancer drugs in South Korea. Expert Rev. Pharmaco. Outcomes Res.17 (4), 411–419. doi: 10.1080/14737167.2017.1277144
50
LakdawallaD. N.DoshiJ. A.GarrisonL. P.Jr.PhelpsC. E.BasuA.DanzonP. M. (2018). Defining elements of value in health care—a health economics approach: an ISPOR Special Task Force report [3]. Value Health21 (2), 131–139. doi: 10.1016/j.jval.2017.12.007
51
MarshK.LanitisT.NeashamD.OrfanosP.CaroJ. (2014). Assessing the value of healthcare interventions using multi-criteria decision analysis: a review of the literature. Pharmacoeconomics32 (4), 345–365. doi: 10.1007/s40273-014-0135-0
52
MarshK.IJzermanM.ThokalaP.BaltussenR.BoysenM.KalóZ.et al. (2016). Multiple criteria decision analysis for health care decision making—emerging good practices: report 2 of the ISPOR MCDA Emerging Good Practices Task Force. Value \Health19 (2), 125–137. doi: 10.1016/j.jval.2015.12.016
53
MarshK.CaroJ. O.HamedA.ZaiserE. (2017). Amplifying each patient’s voice: a systematic review of multi-criteria decision analyses involving patients. Appl. Health Econ. Health Policy15 (2), 155–162. doi: 10.1007/s40258-016-0299-1
54
MarshK. D.SculpherM.CaroJ. J.TervonenT. (2018). The use of MCDA in HTA: great potential, but more effort needed. Value Health21 (4), 394–397. doi: 10.1016/j.jval.2017.10.001
55
Memorial Sloan Kettering Cancer Center (MSKCC) (2015). DrugAbacus tool. DrugPricingLab (Downloaded: 16 March 2019). Available from: https://drugpricinglab.org/tools/drug-abacus/methods.
56
NémethB.MolnárA.BozókiS.WijayaK.InotaiA.CampbellJ. D.et al. (2019). Comparison of weighting methods used in multicriteria decision analysis frameworks in healthcare with focus on low-and middle-income countries. J. Comp. Effect. Res.8 (4), 195–204. doi: 10.2217/cer-2018-0102
57
NeumannP. J.WillkeR. J.GarrisonL. P.Jr. (2018). A health economics approach to US value assessment frameworks—introduction: an ISPOR Special Task Force report [1]. Value Health21 (2), 119–123. doi: 10.1016/j.jval.2017.12.012
58
OortwijnW.Sampietro-ColomL.HabensF. (2017). Developments in value frameworks to inform the allocation of healthcare resources. Int. J. Technol. Assess. Health Care33 (2), 323–329. doi: 10.1017/S0266462317000502
59
RadaelliG.LettieriE.MasellaC.MerlinoL.StradaA.TringaliM. (2014). Implementation of EUnetHTA core model® in Lombardia: the VTS framework. Int. J. Technol. Assess. Health Care30 (1), 105–112. doi: 10.1017/S0266462313000639
60
RaduC.-P.ChiriacN. D.PravatA. M. (2016). The development of the Romanian scorecard HTA system. Value Health Reg. Issues10, 41–47. doi: 10.1016/j.vhri.2016.07.006
61
RamliA.AljunidS. M.SulongS.YusofF. A. M. (2013). National Drug Formulary review of statin therapeutic group using the multiattribute scoring tool. Ther. Clin. Risk Manage.9, 491. doi: 10.2147/TCRM.S52078
62
RudmikL.DrummondM. (2013). Health economic evaluation: important principles and methodology. Laryngoscope123 (6), 1341–1347. doi: 10.1002/lary.23943
63
SorensonC.KanavosP.KaramalisM. (2009). HTA in central and Eastern Europe: current status, challenges and opportunities. J. Med. device Regul.6 (1), 34–45.
64
SussexJ.RolletP.GarauM.SchmittC.KentA.HutchingsA. (2013). A pilot study of multicriteria decision analysis for valuing orphan medicines. Value Health16 (8), 1163–1169. doi: 10.1016/j.jval.2013.10.002
65
TaniosN.WagnerM.TonyM.BaltussenR.van TilJ.RindressD.et al. (2013). Which criteria are considered in healthcare decisions? Insights from an international survey of policy and clinical decision makers. Int. J. Technol. Assess. Health Care29 (4), 456–465. doi: 10.1017/S0266462313000573
66
ThokalaP.DevlinN.MarshK.BaltussenR.BoysenM.KaloZ.et al. (2016). Multiple criteria decision analysis for health care decision making—an introduction: report 1 of the ISPOR MCDA Emerging Good Practices Task Force. Value Health19 (1), 1–13. doi: 10.1016/j.jval.2015.12.003
67
TonyM.WagnerM.KhouryH.RindressD.PapastavrosT.OhP.et al. (2011). Bridging health technology assessment (HTA) with multicriteria decision analyses (MCDA): field testing of the EVIDEM framework for coverage decisions by a public payer in Canada. BMC Health Serv. Res.11 (1), 329. doi: 10.1186/1472-6963-11-329
68
ToumiM.RémuzatC. (2017). Value Added Medicines - Time to Adjust the HTA Decision Frameworks (17 March 2019). Available from: https://www.medicinesforeurope.com/wp-content/uploads/2017/09/White-paper_HTAonValueaddedmedicines_Final.pdf.
69
Velasco-GarridoM.BusseR. (2005). Health technology assessment: an introduction to objectives, role of evidence, and structure in Europe. Health Technol. Assess.: an introduction to objectives role evidence structure Europe. 24 p.
70
WilliamsP.MauskopfJ.LebieckiJ.KilburgA. (2014). Using multicriteria decision analysis during drug development to predict reimbursement decisions. J. Market Access Health Policy2 (1), 25270. doi: 10.3402/jmahp.v2.25270
71
WisemanV.MittonC.Doyle-WatersM. M.DrakeT.ContehL.NewallA. T.et al. (2016). Using economic evidence to set healthcare priorities in low-income and lower-middle-income countries: a systematic review of methodological frameworks. Health Econ.25, 140–161. doi: 10.1002/hec.3299
72
Working Group on Mechanism of Coordinated Access to Orphan Medicinal Products (MoCA-OMP) (2014). Transparent Value Framework (17 March 2019). Available from: http://download2.eurordis.org.s3.amazonaws.com/moca/history/WG%20MoCA-OMP%20Transparent%20Value%20Framework.pdf
73
World Bank Country and Lending Groups Country Classification (2020). [cited 2019 17 Sept]. Available from. https://datahelpdesk.worldbank.org/knowledgebase/articles/906519-world-bank-country-and-lending-groups.
74
YoungkongS. (2014). Application of HTA research on policy decision-making. J. Med. Assoc. Thailand= Chotmaihet thangphaet97, S119–S126.
75
ZahV.ThompsonS.BerjanM.GoereeR. (2015). Zrx Mcdm: a fully flexible tool to support the local adaptation of multiple-criteria decision criteria in health care. Value Health18 (7), A703. doi: 10.1016/j.jval.2015.09.2634
76
ZeleiT.MolnárM. J.SzegediM.KalóZ. (2016). Systematic review on the evaluation criteria of orphan medicines in Central and Eastern European countries. Orphanet. J. Rare Dis.11 (1), 72. doi: 10.1186/s13023-016-0455-6
Summary
Keywords
health technology assessment, pharmaceutical, reimbursement, developing country, multiple criteria
Citation
Jakab I, Németh B, Elezbawy B, Karadayı MA, Tozan H, Aydın S, Shen J and Kaló Z (2020) Potential Criteria for Frameworks to Support the Evaluation of Innovative Medicines in Upper Middle-Income Countries—A Systematic Literature Review on Value Frameworks and Multi-Criteria Decision Analyses. Front. Pharmacol. 11:1203. doi: 10.3389/fphar.2020.01203
Received
06 March 2020
Accepted
23 July 2020
Published
14 August 2020
Volume
11 - 2020
Edited by
Sam Salek, University of Hertfordshire, United Kingdom
Reviewed by
Mohamed Izham Mohamed Ibrahim, Qatar University, Qatar; Luis Laranjeira, Eli Lilly, Portugal
Updates
Copyright
© 2020 Jakab, Németh, Elezbawy, Karadayı, Tozan, Aydın, Shen and Kaló.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Ivett Jakab, ivett.jakab@syreon.eu
This article was submitted to Pharmaceutical Medicine and Outcomes Research, a section of the journal Frontiers in Pharmacology
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.