SYSTEMATIC REVIEW article

Front. Public Health, 20 October 2021

Sec. Health Economics

Volume 9 - 2021 | https://doi.org/10.3389/fpubh.2021.689123

A Systematic Review of Methods and Study Quality of Economic Evaluations for the Treatment of Schizophrenia

  • 1. School of International Pharmaceutical Business, China Pharmaceutical University, Nanjing, China

  • 2. Center for Pharmacoeconomics and Outcomes Research, China Pharmaceutical University, Nanjing, China

  • 3. Sumitomo Pharma (Suzhou) Co., Ltd., Shanghai, China

Abstract

Background: Schizophrenia is a severe and complex disease with substantial economic and social burdens. Despite multiple treatment choices, adverse events, and impaired social functions are still challenges in clinical therapy. Pharmacoeconomic evaluations could provide evidence to help decision makers improve the utilization of scarce resources. However, there remains some challenges especially in modeling due to uncertainties in progression of schizophrenia. There are limited summaries about the overall methodologies of schizophrenia economic evaluations.

Objective: The aim of this study is to review the existing economic evaluations of antipsychotics for the treatment of schizophrenia and summarize the evidence and methods applied.

Methods: An electronic literature search was performed in PubMed, Web of Science, EBSCO host, The Cochrane Library and ScienceDirect from January 2014 to December 2020. Search terms included “schizophrenia,” “schizophrenic,” “pharmacoeconomic,” “economic evaluation,” “cost-effectiveness,” and “cost-utility.” The Literature was screened and extracted by two researchers independently and assessed with the Quality of Health Economic Studies (QHES) List and Consolidated Health Economic Evaluation Reporting Standards (CHEERS) Statement.

Results: A total of 25 studies were included in the review. The regions included Europe, North America, Asia and Africa. Most of the studies chose second-generation antipsychotics as comparators and integrated treatment sequences. Time horizons varied from 1 year to lifetime. The healthcare sector was the most common perspective, accordingly, most of the evaluations considered only direct medical costs. The Markov model and decision tree model were the most common choices. Adverse events, compliance and persistence were considered important parameters. Quality-adjusted life-years were the major outcomes applied to the economic evaluations. All utilities for health states and adverse events were collected from published literature. All of the studies applied uncertainty analysis to explore the robustness of the results. The quality of the studies was generally satisfactory. However, improvements were needed in the choice of time horizons, the measurements of outcomes and the descriptions of assumptions.

Conclusions: This study highlights the methodology of economic evaluation of schizophrenia. Recommendations for modeling method and future study are provided.

Introduction

Schizophrenia is a severe and complex mental illness with early onset coupled with behavior or cognitive disorders that have a significant impact on patients' family and society. A systematic review reported that the global age-standardized prevalence of schizophrenia was 0.28% and the prevalence of cases rose from 13.1 million cases in 1990 to 20.9 million cases in 2016 (1). The average annual healthcare costs were estimated to be between $23,887 and $24,988 according to a real-world retrospective study in the US (2). Patients may incur higher expenditures due to comorbidities that are common among them (3). Furthermore, indirect medical costs related to productivity lost or caregiving were 8.5 times higher compared with direct medical costs according to a retrospective study based on medical insurance database in Guangzhou, China (4). Also, schizophrenia has significant impact on caregivers of the patients. The well-being of both patients and caregivers could be affected during their cognitive appraisal processes of the illness, help-seeking experience and the interaction within the families (5) and the burden of caregivers exists in physical and mental health, social relationship, and financial life (6).

Economic evaluations could generate evidence incorporating both costs and consequences for decision makers to clarify different uses for scarce resources (7). Despite the multiple choices of medications in schizophrenia treatment, there still exist substantial burdens and difficulties in clinical therapies due to low adherence and adverse events. Thus, pharmacoeconomic evidence is required to balance the clinical effects with the resources consumed. However, there remain some challenges especially in modeling due to uncertainties in the progression of the diseases, emphasizing the requirements for systematic reviews of the methods applied in the analysis.

Previous systematic reviews have evaluated studies published since 2000 (810). However, none of them fully discussed the treatment sequences or methods applied. Furthermore, most of them adopted an extensive range of years of publication, which may not characterize the studies in recent years. Therefore, this study was conducted to review the model-based economic evaluations published recently for antipsychotics and summarize the modeling techniques, including model structures, basic settings, integration and translation of the clinical events, and selection of utility values. In addition, the review also aimed to assess the quality of the studies.

Methods

The systematic review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) statement developed by Moher et al. (11).

Eligibility Criteria

The inclusion criteria were as follows: (a) economic evaluations adopting cost-effectiveness analysis (CEA) or cost-utility analysis (CUA) approach; (b) patients diagnosed with schizophrenia with no limitation on gender or age; (c) intervention including all antipsychotics; and (d) outcomes presented as incremental cost-effectiveness (ICER). Studies were excluded if they met the any of the following criteria: (a) not reported in English; (b) not related to economic evaluation; (c) cost of illness, health-related quality of life or budget impact analysis studies; (d) abstracts or studies with full-text unavailable; (e) not model-based studies; and (f) chose clinical effect as the only outcome.

Search Strategy

An electronic literature search was performed in PubMed, Web of Science, EBSCO host, The Cochrane Library, ScienceDirect from January 2014 to December 2020. Search items included “schizophrenia,” “schizophrenic,” “pharmacoeconomics,” “economic evaluation,” “cost-effectiveness,” and “cost-utility.” The detailed strategy is provided in Supplementary Material. In addition, references from retrieved studies were searched manually to avoid missing data.

Data Extraction and Analysis

The included studies were screened, extracted and double checked by two researchers independently. Disagreements were resolved by discussion or by consulting a third researcher. General information was collected including title, first author's surname, year of publication, country or region, intervention, and treatment sequences. To summarize the methods applied, characteristics such as perspectives, type of costs, outcomes, model structures, and necessary parameters were recorded. The results and conclusions of studies were included in the extracted form but were not reported as main outcomes due to the arguments regarding the extrapolation of evaluation results (12). All the information was recorded and compared using Microsoft Excel 2016.

Quality Assessment

According to the review of quality assessment tools conducted by Walker et al. (13) in 2012, the Quality of Health Economic Studies (QHES) List was recommended to discriminate the quality of studies as a quantitative measurement. The Consolidated Health Economic Evaluation Reporting Standards (CHEERS) Statement (14) published in 2012 was recommended by the International Society for Pharmacoeconomics and Outcomes research (ISPOR) as a report checklist and for guidance to optimize the quality of reporting. Both QHES and CHEERS were applied to quantitatively and qualitatively assess the quality of studies.

Results

A total of 1,086 citations were retrieved from five electronic databases. After removing duplicates, 610 studies were eligible to enter the screening process and judgements were generated according to the titles and abstracts. Finally, 25 articles published in English were identified and included in the systematic review. A flow of the literature screening was provided in Figure 1 according to the PRISMA statement (11).

Figure 1

Basic Characteristics of Included Studies

The characteristics of the included studies were summarized in Table 1. The studies covered the regions of Europe (15, 18, 21, 2328, 32, 3436, 39), North America (16, 17, 31), Asia (19, 22, 29, 30, 33, 37, 38), and Africa (20). Eleven (44%) (16, 17, 1922, 29, 32, 36, 37, 39) of the 25 studies specified the onset or averaged ages of patients and four studies (16%) (15, 18, 23, 24) focused on patients in acute or relapsed states. Second-generation antipsychotics (SGAs) also known as atypical antipsychotics were the most common interventions chosen in the analyses, where seven studies (28%) (15, 18, 21, 24, 25, 27, 30) compared the cost-effectiveness of long-acting-injections (LAIs). Eight studies (32%) (1822, 25, 28, 39) included both first-generation and second-generation antipsychotics as compared interventions where haloperidol was the commonest (n = 7, 87.5%). Most of the studies measured the outcomes using quality-adjusted life-years (QALYs) (n = 23, 92%), while only 2 studies (8%) (19, 20) applied disability-adjusted life-years (DALYs). In addition, nine studies (18, 21, 24, 25, 27, 28, 30, 31, 33) reported both utility and clinical effects, where yearly relapse or relapse avoidance was the most frequent choice.

Table 1

ReferencesCountry/regionPatientsInterventionsPerspectivesType of CostsOutcome measurementDiscount rate
Einarson et al. (15)SwedenSchizophrenia patients with relapsePaliperidone LAI, olanzapine LAI, risperidone LAI, haloperidol LAI, oral olanzapineHealth care sector perspectiveDirect and indirect costsQALYsNA
Lachaine et al. (16)CanadaModerate to severe schizophrenia patients above 40Asenapine, olanzapineHealth care sector perspective, societal perspectiveDirect and indirect costsQALYs5%
Park et al. (17)United StatesSchizophrenia patients above 40Olanzapine, risperidone, quetiapine, ziprasidoneHealth care sector perspectiveDirect costsQALYs3%
Dilla et al. (18)SpainSchizophrenia patients in relapse due to low complianceOlanzapine LAI, risperidone LAIHealth care sector perspectiveDirect costsQALYs, relapse averted, life years3%
Anh et al. (19)VietnamSchizophrenia patients above 15Chlorpromazine, haloperidol, levopromazine, risperidone, clozapine, olanzapineHealth care sector perspectiveDirect costsDALYs3%
Lubinga et al. (20)UgandaSchizophrenia patients with average age of 25Chlorpromazine, haloperidol, risperidone, olanzapine, quetiapineSocietal perspectiveDirect and indirect costsDALYs3%
Druais et al. (21)FranceStable schizophrenia patients with average age of 38Paliperidone LAI, risperidone LAI, aripiprazole LAI, olanzapine LAI, haloperidol LAI, oral olanzapinePayer perspectiveDirect costsQALYs, relapse averted4%
Lin et al. (22)SingaporeSchizophrenia patients with average age of 37Amisulpride, aripiprazole, chlorpromazine, olanzapine, paliperidone, quetiapine, risperidone, sulpiride, trifluoperazine, ziprasidoneHealth care sector perspectiveDirect costsQALYs3%
Rajagopalan et al. (23)Scotland and WalesSchizophrenia patients in relapseLurasidone, aripiprazolePayer perspectiveDirect costsQALYs3.50%
Einarson et al. (24)FinlandSchizophrenia patients in relapseAripiprazole LAI, paliperidone LAI, olanzapine LAI, risperidone LAIHealth care sector perspectiveDirect costsQALYs, relapse avertedNA
Einarson et al. (25)PortugalSchizophrenia patientsPaliperidone LAI, risperidone LAI, haloperidol LAI, oral olanzapineHealth care sector perspectiveDirect costsQALYs, relapse avertedNA
Barnes et al. (26)United KingdomPatients unresponsive to clozapineOlanzapine, amisulprideSocietal perspectiveDirect costsQALYsNot specified
Einarson et al. (27)SpainSchizophrenia patientsPP3M, PP1MHealth care sector perspectiveDirect costsQALYs, relapse averted, hospitalization avertedNA
Einarson et al. (28)NetherlandsSchizophrenia patientsPP3M, PP1M, haloperidol LAI, risperidone microspheres, oral olanzapinePayer perspectiveDirect costsQALYs, relapse (hospitalization treated or out-patient treated)NA
Wiwat et al. (29)ThailandStable schizophrenia patients above 15Aripiprazole, risperidoneSocietal perspectiveDirect costsQALYs3%
Nuhoho et al. (30)United Arab EmiratesStable schizophrenia patientsPaliperidone LAI, other oral antipsychoticsPayer perspectiveDirect costsQALYs, rate of hospitalization, relapse, emergencyNA
Aigbogun et al. (31)United StatesStable schizophrenia patientsBrexpiprazole, cariprazine, lurasidonePayer perspectiveDirect costsQALYs, relapse averted, hospitalization avertedNA
Németh et al. (32)HungaryPatients with negative symptoms of schizophrenia with average age of 40Cariprazine, risperidonePayer perspectiveDirect costsQALYs4%
Zhao et al. (33)ChinaSchizophrenia patientsOlanzapine ODT, olanzapine SOT, aripiprazole SOTPayer perspectiveDirect costsQALYs, averaged annual relapseNA
Abdall-Razak et al. (34)United KingdomNot specifiedPaliperidone, amisulpridePayer perspectiveDirect costsQALYsNA
Dutina et al. (35)SerbianAdult patients about to receive for the second-line treatmentAripiprazole, olanzapinePayer perspectiveDirect costsQALYs3%
Arteaga et al. (36)FranceAdult chronic schizophrenic patients stabilized on PP1M with baseline age of 38.75PP3M, PP1MPayer perspectiveDirect costsQALYs4%
Yi et al. (37)ChinaSchizophrenia patients with starting age of 35Amisulpride, olanzapinePayer perspectiveDirect costsQALYs3%
Lin et al. (38)ChinaNot specifyAripiprazole ODT, aripiprazole SOT, olanzapine SOTPayer perspectiveDirect costsQALYNA
Jin et al. (39)United KingdomIndividuals referred to secondary care mental health services with mean age of 23.5Amisulpride, aripiprazole, haloperidol, olanzapine, quetiapine, risperidone, placebo, clozapinePayer perspectiveDirect costsQALYs3.5%

Basic characteristics.

LAI, long-acting-injection; ODT, orally disintegrating antipsychotic tablets; SOT, standard oral tablets; PP3M, paliperidone administered every 3 months; PP1M, paliperidone administered every month; QALY, quality-adjusted life year; DALY, disability-adjusted life year; NA, not applicable.

Due to the diverse efficacy and low compliance rate, therapeutic changes are common among schizophrenia patients, which makes treatment sequences worth consideration. Nineteen studies (76%) (1518, 2025, 2731, 33, 35, 36, 38) specified treatment sequences in the models (Table 2). The methods of setting second-line medications were flexible, including changing to drugs that differed from the first-line or applying mixed therapies based on market share (22, 30, 36) or simple averaging (22, 31). Clozapine was the most common last-line therapy.

Table 2

ReferencesConsideration of the treatment sequenceFirst-line treatmentSecond-line treatmentThird-line treatment
Einarson et al. (15)YesPaliperidone LAI or olanzapine LAI or risperidone LAI or oral olanzapine or haloperidol LAIOlanzapine LAI or paliperidone LAI or haloperidol LAI or oral olanzapine*Clozapine
Lachaine et al. (16)YesAsenapine or olanzapineAripiprazole or ziprazidone or risperidone or quetiapineNA
Park et al. (17)YesOlanzapine or risperidone or quetiapine or ziprasidoneOlanzapine or risperidone or quetiapine or ziprazidone*Clozapine
Dilla et al. (18)YesOlanzapine LAI or risperidone LAIOther antipsychoticsNA
Anh et al. (19)NoChlorpromazine or haloperidol or levopromazine or risperidone or clozapine or olanzapineNANA
Lubinga et al. (20)YesChlorpromazine or haloperidol or risperidone or olanzapine or quetiapineRisperidone or haloperidol*Perphenazine
Druais et al. (21)YesPaliperidone LAI or risperidone LAI or aripiprazole LAI or olanzapine LAI or haloperidol LAI or oral olanzapine25%paliperidoneLAI+25%risperidone LAI+25%aripiprazoleLAI+25%olanzapine LAI**NA
Lin et al. (22)YesAmisulpride or aripiprazole or chlorpromazine or haloperidol or olanzapine or paliperidone or quetiapine or risperidone or sulpiride or trifluoperazineor ziprazidoneThe other drugs excluded the first-line drugs***Clozapine
Rajagopalan et al. (23)YesLurasidone or aripiprazoleAmisulprideClozapine
Einarson et al. (24)YesAripiprazole or paliperidone or olanzapine or risperidoneOlanzapine or risperidone*Clozapine
Einarson et al. (25)YesPaliperidone or risperidone or haloperidol or olanzapineHaloperidol or olanzapine*Clozapine
Barnes et al. (26)NoOlanzapine or amisulprideNANA
Einarson et al. (27)YesPP3M or PP1MAripiprazoleClozapine
Einarson et al. (28)YesPP3M or PP1M or haloperidol or risperidone or olanzapineHaloperidol or oral olanzapineClozapine
Wiwat et al. (29)YesAripiprazole or risperidoneClozapineNA
Nuhoho et al. (30)YesPaliperidone LAI or paliperidone LAI plus oral antipsychoticsRisperidone or paliperidone or aripiprazole or olanzapine or quetiapine***Risperidone or paliperidone or aripiprazole or olanzapine or quetiapine*
Aigbogun et al. (31)YesBrexpiprazole or cariprazine or lurasidoneOlanzapine or risperidone or quetiapine or ziprazidone or aripiprazole**NA
Németh et al. (32)NoCariprazine or risperidoneNANA
Zhao et al. (33)YesOlanzapine ODT or olanzapine SOT or aripiprazole SOTAripiprazole or amisulpride or ziprazidone or clozapineNA
Abdall-Razak et al. (34)NoPaliperidone or amisulprideNANA
Dutina et al. (35)YesAripiprazole or olanzapineClozapineNA
Arteaga et al. (36)YesPP3M or PP1MPaliperidone, olanzapine, aripiprazole, risperidone***NA
Yi et al. (37)NoAmisulpride or olanzapineNANA
Lin et al. (38)YesAripiprazole ODT or aripiprazole SOT or olanzapine SOTNot specifyNA
Jin et al. (39)NoAmisulpride, aripiprazole, haloperidol, olanzapine, quetiapine, risperidone, placebo, clozapineNANA

Treatment sequences# in the included studies.

LAI, long-acting-injection; ODT, orally disintegrating antipsychotic tablets; SOT, standard oral tablets; PP3M, paliperidone administered every 3 months; PP1M, paliperidone administered every month; NA, not applicable.

#

If patients show no response to the therapies or relapse during the current treatment, they will change to the next-line treatment.

*

A single drug different from the previous treatment line was chosen as the current treatment line.

**

Clinical and economic inputs were determined by a weighted average with equal proportions of data for the drugs.

***

Clinical and economic inputs were determined by market share of the drugs.

Results of Methodology Review

Perspectives and Related Costs

Various terms were used to define the study perspectives of the reviewed articles. Therefore, the terms were classified into three categories in this review, which were defined as health care sector perspective (including healthcare system, ministry of health, national health service, and government), payer perspective (including payer, third-party payer, and health insurance), and societal perspective (including societal, modified societal, and broadly societal perspective) based on the report recommended by ISPOR (40). Notably, studies of single-payer health system countries using both the healthcare system and payer were classified into the payer perspective. The perspectives determine the types of costs considered in the analysis (7, 40). Direct medical costs, both direct and indirect costs, and costs paid by payers are most relevant to the health care sector perspective, the societal perspective, and the payer perspective, respectively (7, 41).

Among the eight studies (32%) choosing the health care sector perspective (15, 1719, 22, 24, 25, 27), seven studies adapted direct medical costs, while one study (15) included indirect costs which is inconsistent with the perspective. The payer perspective was chosen by 13 studies (52%) (21, 23, 28, 3039) where six studies specified costs paid by payers and seven studies (28, 31, 33, 3538) merely included direct medical costs. Among the 3 (14.3%) societal perspective-studies (20, 26, 29), only 1 study (20) took both direct and indirect costs into account, indicating that some confusion existed in distinguishing perspectives. One study chose both the healthcare sector and the societal perspective with direct and indirect costs (16).

Types of Models and Health States

The characteristics of the models and related health states are summarized in Table 3. The Markov model (17, 19, 20, 22, 23, 26, 29, 32, 3537) and the decision tree model (15, 24, 25, 27, 28, 30, 34) were the most common choices. One study (16) combined the decision tree model with the Markov model to better reflect complication-related treatment switches within the first year of treatment and long-term metabolic complications. Most of the Markov models consisted of a series of health states, which represent treatment sequences, disease progression or related adverse events and were connected by probabilities based on the averaged cohort level (27). Microsimulation models (18, 33, 38, 39) are more flexible and natural for simulating clinical reality by incorporating patient-level characteristics. Due to the differentiation in the choice of therapy and the treatment effect among patients, a patient-level simulation model as well as a Markov model incorporating treatment sequences, relapse, remission, adherence and adverse events could be more suitable for clinical practice.

Table 3

ReferencesTypes of modelModel statesTime horizonCycle length
Einarson et al. (15)Decision tree, cohortIncorporating clinical events including discontinuation, exacerbation, compliance, hospitalization1 yearNA
Lachaine et al. (16)Decision tree combined with 9-state Markov model, cohortDiabetes, stroke, CHDs, hypertension, no comorbidity, 2/3/4 comorbidities, death5–10 years1 year
Park et al. (17)9-state Markov model, cohortFirst line treatment with/without irreversible SE, 2nd line treatment with/without irreversible SE, clozapine treatment with/without irreversible SE, uncontrolled state with/without SE, death10 years18 weeks
Dilla et al. (18)Discrete event simulation, microsimulationTreatment, treatment emergent adverse events, relapse, doctor-initiated treatment re-evaluation, patient-initiated treatment discontinuation5 yearsNA
Anh et al. (19)3-state Markov model, cohortSchizophrenia patients, recovery patients, schizophrenia-specific and other causes of deathsLifetime1 year
Lubinga et al. (20)10-state Markov model, cohortResidual on/off 1st line AP, acute on/off 1st line AP, residual on/off 2nd line AP, acute on/off 2nd line AP, residual on 3rd lineLifetime1 year
Druais et al. (21)4-state Markov model, cohortStable treated, stable non-treated, relapse, death5 years3 months
Lin et al. (22)4-state Markov model, cohortStable treated, stable non-treated, relapse, deathLifetime1 year
Rajagopalan et al. (23)5-state Markov model, cohortNon-stable/relapse trial of antipsychotic agents, stable/adherent, stable/non-adherent, relapse, death10 years6 weeks
Einarson et al. (24)decision tree, cohortIncorporating clinical events including discontinuation, exacerbation, compliance, hospitalization1 yearNA
Einarson et al. (25)decision tree, cohortIncorporating clinical events including discontinuation, exacerbation, compliance, hospitalization1 yearNA
Barnes et al. (26)3-state Markov model, cohortSymptom response, SEs, death1–10 years3 months
Einarson et al. (27)decision tree, cohortIncorporating clinical events including discontinuation, exacerbation, compliance, hospitalization1 yearNA
Einarson et al. (28)decision tree, cohortIncorporating clinical events including stable, intolerant, relapse treated as out-patient, relapse requiring hospitalization and dropout1 yearNA
Wiwat et al. (29)4-state Markov model, cohort modelRemission with 1st antipsychotics, relapse, remission with clozapine, deathLifetime4 weeks
Nuhoho et al. (30)Decision tree, cohortIncorporating clinical events including adherence, exacerbation, hospitalization1 year3 months
Aigbogun et al. (31)Decision-analytic model, cohortIncorporating clinical events including treatment discontinuation, relapse/impending relapse, AEs1 yearNA
Németh et al. (32)8-state Markov model, cohortConstructed according to both severity of symptoms and disease types1–10 years1/12 weeks
Zhao et al. (33)Decision-analytic model, microsimulationIncorporating adherence levels, relapse with/without hospitalization, treatment discontinuation, AEs suicide risk1 year3 months
Abdall-Razak et al. (34)Decision tree, cohortIncorporating relapse, remission, AEs, diabetes complications1 yearNA
Dutina et al. (35)5-state Markov model, cohortRemission without AEs, remission with AEs, relapse, second response, death10 years3 months
Arteaga et al. (36)5-state Markov model, cohort1st-line treatment, no active treatment, 2nd-line treatment, relapse, death5 years1 month
Yi et al. (37)5-state Markov model, cohortAcute phase, remission, relapse, deathLifetime1 year
Lin et al. (38)Discrete event simulation, microsimulationIncorporating adherence levels, relapse with/without hospitalization, stable and adverse events1 yearNA
Jin et al. (39)Discrete event simulation, microsimulationIncorporating 4 module for different pathway with relevant interventionsLifetimeNA

Basic characteristics of the models in the included economic evaluations.

CHDs, coronary heart diseases; AP, antipsychotics; SEs, side effects; AEs, adverse events; NA, not applicable.

Time Horizon and Cycle Length

As summarized in Tables 3, 6 (24%) studies (19, 20, 22, 29, 37, 39) chose the lifetime horizon in the model. However, considering the low adherence and frequent changes of medication, a certain number of studies (15, 16, 18, 21, 2427, 3033, 35) chose a relatively short time horizon. Among these, most Markov model-based studies explored uncertainty by extending the time horizon as a complementary analysis, while most decision tree model-based or microsimulation model-based studies conducted mere 1-year analyses (15, 24, 25, 27, 30, 31, 33, 34). As a chronic mental disease, schizophrenia should be simulated for a long time period or even lifetime in the model. However, due to the uncertainty in therapy and disease progression, it might be challenging to simulate further into the future as it becomes more unpredictable. Therefore, selecting the appropriate time frame covering events in the near future and then exploring time horizon uncertainty may be a reasonable method for economic evaluations for schizophrenia.

The cycle lengths of Markov models in the included studies varied from 4 weeks to 1 year, where 3 months (21, 26, 30, 32, 33, 35) and 1 year (16, 19, 20, 22, 37) were the most frequently used. The reasons for 3-month cycle selection included appropriate capture the both clinical practice and associated events according to clinical opinion (21, 30), consistent with clinical trials (26), while explanation for 1-year cycle selection was consideration of the realistic treatment management of schizophrenia (37). The length of cycle selection should be depend on both disease and intervention (7), and should be short enough to avoid multiple changes within a single cycle (42). Therefore, a 1-year cycle length may be less preferable compared to a 3-month cycle length.

Adverse Events

Adverse events (AEs) could impact adherence, efficacy and therapy changes as well as health-related quality of life, especially for schizophrenia. Thus, it is meaningful to take relevant AEs into account in the models. The AEs selected in the evaluations are listed in Table 4. The effect of the extrapyramidal system (EPS), weight gain, and diabetes were the most common adverse events considered. A few studies built separate health states for possible AEs to reflect the influence by applying transition probabilities, costs and utility of the health states (16, 22, 26). The majority of studies incorporated the prevalence, costs and disutility of AEs in each cycle to reflect the impact on disease progression. Both methods were acceptable as long as the choice was made based on proper consideration of the disease progression and available evidence (42).

Table 4

ReferencesConsideration and description of AEsTypes of AEs
EPS*Weight gainDiabetesHyperprolactinemiaMetabolic events**Others
Einarson et al. (15)××××××NA
Lachaine et al. (16)×NA
Park et al. (17)××Agranulocytosis
Dilla et al. (18)×××Somnolence, sexual dysfunction, postinjection syndrome, suicide
Anh et al. (19)×××Agranulocytosis
Lubinga et al. (20)××Ischemic heart disease
Druais et al. (21)××NA
Lin et al. (22)××NA
Rajagopalan et al. (23)××NA
Einarson et al. (24)××××××NA
Einarson et al. (25)××××××NA
Barnes et al. (26)××Sexual dysfunction, aversive subjective experience, cardiac symptoms
Einarson et al. (27)××××××NA
Einarson et al. (28)××××××NA
Wiwat et al. (29)×NA
Nuhoho et al. (30)××××××NA
Aigbogun et al. (31)××NA
Németh et al. (32)××××NA
Zhao et al. (33)×NA
Abdall-Razak et al. (34)××Diabetes complications: amputation, MI, stroke, IHD, HF
Dutina et al. (35)×××Neutropenia
Arteaga et al. (36)×NA
Yi et al. (37)×Liver function damage
Lin et al. (38)NA
Jin et al. (39)Neutropenia

Summary of the adverse events considered in the included economic evaluations.

AEs, adverse events; MI, myocardial infarction; IHD, ischemic heart disease; HF, heart failure; NA, not applicable.

*

EPS (effects of the extrapyramidal system) including akathisia, and tardive dyskinesia.

**

Metabolic events including pathoglycemia, dyslipidemia and hypertension.

However, some studies did not fully describe the consideration of AEs (27, 30) or even did not include AEs (15, 24, 25, 28) due to the similar incidence rates, small expenditures or short time horizon. It should be discussed whether the 1-year time horizon was sufficient to capture all important and interesting outcomes, since such a time horizon might fail to capture the impact on both health-related quality of life and costs of relevant clinical events among drugs. It is also notable that AEs with different durations should be clearly described to calculate the respective additional costs and disutilities. Park et al. (17) and Arteaga et al. (36) classified the reversible and irreversible AEs and assumed that reversible AEs would last for 18 weeks or 3 months while irreversible AEs would remain for the rest of the period. Rajagopalan et al. (23) classified the AEs into one-off AEs (such as EPS), persistent AEs (such as weight gain), and cumulatively occurred AEs (such as diabetes). Abdall-Razak et al. (34) clearly described the health-related utilization of the AEs and summarized the calculation of utilities of each AEs. However, more than half of the studies failed to clearly defined both costs and utilities considerations of the AEs.

Compliance and Persistence

Medication compliance (also known as adherence) can be defined as the extent to which the medication-taking of a patient matches that defined by the prescriber while medication persistence (also known as continuous adherence or discontinuation rate) refers to the act confirming to the recommended continuing treatment for the duration of time from the prescriber (43, 44). Both of them could influence the risk of relapse, rehospitalization, costs, quality of life through different aspects, especially for chronic disorders such as schizophrenia (44). Due to the different definitions and roles, it is important to not only take both compliance and persistence into account but also distinguish them when building economic models.

Information about the adoption of compliance or persistence in the included studies is listed in Table 5. The management of medication compliance or medication persistence can be summarized into three types: (1) Patients classified based on the compliance rates (adherence rates) from existing literature and then search for inputs under relevant compliance (15, 24, 25, 30, 33, 38). Such a method is usually applied in decision tree models and microsimulation models. (2) Non-persistence rates (discontinuation or dropout rates) are used rather than compliance to reflect the medication behaviors (17, 18, 2022, 27, 28, 31, 36, 39). In such studies, non-persistence rates were divided into lack of efficacy, adverse events, intolerance, patients' choice and so on. Patients with treatment withdrawn due to lack of efficacy, adverse events or intolerance would change their therapies according to the treatment sequence. Patients who stopped treatment due to their own choices or for other reasons were defined as not receiving any therapies in the next simulation. Such methods were found in Markov model evaluations (3). Compliance not considered in the analysis. Lachaine et al. (16) assumed that patients remained on their medication continually for 5 years which was chosen as relatively short period to reduce uncertainty. Additionally, Abdall-Razak et al. (34) and Németh et al. (32) did not introduce the non-persistence rate due to a lack of relevant data.

Table 5

ReferencesExpressionRoles in the modelSource of the dataStatistical methods
CompliancePersistenceRetrospective studyClinical trialsObservational studyReview
Einarson et al. (15)(a) Act as branches behind chance nodes, (b) influence the probabilities for eventsSimple average
Lachaine et al. (16)××NA
Park et al. (17)×Lead to therapy changesKaplan-Meier discontinuation curves
Dilla et al. (18)×Lead to therapy changes
Anh et al. (19)×Not specifiedSimple average
Lubinga et al. (20)×Lead to relapseNA
Druais et al. (21)×(a) Act as transition probabilities, (b) influence relapse of the diseaseNA
Lin et al. (21)×Lead to change or discontinuation of the therapyNA
Rajagopalan et al. (23)×Act as transition probabilitiesRegression analysis, partial assumption
Einarson et al. (24)(a) Act as branches behind chance nodes, (b) influence the probabilities of eventsPartial assumption, simple average
Einarson et al. (25)(a) Act as branches behind chance nodes, (b) influence the probabilities of eventsPartial assumption, simple average
Barnes et al. (26)Not specifiedNA
Einarson et al. (27)×(a) Act as branches behind chance nodes, (b) influence the probabilities of eventsNA
Einarson et al. (28)×(a) Lead to change or discontinuation of the therapy, (b) influence the relapse of the diseaseNA
Wiwat et al. (29)Influence relapse of the diseaseNA
Nuhoho et al. (30)(a) Act as branches behind chance nodes, (b) influence the probabilities of eventsNA
Aigbogun et al. (31)×Lead to change or discontinuation of the therapyIndirect comparison based on data from clinical trials
Németh et al. (32)××NA
Zhao et al. (33)Classified different types of patientsAssumption
Abdall-Razak et al. (34)××NA
Dutina et al. (35)Not specifiedNA
Arteaga et al. (36)×Act as transition probabilitiesNA
Yi et al. (37)×Act as transition probabilitiesNA
Lin et al. (38)Classified different types of patientsNA
Jin et al. (39)Persistence rate acting as transition probabilities while non-compliance seen as reason for non-persistenceNA

Summary of the methods used to integrate medication compliance.

NA, not applicable.

Various sources of reference for compliance or persistence were adopted in the evaluations, including retrospective studies, clinical trial studies, and naturalistic studies. Simple averaging was the major method adopted for multisource data. Most discontinuation rates were collected from clinical trials. Even though there might be differences between the definitions of compliance and persistence, most of the studies did not explain or discuss this issue. Considering the definition of compliance, data from real-world studies might be superior as no intervention has been implemented to influence the medication behaviors of patients.

Utility

According to the health states defined in the model, the most commonly used utilities in the studies were those for stable schizophrenia, relapse without hospitalization, relapse with hospitalization and the disutilities of EPS, weight gain, and diabetes. Utilities and references are summarized in Tables 6, 7. Utilities of the stable state, relapse without hospitalization and relapse with hospitalization ranged from 0.65 to 0.919, 0.485 to 0.762, and 0.27 to 0.604, respectively. Apart from EPS, weight gain and diabetes, the utilities or disutilities of hypertension, coronary heart disease, stroke, and hyperprolactinemia were also considered in some studies. Utilities from Lenert et al. (45) and Briggs et al. (46) were the most frequently referenced among the studies.

Table 6

StudyStableNon-hospitalized relapseHospitalized relapseReferences
Einarson et al. (15)0.890.6590.49(4549)
Lachaine et al. (16)0.75NANA(45)
Park et al. (17)0.856NA−0.358(46)
Dilla et al. (18)0.77−0.18From SOHO data
Druais et al. (21)0.9190.7620.604(46)
Lin et al. (22)0.8NA0.67(45)
Rajagopalan et al. (23)0.799NA0.67(45, 46, 50)
Einarson et al. (24)0.890.6590.49(4549)
Einarson et al. (25)0.890.6590.49(4549)
Barnes et al. (26)0.696NANAAMICUS trial
Einarson et al. (27)0.7/0.65*0.485/0.469*0.27(21, 51, 52)
Einarson et al. (28)0.890/0.840/0.795/0.790**0.690/0.665/0.643/0.640**0.49(21, 51, 52)
Wiwat et al. (29)0.69NA0.58(22)
Nuhoho et al. (30)0.890.6590.49(4549, 53)
Aigbogun et al. (31)0.880.740.53(45, 54)
Németh et al. (32)Not reportedNot reportedNot reported(39)
Zhao et al. (33)0.880.740.53(45), expert opinion
Abdall-Razak et al. (34)0.7990.670.67(45)
Dutina et al. (35)0.9190.6040.604(46)
Arteaga et al. (36)0.916/0.865##−0.358−0.358(46, 51)
Yi et al. (37)0.920.74#0.60(45, 46)
Lin et al. (38)0.88/0.75/0.75###0.74/0.63/0.63###0.53/0.53/0.42###(45), expert opinion
Jin et al. (39)0.800.67NA(45)

Summary of the utilities for health states of schizophrenia.

NA, not applicable.

*

Data presented as utility for patients receiving injection therapy monthly/utility for patients receiving injection therapy every 3 months.

**

Data presented as utility for patients receiving paliperidone 3-month injection/paliperidone 1-month injection/risperidone long-acting injection/haloperidol decanoate injection, oral olanzapine and clozapine.

#

Data refer to acute phase.

##

Data presented as utility for patients receiving paliperidone 3-month injection/paliperidone 1-month injection.

###

Data presented as utility for full compliance, partial compliance and non-compliance patients.

Table 7

StudyEPSWeight gainDiabetesOthersReferences
Einarson et al. (15)NA
Lachaine et al. (16)−0.074−0.031−0.06/−0.05*Hypertension: −0.02 CHD: −0.07/−0.06* stroke: −0.17/−0.18*(45, 55)
Park et al. (17)−0.256NA−0.151Hyperprolactinemia: −0.089 CHD: −0.151(46)
Dilla et al. (18)−0.054−0.003NASexual disfunction: −0.066From SOHO data
Druais et al. (21)−0.197−0.094−0.15(46)
Lin et al. (22)0.72***0.77***0.77***myocardial infarction: 0.74*** stroke: 0.64*** poststroke: 0.70***(45, 56, 57)
Rajagopalan et al. (23)89%**96%**−0.15NA(45, 46, 50)
Einarson et al. (24)NA
Einarson et al. (25)NA
Barnes et al. (26)NANANAAdverse events-0.006AMICUS trial
Einarson et al. (27)NANANANANA
Einarson et al. (28)NANANANANA
Wiwat et al. (29)0.62***0.66***0.66***Hyperprolactinemia: 0.62(22, 58)
Nuhoho et al. (30)NA
Aigbogun et al. (31)−0.099−0.036NAAkathisia: −0.09 pathoglycemia: −0.067 dyslipidemia: −0.099 sedation: −0.084(45, 5961)
Németh et al. (32)Not reportedNot reportedNot reportedNot reported(62)
Zhao et al. (33)88.8%**95.9%**NANA(45)
Abdall-Razak et al. (34)88.8%**95.9%**0.76***Amputation: −0.109, non-fatal myocardial infarction: −0.129, non-fatal stroke: −0.181, heart failure: −0.108, ischemic heart disease: −0.132(45, 50, 56, 63)
Dutina et al. (35)−0.256NANAMetabolic syndrome: −0.132(46)
Arteaga et al. (36)−0.256−0.089−0.151Prolactin-related syndrome: −0.089(46)
Yi et al. (37)0.72***0.83***NAincreased blood glucose level: 0.77***, liver function damage 0.75***, hyperprolactinemia: 0.82***(45, 64)
Lin et al. (38)88.8%**95.9%**88.8%**hyperlipidemia/hyperprolactinemia: 88.8%**(45)
Jin et al. (39)−0.07−0.03−0.09NA(45, 56)

Summary of the utilities of adverse events or complications.

*

Data presented as utility for male/utility for female.

**

Data presented as percentage reduction in utility for the presence of adverse effects.

***

Data presented as utility values.

It was noteworthy that the utilities for the same disease states varied greatly. Possible reasons included varieties in the classification of the researched states, selection of the population, and different methods or instruments applied among referenced quality of life studies.

Definitions or classifications of the health states were usually developed based on a literature review (45, 46), expert opinions (45, 46) and interviews with patients or laypersons (46) rather than through a unified method. Such states are usually framed by several items or descriptions; however, few studies discussed the applicability for states in decision-analytical models.

Layperson, patients with schizophrenia and caregivers are common responders in research on health-related quality of life. The differentiation of responders could induce heterogeneity among the results. For example, Briggs et al. (46) discovered significant differences between laypersons and patients, especially for relapse and EPS states. Regardless of the doubt regarding the response ability of patients with schizophrenia, studies have demonstrated that schizophrenia patients in the stable stage were able to provide valid and reliable answers, indicating the necessity to include such a population (48, 53).

Due to the specialty of mental disease as well as the choice of population such as laypersons, caregivers, or psychiatrists, the majority of the methods generated utilities using the standard gambling or time trade off approaches. The EuroQol-5 dimensions (EQ-5D) questionnaire is preferred by The National Institute for Health and Care Excellence (NICE). However, the sensitivity of the EQ-5D index to capture both social and psychological well-being for patients with schizophrenia is still controversial (65).

Results of the Economic Evaluations

The included studies compared the cost-effectiveness of commonly used SGAs (including long-acting injections). However, due to economic, political, cultural diversities among the different regions, the results of one economic evaluation may not be applicable beyond the defined setting (12). Thus, the results of the evaluation are not to be introduced here, but detailed information would be provided in the Supplementary Material 1. All of the studies adopted sensitivity analysis to verify the robustness of the base-case results. Notably, a change in the time horizon in different scenarios could lead to inverse results (17, 32). Accordingly, discussion about the time horizon may be required for schizophrenia therapy.

Quality Assessment of the Included Studies

The widely used QHES and CHEERS lists economic evaluation checklists were applied for a quantitative and qualitative review.

Assessed with the QHES list, scores ranged from 60 to 93 for 25 studies, where 21 (84.0%) studies scored between 75 and 93, and 4 (16.0%) studies scored between 60 and 74, indicating the relatively high quality of the majority studies. As summarized in Table 8 and Figure 2, all of the studies met the requirements of item 1, item 4, item 6, and item 15 representing the descriptions of study objective, subgroup analysis, incremental analysis, and conclusion, respectively. However, no more than half of the studies met the requirements of item 8 (choice of appropriate time horizon and discount rate) and item 13 (statement and justification of the choice of model, assumptions, and limitations).

Table 8

StudyQHES itemsScores
12345678910111213141516
Einarson et al. (15)×××81
Lachaine et al. (16)×93
Park et al. (17)×93
Dilla et al. (18)×××85
Anh et al. (19)××××76
Lubinga et al. (20)××87
Druais et al. (21)×93
Lin et al. (22)××85
Rajagopalan et al. (23)××86
Einarson et al. (24)××××73
Einarson et al. (25)×××81
Barnes et al. (26)××86
Einarson et al. (27)××86
Einarson et al. (28)×××78
Wiwat et al. (29)××86
Nuhoho et al. (30)×××77
Aigbogun et al. (31)××86
Németh et al. (32)×××××70
Zhao et al. (33)×××78
Abdall-Razak et al. (34)×××××64
Dutina et al. (35)×××××60
Arteaga et al. (36)×96
Yi et al. (37)××87
Lin et al. (38)××××78
Jin et al. (39)×××82
Qualified studies25/2520/2517/2525/2524/2525/2518/2512/2519/2523/2522/2524/2512/2517/2525/2523/25

Quality assessment of economic evaluations with the QHES checklist.

Figure 2

The quality reports evaluated with the CHEERS checklists were showed in Table 9 and Figure 3. The fulfilled items ranged from 16 to 23 for each study. More than half of the studies failed to meet the requirements for time horizon, preference-based outcome measurement, and assumptions. A considerable number of the studies selected a relatively short time horizon, which may be insufficient to capture necessary events. Most of the studies did not fully explain the reasons for time horizon selections. A large number of the studies chose utility values from the published literature, while some studies did not describe the methods used to elicit preference for outcomes. Studies seldom reported or explained the assumptions in single paragraph or table, adding to the difficulty identifying all assumptions underpinning the decision-analytical model. In addition, descriptions were for the parameters included, such as selection of population, measurement of effectiveness, and choice of model type.

Table 9

StudyCHEERS itemsFulfilled items
123456789101112131415161718192021222324
Einarson et al. (15)××××××18/24
Lachaine et al. (16)×××21/24
Park et al. (17)×××21/24
Dilla et al. (18)×××21/24
Anh et al. (19)××××××18/24
Lubinga et al. (20)×××21/24
Druais et al. (21)×23/24
Lin et al. (22)×××××19/24
Rajagopalan et al. (23)×23/24
Einarson et al. (24)××××××××16/24
Einarson et al. (25)×××××19/24
Barnes et al. (26)×××××××××15/24
Einarson et al. (27)×××21/24
Einarson et al. (28)××××××18/24
Wiwat et al. (29)×××××19/24
Nuhoho et al. (30)×××××××17/24
Aigbogun et al. (31)××22/24
Németh et al. (32)×××××19/24
Zhao et al. (33)××××20/24
Abdall-Razak et al. (34)×××××××××15/24
Dutina et al. (35)××××××××16/24
Arteaga et al. (36)××××××18/24
Yi et al. (37)×××××××17/24
Lin et al. (38)××××××××17/24
Jin et al. (39)×××22/24
Qualified studies22/2523/2523/2515/2520/2523/2525/2512/2522/2525/2516/219/2520/2520/2518/2511/2521/2516/2525/2521/2520/2524/2523/2522/25

Quality assessment report of economic evaluations with the CHEERS checklist.

Figure 3

Unlike the CHEERS checklist acting as a recommendation of the report format, the QHES list was developed to appraise the quality of economic evaluation (13, 14). According to the assessment of the QHES and CHEERS lists, most of the studies could be identified as relatively high-quality analyses, but certain limitations existed regarding the report quality. The selection of an appropriate time horizon and description of assumptions are frequent limitations for the included studies or even for economic evaluations in other chronic diseases (66, 67).

To compare the quality among the included studies, model types, regions, and time horizons were used as the indicators to classify the studies. The average QHES score of Markov model studies was 84.46 (13 studies) which was higher than that of decision tree model studies (77.14, 7 studies). The average score of the studies applying microsimulation was 80.75. The average numbers of the items consistent with the CHEERS recommendations were 19.23, 17.71, and 20 for Markov model studies, decision tree model studies, and microsimulation model studies. As a result, the Markov model and microsimulation model rather than the decision tree model are more appropriate model types for the study of schizophrenia. The quality of studies among different regions was also slightly different. The average QHES scores of the studies from North America, Asia, and European countries were 90.67, 81.00, and 80.07, respectively. It should be noticed that certain discrepancies exist among the scores of the studies from European countries where the maximum and minimum scores were 96 and 64. The numbers of the items consistent with the CHEERS recommendations (21.3, 18.14, and 18.86, respectively) were similar. However, even though the studies from North America seemed to be of higher quality, the numbers of the studies from the three regions differed a lot (three studies from North America, 7 studies from Asia, and 14 studies from European countries) and this might introduce bias when assessing the qualities.

To analyze the quality differences of studies with different time horizons, we classified the studies into two categories: short-term studies, i.e., the time horizon was 1 year or less, and long-term studies, i.e., the time horizon was longer than 1 year. The averaged QHES scores of the short-term studies (78.2, 10 studies) were lower than that of the longer-term studies (84.3, 15 studies). However, the number of the items consistent with the CHEERS recommendations are similar (18.3 for short-term studies vs. 19.53 for long-term studies). As mentioned above, the QHES list was developed to appraise the quality of economic evaluation while the CHEERS checklist was developed to recommend the report format. It can be inferred from the description of the time horizon from the two lists that the QHES list (Did the analytic horizon allow time for all relevant and important outcomes?) was more subjective and focused on the relationship between the item assessed and model outcomes. While the CHEERS checklist [State the time horizon(s) over which costs and consequences are being evaluated and say why appropriate] highlighted the fact that the statement of relevant aspect was provided and with no requirement to judge the appropriateness for the evaluation. Thus, attention should be paid when choosing checklists and interpreting the results.

Discussion

Purpose of Systematic Review of Economic Evaluations

With the increasing number of publications on health economic studies in recent years, systematic reviews in this filed have caught the attention of decision makers as useful tools to generate evidence (68). However, concerns have been voiced regarding whether cost-effectiveness findings can be transferred from one setting to another (12). Anderson et al. discussed the generalizability of the results of health economic studies and concluded that differences in methods, context, intervention costs, and effects contributed to the limitations on evidence synthesis (12). Gomersall et al. also discussed the debate over futility vs. utility of systematic reviews of economic evidence, emphasizing the inability to compare resources between countries and the differences in context and population (69). Pigone et al. also pointed out the challenges in the presentation of economic systematic reviews due to the large amounts of both synthesized data and generated results (70).

Despite challenges in generating results of economic evaluations via systematic reviews, it was suggested that systematic reviews should focus on methods of model development, sources of both efficacy and utility data, and resources used for specific diseases. It might be valuable for researchers and decision makers to identify the differences among studies (69, 70). For model-based economic evaluations, in particular, model structure selection was recommended based on the summarized existing studies, which could provide relatively comprehensive consideration for the necessary model states and link clinical practice and hypothetical model transitions. Notably, compared with differences in national contexts, it was found that variability in published economic evaluations was related more to the variety of study (71), confirming the importance of summarizing the model methods applied to existing studies. Thus, this systematic review focuses on the methods and study quality of the 25 eligible studies.

Main Findings

Among the included studies, most compared the cost-effectiveness of SGAs or long-acting injections. Fifteen studies considered the treatment sequences in model development, which could enlighten the future model development.

Despite the burden and productivity loss due to schizophrenia, few studies chose the societal perspective covering the indirect costs. Due to the early onset of schizophrenia, the average age of these patients is younger than that of patient with other chronic diseases. It would be necessary to include indirect costs in the evaluation. A retrospective study in the US concluded that indirect and non-health care costs were strong contributors and could be more than 70% of the total burden (72). A systematic review of the indirect costs of schizophrenia in Europe found that the average proportion of indirect costs of total disease expenditure was 44% (73). In China, the per case per annum indirect costs of schizophrenia were approximately US$1723.4 in 2013, accounting for 66.6% of the total costs (74). Therefore, a societal perspective covering both direct and indirect costs is preferable.

Description of the treatment sequences from most of the included studies could improve the model design and reflect the clinical prescription especially for chronic diseases or patients with high rate of therapy change. However, there remains challenges considering treatment sequences in economic evaluation. Medication treatment may vary among individuals due to the genotypes, metabolism, comorbidities, adverse events and so on (75). Also, the followed medication therapy should be impact by the previous medication choices. Thus, it may be questioned to apply a uniform sequency for patients especially in the cohort model. Cost and effectiveness data for the multiple drugs and the analytical methods are other challenges for treatment sequences. Even though choice of single drug based on market share or expert opinions and weighted or unweighted averaged data from multiple drugs are common methods in studies, the appropriateness requires further discussion.

According to the summary of the models applied, the Markov model was the most frequently used and treatment sequences, relapse, remission, and adverse events were the important health state elements in model development. The time horizons varied from 5 years to lifetime for the Markov models. While for the decision tree models, a 1 year time horizon was preferred. Due to the uncertainty in the time frame of treatment, it is recommended the impact of the time horizon be explored in a sensitivity analysis. Adverse events such as EPS, weight gain, and diabetes should be considered in the models for schizophrenia since these factors have a recognized influence on the treatment effect. Also, consideration of different types of AEs should be properly defined to estimate the impact on health outcomes and costs in the economic evaluation.

Based on the criteria studies, compliance and persistence were not clearly classified, thus definition is recommended for economic evaluations since it might determine the choice of appropriate data source. When integrating compliance or persistence, data are required on both health outcomes and costs for patients who are non-compliant or discontinued treatment (76). Evidence for compliance or persistence could be collected from retrospective studies, clinical trials, observational studies, or reviews.

Utility values, derived from the literature may contribute to the heterogeneity among results when they are applied to the same health states in different studies. Differences in the classifications of health states, survey responders, elicitation methods, and regions were the main factors influencing the utility values. Thus, it is recommended that researchers choose proper sources based on the factors above, as well as the publication year or update of methods.

Certain limitations to study quality have been identified, such as the description of appropriate time horizon selection, discount rate, statement, and justification of the choice of model type, assumptions and limitations to the evaluations. For the reporting quality of the studies, time horizon, preference-based outcome measurement, and assumptions were the major missing parts. To improve both the quality of the study and the quality of the report, it is suggested that researchers conduct the evaluation and generate the manuscript under the respective guidance and checklists.

Though there exist studies reviewed the economic evaluation of treatment for schizophrenia (810, 77), few fully discussed the treatment sequences, AEs, compliance and persistence of the included studies. Thus, this review provides more comprehensive and detailed information of modeling methodology for economic evaluation.

Limitations

There remain some limitations of this review. First, the review only included studies published after 2014 and does not represent the economic methods used in earlier years. Studies published recently may be more valuable for analysis, considering the relatively high quality, most recent treatment options and updated clinical evidence. Second, this review only included model-based economic evaluations. Even though trial-based economic evaluations for schizophrenia are also important evidence, this study aims to generate summaries and suggestions for model methodology rather than synthesizing economic evidence. In addition, trial-based studies may not provide long-term clinical outcomes and source consumption, especially for chronic diseases.

Suggestions

Based on the results of this review, it is suggested that future research focus on methods to integrate compliance or persistence data for chronic diseases. Due to the diverse utilities cited in the models, characteristics of study groups and measuring approach of preference-based health outcomes from the health-related quality of life research should be explained to provide appropriate options for the studies. Publications of economic evaluations should be designed and reported according to applicable gelines and checklists to improve study quality and provide both scientific and valuable evidence for decision makers. Future research could pay more attention to the economic evaluation of long-acting injection antipsychotics.

Funding

This study received funding from Sumitomo Pharma (Suzhou) Co., Ltd.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Statements

Data availability statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author.

Author contributions

LW and HL contributed to the study design, analysis, and writing. FS, XG, and HX contributed to the review work. JL contributed to the manuscript revise. All authors contributed to the article and approved the submitted version.

Conflict of interest

This study received funding from Sumitomo Pharma (Suzhou) Co., Ltd. The funder had the following involvement with the study: decision to published. JL was employed by company Sumitomo Pharma. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh.2021.689123/full#supplementary-material

References

  • 1.

    CharlsonFJFerrariAJSantomauroDFDiminicSStockingsEScottJGet al. Global epidemiology and burden of schizophrenia: findings from the global burden of disease study 2016. Schizophr Bull. (2018) 44:1195203. 10.1093/schbul/sby058

  • 2.

    ShahAXieLKariburyoFZhangQGoreM. Treatment patterns, healthcare resource utilization and costs among schizophrenia patients treated with long-acting injectable versus oral antipsychotics. Adv Ther. (2018) 35:19942014. 10.1007/s12325-018-0786-x

  • 3.

    LafeuilleMHDeanJFastenauJPanishJOlsonWMarkowitzMet al. Burden of schizophrenia on selected comorbidity costs. Expert Rev Pharmacoecon Outcomes Res. (2014) 14:25967. 10.1586/14737167.2014.894463

  • 4.

    HuangYLiuGLiuYWangCRenXZhangH. Economic burden of schizophrenia: based on medical insurance database from Guangzhou. Chinese Health Econ. (2014) 33:625. 10.7664/CHE20140519

  • 5.

    Caqueo-UrizarARus-CalafellMCraigTKIrarrazavalMUrzuaABoyerLet al. Schizophrenia: impact on family dynamics. Curr Psychiatry Rep. (2017) 19:2. 10.1007/s11920-017-0756-z

  • 6.

    TamiziZFallahi-KhoshknabMDalvandiAMohammadi-ShahboulaghiFMohammadiEBakhshiE. Defining the concept of family caregiver burden in patients with schizophrenia: a systematic review protocol. Syst Rev. (2019) 8:289. 10.1186/s13643-019-1182-6

  • 7.

    DrummondMF. Methods for the Economic Evaluation of Health Care Programmes.Oxford: Oxford University Press (2002). p. 396.

  • 8.

    ZhouJMillierAToumiM. Systematic review of pharmacoeconomic models for schizophrenia. J Mark Access Health Policy. (2018) 6:1508272. 10.1080/20016689.2018.1508272

  • 9.

    JinHTappendenPRobinsonSAchillaEMacCabeJHAceitunoDet al. A systematic review of economic models across the entire schizophrenia pathway. Pharmacoeconomics. (2020) 38:53755. 10.1007/s40273-020-00895-6

  • 10.

    JinHTappendenPRobinsonSAchillaEAceitunoDByfordS. Systematic review of the methods of health economic models assessing antipsychotic medication for schizophrenia. PLoS ONE. (2020) 15:e234996. 10.1371/journal.pone.0234996

  • 11.

    MoherDLiberatiATetzlaffJAltmanDG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. (2009) 6:e1000097. 10.1371/journal.pmed.1000097

  • 12.

    AndersonR. Systematic reviews of economic evaluations: utility or futility?Health Econ. (2010) 19:35064. 10.1002/hec.1486

  • 13.

    WalkerDGWilsonRFSharmaRBridgesJNiessenLBassEBet al. Best Practices for Conducting Economic Evaluations in Health Care: A Systematic Review of Quality Assessment Tools.Rockville, MD: Agency for Healthcare Research and Quality (2012).

  • 14.

    HusereauDDrummondMPetrouSCarswellCMoherDGreenbergDet al. Consolidated health economic evaluation reporting standards (cheers)–explanation and elaboration: a report of the ISPOR health economic evaluation publication guidelines good reporting practices task force. Value Health. (2013) 16:23150. 10.1016/j.jval.2013.02.002

  • 15.

    EinarsonTRVicenteCZilbershteinRPiwkoCCNPudasHet al. Pharmacoeconomics of depot antipsychotics for treating chronic schizophrenia in Sweden. Nord J Psychiat. (2014) 68:41627. 10.3109/08039488.2013.852243

  • 16.

    LachaineJBeaucheminCMathurinKGilbertDBeillatM. Cost-effectiveness of asenapine in the treatment of schizophrenia in Canada. J Med Econ. (2014) 17:296304. 10.3111/13696998.2014.897627

  • 17.

    ParkTKuntzKM. Cost-effectiveness of second-generation antipsychotics for the treatment of schizophrenia. Value Health. (2014) 17:3109. 10.1016/j.jval.2014.02.008

  • 18.

    DillaTMöllerJO DonohoePÁlvarezMSacristánJAHappichMet al. Long-acting olanzapine versus long-acting risperidone for schizophrenia in Spain – a cost-effectiveness comparison. BMC Psychiatry. (2014) 14:298309. 10.1186/s12888-014-0298-4

  • 19.

    AnhNQLinhBNHaNTPhanthunanePHuongNT. Schizophrenia interventions in vietnam: primary results from a cost-effectiveness study. Glob Public Health. (2015) 10(Supppl 1):S2139. 10.1080/17441692.2014.986158

  • 20.

    LubingaSJMutambaBBNganiziABabigumiraJB. A Cost-effectiveness analysis of antipsychotics for treatment of schizophrenia in Uganda. Appl Health Econ Hea. (2015) 13:493506. 10.1007/s40258-015-0176-3

  • 21.

    DruaisSDoutriauxACognetMGodetALanconCLevyPet al. Cost effectiveness of paliperidone long-acting injectable versus other antipsychotics for the maintenance treatment of schizophrenia in France. Pharmacoeconomics. (2016) 34:36391. 10.1007/s40273-015-0348-x

  • 22.

    LinLZhaoYJZhouHJKhooALTengMSohLBet al. Comparative cost-effectiveness of 11 oral antipsychotics for relapse prevention in schizophrenia within singapore using effectiveness estimates from a network meta-analysis. Int Clin Psychopharm. (2016) 31:8492. 10.1097/YIC.0000000000000111

  • 23.

    RajagopalanKTruemanDCroweLSquirrellDLoebelA. Cost-utility analysis of lurasidone versus aripiprazole in adults with schizophrenia. Pharmacoeconomics. (2016) 34:70921. 10.1007/s40273-016-0405-0

  • 24.

    EinarsonTRPudasHGoswamiPvan ImpeKBerezaBG. Pharmacoeconomics of long-acting atypical antipsychotics for acutely relapsed chronic schizophrenia in Finland. J Med Econ. (2016) 19:11120. 10.3111/13696998.2015.1100115

  • 25.

    EinarsonTRMaia-LopesSGoswamiPBerezaBGVan ImpeK. Economic analysis of paliperidone long-acting injectable for chronic schizophrenia in Portugal. J Med Econ. (2016) 19:91321. 10.1080/13696998.2016.1184156

  • 26.

    BarnesTRELeesonVCPatonCMarstonLDaviesLWhittakerWet al. Amisulpride augmentation in clozapine-unresponsive schizophrenia (AMICUS): a double-blind, placebo-controlled, randomised trial of clinical effectiveness and cost-effectiveness. Health Technol Asses. (2017) 21:1. 10.3310/hta21490

  • 27.

    EinarsonTRBerezaBGGarciaLIGonzalezMMBTedouriFVan ImpeK. Cost-effectiveness of 3-month paliperidone treatment for chronic schizophrenia in Spain. J Med Econ. (2017) 20:103947. 10.1080/13696998.2017.1351370

  • 28.

    EinarsonTRBerezaBGTedouriFVan ImpeKDeneeTRDriesP. Cost-Effectiveness of 3-Month Paliperidone Therapy for Chronic Schizophrenia in the Netherlands. J Med Econ. (2017) 20:118799. 10.1080/13696998.2017.1363050

  • 29.

    WiwatTJadesadaLNatthamonTNithiwadeePParaveeWHathaiphatP. Cost-effectiveness analysis of aripiprazole compared with risperidone in the treatment of acute schizophrenia patients in Thailand. Thai J Pharmac Sci. (2018) 42:16975. Available online at: https://www.researchgate.net/publication/327222600_Cost-effectiveness_analysis_of_aripiprazole_compared_with_risperidone_in_the_treatment_of_acute_schizophrenia_patients_in_Thailand

  • 30.

    NuhohoSSaadASaumellGRibesDEl KhouryAC. Economic evaluation of paliperidone palmitate once monthly for treating chronic schizophrenia patients in the United Arab Emirates. Curr Med Res Opin. (2018) 34:60111. 10.1080/03007995.2017.1417246

  • 31.

    AigbogunMSLiuSKamatSASapinCDuhigAMCitromeL. Relapse prevention: a cost-effectiveness analysis of brexpiprazole treatment in adult patients with schizophrenia in the USA. ClinicoEcon Outcomes Res. (2018) 10:44356. 10.2147/CEOR.S160252

  • 32.

    NémethBBendesRNagyBGötzeÁKócziánKHorváthMet al. Cost-utility analysis of cariprazine compared to risperidone among patients with negative symptoms of schizophrenia. Health Policy Techn. (2019) 8:8491. 10.1016/j.hlpt.2019.01.004

  • 33.

    ZhaoJJiangKLiQZhangYChengYLinZet al. Cost-effectiveness of olanzapine in the first-line treatment of schizophrenia in China. J Med Econ. (2019) 22:43946. 10.1080/13696998.2019.1580714

  • 34.

    Abdall-RazakAMacaulayATiefenbachJBorgesKMathemaSZuberiSet al. Cost-utility analysis of amisulpride and paliperidone in the treatment of schizophrenia. Heliyon. (2019) 5:e2343. 10.1016/j.heliyon.2019.e02343

  • 35.

    DutinaAStasevic-KarlicicIPandrcNProkicAJankovicSM. Cost/effectiveness of aripiprazole vs. olanzapine in the long-term treatment of schizophrenia. Srp Ark Celok Lek. (2019) 147:46874. 10.2298/SARH181012065D

  • 36.

    ArteagaDCFakraEVan GilsCGuillonP. The clinical and economic impact of three-monthly long-acting formulation of paliperidone palmitate versus the one-monthly formulation in the treatment of schizophrenia in France: a cost-utility study. Encephale. (2019) 45:45967. 10.1016/j.encep.2019.03.001

  • 37.

    YiZMMenPQuSLiCYuXZhaiS. Comparative cost-effectiveness of amisulpride and olanzapine in the treatment of schizophrenia in China. Expert Rev Pharmacoecon Outcomes Res. (2020) 20:31320. 10.1080/14737167.2020.1752670

  • 38.

    LinZXuanJ. Cost-effectiveness of aripiprazole orally disintegrating tablets in the treatment of schizophrenia in China. Expert Rev Pharmacoecon Outcomes Res. (2020) 20:54957. 10.1080/14737167.2020.1807331

  • 39.

    JinHTappendenPMacCabeJHRobinsonSByfordS. Evaluation of the cost-effectiveness of services for schizophrenia in the UK across the entire care pathway in a single whole-disease model. JAMA Netw Open. (2020) 3:e205888. 10.1001/jamanetworkopen.2020.5888

  • 40.

    GarrisonLJPaulyMVWillkeRJNeumannPJ. An overview of value, perspective, and decision context-a health economics approach: an ISPOR special task force report. Value Health. (2018) 21:12430. 10.1016/j.jval.2017.12.006

  • 41.

    HayJWSmeedingJCarrollNVDrummondMGarrisonLPMansleyECet al. Good research practices for measuring drug costs in cost effectiveness analyses: issues and recommendations: the ISPOR drug cost task force report–part I. Value Health. (2010) 13:37. 10.1111/j.1524-4733.2009.00663.x

  • 42.

    WeinsteinMCO'BrienBHornbergerJJacksonJJohannessonMMcCabeCet al. Principles of good practice for decision analytic modeling in health-care evaluation: report of the ISPOR task force on good research practices–modeling studies. Value Health. (2003) 6:917. 10.1046/j.1524-4733.2003.00234.x

  • 43.

    BrightCE. Measuring medication adherence in patients with schizophrenia: an integrative review. Arch Psychiatr Nurs. (2017) 31:99110. 10.1016/j.apnu.2016.09.003

  • 44.

    HaddadPMBrainCScottJ. Nonadherence with antipsychotic medication in schizophrenia: challenges and management strategies. Patient Relat Outcome Meas. (2014) 5:4362. 10.2147/PROM.S42735

  • 45.

    LenertLSturleyARapaportMChavezSMohrPRupnowM. Public preferences for health states with schizophrenia and a mapping function to estimate utilities from positive and negative symptom scale scores. Schizophr Res. (2004) 71:15565. 10.1016/j.schres.2003.10.010

  • 46.

    BriggsAWildDLeesMReaneyMDursunSParryDet al. Impact of schizophrenia and schizophrenia treatment-related adverse events on quality of life: direct utility elicitation. Health Qual Life Outcomes. (2008) 6:105. 10.1186/1477-7525-6-105

  • 47.

    OhPILanctôtKLMittmannNIskedjianMEinarsonTR. Cost-utility of risperidone compared with standard conventional antipsychotics in chronic schizophrenia. J Med Econ. (2001) 4:13756. 10.3111/200104137156

  • 48.

    RevickiDAShakespeareAKindP. Preferences for schizophrenia-related health states: a comparison of patients, caregivers and psychiatrists. Int Clin Psychopharmacol. (1996) 11:1018. 10.1097/00004850-199606000-00004

  • 49.

    CumminsCStevensAKiselyS. The Use of Olanzapine as a First and Second Choice Treatment in Schizophrenia.York: Centre for Reviews and Dissemination (1998).

  • 50.

    ExcellenceNIFH. Schizophrenia: Core Interventions in the Treatment and Management of Schizophrenia in Primary and Secondary Care (update) (2010).

  • 51.

    OsborneRHDaltonAHertelJSchroverRSmithDK. Health-related quality of life advantage of long-acting injectable antipsychotic treatment for schizophrenia: a time trade-off study. Health Qual Life Outcomes. (2012) 10:35. 10.1186/1477-7525-10-35

  • 52.

    TempestMSapinCBeillatMRobinsonPTreurM. Cost-effectiveness analysis of aripiprazole once-monthly for the treatment of schizophrenia in the UK. J Ment Health Policy Econ. (2015) 18:185200. 10.1016/j.jval.2015.09.979

  • 53.

    MavranezouliI. A review and critique of studies reporting utility values for schizophrenia-related health states. Pharmacoeconomics. (2010) 28:110921. 10.2165/11537300-000000000-00000

  • 54.

    FuriakNMAscher-SvanumHKleinRWSmolenLJLawsonAHMontgomeryWet al. Cost-effectiveness of olanzapine long-acting injection in the treatment of patients with schizophrenia in the United States: a micro-simulation economic decision model. Curr Med Res Opin. (2011) 27:71330. 10.1185/03007995.2011.554533

  • 55.

    SchultzSEKopecJA. Impact of chronic conditions. Health Rep. (2003) 14:4153.

  • 56.

    ClarkePGrayAHolmanR. Estimating utility values for health states of type 2 diabetic patients using the EQ-5D (UKPDS 62). Med Decis Making. (2002) 22:3409. 10.1177/027298902400448902

  • 57.

    Luengo-FernandezRGrayAMBullLWelchSCuthbertsonFRothwellPM. Quality of life after TIA and stroke: ten-year results of the Oxford vascular study. Neurology. (2013) 81:158895. 10.1212/WNL.0b013e3182a9f45f

  • 58.

    DaviesAVardevaKLozeJY. L'Italien GJ, Sennfalt K, Baardewijk M. Cost-effectiveness of atypical antipsychotics for the management of schizophrenia in the UK. Curr Med Res Opin. (2008) 24:327585. 10.1185/03007990802507547

  • 59.

    LevyARChristensenTLJohnsonJA. Utility values for symptomatic non-severe hypoglycaemia elicited from persons with and without diabetes in Canada and the United Kingdom. Health Qual Life Outcomes. (2008) 6:73. 10.1186/1477-7525-6-73

  • 60.

    Ascher-SvanumHFuriakNMLawsonAHKleinTMSmolenLJConleyRRet al. Cost-effectiveness of several atypical antipsychotics in orally disintegrating tablets compared with standard oral tablets in the treatment of schizophrenia in the United States. J Med Econ. (2012) 15:53147. 10.3111/13696998.2012.662923

  • 61.

    TreurMBacaEBobesJCanasFSalvadorLGonzalezBet al. The cost-effectiveness of paliperidone extended release in Spain. J Med Econ. (2012) 15(Suppl 1):2634. 10.3111/13696998.2012.734884

  • 62.

    MohrPEChengCMClaxtonKConleyRRFeldmanJJHargreavesWAet al. The heterogeneity of schizophrenia in disease states. Schizophr Res. (2004) 71:8395. 10.1016/j.schres.2003.11.008

  • 63.

    MataCMRosetGMBadiaLXAntonanzasVFRagelAJ. Effect of type-2 diabetes mellitus on the quality of life of patients treated at primary care consultations in Spain. Aten Primaria. (2003) 31:4939. 10.1016/S0212-6567(03)70722-7

  • 64.

    LamETLamCLLaiCLYuenMFFongDYSoTM. Health-related quality of life of southern Chinese with chronic hepatitis B infection. Health Qual Life Outcomes. (2009) 7:52. 10.1186/1477-7525-7-52

  • 65.

    van de WilligeGWiersmaDNienhuisFJJennerJA. Changes in quality of life in chronic psychiatric patients: a comparison between EuroQol (EQ-5D) and WHOQoL. Qual Life Res. (2005) 14:44151. 10.1007/s11136-004-0689-y

  • 66.

    van der SchansSGoossensLBolandMKocksJPostmaMJvan BovenJet al. Systematic review and quality appraisal of cost-effectiveness analyses of pharmacologic maintenance treatment for chronic obstructive pulmonary disease: methodological considerations and recommendations. Pharmacoeconomics. (2017) 35:4363. 10.1007/s40273-016-0448-2

  • 67.

    MaruSByrnesJWhittyJACarringtonMJStewartSScuffhamPA. Systematic review of model-based analyses reporting the cost-effectiveness and cost-utility of cardiovascular disease management programs. Eur J Cardiovasc Nurs. (2015) 14:2633. 10.1177/1474515114536093

  • 68.

    HutterMRodríguez-IbeasRAntonanzasF. Methodological reviews of economic evaluations in health care: what do they target?Euro J Health Econ. (2014) 15:82940. 10.1007/s10198-013-0527-7

  • 69.

    GomersallJSJadotteYTXueYLockwoodSRiddleDPredaA. Conducting systematic reviews of economic evaluations. Int J Evid Based Healthc. (2015) 13:1708. 10.1097/XEB.0000000000000063

  • 70.

    PignoneMSahaSHoergerTLohrKNTeutschSMandelblattJ. Challenges in systematic reviews of economic analyses. Ann Intern Med. (2005) 142:10739. 10.7326/0003-4819-142-12_Part_2-200506211-00007

  • 71.

    BoehlerCELordJ. Mind the gap! a multilevel analysis of factors related to variation in published cost-effectiveness estimates within and between countries. Med Decis Making. (2016) 36:3147. 10.1177/0272989X15579173

  • 72.

    CloutierMAigbogunMSGuerinANitulescuRRamanakumarAVKamatSAet al. The economic burden of schizophrenia in the United States in 2013. J Clin Psychiatry. (2016) 77:76471. 10.4088/JCP.15m10278

  • 73.

    FasseehANemethBMolnarAFrickeFUHorvathMKoczianKet al. A systematic review of the indirect costs of schizophrenia in Europe. Eur J Public Health. (2018) 28:10439. 10.1093/eurpub/cky231

  • 74.

    ZhaiJGuoXChenMZhaoJSuZ. An investigation of economic costs of schizophrenia in two areas of China. Int J Ment Health Syst. (2013) 7:26. 10.1186/1752-4458-7-26

  • 75.

    National Institute for Health and Care Excellence. Psychosis and Schizophrenia in Adults: Prevention and Management.London: National Institute for Health and Care Excellence (2014).

  • 76.

    HughesDCowellWKonczTCramerJ. Methods for integrating medication compliance and persistence in pharmacoeconomic evaluations. Value Health. (2007) 10:498509. 10.1111/j.1524-4733.2007.00205.x

  • 77.

    HenriqueIde MendoncaLTde MeloDOAguiarPM. Economic evaluations on the use of aripiprazole for patients with schizophrenia: a systematic review. J Clin Pharm Ther. (2020) 45:115. 10.1111/jcpt.13034

Summary

Keywords

schizophrenia, economic evaluation, systematic review, modeling, method review

Citation

Wang L, Shi F, Guan X, Xu H, Liu J and Li H (2021) A Systematic Review of Methods and Study Quality of Economic Evaluations for the Treatment of Schizophrenia. Front. Public Health 9:689123. doi: 10.3389/fpubh.2021.689123

Received

31 March 2021

Accepted

21 September 2021

Published

20 October 2021

Volume

9 - 2021

Edited by

Kevin Lu, University of South Carolina, United States

Reviewed by

Zafer Çaliskan, Hacettepe University, Turkey; Nan Yang, Sichuan University, China

Updates

Copyright

*Correspondence: Hongchao Li

This article was submitted to Health Economics, a section of the journal Frontiers in Public Health

Disclaimer

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

Outline

Figures

Cite article

Copy to clipboard


Export citation file


Share article

Article metrics