Systematic Review and Quality Evaluation of Pharmacoeconomic Studies on Traditional Chinese Medicines

Objectives: This study was aimed to find and appraise the available published pharmacoeconomic research on Traditional Chinese Medicine (TCM), to identify related issues and make suggestions for improvement in future research. Methods: After developing a search strategy and establishing inclusion and exclusion criteria, pharmacoeconomic studies on TCM were sourced from seven Chinese and English databases from inception to April 2020. Basic information about the studies and key pharmacoeconomic items of each study were extracted. The quality of each study was evaluated by using the British Medical Journal economic submissions checklist for authors and peer reviewers, focusing on factors such as study design, research time horizon, sample size, perspective, and evaluation methods. Results: A total of 431 published pharmacoeconomic articles with 434 studies on topics including cost-effectiveness, cost-benefit, cost-minimization, cost-utility, or combination analyses were identified and included in this review. Of these, 424 were published in Chinese and 7 in English. These studies conducted economic evaluations of 264 Chinese patent medicines and 70 types of TCM prescriptions for 143 diseases, including those of the central nervous, cardiovascular, respiratory, gynecologyical, and other systems. The studied TCMs included blood-activating agents (such as Xuesaitong tablet, Fufant Danshen tablet, and Danhong Injection), blood circulation promoting agents (such as Shuxuetong injection, Rupixiao tablet, and Fufang Danshen injection), and other therapeutic agents. The overall quality score of the studies was 0.62 (range 0.38 to 0.85). The mean quality score of studies in English was 0.72, which was higher than that of studies in Chinese with 0.62. Conclusions: The quality of pharmacoeconomic studies on TCM was relatively, generally low. Major concerns included study design, inappropriate pharmacoeconomic evaluation, insufficient sample size, or non-scientific assessment. Enhanced methodological training and cooperation, the development of a targeted pharmacoeconomic evaluation guideline, and proposal of a reasonable health outcome index are warranted to improve quality of future studies.


INTRODUCTION
Pharmacoeconomics is a complex science that provides evidences for the optimal allocation and efficient utilization of medical resources. Since the 1970s, it has been increasingly accepted and applied by health care providers, health policy makers, and medical insurance institutions. It has also formed part of the basis of new drug applications, drug pricing, drug purchasing by medical institutions, National Essential Medicines List formulation, medical insurance, and post-marketing drug evaluation (1)(2)(3).
With the increasing popularity of and substantial expenditures on complementary and alternative medicine (CAM) therapies globally, the availability of economic evaluations becomes increasingly important (4,5). However, the quality of those evaluations has significant influence on the popularization and application of the results. A few systematic reviews on economic evaluations of CAM have described findings and quality of available research and have made recommendations for future research (6)(7)(8)(9)(10).
Traditional Chinese Medicine (TCM) as one kind of CAM plays an important role in preventing, treating, and curing disorders and diseases (11,12). Since 1984, the Drug Administration Law of the People's Republic of China has established the policy governing the development both modern medicines and traditional medicines. In 2020, Chinese patent medicine accounted for more than one third of all the drug approvals in China (13) and for 49.07% of all drugs listed in the National Drug Catalog for Basic Medical Insurance of China (14), equaling the proportion of Western medicines. In recent years, with more attention on economic evaluation of medicines, an increasing amount of pharmacoeconomic research has comprehensively evaluated the efficacy, safety and affordability of TCM. This research provides evidence for the rational use of health resources, essential medicine and state medical insurance catalog selection and national drug price negotiations.
The present study aims to find and appraise the available published pharmacoeconomic studies on TCM, analyze the limitations of the research and make suggestions for improved pharmacoeconomic evaluation on TCM in future.

Data Sources
This systematic review was conducteded in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. We searched the following databases between their inception (years in parentheses) and Abbreviations: CAM, complementary and alternative medicine; TCM, traditional Chinese medicine; PRISMA, preferred reporting items for systematic reviews and meta-analyses; CNKI, China national knowledge infrastructure; VIP, Chongqing VIP information; BMJ, British medical journal; QHES, quality of health economics studies; CHEC, consensus on health economic criteria; ISPOR, international society of pharmacoeconomics and outputs research; CHEERS, consolidated health economic evaluation reporting standards; CEA, cost-effectiveness analysis; CMA, cost analysis; CBA, cost-benefit analysis; CUA, cost-effectiveness analysis; DT, decision tree. The search keywords was ("Chinese medicine" OR "Traditional Chinese Medicine" OR "Chinese patent medicine" OR "herbal medicine" OR "natural medicine" OR "Chinese Medicinal Materials" OR "Botanical medicine" OR "Chinese herbal pieces" OR "Chinese medicinal granula") and ("pharmacoeconomic" OR "economics" OR "Cost Effectiveness Analysis" OR "Cost Utility Analysis" OR "Cost Benefit Analysis" OR "Cost Minimization Analysis" OR "Markov" OR "Decision Tree" OR "Partitioned Survival Model"). Chinese and English terms for search keywords were used when searching the databases. In addition, the references lists of retrieved articles were also searched.

Study Selection
We included the articles describing pharmacoeconomic evaluation articles on TCM, defined as medicines made according to Chinese medicine theory, such as Chinese patent medicine and Chinese medicine prescription or formula. Articles describing research on traditional Chinese treatment or therapy as interventions such as acupuncture, Guasha or massage or theory papers, reviews, reports, protocols, news, and opinion articles, were excluded. Partial economic evaluations, such as outcome, cost, cost-outcome description, effectiveness (or efficacy) evaluation, and cost analysis, quality of life research, and budget impact analysis were also excluded. Besides, the study country was limited in China and study language was limited in Chinese and English.

Data Extraction and Quality Assessment
All articles included in the study were read in full and independently judged against inclusion and exclusion criteria of the individual phase by two authors (HH Zhang and TY Deng). Any difference during assessment between the two reviewers was discussed or resolved by a third dependent reviewer (N Yang). A data extraction table was designed using Excel software, and information extracted from each article including title, first author's name and affiliation, journal and year of publication, information about disease, sample size, study design, intervention, and information related cost, health outcomes, uncertainty analysis, perspective, evaluation techniques, and other factors. The data were descriptively analyzed using Microsoft Excel.
The five most representative checklists or guidelines for evaluation of pharmacoeconomic research are the British Medical Journal (BMJ) checklist (15), Quality of Health Economic Studies (QHES) instrument (16), Consensus Health Economic Criteria (CHEC) (17), Philips guideline (18), and Consolidated Health Economic Evaluation Reporting Standards (CHEERS) (19). Among them, the BMJ checklist was the first to emerge and to be used. Published in 1996, it includes 35 items to assess the quality of articles on economic evaluation, and to date this has been the mostly cited of these appraisal methods (20). Each item of the checklist requires a "yes, " "no, " "unclear, " and "not applicable" response, and this broad form of grading means that the checklist may be used to assess various economic evaluations. Due to its wide application, general recognition in the pharmacoeconomic research field, and flexibility of evaluation items, we used BMJ checklist to evaluate the quality of studies included in this review. The percentage of the applicable items on the BMJ checklist meeting by each study represented the quality score of the study.
The 35 checklist items are organized into 10 sections under three headings including study design, data collection, and analysis and interpretation of results. One mark was awarded for each item with a "yes" response, and a mark of zero for "no" or "unclear." Each study was scored based on its maximum possible score on the checklist, excluding items irrelevant to the study. The overall quality score was calculated as the mean of the scores of all included studies. A score <0.90 was taken to indicate high quality. Scores of individual items were also recorded.

Study Description
According to the pre-defined search strategy and selection criteria, the databases were searched and identified articles were screened (Figure 1). We eventually included a total of 431 articles including 424 articles in Chinese language and seven articles in English.
The first pharmacoeconomic study on TCM in Chinese was published in Chinese Journal of Pharmacoepidemiology in 1997, and focused on Ahylysantinfarctase and Fufang Danshen injection for cerebral infarction patients (21). The first TCM pharmacoeconomic study in English was published in the Chinese Journal of Integrative Medicine in 2014, and was focusing on Chinese medicine and Western medicine for ischemic stroke patients (22). Since 2002, the number of published pharmacoeconomic studies on TCM has gradually increased ( Figure 2).
In addition to 25 dissertations, other pharmacoeconomic evaluation literatures on TCM were published in 173 journals, among which 178 were Chinese journals and 5 English journals.
More than 80.97% of the pharmacoeconomic evaluation studies on TCM were carried out by teams with the first author affiliated with a hospital (349), and the remaining studies' first authors were affiliated with universities or research institutions (n = 72, 16.71%) or enterprises (n = 10, 2.32%). All of the first authors of English articles were affiliated with universities.
Among 431 articles, only 66 (15.56%) were reported as having financial support, including nine nationally funded projects, 22 provincial and ministerial funded projects, 22 municipally funded projects, four university funded projects, four funded projects by the authors' units, three projects supported by a foundation, and two projects supported by Enterprise funding.

Diseases
According to the International Classification of Diseases (ICD-10), the studies focused on a range of 148 kinds of diseases, among which most frequently studied were diseases of the nervous system, cardiovascular system, respiratory system, gynecological, and alimentary systems as shown in Figure 3.

Interventions
The medical interventions involved in the studies were complex and diverse. Nearly half of the studies compared  two interventions, while others compared three or more interventions. However, 186 studies did not describe reasons for choosing the specific program or interventions for comparison. More than a third of the studies used TCM as the control group, and in most of these cases the TCM was not recommended by relevant clinical guidelines ( Table 1).
The studies included 264 types of Chinese patent medicines and 70 types of TCM prescriptions (such as prescription, pieces decoction, powder, extraction, tincture, and pills). Table 2 shows the Chinese patent medicines that were the focus of more than eight

Study Characteristics
Study characteristics of the 434 studies are presented in Table 3 including the study design, duration, sample size, and viewpoint/perspective. While 24 studies were based on data from previous research, most were prospective or retrospective clinical trials. Seventeen studies did not specify the study duration, but of the remainder most were <3 months. Excluding the 27 studies in which data were extracted from previous research, the sample size of included studies ranged from 38 to 2,820 and subjects' age ranged from 0 to 90 years old. Most (n = 377; 86.87%) of the studies did not specify the perspective of their pharmacoeconomic evaluations. More than 85% of the studies used cost-effectiveness analysis (CEA), and the remainder study used at least  Table 3.

Health Outcomes
A total of 226 studies used a single end-point, 156 studies used double end-points, 26 studies used three end-points, and the other 26 studies used four or more end-points at the same time.
Of 400 CEA studies with a health-related the final end-points, 264 studies included clinical efficiency/improvement rate, 132 included adverse reactions rate, 47 included efficiency rate of TCM syndromes, 11 included recurrence rate, and 16 included disease or drug withdrawal rate, and 5 included death rate.

Uncertainty Analysis and Generalizability
Sixty seven percent of the included studies (n = 289) carried out an uncertainty price analysis, the main variables of which are shown in Table 3. In 220 (76.12%) studies, uncertainty analysis were conducted by using one-way sensitivity analysis with reducing drug prices by 10-20%, while 69 (23.88%) of studies used multi-way sensitivity analysis for uncertainty. A total of 102 articles (23.67%) discussed the limitations of their studies, and 12 articles discussed the generalizability of their results. Although 66 studies included children under 18 years of age, none of these acknowledged the related ethical issues.

Quality Assessment of Pharmacoeconomic Evaluation Studies on TCM
The average overall quality score of 431 pharmacoeconomics studies on TCM was 0.62.with a range from 0.38 to 0.85. Scores on most of the studies were from 0.5 to 0.7. The distribution is shown in Table 4. On items 2.5, 2.9, 2.12, 3.2, 3.3, 3.4, and 3.5 the score was below 0.1. On items 1.3, 2.7, 2.8, and 3.14 it was no more than 0.5 and on items 1.4, 2.1, 3.6, 3.7, and 3.8 it was no more than 0.7 ( Table 4).
The quality of the studies was not significantly improved over the past 20 years, as shown in Table 5. The quality of studies with first authors affiliated with universities researchers was much higher than that of studies with first authors affiliated with hospitals. The mean score of the seven studies in English was 0.72, while that of studies published in Chinese journals was 0.62. The scores of studies in English were higher than those in Chinese on items 3.2, 3.2, 3.4 3.5, and 3.14 and scores were relatively high in the Chinese journals on items 1.2, 2.3, 2.7, and 2.8. However, scores on items 2.7, 2.8, 2.9, 3.2, 3.3, and 3.4 were low in both Chinese and English studies, suggesting a need for improvement ( Table 4).  (23), 2015 (24), and 2020 (25). And compared with the studies of Chen (26), Li (27), Lei (28) in 2004, 2008, and 2010 respectively, the quality of pharmacoeconomics research articles on TCM was also lower than that on chemical drugs. Three main limitations in quality were identified. Firstly, many of the included studies were not well-designed and lacked clear research perspective. Only four studies described the method of sample size estimation based on China guidelines for pharmacoeconomic evaluation, which would affect cost identification and measurement and thereby the integrity and reliability of the studies (29). Many studies did not report the randomization method, and failed to justify the selected intervention.
Many studies with first authors affiliated with a hospital appeared to not fully understand the basic principles of pharmacoeconomics and did not design their clinical trials appropriately, despite pharmacoeconomic data being relatively easy to obtain in a hospital. Some studies failed to implement and apply pharmacoeconomic research methods correctly. There were limitations or errors in cost identification and measurement in some studies, more than half of which included only drug costs. Due to these issues, some researchers misunderstood or were confused about cost assessment in pharmacoeconomic evaluation. Even in the study of chronic diseases, people generally did not pay enough attention to discount. In addition, the choice of discount rate is also based on different countries and different perspectives, which is also a controversial issue in the field of Pharmacoeconomics.
Some studies assessed TCM as an adjunct to Western medicines, which made the cost difficult to calculate. In addition, the lack of a specific and unique pharmacoeconomic evaluation guideline for TCM was also an important limitation. In 2013, Xie et al. published a "technical specification for pharmacoeconomic evaluation on post marketed Chinese patent medicine (Draft)" in the China Journal of traditional Chinese Medicine (30). This guideline considered the basic ideology and characteristics of traditional Chinese medicine to a certain extent, but did not solve the specific problems in the pharmacoeconomic evaluation of TCM, such as how to map the effectiveness and health outcome of TCM and chemical medicines.
Finally, some TCM clinical trials were found to be limited in terms of sample size, comparison type, or the use of nonscientific assessments. The short durations of most studies were not well-aligned with the chronic nature of disease on which they focused, and did not fully reflect the advantages of TCM in improving long-term clinical efficacy and potential safety. Most of the studies were retrospective, with small samples and no follow-up. Common defects were in protocol design, lack of trial registration, study reporting, and quality control. Frequently used outcome indicators were overall efficiency and recurrence rate, and were not sufficiently specific to reflect outcomes of different types of TCM treatment.
There were some limitations in our study. First, the BMJ checklist was published in the 1990s, and its applicability may deviate to some extent from the current research including pharmacoeconomic evaluations on TCM. Second, this study included only published research and not unpublished research reports. Some outcome indicators were not addressed by the included studies, resulting in low scores for those indicators, which may have impacted negatively on the evaluation results of this study.

CONCLUSIONS
Although limitations and deficiencies are found in the current pharmacoeconomic evaluation of TCM, the body of research still provides a lot of valuable evidence for the rational use of health resources, essential medicine and state medical insurance catalog selection and national drug price negotiations. The following recommendations may improve the quality of research in this area. In the pharmacoeconomics evaluation of TCM, the following recommendations are made.
With regard to the point discussed above, we call for enhanced methodological training and cooperation to improve the quality of research and reporting quality. A pharmacoeconomic evaluation guideline conforming to the theoretical characteristics of TCM should be established to reflect the economic evaluation results of TCM objectively. In addition, importantly, a reasonable health outcome index for TCM should be developed, balancing between the specificity and quantification of TCM indicators, mapping of indicators between Chinese patent medicine, and Western medicines, and enhancing clinical comparability of health indicators.

DATA AVAILABILITY STATEMENT
The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author/s.

AUTHOR CONTRIBUTIONS
MH and NY conceived the study design. HZ and TD searched and selected the articles, extracted, analyzed, and interpreted the data. NY drafted the manuscript. MH and JG critically reviewed the manuscript. All authors read and approved the final version of the manuscript. Medicine (Project Number: 2018ZC004). The funders will not be involved in the analysis and interpretation of data, or in the decision to publish the results.