Developing a value assessment index system of anti-tumour commercial Chinese polyherbal preparation in China: a modified Delphi study

Objective: To develop a value assessment index system of anti-tumour commercial Chinese polyherbal preparation (CCPP).

Methods: A modified two-round Delphi method was conducted to establish consensus within a field to reach an agreement via questionnaire among a multidisciplinary panel of experts. In addition, the analytic hierarchy process was used to conduct weight analysis of indicators.

Results: In two rounds of Delphi consultation, the experts’ positive coefficient was over 90%, and the authoritative coefficient was over 0.70. The Kendall’s W of two rounds was 0.401 and 0.438, and the P-value of Kendall’s W test was all <0.001 for each round. The final value assessment index system consisted of 7 primary indicators, 24 secondary indicators, and 50 tertiary indicators.

Conclusion: This value assessment index system could promote the clinical rational use of CCPP and the development of the drug catalog and clinical guideline in China.

Introduction

Traditional Chinese Medicine (TCM), with a history spanning over five millennia in China, complements modern medicine through its unique theoretical system by integrating prevention, healthcare, treatment, and rehabilitation to enhance public health. China has published several policy documents, including the “Essentials of the Development Strategy for TCM(2016–2030)”, to promote TCM’s sustainable development and enhance its global recognition and acceptance (The State Council of the People’s Republic of China, 2016). Over centuries, TCM has become the most complete, influential, and widely used traditional medical system in the world today, which has spread to 193 countries and areas, including the United States, the European Union, Africa, and Southeast Asia through trade routes and modern globalization (Yagüe et al., 2022; Kadier et al., 2025). Based on TCM’s theory, commercial Chinese polyherbal preparation (CCPP) refers to Chinese medicinal products, which have been manufactured into pills, powders, granules, capsules, tablets, etc. It features convenient administration and storage, along with relatively low cost. Certain CCPP have been registered either as prescription drugs or over-the-counter medications in Russia, Canada, and the European Union, such as Huatuo Zaizao pill, Danshen Capsule, and Xiaoyao Pian (Zhang B. et al., 2022).

According to the latest data from the International Agency for Research on Cancer, projections estimate that by 2050, the annual number of new cancer cases will increase by 77% compared to 2022 levels (Bray et al., 2024). Cancer treatment faces long-term challenges, including inequitable healthcare resource allocation, high treatment costs, and compromised quality of life (Schütte et al., 2021; Bizuayehu et al., 2024). TCM demonstrates unique advantages as an integral part of comprehensive cancer treatment, which include improving patients’ quality of life, reducing risks of recurrence and metastasis, and prolonging progression-free survival, through modulation of the immune microenvironment and alleviation of radiotherapy- and chemotherapy-induced adverse reactions (Zhang et al., 2021; Xi et al., 2025; Gao et al., 2021).

However, several concerns have emerged regarding the increasing clinical utilisation rate of anti-tumour CCPP (Wu et al., 2022; Wang et al., 2024), including a large number of specifications, inconsistent drug standards, uneven drug quality, and the lack of standardisation in selection mechanisms and post-marketing evaluations (Chen et al., 2023). These result in insufficient reliable evidence for the inclusion of CCPP in critical decision-making tools within the clinical practice guidelines.

Relevant administrative departments in China have successively published multiple evaluation guidelines, based on the Health Technology Assessment (HTA) and Value-Based Healthcare over the years (Zhen et al., 2018; Guo et al., 2023), which focus on the value of health and healthcare interventions (including drugs, devices, diagnosis, and treatment plans). Previous studies developed an HTA-based methodology to develop value evaluation of drugs (Maxion-Bergemann et al., 2006; Zhang C. et al., 2022; Su et al., 2024). However, there are currently no validated, quantitative frameworks to guide CCPP value evaluation. Therefore, this study aimed to construct a value assessment index system of anti-tumour CCPP, which optimised the allocation of health resources, enhanced rational clinical utilisation of anti-tumour CCPP, and provided a reference for selecting anti-tumour CCPP.

Methods

Literature review and construction of preliminary indicator pool

A systematic literature review was conducted to develop an initial indicator pool. The search included Web of Science, PubMed, Embase, Cochrane Library, CNKI, Wanfang, VIP, and related professional websites, which lasted until September 2023. The related websites included National Comprehensive Cancer Network, American Society of Clinical Oncology, China Anti-cancer Association, and Chinese Society of Clinical Oncology. We used the following Chinese and English search terms: cancer, CCPP, drug, Health Technology Assessment, value framework, value evaluation. Based on the literature review, we held a small-scale expert consultation meeting to discuss the importance and suitability of the indicators. None of these experts participated in the follow-up Delphi expert consultation.

The flow chart of the literature search is shown in Figure 1, and the 21 studies were finally included in the review (Supplementary Table S1). Finally, a preliminary indicator pool of value assessment of anti-tumour CCPP was developed, consisting of 7 first-level indicators, 26 second-level indicators, and 62 third-level indicators.

Figure 1

Figure 1. The flow chart of initial indicator construction.

Expert selection

We recruited 32 consulting experts nationwide, including clinical medicine, clinical pharmacy, pharmacy administration, evidence-based medicine, and pharmacoeconomics. The inclusion criteria for Delphi consultation experts were as follows: (1) being familiar with the research topic; (2) intermediate or higher professional title, and ensure that the proportion of experts with associate senior or higher professional titles is greater than 50%; (3) having >5 years of working experience in their field, and ensure that more than 50% of experts have more than 10 years of work experience.

Data collection

The expert questionnaire included three parts: (1) introduction of the background and purpose of the study; (2) basic information of the experts: including title, years of experience, and research field, etc. (3) the expert consultation content: the framework of the three-level indicators, the experts rated the importance and operability of the indicators with Likert scoring method (Ruan et al., 2024). The experts’ familiarity scale with indicators ranging from 1 (very unfamiliar) to 5 (very familiar), and the experts’ basis of judgement included theoretical analysis, practical experience, peer understanding, and individual intuition.

This study completed 2 rounds of expert consultation using the modified Delphi method, with a return time of 2 weeks. Indicators included in the analysis had an average importance score ≥3.5 and the coefficient of variation (CV) ≤ 0.25 (Hommel et al., 2016), and the indicators would be revised, deleted, or added following a deliberation undertaken by the research team, when the experts suggested. Combining the results of the expert questionnaires and group discussion, the index system was finally determined.

Data analysis

SPSS 22.0 was used to analyse and process the data. The coefficient of the authority (Cr) of the experts is the arithmetic average of the judgment coefficients (Ca) and the familiarity coefficients (Cs). In addition, the coefficient of the coordination of the experts’ opinions (Kendall’s W) indicated the degree of coordination of the experts’ opinions, that is, W = 12∗ ${\sum }_{i=1}^n{d}_i^2$/ ${\mathrm{m}}^2\left({\mathrm{n}}^3-\mathrm{n}\right)$. Where “m” is the total number of experts, “n” is the number of lists, and “di” is the value of the evaluation levels of “m” participants for the “i” lists minus the arithmetic average of the evaluation levels for all criteria.

Then, we determined the weights of the dimensions and indicators by the Analytic Hierarchy Process (AHP). First, we constructed a judgment matrix to compare indicators two by two using Saaty’s 1-9 scale. Second, the weights for each level of indicators were calculated. Third, the consistency ratio (CR) was used to test the logical consistency of the judgement matrix. The CR < 0.10 was considered acceptable, indicating that the pairwise comparisons are sufficiently consistent. At last, the weight for each sub-criterion was calculated by multiplying its local weight by the weight of its parent criterion in the level above, and then aggregating these results across the hierarchy.

Results

Characteristics of the experts

A total of 32 experts from various fields were selected to participate in this survey, whose research fields were mainly related to clinical pharmacy (10, 31.25%), clinical medicine (10, 31.25%), evidence-based medicine (4, 12.50%), pharmacy management (5, 15.62%), and pharmacoeconomics (3, 9.38%). The educational background of the specialists varied, with 18 (56.25%) holding a PhD, 12 (37.50%) holding a master’s degree. Of the participants, 19 (59.38%) held a senior associate position. The selected experts came from 11 provinces and cities, including Beijing, Shanghai, Chongqing, Sichuan, Jiangsu, Fujian, etc., with good geographical representation. More information about the expert group is shown in Table 1.

Table 1

Table 1. General information of experts.

Experts’ motivation and authority coefficient

In two rounds of the Delphi survey, 32 experts completed the questionnaire (100%), which demonstrates experts’ high motivation. The coefficient of the authority of the experts (Cr) was 0.84, which is higher than 0.70, indicating the reliability of consulting outcomes (Table 2).

Table 2

Table 2. The authority coefficient of consulting experts.

Consistency and coordination of experts’ opinions

In the first round, the mean importance score of each indicator ranged from 3.60 to 4.94, and CV ranged from 5.00% to 25.65%. In the second round, the mean importance score of each indicator ranged from 3.88 to 4.96, and CV ranged from 3.88% to 22.94%. The Kendall’s W coefficient was 0.401 and 0.438, suggesting that the consistency of experts’ opinions was acceptable (Table 3).

Table 3

Table 3. Consistency coefficient of expert comments in two rounds of consultations.

Indicator screening

Based on the first round of expert consultation, two secondary indicators were deleted (drug interactions, incremental cost-effectiveness ratio), and eleven tertiary indicators were deleted (defined daily doses, median disease-free survival, median duration of response, distant metastasis free survival, time to failure, time to progress, numbers of pharmaceutical enterprises, numbers of pharmaceutical circulation enterprise, off-label use, interactions with chemical drugs, medical staff medication experience). After the two rounds of expert consultations, the final value assessment index system of anti-tumour CCPP included 7 primary indicators, 24 secondary indicators, and 50 tertiary indicators.

Final indicator weights

The results of the weight calculation indicated that the weights of primary indicators were 0.3341 (safety), 0.2880 (efficacy), 0.0792 (economics), 0.0849 (appropriateness), 0.0469 (innovation), 0.1089 (accessibility), and 0.0661 (Characteristics of Chinese medicine). And the consistency ratio value of every matrix’s consistent ratio was <0.1, which met the consistency verification requirements (Supplementary Table S2).

Discussion

Main findings

This research innovatively constructed a system for multiple dimensions of comprehensive assessment of anti-tumour CCPP of clinical application based on this realistic background, which shows significant practical value. First, the system established a value evaluation framework of dimensions, including pharmacoeconomics, clinical efficacy, safety, and characteristics of TCM, which provided a standardized decision-making tool for the entry of drugs into medical institutions. This structured evaluation model can effectively enhance the scientificity and standardization of selection for anti-tumour CCPP. Second, the evaluation system can serve multiple dimensions setting of the drug Administration setting. It can provide a quantitative basis for the substitution of drug varieties in hospitals as well as evidence-based support for the formulation of drug directories by government departments.

Strengths and limitations

Firstly, we invited 32 experts from 11 provinces and cities in China to form an expert group, adopting the Delphi method. The experts’ research directions involved seven specialised fields, including clinical medicine (oncology and TCM-related), clinical pharmacy (oncology and TCM-related), evidence-based medicine, pharmaceutical administration, and pharmacoeconomics. The experts, with 78.13% serving in senior professional titles, and 87.50% having more than 10 years of work experience, indicate that the selected experts have good professionalism and authority. The recovery rate of the consultation forms for both rounds of consultation is 100%, and the recovery time is basically around 2 weeks, indicating that the experts have good enthusiasm. The Cr value stands for the degree of expert authority, which is a higher authority when it is more than 0.7. After two rounds of expert consultation, Cr is 0.84, which demonstrates the superior authority of the experts and credibility of the research results. In this research, the Kendall’s W coefficient of expert opinion coordination was 0.401 and 0.438, respectively, and P < 0.001 during the comparison of two rounds. It indicates the high consistency of expert opinions and the stability of evaluation results.

Secondly, in order to emphasise the characteristics and advantages of CCPP, the first-level indicator (Characteristics of Chinese medicine) was specifically added to the value assessment index system, with all experts agreeing.

Thirdly, a reasonable weight setting is crucial for establishing an index system. The combined dominance of the “Safety” and “Efficacy” criteria (over 60%) indicates a strong consensus among experts that safety and efficacy are the fundamental prerequisites for further value consideration. It suggests that in a multi-criteria decision analysis framework, a poor performance on either of these criteria could act as a potential veto, even if the technology scores highly on other aspects like cost-effectiveness or innovation. This validates the structure of our criteria framework, confirming that the panel prioritised the core requirements for regulatory approval and clinical adoption. The safety and efficacy of our weighting scheme show strong alignment with the core evaluation pillars of major international HTA bodies. Such as the UK’s National Institute for Health and Care Excellence and Canadian Agency for Drugs and Technologies in Health, which base their assessments fundamentally on clinical effectiveness (encompassing efficacy) and safety.

Fourthly, we collected experts’ feedback during each round to gain a deeper understanding of indicators, as well as ensuing guidance for satisfactory practice, so that the index system could be more appropriate for value assessment.

However, the study also has some limitations. First of all, no expert centralised discussions were organised to address different viewpoints, which made it more difficult to reach a consensus, resulting in insufficient consistency of results. In future research, we suggest that implementing sensitivity analysis could more effectively address the complexities and uncertainties involved in evaluating the value of anti-tumour CCPP. Secondly, a formal content validity study before the Delphi rounds would have further strengthened the initial questionnaire. Thirdly, the weight assigned to the Characteristics of Chinese medicine was 0.0661, which is significantly lower than that for safety and efficacy. This discrepancy may stem from the limited representation of Chinese medicine experts among the Delphi panellists in this study. Fourthly, the absence of patient or policymaker perspectives may limit comprehensiveness, although we aim to develop an assessment tool for value evaluation of anti-tumour CCPP from the medical institutions perspective, which can provide a basis for hospital drug access decisions and clinical medication recommendations. The perspectives of patients or policymakers will be considered for inclusion in the evaluation system. In subsequent research, the reliability and validity of the anti-tumour CCPP evaluation system will be verified using selected anti-tumour CCPPs, which are selected from the national drug directory.

Applications in research and clinical practice

The development of the value assessment index system is not merely an academic exercise but provides a foundational instrument with significant potential to guide future research and enhance real-world clinical decision-making. Its applications can be explored in several key directions: ⅰ) The tool’s structured criteria can help identify specific evidence deficiencies for individual drugs, particularly regarding comparative effectiveness, safety in real-world settings, and pharmacoeconomic value. For instance, for a CCPP scoring high on traditional evidence but low on modern pharmacological mechanism validation, future research can be directed towards employing advanced techniques like bioinformatics and molecular biology to elucidate its pathways and potential drug targets. ⅱ) In clinical practice, a validated value assessment index system can be integrated into Clinical Decision Support systems to provide clinicians with a standardised, evidence-based evaluation of CCPP. ⅲ) The tool offers a transparent methodology for drug formulary reviews and health technology assessment. Its structured evaluation of safety, efficacy, and especially economic value can provide critical data to inform reimbursement decisions by national insurance schemes and guide the inclusion of CCPP into the essential drug list and clinical practice guideline.

Conclusion

In summary, this study developed a value evaluation system of anti-tumour CCPP through two rounds of Delphi expert consultation and the analytic hierarchy process. The system encompasses 7 first-level indicators, 24 second-level indicators, and 50 third-level indicators. And we confirmed that the value assessment index system could provide a reference to promoting the clinical rational use of CCPP and the development of the clinical guideline.

Data availability statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author.

Ethics statement

The studies involving humans were approved by The institutional review board of the Second Affiliated Hospital of Army Medical University. The studies were conducted in accordance with the local legislation and institutional requirements. Written informed consent for participation was not required from the participants or the participants’ legal guardians/next of kin in accordance with the national legislation and institutional requirements.

Author contributions

TX: Investigation, Writing – original draft, Writing – review and editing. MH: Data curation, Investigation, Writing – original draft. CL: Data curation, Writing – original draft. MY: Data curation, Writing – review and editing. MC: Data curation, Writing – original draft. XX: Data curation, Writing – original draft. ZZ: Funding acquisition, Project administration, Resources, Validation, Writing – original draft, Writing – review and editing.

Funding

The author(s) declared that financial support was received for this work and/or its publication. This research was supported by the Chongqing Technology Foresight and Institutional Innovation Project (CSTB2023TFII-OIX0054), 2024 Chongqing Graduate Research Innovation Project (CYS240826), and Chongqing Key Specialty Construction Project of Clinical Pharmacy, People’s Republic of China.

Acknowledgements

We thank all the experts who participated in our Delphi study for sharing their time and expertise.

Conflict of interest

The authors declared that this work was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The author(s) declared that generative AI was not used in the creation of this manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphar.2025.1681174/full#supplementary-material

References

Bizuayehu, H. M., Ahmed, K. Y., Kibret, G. D., Dadi, A. F., Belachew, S. A., Bagade, T., et al. (2024). Global disparities of cancer and its projected burden in 2050. JAMA Netw. Open 7 (11), e2443198. doi:10.1001/jamanetworkopen.2024.43198

PubMed Abstract | CrossRef Full Text | Google Scholar

Bray, F., Laversanne, M., Sung, H., Ferlay, J., Siegel, R. L., Soerjomataram, I., et al. (2024). Global cancer statistics 2022: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J. Clin. 74 (3), 229–263. doi:10.3322/caac.21834

PubMed Abstract | CrossRef Full Text | Google Scholar

Chen, Z., Vong, C. T., Zhang, T., Yao, C., Wang, Y., and Luo, H. (2023). Quality evaluation methods of chinese medicine based on scientific supervision: recent research progress and prospects. Chin. Med. 18 (1), 126. doi:10.1186/s13020-023-00836-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Gao, R., Zhang, Y., Hou, W., Li, J., Zhu, G., Zhang, X., et al. (2021). Combination of first-line chemotherapy with kanglaite injections versus first-line chemotherapy alone for advanced non-small-cell lung cancer: study protocol for an investigator-initiated, multicenter, open-label, randomized controlled trial. Trials 22 (1), 214. doi:10.1186/s13063-021-05169-w

PubMed Abstract | CrossRef Full Text | Google Scholar

Guo, W., Wang, P., Zhang, Y., Li, X., Wang, Y., Zhao, K., et al. (2023). Health technology assessment in China's health care sector: development and applications. Health Syst. Reform 9 (3), 2327099. doi:10.1080/23288604.2024.2327099

PubMed Abstract | CrossRef Full Text | Google Scholar

Hommel, I., van Gurp, P. J., Tack, C. J., Wollersheim, H., and Hulscher, M. E. (2016). Perioperative diabetes care: development and validation of quality indicators throughout the entire hospital care pathway. BMJ Qual. Saf. 25 (7), 525–534. doi:10.1136/bmjqs-2015-004112

PubMed Abstract | CrossRef Full Text | Google Scholar

Kadier, A., Ablimit, M., Tursun, H., and Ablat, N. (2025). The Western transmission of traditional Chinese medicine: an investigation of the cultural elements of traditional Chinese medicine in biomedical systems of cross-Asia countries. Front. Pharmacol. 16, 16–2025. doi:10.3389/fphar.2025.1589275

PubMed Abstract | CrossRef Full Text | Google Scholar

Maxion-Bergemann, S., Bornhöft, G., Sonderegger, E., Renfer, A., Matthiessen, P. F., and Wolf, U. (2006). Traditional Chinese medicine (phytotherapy): Health technology assessment report - selected aspects. Forsch Komplementmed 13 (Suppl. 2), 30–41. doi:10.1159/000093591

PubMed Abstract | CrossRef Full Text | Google Scholar

Ruan, J., Chen, S., Ho, Y. S., Wong, V. T., Lam, M. Y., Tsang, H. W. H., et al. (2024). Chinese medicine practitioners’ consensus on traditional Chinese medicine diagnostic patterns, symptoms, and herbal formulas for COVID-19 survivors: a Delphi study. Eur. J. Integr. Med. 66, 102339. doi:10.1016/j.eujim.2024.102339

CrossRef Full Text | Google Scholar

Schütte, K., Schulz, C., and Middelberg-Bisping, K. (2021). Impact of gastric cancer treatment on quality of life of patients. Best. Pract. Res. Clin. Gastroenterol. 50-51, 101727. doi:10.1016/j.bpg.2021.101727

PubMed Abstract | CrossRef Full Text | Google Scholar

Su, P., Zhi, K., Xu, H., Xiao, J., Liu, J., Wang, Z., et al. (2024). The application of multi-criteria decision analysis in evaluating the value of drug-oriented intervention: a literature review. Front. Pharmacol. 15, 1245825. doi:10.3389/fphar.2024.1245825

PubMed Abstract | CrossRef Full Text | Google Scholar

The State Council of the People’s Republic of China. (2016). E. coli. Available online at: https://www.gov.cn/gongbao/content/2016/content_5054716.htm (Accessed December 13, 2024).

Google Scholar

Wang, J. X., Sun, R., Si, D. X., Guo, M. P., Zhang, L., Sun, Y. Y., et al. (2024). Clinical practice guidelines of Chinese patent medicine in China: a critical review. Complement. Ther. Med. 85, 103077. doi:10.1016/j.ctim.2024.103077

PubMed Abstract | CrossRef Full Text | Google Scholar

Wu, J., Fan, D., Shao, Z., Xu, B., Ren, G., Jiang, Z., et al. (2022). CACA guidelines for holistic integrative management of breast cancer. Holist. Integr. Oncol. 1 (1), 7. doi:10.1007/s44178-022-00007-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Xi, Z., Dai, R., Ze, Y., Jiang, X., Liu, M., and Xu, H. (2025). Traditional Chinese medicine in lung cancer treatment. Mol. Cancer 24 (1), 57. doi:10.1186/s12943-025-02245-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Yagüe, E., Sun, H., and Hu, Y. (2022). East wind, west wind: toward the modernization of traditional Chinese medicine. Front. Neurosci. 16, 1057817. doi:10.3389/fnins.2022.1057817

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhang, X., Qiu, H., Li, C., Cai, P., and Qi, F. (2021). The positive role of traditional Chinese medicine as an adjunctive therapy for cancer. Biosci. Trends 15 (5), 283–298. doi:10.5582/bst.2021.01318

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhang, B., Pei, W., Cai, P., Wang, Z., and Qi, F. (2022a). Recent advances in Chinese patent medicines entering the international market. Drug Discov. and Ther. 16 (6), 258–272. doi:10.5582/ddt.2022.01115

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhang, C., Ma, E. L., Liu, B. L., Wu, B., Gu, Z. C., and Lin, H. W. (2022b). Framework development for clinical comprehensive evaluation of Drugs-a study protocol using the Delphi method and analytic hierarchy process. Front. Pharmacol. 13, 869319. doi:10.3389/fphar.2022.869319

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhen, X., Sun, X., and Dong, H. (2018). Health technology assessment and its use in drug policies in China. Value Health Reg. Issues 15, 138–148. doi:10.1016/j.vhri.2018.01.010

PubMed Abstract | CrossRef Full Text | Google Scholar