Osteoporosis guidelines on TCM drug therapies: a systematic quality evaluation and content analysis

Objective The aims of this study were to evaluate the quality of osteoporosis guidelines on traditional Chinese medicine (TCM) drug therapies and to analyze the specific recommendations of these guidelines. Methods We systematically collected guidelines, evaluated the quality of the guidelines using the Appraisal of Guidelines Research and Evaluation (AGREE) II tool, and summarized the recommendations of TCM drug therapies using the Patient-Intervention-Comparator-Outcome (PICO) model as the analysis framework. Results and conclusions A total of 20 guidelines were included. Overall quality evaluation results revealed that four guidelines were at level A, four at level B, and 12 at level C, whose quality needed to be improved in the domains of “stakeholder involvement”, “rigor of development”, “applicability” and “editorial independence”. Stratified analysis suggested that the post-2020 guidelines were significantly better than those published before 2020 in the domains of “scope and purpose”, “stakeholder involvement” and “editorial independence”. Guidelines with evidence systems were significantly better than those without evidence systems in terms of “stakeholder involvement”, “rigor of development”, “clarity of presentation” and “applicability”. The guidelines recommended TCM drug therapies for patients with osteopenia, osteoporosis and osteoporotic fracture. Recommended TCM drugs were mainly Chinese patent medicine alone or combined with Western medicine, with the outcome mainly focused on improving bone mineral density (BMD).


Introduction
Osteoporosis is a systemic skeletal disease characterized by decreased bone mass and microarchitectural deterioration of bone tissue, leading to increased bone fragility and fracture risk (1).An estimated 200 million people worldwide suffer from osteoporosis (2), and osteoporotic fractures account for 34.8% of the global noncommunicable disease burden worldwide (3).In Europe, over 230,000 men and women are at high risk of osteoporotic fractures (4), which account for more Disability Adjusted Life Years (DALYs) lost than common cancers (excluding lung cancer) (3).Additionally, osteoporosis and osteoporotic fractures cost European healthcare systems more than EUR 5.6 billion per year (4).It is estimated that the annual direct medical costs of osteoporosis-related fractures in the United States will reach $2.5 billion by 2025 (5).
As a treatment for osteoporosis, TCM drug therapies have garnered increasing attention (6).Studies have shown that TCM has a better pain-relieving effect, a higher total effective rate, and a lower risk of adverse events compared with regular Western medicine treatment (7,8).With the widespread use of TCM drug therapies for osteoporosis, there is an urgent need for guidance on the best available methods.Thus, osteoporosis guidelines on TCM drug therapies were published, but their quality varied.Different guidelines even gave inconsistent recommendations (9)(10)(11).Additionally, the main treatment plan, indications, and efficacies of TCM still remain unknown.
Therefore, the study aimed to systematically collect relevant osteoporosis guidelines on TCM drug therapies, evaluate them using the Appraisal of Guidelines Research and Evaluation (AGREE) II tool, and analyze the specific factors that affect the quality of the guidelines.The ultimate goal was to provide a reference for future osteoporosis guideline development.Meanwhile, this study summarized the high-quality recommendations of TCM drug therapies in the guidelines, providing up-to-date and essential information for clinical decision-making.

Literature acquisition 2.1.1 Search strategy
A comprehensive search was conducted using PubMed, EMBASE, Web of Science, CBM, CNKI, VIP, and Wanfang Database to search for osteoporosis guidelines that included TCM drug therapies.To search guidelines repositories and grey literature, we used the National Institute for Health and Care Excellence (NICE), Guidelines International Network (G-I-N), World Health Organization (WHO), Scottish Intercollegiate Guidelines Network (SIGN), Medlive and Google Scholar.All relevant articles from inception to June 2023 were retrieved.
Search terms included: Osteoporosis, Bone Loss, Guideline, Guidance, Recommendation, etc.The full search strategy is presented in Appendix 1.

Inclusion criteria and exclusion criteria
The inclusion criteria included: (1) published guidelines related to osteoporosis; (2) written in Chinese and English; (3) guidelines on TCM drug therapies, such as herbs, herbal extracts, TCM prescriptions, Chinese patent medicines, etc.

Literature screening and data extraction
Two researchers (LZ, JL) independently screened the literature and extracted the basic information of the included guidelines (e.g., title, publication year, main developing organization, fund, etc.).In case of disagreement, the decision was taken by a third researcher.

Overall evaluation of AGREE II
Two reviewers independently evaluated the 20 included guidelines using the guideline quality assessment tool AGREE II, which comprises six domains (scope and purpose, stakeholder involvement, rigor of development, clarity of presentation, applicability and editorial independence), including 23 items.The minimum score for each item is one point and the maximum score is seven points.The higher the score, the more consistent the content of the guideline with the requirements of the item.Score of each domain of a guideline is the percentage of the sum of the points of all items in that domain to the highest possible score in the domain (12).
According to the scores of the six domains, a guideline was classified into three levels: if six domains scored ≥60%, the guideline would be classified as A (recommended); if a guideline scored between 30% and 60% in three or more domains, it would be classified as B (recommended after modification and improvement); if a guideline had scores of ≤30% in three or more domains, it would be classified as C (not recommended).
Descriptive statistical analysis was then performed by calculating the total score of each domain, presented as mean ± standard deviation (SD).

Stratified analysis
Guidelines were stratified according to the following stratification factors: (1) publication year, either before or after 2020; (2) availability of funding; (3) whether an evidence system was used or not, and (4) whether they were developed by an organization (a society or an association) or by an individual or expert group.After stratification, a category wise statistical comparison of the scores of all domains were calculated (P<0.05).

Content analysis
Content analysis on the recommendations of TCM drug therapies in A-and B-level guidelines followed these two steps: (1) remove recommendations with incomplete information, low-and very lowquality evidence, before extracting the remaining medium-and highquality recommendations; (2) summarize and analyze the extracted recommendations using the PICO model as an analysis framework.

Literature screening and data extraction
The search obtained 360 records, of which 138 were duplicates.According to the inclusion and exclusion criteria, 166 articles were excluded during the title abstract screening, 36 articles were removed during full-text screening, and 20 guidelines were finally included.The Preferred Reporting Items for Systematic Reviews and Meta Analyses (PRISMA) flowchart for the selection of guidelines is shown in Figure 1, and the basic characteristics of the included guidelines are shown in Table 1.

Overall evaluation of AGREE II
The two reviewers independently evaluated the included guidelines with AGREE II.The ICC were between 0.878~0.982,indicating a high consistency.Among the 20 guidelines, four were recommended at level A (recommended), four were recommended at level B (recommended after modification), and 12 were recommended at level C (not recommended).Consistency and quality evaluation results are shown in Table 2.

Scope and purpose
Mean score for the "scope and purpose" domain was 64.65% (SD 17.25%).While the guidelines clearly described their overall purpose, covered health issues and the target population, they rarely expressed health issues in the form of the PICO.

Stakeholder involvement
Mean score for "stakeholder involvement" was 43.75% (SD 21.89%).Most of the guidelines clearly indicated the users of the guidelines, providing names, institutions, and regions of the members of the expert group, but did not collect the preferences and views of the target population.Descriptions of staffs responsibilities and research fields were incomplete.

Rigor of development
The "rigor of development" domain, which is the most important area for guideline development, had the lowest average score of 31.35% (SD 28.71%).Five guidelines (10, 14-16, 18) scored over 60% in this domain, indicating a rigorous development process.However, most guidelines failed to use systematic search methods to retrieve evidence, establish clear evidence selection Frontiers in Endocrinology frontiersin.orgcriteria, undergo external expert review, and provide specific updates.

Clarity of presentation
The "clarity of presentation" domain had the highest average score of 79.03%, and the lowest SD of 16.85%.The recommendations given by the guidelines were clear and unambiguous, and different options were given for different clinical scenarios.However, some important recommendations were not presented in forms such as tables, special fonts, or flowcharts to enhance readability.

Applicability
Mean score for "applicability" was 32.92% (SD 22.20%).The scores in this area were low.Most guidelines failed to address barriers and facilitators, or potentially related resources and other issues for guideline application, or to provide supporting tools for implementation.

Editorial independence
Mean score for "editorial independence" was 40.00% (SD 30.69%).This domain requires that conflicts of interest among members of the guideline development team are documented and disclosed so that the content of the guidelines is not influenced by the sponsorship.Half of the guidelines reported sponsors and conflicts of interest, but did not adequately explain their role on guideline development.

Stratified analysis
There were statistically significant differences in the "scope and purpose", "stakeholder involvement" and "editorial independence"  domains when comparing guidelines published before and after 2020 (P<0.05).Guidelines published after 2020 were found to be better than those before 2020.Additionally, guidelines with evidence systems significantly outscored those without in terms of "stakeholder involvement", "rigor of development", "clarity of presentation", and "applicability" (P<0.05).Besides, there was no statistically significant difference in all domains of AGREE II in terms of whether the guideline was funded or not, and whether the development was by an association or an individual or expert group.Stratified analysis results are shown in Table 3.

Drug recommendations
After removing recommendations for TCM drug therapies with incomplete information, low-and very low-quality evidence, the remaining medium-and high-quality recommendations were summarized and analyzed, resulting in 16 recommendations for TCM drug therapies, which were mainly from three guidelines (guideline numbers 2, 3 and 5).The target population of the recommendations is mainly divided into three categories: population susceptible to osteoporosis (osteopenia patients), patients with osteoporosis, and patients with osteoporotic fracture (OPF), the most common complication of osteoporosis.Patients with osteoporosis include primary osteoporosis (POP), postmenopausal osteoporosis (PMOP) and senile osteoporosis (SOP).Specific recommended drugs and outcome measures are detailed in Tables 4, 5.

Quality analysis 4.1.1 Progress of osteoporosis guidelines on TCM drug therapies
The number of TCM drug therapy guidelines has been increasing steadily, with the initial appearance in 2006 and a slower growth rate until 2020.From 2020 to 2023, the number of guidelines doubled, reflecting a growing emphasis on standardized diagnosis and treatment by medical professionals.Stratified analysis shows that the quality of guidelines has improved over time, with post-2020 guidelines scoring significantly higher in "scope and purpose" , " stakeholder involvement" , and " editorial independence" domains.This suggests that developers have recognized previous deficiencies and implemented improvements.
In addition, the introduction of evidence systems has significantly enhanced guideline quality.Guidelines with evidence systems generally exhibited higher overall quality compared to those without, and all were classified as A or B grade guidelines.They ranked higher in "stakeholder involvement", "rigor of development", "clarity of presentation", and "applicability" domains.Guidelines with evidence-based systems are more focused on guideline methodology than consensus guidelines and can significantly improve the scientificity, transparency, and applicability of guidelines.Based on the comprehensive quality assessment, guidelines scored highly in the "scope and purpose" and "clarity of presentation" domains, with low variation.This indicated that the guidelines included in this study were particularly focused more on these two domains.Furthermore, these two domains emphasized standardization of guideline writing, indicating a higher level of standardization in the composition of osteoporosis guidelines on TCM drug therapies.

The main deficiencies and targeted measures of the current osteoporosis guidelines on TCM drug therapies
This study included 20 guidelines, of which 17 were from China and only 3 were international guidelines.To a certain extent, this reflected the limited international recognition of TCM drug therapies for osteoporosis.Thus, it is necessary to further promote the development of international guidelines for TCM drug therapies.
At the same time, of the 20 guidelines, 12 are C-level guidelines, more than half.These low-quality guidelines may have potentially impacted the adoption and endorsement of TCM recommendations.Our comprehensive quality assessment revealed that both the overall guidelines and the C-level guidelines obtained low scores in the domains of "stakeholder involvement", "rigor of development", "applicability", and "editorial independence".Notably, the C-level guidelines scored even lower than the overall guidelines in these domains, particularly in the "rigor of development" domain.This pattern implies that the absence of C-level guidelines in these domains may be influencing the overall quality of guidelines in these areas.
Therefore, in order to enhance the overall quality of the guidelines, we propose targeted measures in these four domains: (1) Stakeholder involvement: the guidelines should furnish detailed information about the expert group members, especially regarding their role in guideline development and research fields.(2) Rigor of development: guideline developers should incorporate evidence and recommendation systems, detail literature search strategies and inclusion criteria, and elaborate on the specific methods used to formulate recommendations.Guidelines should undergo external reviews before publication and provide an explicit update process (including method, timing, and frequency).( 3) Applicability: during guideline development, potential implementation obstacles should be fully considered; effective and explicit implementation tools should be provided, and the cost of implementing the recommendations should be thoroughly evaluated.(4) Editorial independence: the development organization should disclose any influence from sponsors and conflicts of interest among members to ensure the objectivity of the guidelines.

Active ingredients and mechanisms of action of TCM
In the guideline, we recommended six Chinese patent medicines and two decoctions.TCM are characterized by their synergistic effects on multiple pathways, targets, and signaling pathways through Xianling Gubao capsule: The flavonoids in the core active ingredients increase BMD, bone volume/tissue volume (BV/TV), trabecular number (Tb.N), and trabecular thickness (Tb.Th), and decrease bone surface/bone volume (BS/BV) in osteoporotic rats; regulate the expression of osteoprotegerin (OPG) and receptor activator of nuclear factor-kB(RANKL) proteins, thereby inhibiting osteoclast generation, decreasing bone resorption, and inhibiting the development of osteoporosis (33).Furthermore, key active ingredients such as psoralen, isostatin, and sulforaphane activate adenylate cyclase to promote the biosynthesis of various sex steroid hormones, elevating the levels of 17b-estradiol, luteinizing hormone, 12a-hydroxyprogesterone, and androstenedione (43).Capsule also improves the bone growth factor bone morphogenetic protein 2 (BMP-2) expression in osteoporotic rats with fractures, enhances insulin-like growth factor-1 (IGF-1) expression, and promotes bone  (45).EXD exerts estrogen-like effects and is effective in reducing bone loss by increasing BMD and improving bone microarchitecture as well as restoring serum levels of the osteoblast-secreted (OCN) protein (44); it also enhances the proliferation rate of osteoblast-like UMR-0 cells in rats (46), and promotes the proliferation of mouse embryonic osteoblast precursor cells, Mc3t3-e1 cells, and osteogenic differentiation by regulating the BK channel (47).EXD also activates the Insulin-like growth factor 1 receptor/Phosphatidylinositol 3-kinase/Protein kinase B(IGF1/PI3K/AKT) signaling pathway by increasing serum IGF1 concentration and tibial Insulin-like growth factor 1 receptor (IGF1R), PI3K, and AKT expression, thereby promoting osteogenic differentiation and proliferation of osteoblasts.Moreover, it regulates the concentration of medium and long-chain free fatty acid (MLCFA) and inhibits Stearoyl-CoA desaturase 1 (SCD1) activity, restoring disturbed lipid metabolism in adipose tissue and promoting fatty acid synthetase expression through activation of the IGF1/PI3K/AKT signaling pathway to alleviate osteoporosis symptoms (45).

Clinical efficacy of TCM
The TCM in the recommended guidelines had several clinical randomized controlled trials (RCTs) to verify its efficacy.The main clinical outcome of the clinical studies is the BMD value, combined with bone pain score, clinical effectiveness rate and osteocalcin, blood calcium and phosphorus.Systematic evaluation and evidence quality grading showed that the quality of evidence was moderate.
For example, in the case of Xianling Gubao capsules, five RCTs (48-52) for the treatment of patients with POP were conducted, with a total sample size of 610 cases in Meta-analysis (304 cases in the experimental group and 306 cases in the control group).The largest sample size in a single clinical trial was 192 cases, and the smallest sample size was 60 cases.The results showed that the use of TCM alone can significantly increase the BMD values of patients.For studies on the treatment of patients with SOP, a meta-analysis of 3 RCTs (53-55) was conducted, with a total sample size of 292 cases (146 cases in the test group and 146 cases in the control group).The largest sample size in a single clinical trial was 160 cases, and the smallest sample size was 64 cases.The results showed that the use of TCM alone could significantly increase the BMD values of the patients; furthermore, a Meta-analysis (56) of 54 clinical studies (including 22 RCTs) was conducted, with a total sample size of 5110 cases (2583 cases in the test group and 2527 cases in the control group).The largest sample size in a single clinical trial was 300 cases, and the smallest sample size was 50 cases.The results showed that the combination of TCM and Western medicine could significantly improve patients' bone mineral density values, VAS for pain, and levels of alkaline phosphatase, osteocalcin, and blood calcium and phosphorus.For studies on the treatment of patients with PMOP, a meta-analysis of 2 RCTs (57, 58) was concluded, with a total sample size of 699 cases (231 cases in the experimental group and 468 cases in the control group).The largest sample size in a single clinical trial was 474 cases, and the smallest sample size was 225 cases.The results showed that using TCM alone can significantly improve the BMD values of patients.A meta-analysis of the 2 RCTs (59, 60) was concluded, with a total sample size of 304 cases (152 cases in the experimental group and 152 cases in the control group).The largest sample size in a single clinical trial was 200 cases, and the smallest sample size was 104 cases.The results showed that the combination of TCM and Western medicine could significantly improve the BMD values of patients.After checking the original studies, all the above RCT trial groups were comparable with the control group at baseline (48)(49)(50)(51)(52)(53)(54)(55)(56)(57)(58)(59)(60).All used objective BMD as the primary clinical outcome with high stability, confidence, and comparability (48)(49)(50)(51)(52)(53)(54)(56)(57)(58)(59)(60).Two RCTs (57, 58) utilized a double-blind design.
In summary, the basic studies confirmed that multiple active ingredients of TCM exert synergistic effects of multiple pathways, targets, and signaling pathways in the prevention and treatment of osteoporosis.The level of clinical research evidence is moderate, with some shortcomings in trial design and control of confounding factors.More high-quality and rigorous clinical observational studies are still needed.Several clinical RCT have demonstrated that the use of TCM alone or in combination with Western medicine can improve patients' bone density values and improve clinical symptoms.From a pharmacoeconomic perspective, TCM is more cost-effective when the individual willingness-to-pay threshold meets certain requirements (61,62).Western medicine, Western medicine basic treatment of osteoporosis, Including calcium, active vitamin D and its analogs, bisphosphonates (including alendronate sodium, zoledronic acid, etc.), calcitonin and so on.WM, Western Medicine; BPSS, Bone Pain Symptom Score; TCMSS, Traditional Chinese Medicine Symptom Score; BMD, Bone Mineral Density; LV, Lumbar Vertebra; ODI, Oswestry Disability Index.The author(s) declare financial support was received for the research, authorship, and/or publication of this article.This study was supported by the National Natural Science Foundation of China (72204061), Guangdong Medical Science and Technology Research Fund Project (B2021466), Science and Technology Planning Project of Guangdong Province (2021B1212100007), and State Key Laboratory of Traditional Chinese Medicine Syndrome, "Research on Basic Standards of Traditional Chinese Medicine Syndrome" Project.The funders had no role in the study design, collection, analysis or interpretation of the data.The funders did not write the paper and had no role in the decision to submit the paper for publication.

FIGURE 1 PRISMA
FIGURE 1PRISMA flowchart for the selection of guidelines.

TABLE 1
The basic characteristics of the guidelines.
GN, Guideline Number; ES, Evidence System; GRAGE, Grading of Recommendations, Assessment, Development, and Evaluations; TMES, Traditional Medicine Evidence System suggested by Jianping Liu; R, Repot; NR, Not Report; CACM, China Association of Chinese Medicine; CACMS, China Academy of Chinese Medical Sciences; SP, Standardization Project of Clinical Application Guidelines for the Treatment of Superior Diseases with Chinese Patent Medicines; CAGG, Chinese Association of Gerontology and Geriatrics; WHO, World Health Organization; WFCMS, World Federation of Chinese Medicine Societies; BMA, Beijing Medical Association; CHPF, China Health Promotion Foundation; CSOBMR, Chinese Society of Osteoporosis and Bone Mineral Research; BPWF, Bethune Public Welfare Foundation; CAITWM, Chinese Association of Integrated Traditional and Western Medicine; CAITWMZP, Chinese Association of Integrated Traditional and Western Medicine in Zhejiang Province; IARASM, Institute for Advanced Research in Asian Science and Medicine.

TABLE 2 AGREE
II domain scores and consistency tests for guidelines.

TABLE 3
Stratified analysis results of the AGREE II domain scores of guidelines.
NR, Not report; R, Report; *The difference was statistically significant.

TABLE 4
Drug recommendations for osteoporosis patients.