Guidelines meeting the eligibility criteria were searched in English and Chinese using a computer program to avoid subjective interpretation. A total of 11 databases, including the National Institute for Health and Care Excellence (NICE), Agency for Healthcare Research and Quality (AHRQ), PubMed, Embase, AMED (Allied and Alternative Medicine), Cumulative Index to Nursing and Allied Health Literature (CINAHL), WanFang Data, China National Knowledge Infrastructure (CNKI), Chinese Technical Periodicals (VIP), and Chinese Biomedical Literature Database (CBM), collection of clinical practice guidelines for Tuina at CHINA and abroad, were searched for articles from inception until March 2021. At the same time, the Google Academic and the Yimaitong databases were searched to supplement the acquisition of relevant guidelines. The search adopts a combination of subject terms and free words. The search terms include “massage” or “Tuina” or “Chinese manual therapy” AND “guideline” or “guidance” or “recommendation” or “consensus” or “policy”.

We included clinical practice guidelines published by international, national, or regional groups for the application of Tuina. The guidelines were included if they met the following criteria: (1) published in English or Chinese language, (2) peer-review publications, and (3) published between 2000 and 2021, and we also excluded patient-used guidelines, guideline commentaries, guideline interpretations, and translated versions of original guidelines.

Two researchers (Mingwang Qiu and Yue Zhang) used the document management software EndNote X9 to screen the documents and extract data independently. Disagreement between the two parties shall be resolved through a discussion by a third party (Fan Huang).

In total, four researchers (Mingwang Qiu, Yue Zhang, Fan Huang, and Siyi Zhao) used AGREE II to evaluate the quality of the inclusion guidelines, including 23 items in 6 areas and 2 overall assessment items. The minimum score for each item was 1 point, and the maximum score was 7 points (9). We have calculated the final score of each field according to the formula as follows: each field score = (actual score-minimum possible score)/(maximum possible score-minimum possible score) × 100% (9).

The AGREE II instrument does not offer guidance on the cutoff scores to determine the quality of each domain; to determine the overall quality and recommendation level, we based our approach on the methods of previous research, guidelines were classified as strongly recommended when the standardized percentages of all six domains were above 60% and recommended when the standardized percentages ranged from 30% to 60% in more than three domains. However, guidelines are not recommended when the standardized percentages were < 30% in more than three domains.

Before the formal evaluation, all investigators were trained, and one guide was independently pre-scored. Then the group discussed and negotiated to ensure that the four evaluators had basically the same understanding of each item and had the same evaluation criteria.

Statistical analysis used IBM SPSS Statistics 25 software to calculate the ICC value to verify the consistency of the evaluators when using the evaluation tool. Using Excel 2016 for calculating the scores of the AGREE II tool, the mean and standard deviation of the scores in each field and the proportion of each part were calculated. The agreement between the four reviewers was measured by the intra-group correlation coefficient (ICC) and 95% confidence interval (CI). The degree of agreement between 0.01 and 0.20 was considered minor, the degree of proportionality between 0.21 and 0.40 was considered moderate, the degree of proportionality between 0.41 and 0.60, the substantive degree between 0.61 and 0.80, and the agreement between 0.81 and 1.00 were considered very good. P < 0.05 indicates statistical significance. All tests were double-sided. We used SPSS version 25.0 for statistical analysis. The ICC of the scores given by the four evaluators would be analyze.

No patient was involved.

The initial search detected 239 related publications, and EndNote X9 excluded 160 duplicate records. After reading the title and abstract, 13 records were excluded from the preliminary screening. After the full-text screening, a total of eight Tuina guidelines were included (12–19) for SR through AGREE II. The literature search and screening process are shown in Figure 1.

As shown in Table 1, all the guidelines evaluated in this study originated from China and were published from 2010 to 2020. Three of the guidelines (14–16) were published in the same journal in 2010. However, a major shortcoming of the former three articles was that there is no evidence of the source of the guideline writing. In total, seven of the eight guidelines (9, 11–16, 18, 19) were aimed at clinicians or Tuina practitioners specifically. Seven (12–17, 19) guidelines were from different Chinese medicine universities or affiliated hospitals, and the remaining one (18) was from the China HQCC.

Table 2 shows the AGREE II standardization field score for each Tuina CPG and its overall recommendations. The scope and purpose of the field and the clarity and presentation achieved the highest average scores of 70% and 67% (ranged 60–82% and 51–82%, respectively). The average score for stakeholder participation in the domain was 51% (ranged 33–61%), and only one guide scored more than 60%. The largest score range was editorial independence (17–94%). A total of three guides (37.5%) scored 17, and five guides (62.5%) scored <50%. Editorial independence and applicability produced the lowest average scores of 40% and 47% (ranged 17–94% and 21–64%, respectively). Unexpectedly, the four guidelines (26.7%) had the lowest score due to the failure to describe the criteria for selecting evidence and making recommendations clearly.

In general, the Practical Guidelines for Treating Prophylactic Diseases of Tuina Intervention of Spleen Deficiency (13) in Children's Guide had high scores in all areas and is listed as “strongly recommended” in clinical practice, “recommendation of 3 types of Tuina CPG (37.5%),” and three types (37.5%) “not recommended.” The agreement of the overall reviewer was very good (ICC:0.901, 95% CI).

After four reviewers' SR, we obtained the AGREE II scores and total scores in each field of the eight guides, as shown in Table 3. Chen et al. (13) and Qin et al. (17) served as the first author's guide for a total score of more than 400; therefore, we can strongly recommend using these two Tuina guidelines. The guidelines with Li (12), Yan et al. (18), and Liu (19) as the first authors had a total score of < 300, so we do not recommend using these three guidelines. The guide, led by Lei et al. (14), Sun et al. (15), and Shen et al. (16) as the first author, scored between 300 and 400. These three guides can be recommended after upgrading and improving modifications.

The agreement between the four reviewers was measured by the intra-group correlation coefficient (ICC) and 95% confidence interval (CI). The degree of agreement between 0.01 and 0.20 was considered minor; the degree of proportionality between 0.21 and 0.40 was moderate; and the degree of proportionality between 0.41 and 0.60, the substantive degree between 0.61 and 0.80, and the agreement between 0.81 and 1.00 were very good. P < 0.05 indicates statistical significance. All tests were double-sided. We used SPSS version 25.0 for statistical analysis. By analyzing the intra-group correlation coefficients of the scores given by the four evaluators, we can see that the ICC value of each field was >0.81, as shown in Table 3. It can be considered that the scores given by the evaluators within the group were highly consistent.

As shown in Figure 2, the abscissa axis is the names of the diseases included in the literature, and the ordinate axis is the frequency with which they appear in the literature. A total of 35 diseases were included; of which, 13 were specific to children.