The following seven databases were searched from inception to November 2021: MEDLINE (via PubMed), the Cochrane Central Register of Controlled Trials (CENTRAL), EMBASE (via OVID), China National Knowledge Infrastructure (CNKI), WanFang Database, Chinese Scientific Journals Database (VIP), and Chinese Biomedical Literature database (SinoMed). Literature was searched using the combination of medical subject headings (MeSH), free-text words, and publication types. Only Chinese and English articles were retrieved. Reference lists of relevant systematic reviews and meta-analysis identified through screening were also checked manually. Full details of the search strategies used for each database are provided in Supplementary File S2 .

EndNote (EN) X9.3.3 was used to manage literature. One review author (YFX) excluded ineligible studies first by screening titles and abstracts. This was followed by two review authors (YRC and YFX) independently identifying eligible studies through full-text review. Disagreements were resolved through discussion or by referral to a third author (ZLL).

Two review authors (YFX and YXS) independently extracted data from eligible studies. Data were cross-checked for accuracy, and disagreements were resolved through discussion. The following data items were extracted: (1) publication information including first author and year of publication; (2) study characteristics including sample size, follow-up duration, randomization procedure, and blinding procedure; (3) patient characteristics including age and sex; (4) intervention and comparator characteristics including dose and course; and (5) outcome measurements.

Two authors (YXS and CYL) independently assessed risk of bias for each study using the revised Cochrane risk-of-bias tool for randomized trials (RoB 2) (18). The following five domains were assessed within each included study under the official guidance document (19): (1) bias arising from the randomization process, (2) bias due to deviations from intended interventions, (3) bias due to missing outcome data, (4) bias in measurement of the outcome, and (5) bias in selection of the reported result. An overall risk-of-bias judgment was made on each study as “low risk of bias”, “some concerns”, or “high risk of bias”. Disagreements were resolved through discussion or by consulting a third author (DMQ) for consensus.

Two review authors (BFL and YBH) independently assessed the confidence in the body of evidence using the Confidence in Network Meta-Analysis (CINeMA) web application, recommended by the Cochrane handbook for undertaking NMA (20). Disagreements were discussed mutually or by inviting a third author (JPL) to reach a consensus. The methodological framework of CINeMA evaluates confidence in the NMA findings based on the contribution matrix of included studies with consideration of the following six domains: within-study bias, reporting bias, indirectness, imprecision, heterogeneity, and incoherence (21).

We performed a standard pairwise meta-analysis using STATA 16.0. A Bayesian NMA was conducted using R software 4.1.0 via Just Another Gibbs Sampler (JAGS). The BUGSnet package was used in R (22). We calculated the risk ratio (RR) with 95% confidence intervals (CIs) for the rate of clinical effectiveness, KPS improvement, and AEs. A random-effects model was analyzed to estimate effects among multiple comparisons using the Markov chain Monte Carlo (MCMC) method. We set an uninformative prior distribution for four Markov chains running 250,000 iterations (burn-in iterations = 50,000, thinning factor = 1). Convergence was assessed by the Brooks-Gelman-Rubin diagnosis plot and potential scale reduction factor (PSRF), with a PSRF value close to 1 indicating convergence (23). For the dichotomous outcome measurements among mixed comparisons, RR with 95% credible intervals (CrIs) were presented within league tables. We also calculated surface under the cumulative ranking curve (SUCRA) probability values to estimate rankings of competing interventions. The BUGSnet R package was used to draw SUCRA plots. In our study, higher SUCRA values reflect a higher associated clinical effective rate, higher KPS improvement rate, and a lower rate of adverse events. A network geometry plot was drawn to summarize the treatment network using STATA. Each node represents an intervention, and each edge represents a head-to-head comparison between two different interventions (24). The sizes of nodes and edges display the numbers of patients receiving the treatment and the number of studies for the comparison, respectively (24). We split three-arm studies into two pairwise comparisons by equally dividing the number of patients receiving DDP. Since there were no “closed loops” in the network plot, we were unable to assess inconsistency among direct and indirect comparisons. Statistical heterogeneities were tested using the χ2 test with a significance level of 0.1 and quantified using I ² statistics. Substantial heterogeneities were considered with I ² greater than 50%. There was insufficient information in included studies for conducting subgroup analysis considering different lung cancer subtypes or treatment duration. A subgroup analysis considering different doses of DDP was conducted to identify substantial sources of clinical heterogeneity. Comparison-adjusted funnel plots were presented to assess small study effects and potential publication bias using STATA.

A total of 7,456 citations were identified from seven databases. After removing 1,364 duplicates, a further 5,778 were excluded due to irrelevancy based on their titles and abstracts. The full text of the remaining 314 studies was screened, of which 44 RCTs were deemed eligible. The PRISMA flow diagram for the study selection process is shown in Figure 1 .

A total of 2,573 lung cancer patients and 6 kinds of CHIs were involved in the 44 RCTs in which all the patients were in advanced stage. The average age of patients in the vast majority of included studies fluctuated between 50 and 70. All patients received treatment for at least 2 weeks. In terms of treatment, 1,258 patients used DDP alone, 1,096 patients were treated with CHIs combined with DDP, and 219 patients received only CHIs. For the outcomes, 43 studies (97.7%) reported clinical effective rate, 30 studies (68.2%) evaluated the improvement rate of KPS score, and 33 studies (75.0%), 25 studies (56.8%), 26 studies (59.1%), 21 studies (47.7%) assessed the incidence of gastrointestinal reactions, leukopenia, chest pain, and fever, respectively. Details of the baseline characteristics of the studies are shown in Table 1 .

Of the 44 RCTs included, all were two-arm studies except for one (51) three-arm study. The three-arm study administered YDZ combined with DDP, DDP alone, and YDZ alone. The interventions for all the two-arm studies were either combined therapies of CHIs and DDP or CHIs alone, compared to DDP alone. Among the combined therapies, there were six kinds of CHIs: AD combined with DDP [10 RCTs (25–30, 43, 46, 60, 61)], FFKS combined with DDP [11 RCTs (31–37, 42, 45, 48, 59)], HCS combined with DDP [one RCT (38)], KA combined with DDP [two RCTs (39, 40)], KLT combined with DDP [one RCT (41)], and YDZ combined with DDP [12 RCTs (44, 47, 49–58)]. As for the studies that used CHIs alone, there were four kinds of CHIs: KLT [one RCT (62)], AD [three RCTs (63, 65, 68)], FFKS [two RCTs (64, 66)], and YDZ [two RCTs (51, 67)]. The detailed information about compositions, indications, and mechanisms of the CHIs is described in Supplementary File S3 .

Considering the bias generated by the randomization process, all studies had adopted a randomized approach, and reported that the baselines of the two groups were comparable. However, due to the lack of specified methods for generating allocation sequence and concealment, 41 of 44 RCTs were assessed as “some concerns”. Two RCTs (45, 56) were classified as low risk with envelopes for concealment and double-blind procedure mentioned, respectively. One RCT (60) was classified as high risk because of collecting data retrospectively. About the bias due to deviations from intended interventions, all included studies reported no deviations from allocated interventions and used an appropriate method to analyze treatment effects. Thus, all studies were regarded as “low risk”. In terms of bias due to missing outcome data and bias in measurement of the outcome, we could get complete data in all studies; moreover, the measurement or determination of the outcomes in the two groups is consistent and objective; hence, all studies were evaluated as “low risk”. As for the bias in selection of the reported results, there were no pre-reported study protocols identified; thus, all RCTs were rated as “some concerns”. Details of the risk of bias assessment are shown in Supplementary File S4 .

We performed a direct comparison of interventions with different CHIs compared with DDP in the six outcomes. The forest plot and detailed information of the heterogeneity analysis for the six outcomes are shown in Supplementary File S5 . Most of the comparisons between the two groups showed no significant heterogeneity, except for FFKS compared to DDP for clinical effective rate (I ² = 69%), YDZ compared to DDP for the improvement rate of KPS score (I ² = 90.9%), and FFKS+DDP compared to DDP for the incidence of gastrointestinal reactions (I ² = 59.5%). Thus, the fixed-effects model for meta-analysis was used. Subgroup analysis and sensitivity analysis was conducted when there was heterogeneity. Since the tumor stages included in this study were all stage IV, which were consistent and had no obvious clinical heterogeneity, and different doses and courses of chemotherapy may be substantial sources of clinical heterogeneity, a subgroup analysis conducted on the total dose of DDP with sufficient studies indicated that the dose was the likely cause of the heterogeneity. Changing the effect model and eliminating the literature effect size one by one revealed that the original results were not changed (p < 0.05), indicating that the sensitivity analysis results were negative, and the results were relatively robust and reliable. The details of subgroup analysis and sensitivity analysis are shown in Supplementary File S6 .

Network graphs comparing CHIs for lung cancer patients with MPE in each of the six outcomes are shown in Figure 2 . The network graphs were generated using Stata 16.0. Each intervention was shown by a circular node, and each connection represented a contrast. The diameter of the circular node was positively correlated with the number of patients included, and line thickness was positively related to the number of direct comparisons.

It can be seen from Figure 2 that DDP was used as the comparator arm in all studies, but as there was no direct comparison between any two interventions, no closed loop existed. As a result, an inconsistency test was not required for this study. Based on the heterogeneity results and the baseline data of the studies shown in Table 2 , we believe that the homogeneity and similarity assumptions between the studies were sufficient in the NMA, and therefore, the consistency model and random-effects model were chosen to build Bayesian models. The maximum number of iterative calculations during the model building process was 250,000.

RRs (95% CrIs) of all interventions for the six outcomes in our NMA are shown in Table 2 . The results of the ranking probabilities based on SUCRA are shown in Table 3 and Figure 3 . We also provided the rankograms in Figure S7 .

Comparison-adjusted funnel plots were used to detect whether there was publication bias in the six outcomes and are provided in Supplementary File S8 . It can be seen in Figure S8 that there are different angles between the calibration auxiliary line and the center line, indicating that this study may have potential publication bias and small study effects in the six outcomes.

The grading of the comparisons with CINeMA displayed mainly “low” to “very low” confidence ratings. This was due to the network without closed loops of evidence (without mixed evidence); hence, inconsistency cannot be assessed. Thus, the “Incoherence” levels were all illustrated as “Some concerns”. There were “Major concerns” about “Imprecision,” usually related to the low numbers of trials available for some comparisons in this study. Details are provided in  Supplementary File S9 .

This NMA evaluated six types of CHIs as adjuvant and four types of CHIs as alternative treatments when compared to DDP alone for lung cancer patients with MPE. The CHIs included AD, FFKS, HCS, KA, KLT, and YDZ. The six outcomes assessed included clinical effective rate, the improvement rate of KPS score, and the incidence of gastrointestinal reactions, leukopenia, chest pain, and fever. The overall heterogeneity between the different comparisons of drugs was found to be low in our NMA. With respect to improvements in clinical effective rate, the NMA results concluded that HCS combined with DDP performed the best. Modern pharmacological studies have shown that cininobufosin and its active compounds (such as bufalin and cininobufosin) have significant antitumor activities and can reverse the regulation of multidrug resistance and immune response. Moreover, some clinical data have indicated that cinocobalamin may have effective anticancer activity, with low toxicity and few adverse effects (69). In the aspect of KPS score, KA combined with DDP might be the best choice. KA is an intravenous fluid made from an extraction of three Chinese herbs (ginseng, astragalus, and matrine), which has a variety of pharmacological effects including antitumor, reductions in adverse reactions caused by chemotherapy, and improvements in the body’s immune function (70). In relation to reducing the incidence of adverse reactions, FFKS alone showed the best results in reducing gastrointestinal reactions, leukopenia, and chest pain, and FFKS combined with DDP demonstrated the best safety when it comes to fever. The main components of FFKS are oxymatrine, matrine, and other alkaloids, which could induce cell apoptosis and enhance the effects of DDP in non-small-cell lung cancer (NSCLC) cells (71), and prevent or reduce chemotherapy- and/or radiotherapy-induced toxicity when combined with chemotherapeutic drugs (72). Apart from this, other CHIs are able to exert their antitumor and reduce side effects through various mechanisms. The AD contains multiple active ingredients, including astragaloside (Re, Rb1, and Rg1), ginsenoside, cantharidin, eleutheroside E, and syringin, which significantly inhibit the proliferation of various tumor cells, induced cell apoptosis, and have shown outstanding antitumor properties, immune regulation functions, and decrease in chemotherapy-related ADRs (73). Coixenolide is the main active ingredient of KLT, which exhibits anticancer and immunomodulatory properties. The induction of NF-κB-mediated gene transcription in CD4+ T cells participates in the immunomodulatory activity of KLT (74). Research has shown that YDZ could induce the death of cancer cells through a variety of mechanisms, and exhibited higher activity and a broader antitumor spectrum in vitro ( 75).

As the rank probability of six outcomes suggested, CHIs combined with DDP or single-use CHIs were superior than the use of DDP alone in improving the effective rate and KPS score and reducing the incidence of adverse reactions. However, several CHIs did not show statistical significance when compared with DDP alone in the pairwise meta-analysis. Moreover, because of the wide confidence intervals in the NMA due to the small sample size of included patients and the low incidence of adverse events, the rank results need to be carefully considered. One previous simulation study found that the rank probability of the treatment was underestimated when being tested in the largest number of studies in a given network and overestimated for the treatment included in the smallest number of studies. The results can only be reliable when each treatment involved in the analysis has direct evidence or has obvious advantages in effectiveness (76). In this NMA, there was only one RCT of HCS combined with DDP, one RCT of KA combined with DDP, and two RCTs of FFKS alone included where analysis lacked direct comparisons between certain interventions. The grading of the comparisons with CINeMA showed primarily “low” to “very low” confidence ratings, and as a result, the conclusions based on this NMA may not be trustworthy. We suggest clinicians should choose different treatment methods according to the specific requirements of their patients.

In comparison with published research, this is the first NMA, to our knowledge, that compares different CHIs as an adjuvant or alternative treatment to DDP in the treatment of lung cancer patients with MPE (77, 78). Our research has ascendency. Firstly, strict eligibility criteria were used, particularly inclusion of only patients with pleural effusion caused by lung cancer, and DDP as a fixed control. This ensured consistency of the disease conditions and interventions included in the RCTs, which could decrease clinical heterogeneity. Only antitumor drugs listed by the NHSA in the catalog of Chinese patent medicines were included, to ensure conformity with actual clinical usage and provide relevancy for future clinical practice. Furthermore, the six outcome indicators, clinical effective rate, improvement rate of KPS score, and the incidence of gastrointestinal reactions, leukopenia, chest pain, and fever, were chosen on the basis of whether they could provide comprehensive information to recommend as realistic treatment recommendations.

Nevertheless, limitations and shortcomings existed in our research. Firstly, the overall risk of bias was assessed as some concerns. Secondly, the sample size of included studies was relatively small, and the number of qualified studies included were not sufficient. We believe that the credibility of the NMA could be improved if the sample size was increased, and more eligible studies and more RCTs of different types of CHIs were included. In addition, more ranking comparison on dosage and treatment duration could also be considered. Thirdly, as indicated by our results, the network diagram does not form a typical closed loop, such that the research inconsistencies and credibility of our conclusions cannot be checked. Fourthly, long-term survival outcomes are critical for clinical decision-making, and most studies included in our MNA were primarily focused on the short-term therapeutic outcomes due to the relatively limited treatment course and follow-up time. Finally, owing to the limited scope of application of CHIs, all included studies were carried out in China and all patients were Chinese, which may introduce some degree of selection bias to the results. Notably, the Food and Drug Administration (FDA) of the United States approved the clinical trial of KLT in 2001, and a phase II study in patients with advanced pancreatic cancer has been completed in 2014 (79). The Russian Federation approved the clinical trial of KLT in 2002, and KLT has been marketed in Russia since 2005 with a positive response (80). However, the clinical application of KLT still seems limited outside of China with little information being reported officially, and there is no international multicenter study concerning the effect of KLT on MPE. The conclusions drawn from the results, therefore, cannot be generalized on a large scale worldwide.