Clinical Utility of Molecular Tests for Guiding Therapeutic Decisions in Bloodstream Staphylococcal Infections: A Meta-Analysis

Background: Treatment of bloodstream staphylococcal infections (BSI) necessitates the prompt initiation of appropriate antimicrobial agents and the rapid de-escalation of excessive broad-spectrum coverage to reduce the risk of mortality. We, therefore, aimed to demonstrate the diagnostic accuracy of nucleic acid amplification tests (NAAT) for the identification of methicillin-resistant S. aureus (MRSA) and methicillin-sensitive S. aureus (MSSA) in clinically suspected patients. Methods: Until November 23, 2020, databases including PubMed, Scopus, Embase, and Web of Science were scanned for eligible studies. A bivariate random-effects model was used for meta-analysis of the 33 included studies obtained from 1606 citations, and pooled summary estimates with 95% confidence intervals (CI) were generated. Results: Twenty-three studies (n = 8,547) assessed NAAT accuracy for MSSA detection, while three studies (n = 479) evaluated MRSA detection in adults. The pooled NAAT sensitivity and specificity for MRSA in adults was higher [sensitivity: 0.83 (95% CI 0.59–0.96), specificity: 0.99 (95% CI 0.98–1.0)] as compared to MSSA [sensitivity: 0.76 (95% CI 0.69–0.82), specificity: 0.98 (95% CI 0.98–0.99)]. Similarly, eight studies (n = 4,089) investigating MSSA in pediatric population reported higher NAAT accuracy [sensitivity: 0.89 (95% CI 0.76–0.96), specificity: 0.98 (95% CI 0.97–0.98)] compared to adults. Among NAA tests, SeptiFast (real-time PCR, commercial) was frequently applied, and its diagnostic accuracy corresponded well to the overall summary estimates. A meta-regression and subgroup analysis of study design, sample condition, and patient selection method could not explain the heterogeneity (P > 0.05) in the diagnostic efficiency. Conclusions: NAAT could be applied as the preferred initial tests for timely diagnosis and BSI management.


INTRODUCTION
Bloodstream staphylococcal infection (BSI) is an urgent medical issue due to its rising incidence, associated poor outcome and the emergence of high rates of secondary infections such as osteomyelitis, septic arthritis, infective endocarditis and septic metastases (1,2). Many surveillances worldwide recorded a rise in the incidence of BSI to varying degrees, including Thailand (27.4%) (3), France (24%) (4) and Brazil (40-70%) (5), but the majority of documented cases are in adults. Pediatric incidence statistics are minimal, but infants younger than 1 year have been reported as having a higher incidence compared to adolescents (6). Especially in developed countries, methicillin-resistant S. aureus (MRSA) has been observed in over 60% of all isolated S. aureus, and associated MRSA mortality rates have reportedly been higher than methicillin-susceptible S. aureus (MSSA) (7). The burden of BSI is utterly immense in both clinical (8) and economic terms (9), with mortality rates ranging between 20 and 70% (10,11). Thus, rapid identification of the causative agents and detection of resistance markers in patients with BSI, such as the mecA gene, may provide clinically vital information to guide effective care on time, leading to improved patient outcomes.
Vancomycin, a glycopeptide antimicrobial agent, has frequently been included as empirical therapy for suspected BSI (12), along with the awareness that delays in initiating effective antimicrobial therapy may affect patient outcomes. Once identification and susceptibility of microorganisms are established, treatment can be optimized to target isolated bacteria, including discontinuation of vancomycin when MSSA is present. Though safe, this approach exposes the patient to widespectrum antibiotic overuse. Even short anti-MRSA treatment courses may alter host flora, expose to drug-induced toxicity, escalate multidrug-resistant pathogens, treatment-related side effects, and increase hospitalization costs (13). Vancomycin, the preferred antibiotic solution for MRSA infections, is also less effective than oxacillin in the treatment of MSSA infections (14). If the initial antibiotics are inadequate and changed after the diagnostic tests are available, the mortality rate does not improve significantly. Therefore, it is increasingly important to balance these two competing interests, namely the need for complete coverage while avoiding unnecessary medications.
Blood cultures currently represent the primary method for determining the etiology of the BSI (15). This traditional culturebased approach can be time-consuming, including growthbased assays, colony morphology and microdilution resistance tests. It takes about 48-72 h to classify the causative organism, even with a positive blood culture. A positive microbiological diagnosis in BSI can only be made in ∼20-30% of cases with a substantially higher false-negative rate (16,17), and requires a certain amount of sample to begin the cultivation process. Nevertheless, obtaining adequate amounts of blood from neonates for culture is often difficult (18). Furthermore, samples collected after antibiotic exposure may reduce culturebased bacterial detection. As a result, the patients will likely miss the optimal chance of treatment. Therefore, if no pathogenic bacterial agent is detected, sepsis diagnosis is based solely on clinical symptoms, often coupled with an increase in essential biomarkers such as C-reactive protein or procalcitonin (19). For these reasons, there is an obvious need for a more rapid, yet precise tool that uses a limited blood volume to detect organism and guide antibiotic choices in patients with suspected BSI.
While the pathogen culture remains the gold standard, molecular amplification tests, which typically have a shorter turnaround time, can drastically reduce the critical time to initiate preventive and therapeutic strategies, including appropriate antibacterial therapy. Unlike traditional blood culture, NAAT relies on detecting bacterial DNA rather than the recovery of viable bacteria and is less affected by antibiotic preadministration. A previous study demonstrated a decrease in the length of anti-MRSA antibiotics in patients with BSI following a rapid diagnostic test (20). Several studies have assessed the importance of molecular techniques including PCR, real-time PCR, GeneXpert, LAMP, and FilmArray (21)(22)(23)(24)(25), but literature on the relevance of these tests to timely BSI management is widely scattered for any meaningful interpretation. Given the need to make clinical decisions among the pediatric and adult population, we have systematically reviewed and analyzed the available data to demonstrate NAAT diagnostic accuracy compared with microbiological culture.

Study Selection
The MeSH terms were used to search through electronic databases for all relevant citations, and duplicates were carefully removed using the EndNote X9 software (Thomson Reuters, New York, NY, USA). The records obtained were initially scrutinized by reviewing titles and abstracts, and subsequent analysis excluded irrelevant studies. The full-text of potentially eligible studies for accuracy data was retrieved and carefully analyzed. The data collected by two independent researchers (K. Chen and A. A. Malik) were compared, and any comparative discrepancies were resolved by mutual consensus.

Inclusion Criteria
We included full-text, peer-reviewed, cross-sectional, randomized controlled, and case-control studies using NAAT to diagnose and compare BSI with a culture reference standard. For the index test, the studies explicitly provided True positive (TP), True negative (TN), False positive (FP) and False negative (FN) values or included sufficient information to derive 2 × 2 contingency tables. All studies that met the standard BSI definition were eligible for inclusion, including fever, chills, palpitations, rapid breathing, gastrointestinal symptoms, confusion and shock.

Exclusion Criteria
Studies were considered for exclusion if they were conference proceedings, commentaries, reviews, meta-analysis, editorials, case reports, mechanism, and animal experimentation. Studies providing insufficient data for constructing a 2 × 2 contingency table and comprising <10 specimens were excluded. Noninterpretable test results by both index test and microbiological reference standard were also not included.

Data Extraction
Two independent analysts (K. Chen and A. A. Malik) scanned all the related papers with pre-specified eligibility requirements in order to ensure the reproducibility of study selection. Disagreed studies were resolved by consultation with a third investigator (S. C. Ojha). The data were collected from eligible studies including authorship, publishing year, country, settings, study type, patient selection, patient characteristics, sample type, sample size, NAAT specifics, potential risks, and information for construction of 2 × 2 contingency table. For missing details, ambiguous reference standards, and specimen preparation techniques, the authors were consulted individually. Contingency tables for NAAT performance compared to microbiological reference standards were constructed on the basis of available data from the qualifying studies. Studies involving different index tests as compared to the specific reference standard were considered separate studies.

Quality Assessment
The methodological quality of the studies was assessed using the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2), a validated diagnostic study tool (27). The four QUADAS-2 domains were: patient selection, index test, reference standard, and flow and timing. All four domains for the possible risk of bias and the first three domains for applicability concerns were evaluated by reviewing authors (K. Chen and A. A. Malik). Each domain was assessed in terms of risk of bias using signaling questions that are answered with "yes, " "no, " or "unclear" and are judged as "low, " "high, " or "unclear, " respectively. The first three domains are simultaneously evaluated in terms of applicability concerns, which are also graded as "low, " "high, " or "unclear" with identical characteristics. The differences between the reviewing authors were settled by consensus.

Statistical Analysis
The following software was used for data analysis: RevMan 5.4 (Nordic Cochrane Centre, Copenhagen, Denmark) for the methodological quality assessment of included studies and summary plots generation (28). Meta-DiSc 1.4 (Cochrane Colloquium, Barcelona, Spain) for computation of pooled summary estimates including specificity, sensitivity, likelihood ratios, diagnostic odds ratio (DOR), and heterogeneity amongst data (29). Diagnostic accuracy of NAAT in association with 95% CI was computed against microbiological culture using a random-effects model. Furthermore, the I-square (I 2 ) statistics were used to evaluate the heterogeneity of the included studies, where I 2 values ranging from 0 to 40% indicate low heterogeneity, 30-60% moderate heterogeneity, 50-90% substantial heterogeneity, and values >90% signify considerable heterogeneity (30). Using subgroup analysis, different specimen conditions, patient selection, study design, and patient type were analyzed as possible heterogeneity sources. Publication bias was inspected using Deeks' funnel plot asymmetry test (31). A P-value of < 0.05 was generally considered to be statistically significant.

Literature Selection
In total, our search identified 1,606 unique records (PubMed, 402; Scopus, 818; Embase, 180; Web of Science, 206) (Figure 1). Of which, due to duplication in databases, 496 citations were eliminated. Following the scanning of the titles and abstracts of 1,110 publications, 156 studies considered potentially significant were subjected to a full-text revision. Supplementary Table 1 summarizes the studies reviewed, includin why these studies were excluded (see Supplementary Table 1). Essentially, 33 studies were included in subsequent analyses that met all the inclusion criteria (21,.

Quality Appraisal
We applied the QUADAS-2 tool to evaluate the methodological quality of included studies against microbiological culture reference standard (see Figure 2). For a few studies in the patient selection domain (34,38,40,48,53,55,63), the risk of bias was high because the studies could not prevent improper sample exclusion. Regarding the applicability concern, all studies included blood samples from patients suspected of BSI, indicating a low risk of bias. Other domains, including the index test and the reference standard, were presumably at  low risk of bias, as NAAT used pre-established binary response investigation criteria. Concerns about the index test conduct's applicability are unclear, as no proven protocol is available for global use. Reference standards of all studies have been carried out in hospitals or university-affiliated reference labs; as such, we expect operator error bias to be of low concern. Subsequently, there was no uncertainty regarding the possible risk of bias in the flow and timing domain as both the index test and the reference standards were applied to the same samples. In general, the quality of the studies included in our meta-analysis met the methodological standards.

Summary Estimates
The studies were rather heterogeneous, so obtaining NAAT pooled summary estimates from integrated pediatric and adult populations was not considered meaningful for antimicrobial therapy. Initially, we concentrated on diagnosing BSI in children as it is often difficult to procure sufficiently large amounts of blood from children for microbiological culture. Subsequently, the pooled summary estimates of NAAT in the adult population to diagnose BSI were assessed. The accuracy of index tests, commercial tests, and a potential source of heterogeneity among studies in predefined subgroups have also been demonstrated.     (Figure 5B).

Diagnostic Accuracy of In-house vs. Commercial Tests
The diagnostic accuracy of studies based on various NAA tests is summarized in

Meta-Regression and Subgroup Analysis
We assessed the possible source of heterogeneity through a meta-regression analysis on predefined subgroups. Metaregression suggested that study design (prospective/others), country (developed/developing), and patient selection (consecutive/convenience) were not the significant source of heterogeneity (meta-regression P = 0.42, P = 0.34, and P = 0.62, respectively) with the exception of the sample condition (fresh/frozen) (P = 0.04).

Publication Bias
Using Deek's funnel plot asymmetry test, publication bias was measured. In our analysis, we did not find striking publication bias (P = 0.25).

DISCUSSION
Despite advances in supportive care, BSI is a leading cause of death worldwide (10). In recent years, most studies have centered on screening positive blood culture to identify causative staphylococcal pathogen (64-66); however, it remains a matter of concern as every hour of delay in initiating effective antimicrobial therapy increases mortality by 7.6% in sepsis patients (67).
Therefore, it is essential to recognize staphylococcal species and their resistance markers rapidly in patients with suspected BSI, as the prompt intervention will lead to improved clinical outcomes with effective antimicrobial therapy. Currently, several multiplex molecular assays have been cleared by the Food and Drug Administration (FDA) that can identify a wide range of microorganisms concurrently with specific resistance genes in blood samples, and several of these assays have been rendered commercially accessible (68). Nevertheless, the numerous case descriptions and the diverse samples used in the various studies make it difficult to compare research results and restrict diseases' management. We, therefore, conducted a systematic review and meta-analysis to evaluate NAAT's diagnostic performance for diagnosing BSI in clinically suspected patients. Our results revealed that NAAT overall summary estimates for MRSA detection [sensitivity: 0.83 (95% CI 0.59-0.96), specificity: 0.99 (95% CI 0.98-1.0), AUC: 0.98 (95% CI 0.96-0.99)] were higher as compared to MSSA [sensitivity: 0.76 (95% CI 0.69-0.82), specificity: 0.98 (95% CI 0.98-0.99), AUC: 0.98 (95% CI 0.97-0.99)] among adults. Relatively smaller sample size, different DNA extraction technique, target genes adopted, and reaction material quality are among factors that could have contributed to greater NAAT accuracy for MRSA detection. Figure 6 shows the pooled sensitivity and specificity of NAAT for BSI detection in children and adults.
In the pediatric population, NAAT displayed consistently higher diagnostic accuracy for MSSA detection [sensitivity: 0.89 (95% CI 0.76-0.96), specificity: 0.98 (95% CI 0.97-0.98), AUC: 0.90 (95% CI 0.74-1.0)]. Albeit not perfect, the higher sensitivity of NAAT among a pediatric population, where the acquisition of large volumes of blood is a major concern, and relatively small non-interpretable results, encourages the use of the test for    (71). However, Chang and colleagues evaluated the accuracy of SeptiFast against a composite reference standard, including bacteremia and fungemia, which was not unique to BSI.
The vital strengths of this study include a rigorous search strategy, utility of systemic guidelines, impartial selection criteria, a precise reference standard, a bivariate random-effects model for data-manipulation, meta-regression analysis on predefined subgroups, and independent analysts' interpretation. Studies that did not adhere to specific guidelines for diagnosing BSI, as well as those with a high risk of bias and high applicability concerns, as judged by the QUADAS-2 tool, were removed from subsequent analysis. We also excluded studies with fewer than ten samples to reduce the effects of publication bias while also discouraging future researchers from conducting small-scale studies, which is consistent with other meta-analyses (72,73). Furthermore, and involved pre-enrichment steps before molecular testing, which may tend to overstate the index test's diagnostic performance, were excluded.
There are a few limitations to our analysis. It is likely that we may have overlooked a few crucial studies through systematic literature searches across databases. Due to the high degree of reporting variability of the included studies, the effect of factors such as sample volume, non-standardized sample preparation, NAA testing expertise, amplification procedures, and laboratory facilities on the accuracy of NAA tests could not be addressed. It should be noted that the gene targets were also different, which could be possible reasons for the heterogeneity. Although the study design, sample condition, and patient selection were not significant sources of heterogeneity in the meta-regression analysis, these variables could enhance heterogeneity and limit the generalizability of the NAAT's overall diagnostic accuracy. In addition, this meta-analysis was constrained due to insufficient MRSA studies among both children and adult populations, and should be interpreted with caution.

CONCLUSIONS
Our findings suggest that currently available molecular assays may not have adequate diagnostic accuracy to replace microbial cultures. However, molecular assays have a shorter turnaround time, a higher proportion of false-positives, a higher specificity, and may detect minute amounts of DNA from a dead organism. Furthermore, because the specimen type and gene target for MRSA detection are the same in adults and children, NAAT's accuracy in adult blood samples could provide a glimpse of its applicability for subsequent detection in the pediatric population. Therefore, NAAT should be considered as the preferred initial tests for diagnosing BSI in order to avoid unnecessary anti-MRSA therapy. The utility of NAAT, in combination with microbiological culture, should be considered whenever feasible. Given the limited data on MRSA, it would be of interest to conduct a detailed investigation using a higher number of prospective studies to fully validate the clinical outcomes associated with NAAT's utility. Additionally, future research should analyze additional measures, including NAAT's impact on decreased hospitalizations, cost-effectiveness, antibiotic-related adverse effects, and electronic clinical decision supporting tools to accelerate therapy adjustment.

DATA AVAILABILITY STATEMENT
The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author/s.

AUTHOR CONTRIBUTIONS
SCO conceptualized the study. KC, AAM, and SCO performed the literature search, analyzed data, and drafted the manuscript. Y-JS, SA, CS, C-LD, and SCO reviewed and edited the manuscript. All authors contributed to the article and approved the submitted version.

FUNDING
This study was funded by the Doctoral Research Fund of Affiliated Hospital of Southwest Medical University to SCO and KC.