One hundred sixty-seven patients admitted to four different ICUs in mainland China during 2018 were retrospectively studied. These cases were reviewed considering the following inclusion criteria: (i) patients with respiratory symptoms requiring oxygen therapy or any other organ support in the ICU, (ii) patients with radiology images (chest x-ray or CT scanning) showing lung infiltration, (iii) patients with a suspected infection causing the conditions described meeting criteria (i) and (ii), and (iv) patients with lower respiratory tract samples collected for both mNGS and CMT.

Demographic information, comorbidities, immune state, disease severity upon ICU admission, duration between ICU admission and attempt to sequence LRT specimen, ICU intervention, clinical outcome, and conventional microbiological test results were all collected from the medical record of each patient through the hospital information system. Disease severity was assessed using an Acute Physiology and Chronic Health Evaluation (APACHE) II score (10) and a Sequential Organ Failure Assessment (SOFA) score (11). A higher score indicates more severe clinical presentation. Furthermore, ICU intervention in this study was listed but not limited to vasopressor utilization, invasive or non-invasive mechanical ventilation, and continuous renal replacement therapy. Clinical outcome was illustrated as the length of stay in the ICU and ICU mortality.

Lower respiratory tract samples such as sputum, trachea aspirates, or bronchoalveolar lavage fluids were investigated in this study. The available CMTs of LRT samples (smear, culture, and multiplex PCR) in participating centers and significant pathogen consideration were consistent with previous publication (Supplementary Tables 1, 2) (12). Microbes detected via CMT were identified as true pathogens rather than commensals or contaminants. Serum antigen test results were also reviewed, while serum antibody test results were excluded due to a failure to distinguish acute infection. More specifically, immunoglobulin G (IgG) antibodies reported in our cohort lacked track of a 4-fold rise while IgM antibody test results were not recommended for the infection diagnoses of adenovirus, Mycoplasma pneumoniae, and Legionella/Chlamydia spp. (13, 14).

After extraction, nucleic acid fragments underwent end repairing, adapter ligation, and amplification to construct the library, quality control of which was assessed by Agilent 2100 Bioanalyzer (Agilent Technologies, Santa Clara, CA, United States). The qualified library was sequenced on the BGISEQ-100/50 platform. A non-template specimen was parallelly run for the purpose of contamination control (non-template control, NTC).

Raw sequencing data were filtered by removing low-quality reads, and clean reads were mapped to the human reference genome (hg19) to subtract the host portion by Burrows-Wheeler Alignment. The remaining reads were classified by simultaneously mapping to four microbial genome databases, composed of 4,152 whole genome sequences of viral taxa, 3,446 bacteria, 206 fungi, and 140 parasites associated with human disease (15).

The mapped reads number of each microbe in each respiratory sample was normalized in three ways (15, 16).

The positive criteria for the mNGS result were set as follows, which were consistent with literature reviews at the species level and those that found that microbes were evidenced of pathogenicity in the lungs (15–17). A value of “1” was adjusted to the RPM of the non-template control with no pathogenic bacteria, fungi, or viruses (18).

A Mann–Whitney U-test was used for non-parametric data analysis. A 2 × 2 contingency table was drawn to calculate sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Data were reported as absolute values with 95% CI (7). The marginal frequencies of the 2 × 2 table were tested by a McNemar test. Significance was considered when p < 0.05. The SPSS 22.0 software was used.

One hundred forty-nine cases were enrolled into the final analysis since the total sequencing reads in the mNGS reports of 18 patients were missing, making both RPM and SDSMRN infeasible to calculate. Among these 149 patients [median age 59, interquartile range (IQR) from 47.5 to 70; 104 (69.8%) male], 75 (50.3%) were at immunocompromised states. Fifty (33.6%) were prescribed prolonged corticosteroids (defined as a minimum dose of 0.3 mg/kg/d of prednisone or an equivalent for 3 weeks at least) (12), constructing the therapy of autoimmune disease [37 (24.8%)] and post-transplantation [8 (5.4%)] or other. Twenty-one (14.9%) were receiving chemotherapy for cancer, leukemia, or lymphoma. The median APACHE II and sofa scores were 20 (IQR from 15 to 24) and 9 (IQR from 6 to 11), respectively. The median ICU stay was 12 days (IQR from 7 to 21.5 days), and ICU mortality was 51%. The LRT samples of patients were retrieved and sent for mNGS 1 day (IQR from 1 to 3 days) after ICU admission (Table 1).

In the metagenomic analysis of these 149 samples, 96 were tested by DNA sequencing (DNAseq), 35 were tested by RNA sequencing (RNAseq), and 18 were tested by DNAseq and RNAseq parallelly. No significant difference was found between the mean total yield of DNAseq and RNAseq (21,261,618 vs. 22,862,055 reads, p = 0.49) and for human proportion (97.1 vs. 96.8%, p = 0.09) (Figures 1, 2).

Streptococcus pneumonia (if using RPMsample/RPM_NTC ratio ≥10 criterion, n = 5; while using SDSMRN ≥3 criterion, n = 8), Haemophilus influenzae, Legionella pneumonia, and Bacillus anthraci were only detected by mNGS other than bacterium culture. Forty-eight (48/149, 32.2%) samples were sent for the L. pneumonia PCR, and only one (1/48, 2.1%) case tested positive. Of the 48 cases with a paired comparison between mNGS and Legionella PCR, both criteria also identified only one legionella positive case, which consisted of the PCR result (Figure 3).

Due to the inability of culturing Pneumocystis jirovecii (19), a gomori methenamine stain or PCR was used for the P. jirovecii pneumonia diagnosis. Fifty-eight (58/149, 38.9%) patients had both their gomori methenamine stain and PCR results reported, among which 25 (25/58, 43.1%) cases were P. jirovecii PCR positive, while the gomori methenamine stains only made 7 (7/58, 12.1%) P. jirovecii pneumonia confirmations. The RPMsample/RPM_NTC ratio ≥1 suggested 26 (26/149, 17.4%) P. jirovecii pneumonia cases with both sensitivity and accuracy [sensitivity (95% CI), 80% (58.7–92.4%); accuracy (95% CI), 91.4% (84.2–98.6%)] slightly lower than that of SDSMRN ≥3 [sensitivity (95% CI), 96% (77.7–99.8%); accuracy (95% CI), 96.6% (91.9–100%)], which identified 34 (34/149, 22.8%) P. jirovecii positive cases. When comparing mNGS and CMT in the paired 58 cases, mNGS performed no better than a simplex PCR but beat the gomori methenamine stain for P. jirovecii pneumonia diagnosis with either positive criterion chosen (RPM ratio vs. stain, p < 0.001; SDSMRN vs. stain, p < 0.001) (Figure 3, Table 4).

Viral etiology was revealed only in one-third (54/149, 36.2%) of the patients in this study when using a multiplex PCR. Viruses of the PCR interests were limited to cytomegalovirus (CMV), influenza A/B, adenovirus, rhinovirus, respiratory syncytial virus (RSV), metapneumovirus, and parainfluenza virus. Because the samples of 96 patients only underwent a metagenomic DNAseq analysis, from which RNA viruses would be impossibly discovered, the accuracy of mNGS was evaluated for DNA virus and RNA virus cases separately. Both an RPMsample/RPM_NTC ratio ≥1 (21/149, 14.1%) and an SDSMRN ≥3 (31/149, 20.8%) identified more DNA virus cases than the PCR (6/51, 11.8%). The mNGS was superior to the PCR detecting DNA viruses after a paired comparison between the two criteria and the multiplex PCR of the 51 cases (RPM ratio vs. PCR, p = 0.007; SDSMRN vs. PCR, p < 0.001). Six (6/53, 11.3%) RNA viral infections were suggested by the RPM ratio criterion and nine (9/53, 17.0%) by SDSMRN ≥1. The PCR revealed less cases (5/46, 10.9%) but without significant difference compared with mNGS (RPM ratio vs. PCR, p = 0.5; SDSMRN vs. PCR, p = 0.313) (Figure 4).

Metagenomic next generation sequencing presented a sensitivity of 100% (95% CI, 51.7–100%) for DNA viruses and a negative predictive value of over 90% for RNA viruses. The mNGS detection accuracy of RNA viruses [RPM ratio (95% CI), 90.9% (78.9–100%); SDSMRN (95% CI), 81.8% (65.7–97.9%)] was higher than that of DNA virus [RPM ratio (95% CI), 86.3% (76.8–95.7%); SDSMRN (95% CI), 74.5% (62.5–86.5%)]. Still, the RPM ration criterion performed better than the SDSMRN criterion in identifying virus cases, with higher accuracy (Table 5).

The bocavirus [n = 1/1 (RPM ratio positive/SDSMRN positive)] and coronavirus (n = 1/2) were the two novel viruses reported by mNGS outranging permits of the multiplex PCR in this cohort (Figure 3).

The majority of studies published by researchers evaluated the diagnostic performance of mNGS by setting up the final call for infection of clinicians as the golden standard. However, the mNGS performance in pathogen detection might have been disvalued if clinical impressions involved bias. By reviewing existing reports of both mNGS and CMTs from suspected severe pneumonia patients, this study utilized direct paired comparisons between mNGS and CMT to reveal whether these two methods were significantly different in causative pathogen detection. The study implied that mNGS performed better than fungi culture. Thus, clinicians who are faced with immunocompromised populations or who are working in less-developed areas without access to simplex fungus PCRs or antigen tests are suggested to order mNGS for LRT specimens.

It was also discovered that RPM could be widely used for mNGS results interpretation among clinicians and that the RPMsample/RPM_NTC ratio criterion outperformed regardless of pathogen categorizing.

The research was clearly limited in several ways. Not all samples from this cohort had a PCR test of the suspected viral pathogen. The mNGS performance in detecting common viruses responsible for LRT infection was only evaluated in a small proportion of patients due to the need to compare with available PCR results. This could impact less in a future prospective study with all samples sent for both mNGS and PCR screens. Also, the criteria that were selected in identifying positive microbes reported by mNGS were documented ones from previous studies. It would be better to come up with a threshold based on the quantity of sequenced species under the circumstance that quantitative PCR was run to validate those in each sample. Lastly, effects on the clinical decision of mNGS results were not discussed, which would be better evaluated in a randomized controlled trial.