Evaluation of PCR in Bronchoalveolar Lavage Fluid for Diagnosis of Pneumocystis jirovecii Pneumonia: A Bivariate Meta-Analysis and Systematic Review

1Pneumocystis jirovecii pneumonia (PJP) remains an important cause of morbidity and mortality in all immunosuppressed patients especially in HIV-infected patients –. In England, laboratory-confirmed cases of P. jirovecii pneumonia had an average annual increase of 7% per year during 2000 to 2010. In the United States, the HIV Outpatient Study (HOPS) reported that the incidence of a first episode of PCP was 3.9 cases per 1,000 person-years for the period from 2003 to 2007. The mortality of in hospital PJP ranged from 7 to 11%. As for critical ill PJP patients, the mortality was as high as 29–62%.

2Since P. jirovecii is hard to be cultured in vitro, the diagnosis of PJP relies on a direct microscopic examination of respiratory specimens. In non-HIV immunocompromised patients, conventional staining methods showed low sensitivity, ranging from 38% to 53% in sputum samples. The microscopic approaches highly depend on quality and type of samples, the skill of observers and the reaction of cysts or trophozoites to the staining methods. Additionally, the diagnosis can be hampered in patients using highly active anti-retroviral therapy and PJP chemoprophylaxis which may lead to low burden of P. jirovecii, especially in sputum and oropharyngeal wash samples. Compared to sputum samples, positive results of BALF and lung biopsy specimens were considered as the“gold standard” for PJP diagnosis.

3As a promising tool, PCR techniques for the diagnosis of PJP have been developed by many studies. However, the role of PCR assays in BALF remains controversial, especially to the patients with the positive PCR assays and the negative staining results. Thus, we systematically reviewed all relevant studies published from 1994 to 2012 to evaluate the diagnostic accuracy of PCR in BALF for the diagnosis of PJP.

4A search was made of PubMed, Embase and Web of Science databases for the English-language literature published up to November 2012 by two investigators, respectively. The syntax for the searches was as follows: “Pneumocystis jirovecii Pneumonia” or “PJP” or “Pneumocystis Carinii Pneumonia” or “PCP” and “Polymerase Chain Reaction” or “PCR” and “bronchoalveolar lavage fluid” or “BALF”. An expanded hand-search of references of eligible trials and relevant articles were also performed.

5Full-text publications were included if: (1) They used PCR in bronchoalveolar lavage fluid for suspected PJP. (2) The diagnosis of PJP had to be made in patients who had (i) relevant clinical manifestations including cough and fever; (ii) detection of the pathogen in lung tissue, BALF specimens or sputum specimens by use of conventional staining methods (Giemsa, Toluidine-blue and Gomori methenamine silver) or indirect immunofluorescence assay; (iii) radiographic findings on chest X-ray and chest computed tomography (CT) compatible with PCP. (3) Studies reported that the PCR diagnostic data that could not make mathematical sense according to known mathematical formulas were excluded.(4)To avoid bias, studies with less than 10 patients suspicious of PJP were excluded. Disagreements were discussed among the group until a consensus was reached.

6Each article that met eligible criteria was assessed using the QUADAS-2 tool by two investigators, respectively. Discrepancies were solved by consensus, and there was an adjudicator in case of persistent disagreement.

7Deek’s funnel plot asymmetry test was performed to detect publication bias.

8Data extraction was done independently by two reviewers with predesigned data extraction forms including the following items: general identification information (patient population, study design, reference standard, blind and proven PJP/total patients), technical details of the PCR methods (BALF sample volume, sample centrifugation, DNA extraction methods, PCR method, target gene and appropriate control). Data was directly extracted from the published data, or estimated using common mathematical formulas.

9We calculated pooled estimates of the sensitivity (SEN), specificity (SPE) by using a bivariate analysis approach. The positive and negative likelihood ratio (PLR and NLR) and the diagnostic odds ratio (DOR) were also reported. We constructed a summary receiver operating characteristics (SROC) curve to see the accuracy of the assay.

10Statistical heterogeneity of the results of the trials was assessed by the chi-square test, expressed with the I2 index, as described by Higgins and his colleagues. When heterogeneity was detected, threshold analysis and multiple covariates bivariate meta-regression models were performed. The main covariates assessed were sample processing (prospective), study design (cohort), DNA extraction method (commercial), type of PCR (qPCR; single-PCR; nested-PCR), target gene (mt-rRNA; DHPS), staining methods (IFS), BALF centrifugation and blind. Forest plots were created for each study, showing individual outcome with confidence intervals (CI) and the overall DerSimmonian-Laird pooled estimate. All P values were two-sided and P<0.05 were considered statistically significant. Statistical analyses were performed using midas module in Stata software v. 12 (Stata Corporation, College Station, TX).

11An initial search identified 398 potentially relevant studies. After reading the titles and abstracts, 327 studies were excluded because of irrelevance. With further screening of full texts, 55 studies were discarded for various reasons. Ultimately, 16 studies with 1857 BALF samples from 1793 patients published between 1994 and 2012 met the inclusion criteria and were included in the systematic review –. The literature screening process was shown in Figure 1.

12The characteristics of the included studies were summarized in Table 1. We included prospective, retrospective, cohort and case-control studies. The populations evaluated in our meta-analysis were consisted of HIV-infected patients and non-HIV immunosuppressed patients who were suspected of having PJP. The criteria used for diagnosis of PJP were almost based on chest radiographic infiltration and clinical presentation along with etiological findings from respiratory samples. Details of the PCR techniques were presented in Table 2.

13Methodology quality of included trials was presented in Figure 2. Based on the methods reported in each trial, each of the 14 components according to QUADAS-2 criteria was graded “yes”, “unclear” or “no”, which meant “low risk of bias”, “uncertain of bias” and “high risk of bias”, respectively.

14Non-publication bias was detected by Deek’s funnel plot asymmetry test (P = 0.671).

15Forest plots of sensitivity and specificity showed each study and the overall studies. For 16 included studies, the sensitivity values ranged from 53% to 100% and the specificity values ranged from 23% to 100%, respectively. The pooled sensitivity and specificity values of the bivariate model were 98.3% (95% CI, 91.3% to 99.7%) and 91.0% (95% CI, 82.7% to 95.5%), respectively (Figure 3). The average positive likelihood ratio of PCR in BALF was 10.894 (95% CI, 5.569–21.309) and the negative likelihood ratio was 0.018 (95% CI, 0.003–0.099). Based on the positive and negative likelihood ratios obtained from the meta-analysis, the interpretations of the BALF-PCR results for the different pretest probabilities could be examined by using Bayes theorem to generate post-test probabilities, PTP = LR×pretest probability/[(1-pretest probability)×(1-LR)]. For instance, in a setting of 20% prevalence of PJP, the probability of PJP would be less than 0.5% if the BALF-PCR test was negative and the probability of PJP would be more than 70% if the BALF-PCR test was positive. The pooled diagnostic odds ratio (DOR) was 595.670 (95% CI, 106.412–3334.420) and the area under the curve (AUC) was 0.98 (95% CI, 0.97–0.99) (Figure 4), indicating that the PCR assay has an excellent level of diagnostic value for PJP. We found significant heterogeneity for all test performances because I-square values were above 50%. In subgroup analyses, the sensitivity of subgroups were all above 90% and the specificity were all above 80%. And except cohort, comEX and nested-PCR subgroups, the SEN and SPE of the remaining ones were all over 90%. Quantitative PCR had the highest SEN of 100% (95% CI, 92%–100%) and SPE of 93% (95% CI, 77%–98%). The SEN of single-PCR was relatively low, 92% (95% CI, 54%–99%), while the SPE was 97% (95% CI, 88%–99%). In the analysis of potential influence of heterogeneity, I-squared of comEX, nested-PCR and single PCR subgroups were over 50%. However, we did not find statistically significant except nested-PCR (P<0.001) (Table 3).

16Despite the introduction of highly active antiretroviral therapy (HAART), Pneumocystis jirovecii still remains a severe opportunistic infection, associated with a high mortality rate. Recently, a paper reported that the proportion of PCP as cause of death significantly increased from 8.7% in pre-HAART (1990–1997) to 31.8% in late-HAART (2002–2011) period (P<0.001). At present, a highly sensitive technique is required for the detection of Pneumocystis jirovecii, especially in patients receiving chemoprophylaxis and aggressive anti-retroviral therapy, because false negative results by tinctorial methods may occur in these patients owing to the low sensitivity of these assays.

17We undertook the meta-analysis aiming to evaluate the overall accuracy of the BALF-PCR for the diagnosis of PJP. By meta-analysis, the pooled sensitivity of the BALF-PCR for the diagnosis of PJP was 98.3% (95% CI, 91.3%–99.7%) and the pooled specificity was 91.0% (95% CI, 82.7%–95.5%), which predicts PCR in BALF is a very good method for diagnosis of PJP.

18We also investigated the causes of heterogeneity. For the whole population, there was a significant heterogeneity due to a threshold effect (p = 0.7969). In addition, we could not extract the exact data for AIDS and non-AIDS patients and the number of patients that were treated with prophylaxis prior to BALF, which maybe the potential causes lead to heterogeneity. Subgroup analysis showed nested PCR had a relatively high sensitivity of 98% (95% CI, 76%–100%) and the lowest specificity of 73% (95% CI, 53%–86%), which was consistent with Moonens’s finding. Meta-regression showed the I-squared (95% CI) of nested-PCR was 82.19 (62.16–100.00) with p<0.001, indicating a significant influence of heterogeneity. Another potential influenced factor was the lack of a universally accepted definition for the diagnosis of PCP. The QUADAS-2 reference standard risk of bias similarly affected results. Results were heterogeneous, and the paucity of studies did not allow for assessment of some covariates.

19In clinical practice, whether BALF-PCR is appropriate as a diagnostic test depends ultimately on the predictive values in the intended setting. Fagan’s nomogram showed that in a setting of 20% prevalence of PJP, the probability of PJP would be less than 0.5% if the BALF-PCR test was negative and the probability of PJP would be more than 70% if the BALF-PCR test was positive, which is very helpful for the diagnosis of PJP (Figure 5). Especially in the pre-HAART era, fibrotic bronchoscopy with BALF should be performed in patients with negative induced sputum results while presenting compatible clinical symptoms.

20P. jirovecii-specific DNA amplification will be helpful if the results of routine microbiological methods are negative to patients clinically suspected with PJP. Discrepancies between positive PCR and negative staining results have always been reported by many investigators. Since neither calcofluor white stain nor the monoclonal antibody (fluorescent) stains can make a definitive identification for PJP, positive PCR results would provide an earlier diagnosis for suspected patients before morphological methods. Lu et al. reviewed the PCR methods for diagnosis of PJP. They found that 31.8% of the patients with current false-positive PCR results had prior or later PJP. Other studies made the similar conclusion,, which suggesting that a possible explanation for the differences in assays’ sensitivity may be caused by the low load of P. jirovecii. In this situation, repeating the staining assay from BALF along with re-evaluating consistent respiratory symptoms and radiographic changes for the diagnosis of PJP are recommended.

21It should be noted that P. jirovecii DNA has been detected in respiratory samples from patients either with colonization or subclinical infection due to the high sensitivity of the molecular methods,. Quantification of the Pneumocystis jirovecii burden by real-time PCR might be helpful to distinguish between colonization and infection and could possibly allow for therapeutic monitoring. Matsumura et al. found that the area under the receiver-operating characteristic curve (AUC), sensitivity and specificity for discriminating definite PJP from colonization were 0.96, 100.0% and 80.0%, respectively, at a cut-off value of 1300 copies/ml. However, Flori P et al. found an overlap cut-off values between in samples obtained from potential carriers and proven PJP. Due to the performance of different in-house PCR assays currently used in microbiology laboratories, the establishment of a uniform PCR assay and a cut-off value to discriminate between disease and colonization for P. jirovecii is utmost importance.

22Our study had some limitations. First, the overall number of patients included in our review was relatively small. Albeit we tried to collect all relevant data, it was hard to ensure that no data was missed. Second, we ruled out non-English-language studies and studies with less than 10 cases, which may lead to bias. Third, the lack of a published gold standard for the diagnosis of PJP may cause underlying heterogeneity in our study. Although we try to include studies using criteria that patients who had relevant clinical and radiological manifestations with detection of the pathogen in lower respiratory secretions, we found that some studies did not adhere to this criteria rigorously.

23In summary, this meta-analysis shows a very high accuracy of PCR in BALF for the diagnosis of PJP to patients at risk. The excellent sensitivity of PCR for PJP in BALF suggests that a negative result in suspected patients should present ruling out the diagnosis of PJP. A positive PCR result should be interpreted in parallel with compatible clinical and radiological findings. Further prospective cohort studies should focus on quantitative PCR standardization and determination of the optimal cut-off for quantitative PCR results for the wide use of this test in clinical practice.