How many biological replicates are needed in an RNA-seq experiment and which differential expression tool should you use?

Abstract

RNA-seq is now the technology of choice for genome-wide differential gene expression experiments, but it is not clear how many biological replicates are needed to ensure valid biological interpretation of the results or which statistical tools are best for analyzing the data. An RNA-seq experiment with 48 biological replicates in each of two conditions was performed to answer these questions and provide guidelines for experimental design. With three biological replicates, nine of the 11 tools evaluated found only 20%–40% of the significantly differentially expressed (SDE) genes identified with the full set of 42 clean replicates. This rises to >85% for the subset of SDE genes changing in expression by more than fourfold. To achieve >85% for all SDE genes regardless of fold change requires more than 20 biological replicates. The same nine tools successfully control their false discovery rate at ≲5% for all numbers of replicates, while the remaining two tools fail to control their FDR adequately, particularly for low numbers of replicates. For future RNA-seq experiments, these results suggest that at least six biological replicates should be used, rising to at least 12 when it is important to identify SDE genes for all fold changes. If fewer than 12 replicates are used, a superior combination of true positive and false positive performances makes edgeR and DESeq2 the leading tools. For higher replicate numbers, minimizing false positives is more important and DESeq marginally outperforms the other tools.

Shown for reading. Its publisher allows quoting it, not republishing it.

Full text

Its licence does not allow us to show the full text. Read it at PubMed Central:

References 71

1. 1.
   Differential expression analysis for sequence count data · 2010

   No notice
2. 2.
   HTSeq—a Python framework to work with high-throughput sequencing data · 2014

   No notice
3. 3.

   Auer PL, Doerge RW. 2010 Statistical design and analysis of RNA sequencing data. Genetics 185: 405–416.

   Not in our library10.1534/genetics.110.114983

4. 4.

   Becker PB, Horz W. 2002 ATP-dependent nucleosome remodeling. Annu Rev Biochem 71: 247–273.

   Not in our library10.1146/annurev.biochem.71.110601.135400

5. 5.

   Bellera CA, Julien M, Hanley JA. 2010 Normal approximations to the distributions of the Wilcoxon statistics: accurate to what N? Graphical insights. J Stat Educ 18.

   Not in our library

6. 6.

   Benjamini Y, Hochberg Y. 1995 Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc B (Methodol) 57: 289–300.

   Not in our library

7. 7.
   Evaluating Gene Expression in C57BL/6J and DBA/2J Mouse Striatum Using RNA-Seq and Microarrays · 2011

   No notice
8. 8.
   Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments · 2010

   No notice
9. 9.

   Burden C, Qureshi S, Wilson SR. 2014 Error estimates for the analysis of differential expression from RNA-seq count data. PeerJ 2: e576.

   Not in our library10.7717/peerj.576

10. 10.

    Busby MA, Stewart C, Miller CA, Grzeda KR, Marth GT. 2013 Scotty: a web tool for designing RNA-Seq experiments to measure differential gene expression. Bioinformatics 29: 656–657.

    Not in our library10.1093/bioinformatics/btt015

11. 11.

    The Cancer Genome Atlas Research Network. 2008 Comprehensive genomic characterization defines human glioblastoma genes and core pathways. Nature 455: 1061–1068.

    Not in our library10.1038/nature07385

12. 12.
    Polymorphic Cis- and Trans-Regulation of Human Gene Expression · 2010

    No notice
13. 13.

    Churchill GA. 2002 Fundamentals of experimental design for cDNA microarrays. Nat Genet 32: 490–495.

    Not in our library10.1038/ng1031

14. 14.

    Colbourn CJ, Dinitz JH. 2007 Handbook of combinatorial designs. Chapman and Hall/Taylor and Francis, Boca Raton, FL.

    Not in our library

15. 15.

    Cui X, Hwang JT, Qiu J, Blades NJ, Churchill GA. 2005 Improved statistical tests for differential gene expression by shrinking variance components estimates. Biostatistics 6: 59–75.

    Not in our library10.1093/biostatistics/kxh018

16. 16.

    De Hertogh B, De Meulder B, Berger F, Pierre M, Bareke E, Gaigneaux A, Depiereux E. 2010 A benchmark for statistical microarray data analysis that preserves actual biological and technical variance. BMC Bioinformatics 11: 17.

    Not in our library10.1186/1471-2105-11-17

17. 17.

    Dillies MA, Rau A, Aubert J, Hennequet-Antier C, Jeanmougin M, Servant N, Keime C, Marot G, Castel D, Estelle J, 2013 A comprehensive evaluation of normalization methods for Illumina high-throughput RNA sequencing data analysis. Brief Bioinform 14: 671–683.

    Not in our library10.1093/bib/bbs046

18. 18.

    Efron B, Tibshirani R. 1993a An introduction to the bootstrap, pp. 203–219. Chapman and Hall, New York.

    Not in our library

19. 19.

    Efron B, Tibshirani R. 1993b An introduction to the bootstrap, pp. 220–224. Chapman and Hall, New York.

    Not in our library

20. 20.

    Flicek P, Amode MR, Barrell D, Beal K, Brent S, Chen Y, Clapham P, Coates G, Fairley S, Fitzgerald S, 2011 Ensembl 2011. Nucleic Acids Res 39: D800–D806.

    Not in our library10.1093/nar/gkq1064

21. 21.

    Frazee AC, Langmead B, Leek JT. 2011 ReCount: a multi-experiment resource of analysis-ready RNA-seq gene count datasets. BMC Bioinformatics 12: 449.

    Not in our library10.1186/1471-2105-12-449

22. 22.

    Frazee AC, Sabunciyan S, Hansen KD, Irizarry RA, Leek JT. 2014 Differential expression analysis of RNA-seq data at single-base resolution. Biostatistics 15: 413–426.

    Not in our library10.1093/biostatistics/kxt053

23. 23.

    Gierliński M, Blaxter M, Cole C, Gharbi K, Owen-Hughes T, Schofield P, Schurch NJ, Sherstnev A, Singh V, Wrobel N, 2015 Statistical models for RNA-seq data derived from a two-condition 48-replicate experiment. Bioinformatics 31: 3625–3630.

    Not in our library10.1093/bioinformatics/btv425

24. 24.

    Gkikopoulos T, Schofield P, Singh V, Pinskaya M, Mellor J, Smolle M, Workman JL, Barton GJ, Owen-Hughes T. 2011 A role for Snf2-related nucleosome-spacing enzymes in genome-wide nucleosome organization. Science 333: 1758–1760.

    Not in our library10.1126/science.1206097

25. 25.

    Guo Y, Li CI, Ye F, Shyr Y. 2013 Evaluation of read count based RNAseq analysis methods. BMC Genomics 14: S2.

    Not in our library10.1186/1471-2164-14-s8-s2

26. 26.

    Hansen KD, Wu Z, Irizarry RA, Leek JT. 2011 Sequencing technology does not eliminate biological variability. Nat Biotechnol 29: 572–573.

    Not in our library10.1038/nbt.1910

27. 27.
    baySeq: Empirical Bayesian methods for identifying differential expression in sequence count data · 2010

    No notice
28. 28.

    Hirschhorn JN, Brown SA, Clark CD, Winston F. 1992 Evidence that SNF2/SWI2 and SNF5 activate transcription in yeast by altering chromatin structure. Genes Dev 6: 2288–2298.

    Not in our library10.1101/gad.6.12a.2288

29. 29.

    Holstege FC, Jennings EG, Wyrick JJ, Lee TI, Hengartner CJ, Green MR, Golub TR, Lander ES, Young RA. 1998 Dissecting the regulatory circuitry of a eukaryotic genome. Cell 95: 717–728.

    Not in our library10.1016/s0092-8674(00)81641-4

30. 30.

    The International HapMap Consortium. 2005 A haplotype map of the human genome. Nature 437: 1299–1320.

    Not in our library10.1038/nature04226

31. 31.

    Jiang L, Schlesinger F, Davis CA, Zhang Y, Li R, Salit M, Gingeras TR, Oliver B. 2011 Synthetic spike-in standards for RNA-seq experiments. Genome Res 21: 1543–1551.

    Not in our library10.1101/gr.121095.111

32. 32.

    Kaisers W, Schwender H, Schaal H. 2014 Hierarchical clustering of DNA k-mer counts in RNA-seq fastq files reveals batch effects. arXiv 1405.0114.

    Not in our library10.3390/ijms19113687

33. 33.

    Kohrer K, Domdey H. 1991 Preparation of high molecular weight RNA. Methods Enzymol 194: 398–405.

    Not in our library10.1016/0076-6879(91)94030-g

34. 34.

    Kvam VM, Liu P, Si Y. 2012 A comparison of statistical methods for detecting differentially expressed genes from RNA-seq data. Am J Bot 99: 248–256.

    Not in our library10.3732/ajb.1100340

35. 35.
    voom: precision weights unlock linear model analysis tools for RNA-seq read counts · 2014

    No notice
36. 36.

    Leng N, Dawson JA, Thomson JA, Ruotti V, Rissman AI, Smits BM, Haag JD, Gould MN, Stewart RM, Kendziorski C. 2013 EBSeq: an empirical Bayes hierarchical model for inference in RNA-seq experiments. Bioinformatics 29: 1035–1043.

    Not in our library10.1093/bioinformatics/btt087

37. 37.

    Li J, Tibshirani R. 2013 Finding consistent patterns: a nonparametric approach for identifying differential expression in RNA-Seq data. Stat Methods Med Res 22: 519–536.

    Not in our library10.1177/0962280211428386

38. 38.

    Li J, Witten DM, Johnstone IM, Tibshirani R. 2012 Normalization, testing, and false discovery rate estimation for RNA-sequencing data. Biostatistics 13: 523–538.

    Not in our library10.1093/biostatistics/kxr031

39. 39.

    Liu Y, Zhou J, White KP. 2014 RNA-seq differential expression studies: more sequence or more replication? Bioinformatics 30: 301–304.

    Not in our library10.1093/bioinformatics/btt688

40. 40.
    Moderated estimation of fold change and dispersion for RNA-seq data with DESeq2 · 2014

    No notice

Cited by papers we hold 15

• Applications of multi‐omics analysis in human diseases · 2023
  No notice
• Secreted Amyloid Precursor Protein Alpha, a Neuroprotective Protein in the Brain Has Widespread Effects on the Transcriptome and Proteome of Human Inducible Pluripotent Stem Cell-Derived Glutamatergic Neurons Related to Memory Mechanisms · 2022
  Not checked
• Divergence of X-linked trans regulatory proteins and the misexpression of gene targets in sterile Drosophila pseudoobscura hybrids · 2022
  Not checked
• A simple guide to de novo transcriptome assembly and annotation · 2022
  No notice
• Exaggerated false positives by popular differential expression methods when analyzing human population samples · 2022
  No notice
• Population Genetic Structure and Gene Expression Plasticity of the Deep-Sea Vent and Seep Squat Lobster Shinkaia crosnieri · 2020
  Not checked
• Limits in the detection of m6A changes using MeRIP/m6A-seq · 2020
  No notice
• Full-length transcript characterization of SF3B1 mutation in chronic lymphocytic leukemia reveals downregulation of retained introns · 2020
  No notice
• West Nile virus infection and interferon alpha treatment alter the spectrum and the levels of coding and noncoding host RNAs secreted in extracellular vesicles · 2019
  Not checked
• Large-scale sequence comparisons with sourmash · 2019
  No notice
• PCR and Omics Based Techniques to Study the Diversity, Ecology and Biology of Anaerobic Fungi: Insights, Challenges and Opportunities · 2017
  Not checked
• Multi-omics approaches to disease · 2017
  No notice
• Resolving host–pathogen interactions by dual RNA-seq · 2017
  No notice
• RNA-Seq differential expression analysis: An extended review and a software tool · 2017
  No notice
• From reads to genes to pathways: differential expression analysis of RNA-Seq experiments using Rsubread and the edgeR quasi-likelihood pipeline · 2016
  No notice

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.