Skip to content
naulon
Get started

No retraction, correction or concern found when last checked on Oct 30, 2026.

Measuring the reproducibility and quality of Hi-C data

Galip Gürkan Yardımcı, Hakan Ozadam, Michael E. G. Sauria and 16 more

Genome Biology · 2019 · CC BY

doi:10.1186/s13059-019-1658-7

Abstract

Background. Hi-C is currently the most widely used assay to investigate the 3D organization of the genome and to study its role in gene regulation, DNA replication, and disease. However, Hi-C experiments are costly to perform and involve multiple complex experimental steps; thus, accurate methods for measuring the quality and reproducibility of Hi-C data are essential to determine whether the output should be used further in a study.

Results. Using real and simulated data, we profile the performance of several recently proposed methods for assessing reproducibility of population Hi-C data, including HiCRep, GenomeDISCO, HiC-Spector, and QuASAR-Rep. By explicitly controlling noise and sparsity through simulations, we demonstrate the deficiencies of performing simple correlation analysis on pairs of matrices, and we show that methods developed specifically for Hi-C data produce better measures of reproducibility. We also show how to use established measures, such as the ratio of intra- to interchromosomal interactions, and novel ones, such as QuASAR-QC, to identify low-quality experiments.

Conclusions. In this work, we assess reproducibility and quality measures by varying sequencing depth, resolution and noise levels in Hi-C data from 13 cell lines, with two biological replicates each, as well as 176 simulated matrices. Through this extensive validation and benchmarking of Hi-C data, we describe best practices for reproducibility and quality assessment of Hi-C experiments. We make all software publicly available at http://github.com/kundajelab/3DChromatin_ReplicateQC to facilitate adoption in the community.

Electronic supplementary material. The online version of this article (10.1186/s13059-019-1658-7) contains supplementary material, which is available to authorized users.

References 48

  1. 1.

    Lieberman-Aiden E, van Berkum NL, Williams L, Imakaev M, Ragoczy T, Telling A, et al. Comprehensive mapping of long-range interactions reveals folding principles of the human genome. Science. 2009;326:289-293.

    Not in our library10.1126/science.1181369

  2. 2.

    Dixon JR, Jung I, Selvaraj S, Shen Y, Antosiewicz-Bourget JE, Lee AY, et al. Chromatin architecture reorganization during stem cell differentiation. Nature. 2015;518:331-336.

    Not in our library10.1038/nature14222

  3. 3.

    Krijger PHL, Di Stefano B, De Wit E, Limone F, Van Oevelen C, De Laat W, et al. Cell-of-origin-specific 3D genome structure acquired during somatic cell reprogramming. Cell Stem Cell. 2016;18:597-610.

    Not in our library10.1016/j.stem.2016.01.007

  4. 4.

    Ma W, Ay F, Lee C, Gulsoy G, Deng X, Cook S, et al. Fine-scale chromatin interaction maps reveal the cis-regulatory landscape of lincRNA genes in human cells. Nat Methods. 2015;12:71-78.

    Not in our library10.1038/nmeth.3205

  5. 5.

    Giorgetti L, Lajoie BR, Carter AC, Attia M, Zhan Y, Xu J, et al. Structural organization of the inactive X chromosome in the mouse. Nature. 2016;535:575-579.

    Not in our library10.1038/nature18589

  6. 6.

    Darrow EM, Huntley MH, Dudchenko O, Stamenova EK, Durand NC, Sun Z, et al. Deletion of DXZ4 on the human inactive X chromosome alters higher-order genome architecture. Proc Natl Acad Sci U S A. 2016;113:E4504–12. http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=PMC4978254.

    Not in our library10.1073/pnas.1609643113

  7. 7.

    Naumova N, Imakaev M, Fudenberg G, Zhan Y, Lajoie BR, Mirny LA, et al. Organization of the mitotic chromosome. Science. 2013;342:948-953.

    Not in our library10.1126/science.1236083

  8. 8.

    Dixon JR, Selvaraj S, Yue F, Kim A, Li Y, Shen Y, et al. Topological domains in mammalian genomes identified by analysis of chromatin interactions. Nature. 2012;485:376-380.

    Not in our library10.1038/nature11082

  9. 9.

    Nora EP, Lajoie BR, Schulz EG, Giorgetti L, Okamoto I, Servant N, et al. Spatial partitioning of the regulatory landscape of the X-inactivation centre. Nature. 2012;485:381-385.

    Not in our library10.1038/nature11049

  10. 10.

    Sexton T, Yaffe E, Kenigsberg E, Bantignies F, Leblanc B, Hoichman M, et al. Three-dimensional folding and functional organization principles of the Drosophila genome. Cell. 2012;148:458-472.

    Not in our library10.1016/j.cell.2012.01.010

  11. 11.

    Rao SSP, Huntley MH, Durand N, Neva C, Stamenova EK, Bochkov ID, et al. A 3D map of the human genome at kilobase resolution reveals principles of chromatin looping. Cell. 2014;59:1665-1680.

    Not in our library10.1016/j.cell.2014.11.021

  12. 12.

    Jin F, Li Y, Dixon JR, Selvaraj S, Ye Z, Lee AY, et al. A high-resolution map of the three-dimensional chromatin interactome in human cells. Nature. 2013;503:290-294.

    Not in our library10.1038/nature12644

  13. 13.

    Schmitt AD, Hu M, Ren B. Genome-wide mapping and analysis of chromosome architecture. Nat Rev. 2016;17:743–55.

    Not in our library10.1038/nrm.2016.104

  14. 14.

    Barski A, Cuddapah S, Cui K, Roh TY, Schones DE, Wang Z, et al. High-resolution profiling of histone methylations in the human genome. Cell. 2007;129:823-837.

    Not in our library10.1016/j.cell.2007.05.009

  15. 15.

    Boyle AP, Davis S, Shulha HP, Meltzer P, Margulies EH, Weng Z, et al. High-resolution mapping and characterization of open chromatin across the genome. Cell. 2008;132:311-322.

    Not in our library10.1016/j.cell.2007.12.014

  16. 16.

    Landt SG, Marinov GK, Kundaje A, Kheradpour P, Pauli F, Batzoglou S, et al. ChIP-seq guidelines and practices of the ENCODE and modENCODE consortia. Genome Res. 2012;22:1813-1831.

    Not in our library10.1101/gr.136184.111

  17. 17.

    Li Q, Brown JB, Huang H, Bickel PJ. Measuring reproducibility of high-throughput experiments. Ann Appl Stat. 2011;5:1752-1779.

    Not in our library10.1214/11-aoas466

  18. 18.

    Qin Q, Mei S, Wu Q, Sun H, Li L, Taing L, et al. ChiLin: a comprehensive ChIP-seq and DNase-seq quality control and analysis pipeline. BMC Bioinformatics. 2016;17:404.

    Not in our library10.1186/s12859-016-1274-4

  19. 19.

    Ji H, Jiang H, Ma W, Johnson DS, Myers RM, Wong WH. An integrated system CisGenome for analyzing ChIP-chip and ChIP-seq data. Nat Biotechnol. 2008;26:1293.

    Not in our library10.1038/nbt.1505

  20. 20.

    Frank CL, Liu F, Wijayatunge R, Song L, Biegler MT, Yang MG, et al. Regulation of chromatin accessibility and Zic binding at enhancers in the developing cerebellum. Nat Neurosci. 2015;18:647-656.

    Not in our library10.1038/nn.3995

  21. 21.

    Bardet AF, He Q, Zeitlinger J, Stark A. A computational pipeline for comparative ChIP-seq analyses. Nat Protoc. 2012;7:45-61.

    Not in our library10.1038/nprot.2011.420

  22. 22.

    Ho JWK, Bishop E, Karchenko PV, Nègre N, White KP, Park PJ. ChIP-chip versus ChIP-seq: lessons for experimental design and data analysis. BMC Genomics. 2011;12:134.

    Not in our library10.1186/1471-2164-12-134

  23. 23.
    Analysis methods for studying the 3D architecture of the genome · 2015
    No notice
  24. 24.

    Lajoie BR, Dekker J, Kaplan N. The Hitchhiker’s guide to Hi-C analysis: practical guidelines. Methods. 2015;72:65-75.

    Not in our library10.1016/j.ymeth.2014.10.031

  25. 25.

    Tjong H, Gong K, Chen L, Alber F. Physical tethering and volume exclusion determine higher-order genome organization in budding yeast. Genome Res. 2012;22:1295-1305.

    Not in our library10.1101/gr.129437.111

  26. 26.

    Hu M, Deng K, Selvaraj S, Qin Z, Ren B, Liu JS. HiCNorm: removing biases in Hi-C data via Poisson regression. Bioinformatics. 2012;28:3131-3133.

    Not in our library10.1093/bioinformatics/bts570

  27. 27.

    Gorkin DU, Leung D, Ren B. The 3D genome in transcriptional regulation and pluripotency. Cell Stem Cell. 2014;14(6):771–5.

    Not in our library10.1016/j.stem.2014.05.017

  28. 28.

    van Berkum NL, Lieberman-Aiden E, Williams L, Imakaev M, Gnirke A, Mirny LA, et al. Hi-C: a method to study the three-dimensional architecture of genomes. J Vis Exp. 2010;6:1869.

    Not in our library10.3791/1869

  29. 29.
    A benchmark for RNA-seq quantification pipelines · 2016
    Corrected
  30. 30.

    Imakaev M, Fudenberg G, McCord RP, Naumova N, Goloborodko A, Lajoie BR, et al. Iterative correction of Hi-C data reveals hallmarks of chromosome organization. Nat Methods. 2012;9:999-1003.

    Not in our library10.1038/nmeth.2148

  31. 31.
    Automatic analysis and 3D-modelling of Hi-C data using TADbit reveals structural features of the fly chromatin colors · 2017
    No notice
  32. 32.
    Comparison of Hi-C results using in-solution versus in-nucleus ligation · 2015
    No notice
  33. 33.

    Yan KK, Yardlmcl GG, Yan C, Noble WS, Gerstein M. HiC-spector: a matrix library for spectral and reproducibility analysis of Hi-C contact maps. Bioinformatics. 2017;33(14):2199–201.

    Not in our library10.1093/bioinformatics/btx152

  34. 34.

    Yang T, Zhang F, Yardimci GG, Song F, Hardison RC, Noble WS, et al. HiCRep: assessing the reproducibility of Hi-C data using a stratum-adjusted correlation coefficient. Genome Res. 2017;gr.220640.117. http://genome.cshlp.org/lookup/doi/10.1101/gr.220640.117.

    Not in our library10.1101/gr.220640.117

  35. 35.

    Ursu O, Boley N, Taranova M, Wang YXR, Yardimci GG, Noble WS, et al. GenomeDISCO: a concordance score for chromosome conformation capture experiments using random walks on contact map graphs. bioRxiv. 2017:181842 Available from: https://www.biorxiv.org/content/early/2017/08/29/181842. [cited 2018 Jan 30]. Cold Spring Harbor Laboratory.

    Not in our library10.1093/bioinformatics/bty164

  36. 36.

    Sauria ME, Taylor J. QuASAR: Quality Assessment of Spatial Arrangement Reproducibility in Hi-C data. bioRxiv. 2017:204438 Available from: https://www.biorxiv.org/content/early/2017/11/14/204438.

    Not in our library

  37. 37.

    Ramírez F, Lingg T, Toscano S, Lam KC, Georgiev P, Chung HR, et al. High-affinity sites form an interaction network to facilitate spreading of the MSL complex across the X chromosome in Drosophila. Mol Cell. 2015;60:146-162.

    Not in our library10.1016/j.molcel.2015.08.024

  38. 38.

    Ay F, Bailey TL, Noble WS. Statistical confidence estimation for Hi-C data reveals regulatory chromatin contacts. Genome Res. 2014;24:999-1011.

    Not in our library10.1101/gr.160374.113

  39. 39.

    Carty M, Zamparo L, Sahin M, González A, Pelossof R, Elemento O, et al. An integrated model for detecting significant chromatin interactions from high-resolution Hi-C data. Nat Commun. 2017;8.

    Not in our library10.1038/ncomms15454

  40. 40.

    Crane E, Bian Q, McCord RP, Lajoie BR, Wheeler BS, Ralston EJ, et al. Condensin-driven remodelling of X chromosome topology during dosage compensation. Nature. 2015;523:240-244.

    Not in our library10.1038/nature14450

All 48 references

Cited by papers we hold 3

Reads

Read by people on this page

None yet.

Checked or read by AI through naulon

None yet.

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.

Are you an author?

Is this your paper?

Claim it to add your note and be paid when agents cite it. Free.