Deep Learning for Whole Slide Image Analysis: An Overview

Abstract

The widespread adoption of whole slide imaging has increased the demand for effective and efficient gigapixel image analysis. Deep learning is at the forefront of computer vision, showcasing significant improvements over previous methodologies on visual understanding. However, whole slide images have billions of pixels and suffer from high morphological heterogeneity as well as from different types of artifacts. Collectively, these impede the conventional use of deep learning. For the clinical translation of deep learning solutions to become a reality, these challenges need to be addressed. In this paper, we review work on the interdisciplinary attempt of training deep neural networks using whole slide images, and highlight the different ideas underlying these methodologies.

Full text

Its licence does not allow us to show the full text. Read it at PubMed Central:

References 53

1. 1.

   Snead RJD Tsang Y Meskiri A Kimani KP Crossman R Rajpoot MN. Validation of digital pathology imaging for primary histopathological diagnosis. Histopathology. (2015 ) 68:1063–72.

   Not in our library10.1111/his.12879

2. 2.

   Pantanowitz L Sinard HJ Henricks HW Fatheree AL Carter BA Contis L. Validating whole slide imaging for diagnostic purposes in pathology: guideline from the college of American pathologists pathology and laboratory quality center. Arch Pathol Lab Med. (2013 ) 137:1710–22.

   Not in our library10.5858/arpa.2013-0093-cp

3. 3.

   Caccomo S FDA Allows Marketing of First Whole Slide Imaging System for Digital Pathology (2017 ). Available online at: https://www.fda.gov/NewsEvents/Newsroom/PressAnnouncements/ucm552742.htm

   Not in our library

4. 4.

   Hamilton WP Bankhead P Wang Y Hutchinson R Kieran D McArt GD. Digital pathology and image analysis in tissue biomarker research. Methods. (2014 ) 70:59–73.

   Not in our library10.1016/j.ymeth.2014.06.015

5. 5.

   Harder N Schönmeyer R Nekolla K Meier A Brieu N Vanegas C. Automatic discovery of image-based signatures for ipilimumab response prediction in malignant melanoma. Sci Rep. (2019 ) 9:7449.

   Not in our library10.1038/s41598-019-43525-8

6. 6.

   Janowczyk A Madabhushi A. Deep learning for digital pathology image analysis: a comprehensive tutorial with selected use cases. J Pathol Informat. (2016 ) 7:29.

   Not in our library10.4103/2153-3539.186902

7. 7.

   Brieu N Gavriel GC Nearchou PI Harrison JD Schmidt G Caie DP. Automated tumour budding quantification by machine learning augments TNM staging in muscle-invasive bladder cancer prognosis. Sci Rep. (2019 ) 9:5174.

   Not in our library10.1038/s41598-019-41595-2

8. 8.

   Caie DP Zhou Y Turnbull KA Oniscu A Harrison JD. Novel histopathologic feature identified through image analysis augments stage II colorectal cancer clinical reporting. Oncotarget. (2016 ) 7:44381–94.

   Not in our library10.18632/oncotarget.10053

9. 9.

   Beck HA Sangoi RA Leung S Marinelli JR Nielsen OT van de Vijver JM. Systematic analysis of breast cancer morphology uncovers stromal features associated with survival. Sci Transl Med. (2011 ) 3:108ra113.

   Not in our library10.1126/scitranslmed.3002564

10. 10.

    Nearchou PI Lillard K Gavriel GC Ueno H Harrison JD Caie DP. Automated analysis of lymphocytic infiltration, tumor budding, and their spatial relationship improves prognostic accuracy in colorectal cancer. Cancer Immunol Res. (2019 ) 7:609–20.

    Not in our library10.1158/2326-6066.cir-18-0377

11. 11.

    Sari TC Gunduz-Demir C. Unsupervised feature extraction via deep learning for histopathological classification of colon tissue images. IEEE Trans Med Imaging. (2019 ) 38:1139–49.

    Not in our library10.1109/tmi.2018.2879369

12. 12.

    Dimitriou N Arandjelović O Harrison JD Caie DP. A principled machine learning framework improves accuracy of stage II colorectal cancer prognosis. npj Digit Med. (2018 ) 1:52.

    Not in our library10.1038/s41746-018-0057-x

13. 13.

    Qaiser T Mukherjee A Reddy PB C Munugoti DS Tallam V Pitkäaho T. HER2 challenge contest: a detailed assessment of automated HER2 scoring algorithms in whole slide images of breast cancer tissues. Histopathology. (2017 ) 72:227–38.

    Not in our library10.1111/his.13333

14. 14.

    Wang D Khosla A Gargeya R Irshad H Beck HA Deep learning for identifying metastatic breast cancer. arXiv. (2016 ).

    Not in our library

15. 15.

    Liu Y Gadepalli K Norouzi M Dahl EG Kohlberger T Boyko A Detecting cancer metastases on gigapixel pathology images. arXiv. (2017 ).

    Not in our library

16. 16.

    Yue X Dimitriou N Caie DP Harrison JD Arandjelovic O Colorectal cancer outcome prediction from H&E whole slide images using machine learning and automatically inferred phenotype profiles. In: Conference on Bioinformatics and Computational Biology. Vol. 60, Honolulu, HI (2019 ). p. 139–49.

    Not in our library

17. 17.

    BenTaieb A Ghassan Hamarneh G Predicting cancer with a recurrent visual attention model for histopathology images. In: Frangi AF, Schnabel JA, Davatzikos C, Alberola-López C, Fichtinger G, editiors. Medical Image Computing and Computer-Assisted Intervention. Vol. 11071 Springer International Publishing (2018 ). p. 129–37. Available online at: https://link.springer.com/book/10.1007/978-3-030-00934-2

    Not in our library

18. 18.

    Tellez D Litjens G van der Laak J Ciompi F. Neural image compression for gigapixel histopathology image analysis. arXiv. (2018 ).

    Not in our library10.1109/tpami.2019.2936841

19. 19.

    Wong FP Wei W Smithy WJ Acs B Toki IM Blenman RMK. Multiplex quantitative analysis of tumor-infiltrating lymphocytes and immunotherapy outcome in metastatic melanoma. Clin Cancer Res. (2019 ) 25:2442–9.

    Not in our library10.1158/1078-0432.ccr-18-2652

20. 20.

    Schlag I Arandjelović O Ancient Roman coin recognition in the wild using deep learning based recognition of artistically depicted face profiles. In: Proceedings of the International Conference on Computer Vision. Venice (2017 ). p. 2898–906.

    Not in our library

21. 21.

    Cooper J Arandjelović O Visually understanding rather than merely matching ancient coin images. In: Proceedings of the INNS Conference on Big Data and Deep Learning. Sestri Levante (2019 ).

    Not in our library

22. 22.
    Image-based consensus molecular subtype (imCMS) classification of colorectal cancer using deep learning · 2020

    No notice
23. 23.

    Bejnordi EB Veta M van Diest JP van Ginneken B Karssemeijer N Litjens G Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. J Am Med Assoc. (2017 ) 318:2199–210.

    Not in our library10.1001/jama.2017.14580

24. 24.

    Bandi P Geessink O Manson Q van Dijk M Balkenhol M Hermsen M. From detection of individual metastases to classification of lymph node status at the patient level: the CAMELYON17 challenge. IEEE Trans Med Imaging. (2019 ) 38:550–560.

    Not in our library10.1109/tmi.2018.2867350

25. 25.

    Aresta G Araújo T Kwok S Chennamsetty SS Safwan M Alex V. BACH: grand challenge on breast cancer histology images. arXiv. (2018 ).

    Not in our library10.1016/j.media.2019.05.010

26. 26.

    Veta M Heng JY Stathonikos N Bejnordi EB Beca F Wollmann T. Predicting breast tumor proliferation from whole-slide images: the TUPAC16 challenge. Med Image Anal. (2019 ) 54:111–21.

    Not in our library10.1016/j.media.2019.02.012

27. 27.

    Campanella G Silva WKV Fuchs JT Terabyte-scale deep multiple instance learning for classification and localization in pathology. arXiv. (2018 ).

    Not in our library

28. 28.

    Bennett W Smith K Jarosz Q Nolan T Bosch W. Reengineering workflow for curation of DICOM datasets. J Digit Imaging. (2018 ) 31:783–91.

    Not in our library10.1007/s10278-018-0097-4

29. 29.

    Kahn EC Carrino AJ Flynn JM Peck JD Horii CS. DICOM and radiology: past, present, and future. J Am Coll Radiol. (2007 ) 4:652–7.

    Not in our library10.1016/j.jacr.2007.06.004

30. 30.

    Lennerz KJ Herrmann DM Clunie AD Fedorov A Doyle WS Pieper S. Implementing the DICOM standard for digital pathology. J Pathol Informat. (2018 ) 9:37.

    Not in our library10.4103/jpi.jpi_42_18

31. 31.

    Clunie AD. Dual-personality DICOM-TIFF for whole slide images: a migration technique for legacy software. J Pathol Informat. (2019 ) 10:12.

    Not in our library10.4103/jpi.jpi_93_18

32. 32.

    Caie DP Schuur K Oniscu A Mullen P Reynolds AP Harrison JD. Human tissue in systems medicine. FEBS J. (2013 ) 280:5949–56.

    Not in our library10.1111/febs.12550

33. 33.

    Magee RD Treanor D Crellin D Shires M Smith EK Mohee K Colour normalisation in digital histopathology images. In: Proc. Opt. Tissue Image Anal. Microsc. Histopathol. Endosc. London (2009 ). p. 100–11.

    Not in our library

34. 34.

    Brieu N Caie DP Gavriel GC Schmidt G Harrison JD Context-based interpolation of coarse deep learning prediction maps for the segmentation of fine structures in immunofluorescence images. In: Medical Imaging 2018: Digital Pathology. Houston, TX (2018 ).

    Not in our library

35. 35.

    Chang YH Jung KC Woo IJ Lee S Cho J Kim WS. Artificial intelligence in pathology. J Pathol Transl Med. (2019 ) 53:1–12.

    Not in our library10.4132/jptm.2018.12.16

36. 36.

    Li Y Ping W Cancer metastasis detection with neural conditional random field. arXiv. (2018 ).

    Not in our library

37. 37.

    Kong B Wang X Li Z Song Q Zhang S Cancer metastasis detection via spatially structured deep network. In: Information Processing in Medical Imaging. Boone, NC: Springer International Publishing (2017 ). p. 236–48.

    Not in our library

38. 38.

    Zanjani GF Zinger S de With HNP Cancer detection in histopathology whole-slide images using conditional random fields on deep embedded spaces. In: Medical Imaging. Vol. 10581, Houston, TX: SPIE (2018 ).

    Not in our library

39. 39.

    Litjens G Kooi T Bejnordi EB Setio AAA Ciompi F Ghafoorian M. A survey on deep learning in medical image analysis. Med Image Anal. (2017 ) 42:60–88.

    Not in our library10.1016/j.media.2017.07.005

40. 40.

    Coudray N Ocampo SP Sakellaropoulos T Narula N Snuderl M Fenyö D. Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning. Nat Med. (2018 ) 24:1559–67.

    Not in our library10.1038/s41591-018-0177-5

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.