Deep learning-based transformation of H&E stained tissues into special stains

Abstract

Pathology is practiced by visual inspection of histochemically stained tissue slides. While the hematoxylin and eosin (H&E) stain is most commonly used, special stains can provide additional contrast to different tissue components. Here, we demonstrate the utility of supervised learning-based computational stain transformation from H&E to special stains (Masson’s Trichrome, periodic acid-Schiff and Jones silver stain) using kidney needle core biopsy tissue sections. Based on the evaluation by three renal pathologists, followed by adjudication by a fourth pathologist, we show that the generation of virtual special stains from existing H&E images improves the diagnosis of several non-neoplastic kidney diseases, sampled from 58 unique subjects (P = 0.0095). A second study found that the quality of the computationally generated special stains was statistically equivalent to those which were histochemically stained. This stain-to-stain transformation framework can improve preliminary diagnoses when additional special stains are needed, also providing significant savings in time and cost.

Full text

Its licence does not allow us to show the full text. Read it at PubMed Central:

References 34

1. 1.

   Global Transformational Health Research Team at Frost & Sullivan. Global Tissue Diagnostics Market, Forecast to 2022 (Frost and Sullivan 2018).

   Not in our library

2. 2.
   Histological Stains: A Literature Review and Case Study · 2015

   No notice
3. 3.

   Walker PD, Cavallo T, Bonsib SM, Ad Hoc Committee on Renal Biopsy Guidelines of the Renal Pathology Society. Practice guidelines for the renal biopsy. Mod. Pathol. 2004;17:1555-1563.

   Not in our library10.1038/modpathol.3800239

4. 4.

   Tao YK, et al. Assessment of breast pathologies using nonlinear microscopy. Proc. Natl Acad. Sci. USA. 2014;111:15304-15309.

   Not in our library10.1073/pnas.1416955111

5. 5.

   Fereidouni F, et al. Microscopy with ultraviolet surface excitation for rapid slide-free histology. Nat. Biomed. Eng. 2017;1:957-966.

   Not in our library10.1038/s41551-017-0165-y

6. 6.

   Glaser AK, et al. Light-sheet microscopy for slide-free non-destructive pathology of large clinical specimens. Nat. Biomed. Eng. 2017;1:1-10.

   Not in our library10.1038/s41551-017-0084

7. 7.

   Rivenson, Y. et al. Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning. Nat. Biomed. Eng.10.1038/s41551-019-0362-y (2019).

   Not in our library10.1038/s41551-019-0362-y

8. 8.

   Zhang Y, et al. Digital synthesis of histological stains using micro-structured and multiplexed virtual staining of label-free tissue. Light. Sci. Appl. 2020;9:78.

   Not in our library10.1038/s41377-020-0315-y

9. 9.

   Bayramoglu, N., Kaakinen, M., Eklund, L. & Heikkilä, J. Towards virtual H and E staining of hyperspectral lung histology images using conditional generative adversarial networks. In IEEE International Conference on Computer Vision Workshops (ICCVW) (Cucchiara R., Matsushita, Y. Sebe, N. & Soatto, S.) 64–71 (IEEE, 2017).

   Not in our library

10. 10.
    PhaseStain: the digital staining of label-free quantitative phase microscopy images using deep learning · 2019

    No notice
11. 11.

    Rana A, et al. Use of deep learning to develop and analyze computational hematoxylin and eosin staining of prostate core biopsy images for tumor diagnosis. JAMA Netw. Open. 2020;3:e205111-e205111.

    Not in our library10.1001/jamanetworkopen.2020.5111

12. 12.

    Borhani N, Bower AJ, Boppart SA, Psaltis D. Digital staining through the application of deep neural networks to multi-modal multi-photon microscopy. Biomed. Opt. Express. 2019;10:1339-1350.

    Not in our library10.1364/boe.10.001339

13. 13.

    Roy-Chowdhuri, S. et al. Collection and handling of thoracic small biopsy and cytology specimens for ancillary studies: guideline from the College of American Pathologists in collaboration with the American College of Chest Physicians, Association for Molecular Pathology, American Society of Cytopathology, American Thoracic Society, Pulmonary Pathology Society, Papanicolaou Society of Cytopathology, Society of Interventional Radiology, and Society of Thoracic Radiology. Arch. Pathol. Lab. Med. 10.5858/arpa.2020-0119-CP (2020).

    Not in our library10.5858/arpa.2020-0119-cp

14. 14.

    Levy, J. J., Jackson, C. R., Sriharan, A., Christensen, B. C. & Vaickus, L. J. Preliminary Evaluation of the Utility of Deep Generative Histopathology Image Translation at a Mid-sized NCI Cancer Center. in Proceedings of the 13th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2020) - Volume 3: BIOINFORMATICS,(eds. Maria, E. D., Fred, A. L. N. & Gamboa, H.) 302–311 (SCITEPRESS, 2020).

    Not in our library

15. 15.

    Lahiani, A., Klaman, I., Navab, N., Albarqouni, S. & Klaiman, E. Seamless virtual whole slide image synthesis and validation using perceptual embedding consistency. IEEE J. Biomed. Health Inform. 10.1109/JBHI.2020.2975151 (2020).

    Not in our library10.1109/jbhi.2020.2975151

16. 16.

    Gadermayr, M., Appel, V., Klinkhammer, B. M., Boor, P. & Merhof, D. In Medical Image Computing and Computer Assisted Intervention – MICCAI 2018 (eds Frangi, A. F., Schnabel, J. A., Davatzikos, C., Alberola-López, C. & Fichtinger, G.) 165–173 (Springer, 2018).

    Not in our library

17. 17.

    Kapil, A. et al. DASGAN–Joint domain adaptation and segmentation for the analysis of epithelial regions in histopathology PD-L1 images. Preprint at arXiv:1906.11118 [cs, eess] (2019).

    Not in our library

18. 18.

    Zhu, J.-Y., Park, T., Isola, P. & Efros, A. A. Unpaired image-to-image translation using cycle-consistent adversarial networks. In 2017 IEEE International Conference on Computer Vision (ICCV) (eds Cucchiara, R., Matsushita, Y., Sebe, N. & Soatto, S.) 2242–2251 (IEEE, 2017).

    Not in our library

19. 19.

    Cohen, J. P., Luck, M. & Honari, S. Distribution Matching Losses Can Hallucinate Features in Medical Image Translation. in Medical Image Computing and Computer Assisted Intervention – MICCAI 2018 (eds. Frangi, A. F., Schnabel, J. A., Davatzikos, C., Alberola-López, C. & Fichtinger, G.) 529–536 (Springer International Publishing, 2018). 10.1007/978-3-030-00928-1_60.

    Not in our library

20. 20.

    Fujitani, M. et al. Re-staining pathology images by FCNN. In 16th International Conference on Machine Vision Applications (MVA) (eds Maki, A. & Favaro, P.) 1–6 (IEEE, 2019).

    Not in our library

21. 21.

    Mercan, C. et al. Virtual Staining for Mitosis Detection in Breast Histopathology. in 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI) (IEEE, 2020). 10.1109/isbi45749.2020.9098409.

    Not in our library

22. 22.

    Bauer TW, et al. Validation of whole slide imaging for primary diagnosis in surgical pathology. Arch. Pathol. Lab. Med. 2013;137:518-524.

    Not in our library10.5858/arpa.2011-0678-oa

23. 23.

    Shaban, M. T., Baur, C., Navab, N. & Albarqouni, S. Staingan: stain style transfer for digital histological images. In 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019) (eds Carbayo, M. L., Ángel, M. & Ballester, G.) 953–956 (IEEE, 2019).

    Not in our library

24. 24.
    Data augmentation using generative adversarial networks (CycleGAN) to improve generalizability in CT segmentation tasks · 2019

    No notice
25. 25.

    Erlandson, R. A. Role of electron microscopy in modern diagnostic surgical pathology. Mod. Surg. Pathol. 10.1016/B978-1-4160-3966-2.00005-9 (2009).

    Not in our library

26. 26.

    Kingma, D. P. & Ba, J. Adam: a method for stochastic optimization. In 3rd International Conference on Learning Representations, ICLR (eds. Bengio, Y. & LeCun, Y.) (2015).

    Not in our library

27. 27.

    Convert RGB color values to YCbCr color space - MATLAB rgb2ycbcr. https://www.mathworks.com/help/images/ref/rgb2ycbcr.html (2020).

    Not in our library

28. 28.

    Wang H, et al. Deep learning enables cross-modality super-resolution in fluorescence microscopy. Nat. Methods. 2019;16:103-110.

    Not in our library10.1038/s41592-018-0239-0

29. 29.

    Register Multimodal MRI Images - MATLAB & Simulink Example. https://www.mathworks.com/help/images/registering-multimodal-mri-images.html (2020).

    Not in our library

30. 30.

    Culley S, et al. Quantitative mapping and minimization of super-resolution optical imaging artifacts. Nat. Methods. 2018;15:263-266.

    Not in our library10.1038/nmeth.4605

31. 31.

    Zoomify — Zoomable web images! http://zoomify.com/ (2020).

    Not in our library

32. 32.

    GIGAmacro: Exploring Small Things in a Big Way. https://viewer.gigamacro.com/ (2020).

    Not in our library

33. 33.

    Pantanowitz L, et al. Validating whole slide imaging for diagnostic purposes in pathology: guideline from the College of American Pathologists Pathology and Laboratory Quality Center. Arch. Pathol. Lab. Med. 2013;137:1710-1722.

    Not in our library10.5858/arpa.2013-0093-cp

34. 34.

    Jodogne S. The orthanc ecosystem for medical imaging. J. Digit. Imaging. 2018;31:341-352.

    Not in our library10.1007/s10278-018-0082-y

Cited by papers we hold 3

• High-plex immunofluorescence imaging and traditional histology of the same tissue section for discovering image-based biomarkers · 2023
  No notice
• Deep learning-enabled virtual histological staining of biological samples · 2023
  No notice
• GANscan: continuous scanning microscopy using deep learning deblurring · 2022
  Not checked

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.