Deep learning-enabled virtual histological staining of biological samples

Abstract

Histological staining is the gold standard for tissue examination in clinical pathology and life-science research, which visualizes the tissue and cellular structures using chromatic dyes or fluorescence labels to aid the microscopic assessment of tissue. However, the current histological staining workflow requires tedious sample preparation steps, specialized laboratory infrastructure, and trained histotechnologists, making it expensive, time-consuming, and not accessible in resource-limited settings. Deep learning techniques created new opportunities to revolutionize staining methods by digitally generating histological stains using trained neural networks, providing rapid, cost-effective, and accurate alternatives to standard chemical staining methods. These techniques, broadly referred to as virtual staining , were extensively explored by multiple research groups and demonstrated to be successful in generating various types of histological stains from label-free microscopic images of unstained samples; similar approaches were also used for transforming images of an already stained tissue sample into another type of stain, performing virtual stain-to-stain transformations. In this Review, we provide a comprehensive overview of the recent research advances in deep learning-enabled virtual histological staining techniques. The basic concepts and the typical workflow of virtual staining are introduced, followed by a discussion of representative works and their technical innovations. We also share our perspectives on the future of this emerging field, aiming to inspire readers from diverse scientific fields to further expand the scope of deep learning-enabled virtual histological staining techniques and their applications.

Full text

Its licence does not allow us to show the full text. Read it at PubMed Central:

References 106

1. 1.

   Bancroft, J. D. & Gamble, M. Theory and Practice of Histological Techniques. 6th edn. (Churchill Livingstone, Edinburgh, 2008).

   Not in our library

2. 2.

   Musumeci G. Past, present and future: overview on histology and histopathology. J. Histol. Histopathol. 2014;1:5.

   Not in our library

3. 3.

   Titford M. Progress in the development of microscopical techniques for diagnostic pathology. J. Histotechnol. 2009;32:9-19.

   Not in our library

4. 4.
   Histological Stains: A Literature Review and Case Study · 2015

   No notice
5. 5.

   Gurcan MN, et al. Histopathological image analysis: a review. IEEE Rev. Biomed. Eng. 2009;2:147-171.

   Not in our library10.1109/rbme.2009.2034865

6. 6.

   Titford M. The long history of hematoxylin. Biotech. Histochem. 2005;80:73-78.

   Not in our library10.1080/10520290500138372

7. 7.

   Foot NC. The Masson trichrome staining methods in routine laboratory use. Stain Technol. 1933;8:101-110.

   Not in our library

8. 8.

   Pollitt CC. Basement membrane pathology: a feature of acute equine laminitis. Equine Vet. J. 1996;28:38-46.

   Not in our library10.1111/j.2042-3306.1996.tb01588.x

9. 9.

   Coons AH, Creech HJ, Jones RN. Immunological properties of an antibody containing a fluorescent group. Exp. Biol. Med. 1941;47:200-202.

   Not in our library

10. 10.

    Ramos-Vara JA, Miller MA. When tissue antigens and antibodies get along: revisiting the technical aspects of immunohistochemistry—the red, brown, and blue technique. Vet. Pathol. 2014;51:42-87.

    Not in our library10.1177/0300985813505879

11. 11.

    Al-Janabi S, Huisman A, Van Diest PJ. Digital pathology: current status and future perspectives. Histopathology. 2012;61:1-9.

    Not in our library10.1111/j.1365-2559.2011.03814.x

12. 12.

    Madabhushi A, Lee G. Image analysis and machine learning in digital pathology: challenges and opportunities. Med. Image Anal. 2016;33:170-175.

    Not in our library10.1016/j.media.2016.06.037

13. 13.

    Wang HD, et al. Deep learning enables cross-modality super-resolution in fluorescence microscopy. Nat. Methods. 2019;16:103-110.

    Not in our library10.1038/s41592-018-0239-0

14. 14.

    Wu YC, et al. Bright-field holography: cross-modality deep learning enables snapshot 3D imaging with bright-field contrast using a single hologram. Light Sci. Appl. 2019;8:25.

    Not in our library10.1038/s41377-019-0139-9

15. 15.

    Jo Y, et al. Label-free multiplexed microtomography of endogenous subcellular dynamics using generalizable deep learning. Nat. Cell Biol. 2021;23:1329-1337.

    Not in our library10.1038/s41556-021-00802-x

16. 16.

    Cheng SY, et al. Single-cell cytometry via multiplexed fluorescence prediction by label-free reflectance microscopy. Sci. Adv. 2021;7:eabe0431.

    Not in our library10.1126/sciadv.abe0431

17. 17.

    Ounkomol C, et al. Label-free prediction of three-dimensional fluorescence images from transmitted-light microscopy. Nat. Methods. 2018;15:917-920.

    Not in our library10.1038/s41592-018-0111-2

18. 18.

    Rivenson Y, et al. Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning. Nat. Biomed. Eng. 2019;3:466-477.

    Not in our library10.1038/s41551-019-0362-y

19. 19.

    Rivenson Y, et al. Emerging advances to transform histopathology using virtual staining. BME Front. 2020;2020:9647163.

    Not in our library10.34133/2020/9647163

20. 20.
    PhaseStain: the digital staining of label-free quantitative phase microscopy images using deep learning · 2019

    No notice
21. 21.
    Deep learning-based transformation of H&E stained tissues into special stains · 2021

    No notice
22. 22.

    Zhang YJ, et al. Digital synthesis of histological stains using micro-structured and multiplexed virtual staining of label-free tissue. Light Sci. Appl. 2020;9:78.

    Not in our library10.1038/s41377-020-0315-y

23. 23.

    Li JX, et al. Biopsy-free in vivo virtual histology of skin using deep learning. Light Sci. Appl. 2021;10:233.

    Not in our library10.1038/s41377-021-00674-8

24. 24.

    Zhu, J. Y. et al. Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV). 2242–2251, 10.1109/ICCV.2017.244 (IEEE, Venice, Italy, 2017).

    Not in our library

25. 25.

    Rivenson, Y. et al. Deep learning-based virtual histology staining using auto-fluorescence of label-free tissue. Print at https://arxiv.org/abs/1803.11293 (2018).

    Not in our library

26. 26.

    Li XY, et al. Unsupervised content-preserving transformation for optical microscopy. Light Sci. Appl. 2021;10:44.

    Not in our library10.1038/s41377-021-00484-y

27. 27.

    Picon A, et al. Autofluorescence image reconstruction and virtual staining for in-vivo optical biopsying. IEEE Access. 2021;9:32081-32093.

    Not in our library

28. 28.

    Meng XY, Li X, Wang X. A computationally virtual histological staining method to ovarian cancer tissue by deep generative adversarial networks. Comput. Math. Methods Med. 2021;2021:4244157.

    Not in our library10.1155/2021/4244157

29. 29.

    Monici M. Cell and tissue autofluorescence research and diagnostic applications. Biotechnol. Annu. Rev. 2005;11:227-256.

    Not in our library10.1016/s1387-2656(05)11007-2

30. 30.

    Croce AC, Bottiroli G. Autofluorescence spectroscopy and imaging: a tool for biomedical research and diagnosis. Eur. J. Histochem. 2014;58:2461.

    Not in our library10.4081/ejh.2014.2461

31. 31.

    Bai BJ, et al. Label-free virtual HER2 immunohistochemical staining of breast tissue using deep learning. BME Front. 2022;2022:9786242.

    Not in our library10.34133/2022/9786242

32. 32.

    Nygate YN, et al. Holographic virtual staining of individual biological cells. Proc. Natl Acad. Sci. USA. 2020;117:9223-9231.

    Not in our library10.1073/pnas.1919569117

33. 33.

    Abraham, T. M. et al. Mode-mapping qOBM microscopy to virtual hematoxylin and eosin (H&E) histology via deep learning. Proceedings of SPIE 12136, Unconventional Optical Imaging III. 121360Q (SPIE, Strasbourg, France, 2022).

    Not in our library

34. 34.

    Borhani N, et al. Digital staining through the application of deep neural networks to multi-modal multi-photon microscopy. Biomed. Opt. Express. 2019;10:1339-1350.

    Not in our library10.1364/boe.10.001339

35. 35.

    Pradhan P, et al. Computational tissue staining of non-linear multimodal imaging using supervised and unsupervised deep learning. Biomed. Opt. Express. 2021;12:2280-2298.

    Not in our library10.1364/boe.415962

36. 36.

    Li D, et al. Deep Learning for virtual histological staining of bright-field microscopic images of unlabeled carotid artery tissue. Mol. Imaging Biol. 2020;22:1301-1309.

    Not in our library10.1007/s11307-020-01508-6

37. 37.

    Zhang GH, et al. Image-to-images translation for multiple virtual histological staining of unlabeled human carotid atherosclerotic tissue. Mol. Imaging Biol. 2022;24:31-41.

    Not in our library10.1007/s11307-021-01641-w

38. 38.

    Boktor M, et al. Virtual histological staining of label-free total absorption photoacoustic remote sensing (TA-PARS). Sci. Rep. 2022;12:10296.

    Not in our library10.1038/s41598-022-14042-y

39. 39.

    Kaza N, Ojaghi A, Robles FE. Virtual staining, segmentation, and classification of blood smears for label-free hematology analysis. BME Front. 2022;2022:9853606.

    Not in our library10.34133/2022/9853606

40. 40.

    Soltani S, et al. Prostate cancer histopathology using label-free multispectral deep-UV microscopy quantifies phenotypes of tumor aggressiveness and enables multiple diagnostic virtual stains. Sci. Rep. 2022;12:9329.

    Not in our library10.1038/s41598-022-13332-9

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.