PhaseStain: the digital staining of label-free quantitative phase microscopy images using deep learning

Abstract

Using a deep neural network, we demonstrate a digital staining technique, which we term PhaseStain, to transform the quantitative phase images (QPI) of label-free tissue sections into images that are equivalent to the brightfield microscopy images of the same samples that are histologically stained. Through pairs of image data (QPI and the corresponding brightfield images, acquired after staining), we train a generative adversarial network and demonstrate the effectiveness of this virtual-staining approach using sections of human skin, kidney, and liver tissue, matching the brightfield microscopy images of the same samples stained with Hematoxylin and Eosin, Jones’ stain, and Masson’s trichrome stain, respectively. This digital-staining framework may further strengthen various uses of label-free QPI techniques in pathology applications and biomedical research in general, by eliminating the need for histological staining, reducing sample preparation related costs and saving time. Our results provide a powerful example of some of the unique opportunities created by data-driven image transformations enabled by deep learning.

Full text

Its licence does not allow us to show the full text. Read it at PubMed Central:

References 49

1. 1.

   Cuche E, Marquet P, Depeursinge C. Simultaneous amplitude-contrast and quantitative phase-contrast microscopy by numerical reconstruction of Fresnel off-axis holograms. Appl. Opt. 1999;38:6994-7001.

   Not in our library10.1364/ao.38.006994

2. 2.

   Popescu G. Quantitative Phase Imaging of Cells and Tissues. 2011.

   Not in our library

3. 3.

   Shaked NT, Rinehart MT, Wax A. Dual-interference-channel quantitative-phase microscopy of live cell dynamics. Opt. Lett. 2009;34:767-769.

   Not in our library10.1364/ol.34.000767

4. 4.

   Wang Z, et al. Spatial light interference microscopy (SLIM). Opt. Express. 2011;19:1016-1026.

   Not in our library10.1364/oe.19.001016

5. 5.

   Greenbaum A, et al. Imaging without lenses: achievements and remaining challenges of wide-field on-chip microscopy. Nat. Methods. 2012;9:889-895.

   Not in our library10.1038/nmeth.2114

6. 6.

   Zheng GA, Horstmeyer R, Yang C. Wide-field, high-resolution Fourier ptychographic microscopy. Nat. Photonics. 2013;7:739-745.

   Not in our library10.1038/nphoton.2013.187

7. 7.

   Tian L, Waller L. Quantitative differential phase contrast imaging in an LED array microscope. Opt. Express. 2015;23:11394-11403.

   Not in our library10.1364/oe.23.011394

8. 8.

   Wang Z, Tangella K, Balla A, Popescu G. Tissue refractive index as marker of disease. J. Biomed. Opt. 2011;16:116017.

   Not in our library10.1117/1.3656732

9. 9.

   Wang Z, Ding HF, Popescu G. Scattering-phase theorem. Opt. Lett. 2011;36:1215-1217.

   Not in our library10.1364/ol.36.001215

10. 10.

    Liu Y., et al. Detecting cancer metastases on gigapixel pathology images. ArXiv: 1703.02442 (2017).

    Not in our library

11. 11.

    Litjens G., et al. A survey on deep learning in medical image analysis. Med. Image Anal.42: 60–88 (2017).

    Not in our library10.1016/j.media.2017.07.005

12. 12.

    Tao YK, et al. Assessment of breast pathologies using nonlinear microscopy. Proc. Natl Acad. Sci. USA. 2014;111:15304-15309.

    Not in our library10.1073/pnas.1416955111

13. 13.

    Giacomelli MG, et al. Virtual hematoxylin and eosin transillumination microscopy using epi-fluorescence imaging. PLoS One. 2016;11:e0159337.

    Not in our library10.1371/journal.pone.0159337

14. 14.

    Orringer DA, et al. Rapid intraoperative histology of unprocessed surgical specimens via fibre-laser-based stimulated Raman scattering microscopy. Nat. Biomed. Eng. 2017;1:0027.

    Not in our library10.1038/s41551-016-0027

15. 15.

    Rivenson Y, et al. Deep learning microscopy. Optica. 2017;4:1437-1443.

    Not in our library10.1364/optica.4.001437

16. 16.

    Rivenson Y, et al. Deep learning enhanced mobile-phone microscopy. ACS Photon. 2018;5:2354-2364.

    Not in our library10.1021/acsphotonics.8b00146

17. 17.

    Wang H., et al. Deep learning enables cross-modality super-resolution in fluorescence microscopy Nat. Methods16, 103–110 (2019).

    Not in our library10.1038/s41592-018-0239-0

18. 18.

    Sinha A, Lee J, Li S, Barbastathis G. Lensless computational imaging through deep learning. Optica. 2017;4:1117-1125.

    Not in our library10.1364/optica.4.001117

19. 19.
    Phase recovery and holographic image reconstruction using deep learning in neural networks · 2017

    No notice
20. 20.

    Wu YC, et al. Extended depth-of-field in holographic imaging using deep-learning-based autofocusing and phase recovery. Optica. 2018;5:704-710.

    Not in our library10.1364/optica.5.000704

21. 21.

    Jo Y., et al. Quantitative phase imaging and artificial intelligence: a review. arXiv: 1806.03982 (2018).

    Not in our library

22. 22.

    Kamilov U, et al. Learning approach to optical tomography. Optica. 2015;2:517-522.

    Not in our library10.1364/optica.2.000517

23. 23.

    Nguyen T., Xue Y. J., Li Y. Z., Tian L., Nehmetallah G. Deep learning approach to Fourier ptychographic microscopy. arXiv: 1805.00334 (2018).

    Not in our library10.1364/oe.26.026470

24. 24.

    Boyd N., Jonas E., Babcock H. P., Recht B. DeepLoco: fast 3D localization microscopy using neural networks. bioRxiv: 267096, 2018. 10.1101/267096.

    Not in our library

25. 25.

    Nehme E, Weiss LE, Michaeli T, Shechtman Y. Deep-STORM: super-resolution single-molecule microscopy by deep learning. Optica. 2018;5:458-464.

    Not in our library10.1364/optica.5.000458

26. 26.

    Ouyang W, Aristov A, Lelek M, Hao X, Zimmer C. Deep learning massively accelerates super-resolution localization microscopy. Nat. Biotechnol. 2018;36:460-468.

    Not in our library10.1038/nbt.4106

27. 27.

    Zahavy T, et al. Deep learning reconstruction of ultrashort pulses. Optica. 2018;5:666-673.

    Not in our library10.1364/optica.5.000666

28. 28.

    Rivenson Y., et al. Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning. Nat. Biomed. Eng. (in the press).

    Not in our library10.1038/s41551-019-0362-y

29. 29.

    Goodfellow I. J., et al. Generative adversarial nets. In Proceedings of the 27th International Conference on Neural Information Processing Systems. (MIT Press: Cambridge, MA, 2014) pp. 2672–2680.

    Not in our library

30. 30.

    Park HS, Rinehart MT, Walzer KA, Chi JTA, Wax A. Automated detection of P. falciparum using machine learning algorithms with quantitative phase images of unstained cells. PLoS One. 2016;11:e0163045.

    Not in our library10.1371/journal.pone.0163045

31. 31.
    Deep Learning in Label-free Cell Classification · 2016

    No notice
32. 32.

    Roitshtain D, et al. Quantitative phase microscopy spatial signatures of cancer cells. Cytometry A. 2017;91:482-493.

    Not in our library10.1002/cyto.a.23100

33. 33.

    Jo Y, et al. Holographic deep learning for rapid optical screening of anthrax spores. Sci. Adv. 2017;3:e1700606.

    Not in our library10.1101/109108

34. 34.

    Javidi B, et al. Sickle cell disease diagnosis based on spatio-temporal cell dynamics analysis using 3D printed shearing digital holographic microscopy. Opt. Express. 2018;26:13614-13627.

    Not in our library10.1364/oe.26.013614

35. 35.

    Tata A, et al. Wide-field tissue polarimetry allows efficient localized mass spectrometry imaging of biological tissues. Chem. Sci. 2016;7:2162-2169.

    Not in our library10.1039/c5sc03782d

36. 36.

    Cree IA, et al. Guidance for laboratories performing molecular pathology for cancer patients. J. Clin. Pathol. 2014;67:923-931.

    Not in our library10.1136/jclinpath-2014-202404

37. 37.

    Patel PG, et al. Preparation of formalin-fixed paraffin-embedded tissue cores for both RNA and DNA extraction. J. Vis. Exp. 2016;2016:e54299.

    Not in our library10.3791/54299

38. 38.

    Ikeda T, Popescu G, Dasari RR, Feld MS. Hilbert phase microscopy for investigating fast dynamics in transparent systems. Opt. Lett. 2005;30:1165-1167.

    Not in our library10.1364/ol.30.001165

39. 39.

    Shaked NT, Zhu YZ, Badie N, Bursac N, Wax A. Reflective interferometric chamber for quantitative phase imaging of biological sample dynamics. J. Biomed. Opt. 2010;15:030503.

    Not in our library10.1117/1.3420179

40. 40.

    Watanabe E, Hoshiba T, Javidi B. High-precision microscopic phase imaging without phase unwrapping for cancer cell identification. Opt. Lett. 2013;38:1319-1321.

    Not in our library10.1364/ol.38.001319

Cited by papers we hold 4

• Deep learning-enabled virtual histological staining of biological samples · 2023
  No notice
• GANscan: continuous scanning microscopy using deep learning deblurring · 2022
  Not checked
• Spectroscopic and deep learning-based approaches to identify and quantify cerebral microhemorrhages · 2021
  Not checked
• Deep learning-based transformation of H&E stained tissues into special stains · 2021
  No notice

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.