Phase recovery and holographic image reconstruction using deep learning in neural networks

Abstract

Phase recovery from intensity-only measurements forms the heart of coherent imaging techniques and holography. In this study, we demonstrate that a neural network can learn to perform phase recovery and holographic image reconstruction after appropriate training. This deep learning-based approach provides an entirely new framework to conduct holographic imaging by rapidly eliminating twin-image and self-interference-related spatial artifacts. This neural network-based method is fast to compute and reconstructs phase and amplitude images of the objects using only one hologram, requiring fewer measurements in addition to being computationally faster. We validated this method by reconstructing the phase and amplitude images of various samples, including blood and Pap smears and tissue sections. These results highlight that challenging problems in imaging science can be overcome through machine learning, providing new avenues to design powerful computational imaging systems.

Full text

Its licence does not allow us to show the full text. Read it at PubMed Central:

References 59

1. 1.

   Gerchberg RW, Saxton WO. A practical algorithm for the determination of phase from image and diffraction plane pictures. Optik 1972; 35: 237.

   Not in our library

2. 2.

   Fienup JR. Reconstruction of an object from the modulus of its Fourier transform. Opt Lett 1978; 3: 27–29.

   Not in our library10.1364/ol.3.000027

3. 3.

   Zalevsky Z, Mendlovic D, Dorsch RG. Gerchberg–Saxton algorithm applied in the fractional Fourier or the Fresnel domain. Opt Lett 1996; 21: 842–844.

   Not in our library10.1364/ol.21.000842

4. 4.

   Elser V. Solution of the crystallographic phase problem by iterated projections. Acta Crystallogr A 2003; 59: 201–209.

   Not in our library10.1107/s0108767303002812

5. 5.

   Luke DR. Relaxed averaged alternating reflections for diffraction imaging. Inverse Probl 2005; 21: 37–50.

   Not in our library

6. 6.

   Latychevskaia T, Fink HW. Solution to the twin image problem in holography. Phys Rev Lett 2007; 98: 233901.

   Not in our library10.1103/physrevlett.98.233901

7. 7.

   Marchesini S. Invited Article: a unified evaluation of iterative projection algorithms for phase retrieval. Rev Sci Instrum 2007; 78: 011301.

   Not in our library10.1063/1.2403783

8. 8.

   Quiney HM, Williams GJ, Nugent KA. Non-iterative solution of the phase retrieval problem using a single diffraction measurement. Opt Express 2008; 16: 6896–6903.

   Not in our library10.1364/oe.16.006896

9. 9.

   Brady DJ, Choi K, Marks DL, Horisaki R, Lim S. Compressive holography. Opt Express 2009; 17: 13040–13049.

   Not in our library10.1364/oe.17.013040

10. 10.

    Szameit A, Shechtman Y, Osherovich E, Bullkich E, Sidorenko P et al. Sparsity-based single-shot subwavelength coherent diffractive imaging. Nat Mater 2012; 11: 455–459.

    Not in our library10.1038/nmat3289

11. 11.

    Candès EJ, Eldar YC, Strohmer T, Voroninski V. Phase retrieval via matrix completion. SIAM J Imaging Sci 2013; 6: 199–225.

    Not in our library

12. 12.

    Rodriguez JA, Xu R, Chen CC, Zou YF, Miao JW. Oversampling smoothness: an effective algorithm for phase retrieval of noisy diffraction intensities. J Appl Crystallogr 2013; 46: 312–318.

    Not in our library10.1107/s0021889813002471

13. 13.

    Rivenson Y, Aviv (Shalev) M, Weiss A, Panet H, Zalevsky Z. Digital resampling diversity sparsity constrained-wavefield reconstruction using single-magnitude image. Opt Lett 2015; 40: 1842–1845.

    Not in our library10.1364/ol.40.001842

14. 14.

    Gabor D. A new microscopic principle. Nature 1948; 161: 777–778.

    Not in our library10.1038/161777a0

15. 15.

    Faulkner HML, Rodenburg JM. Movable aperture lensless transmission microscopy: a novel phase retrieval algorithm. Phys Rev Lett 2004; 93: 023903.

    Not in our library10.1103/physrevlett.93.023903

16. 16.

    Dierolf M, Menzel A, Thibault P, Schneider P, Kewish CM et al. Ptychographic X-ray computed tomography at the nanoscale. Nature 2010; 467: 436–439.

    Not in our library10.1038/nature09419

17. 17.

    Zheng GA, Horstmeyer R, Yang C. Wide-field, high-resolution Fourier ptychographic microscopy. Nat Photonics 2013; 7: 739–745.

    Not in our library10.1038/nphoton.2013.187

18. 18.

    Tian L, Waller L. 3D intensity and phase imaging from light field measurements in an LED array microscope. Optica 2015; 2: 104–111.

    Not in our library

19. 19.

    Misell DL. An examination of an iterative method for the solution of the phase problem in optics and electron optics: I. Test calculations. J Phys D Appl Phys 1973; 6: 2200–2216.

    Not in our library

20. 20.

    Teague MR. Deterministic phase retrieval: a Green’s function solution. J Opt Soc Am 1983; 73: 1434–1441.

    Not in our library

21. 21.

    Paganin D, Barty A, McMahon PJ, Nugent KA. Quantitative phase-amplitude microscopy. III. The effects of noise. J Microsc 2004; 214: 51–61.

    Not in our library10.1111/j.0022-2720.2004.01295.x

22. 22.

    Greenbaum A, Ozcan A. Maskless imaging of dense samples using pixel super-resolution based multi-height lensfree on-chip microscopy. Opt Express 2012; 20: 3129–3143.

    Not in our library10.1364/oe.20.003129

23. 23.

    Rivenson Y, Wu Y, Wang H, Zhang Y, Feizi A et al. Sparsity-based multi-height phase recovery in holographic microscopy. Sci Rep 2016; 6: 37862.

    Not in our library10.1038/srep37862

24. 24.

    Wang H, Göröcs Z, Luo W, Zhang Y, Rivenson Y et al. Computational out-of-focus imaging increases the space–bandwidth product in lensbased coherent microscopy. Optica 2016; 3: 1422–1429.

    Not in our library

25. 25.

    Ferraro P, Miccio L, Grilli S, Paturzo M, De Nicola S et al. Quantitative Phase Microscopy of microstructures with extended measurement range and correction of chromatic aberrations by multiwavelength digital holography. Opt Express 2007; 15: 14591–14600.

    Not in our library10.1364/oe.15.014591

26. 26.

    Luo W, Zhang YB, Feizi A, Göröcs Z, Ozcan A. Pixel super-resolution using wavelength scanning. Light Sci Appl 2016; 5: e16060 doi:10.1038/lsa.2016.60.

    Not in our library10.1038/lsa.2016.60

27. 27.

    Gonsalves RA. Phase retrieval and diversity in adaptive optics. Opt Eng 1982; 21: 215829.

    Not in our library

28. 28.

    Eisebitt S, Lüning J, Schlotter WF, Lörgen M, Hellwig O et al. Lensless imaging of magnetic nanostructures by X-ray spectro-holography. Nature 2004; 432: 885–888.

    Not in our library10.1038/nature03139

29. 29.

    Rosen J, Brooker G. Non-scanning motionless fluorescence three-dimensional holographic microscopy. Nat Photonics 2008; 2: 190–195.

    Not in our library

30. 30.

    Marchesini S, Boutet S, Sakdinawat AE, Bogan MJ, Bajt S et al. Massively parallel X-ray holography. Nat Photonics 2008; 2: 560–563.

    Not in our library

31. 31.

    Popescu G, Ikeda T, Dasari RR, Feld MS. Diffraction phase microscopy for quantifying cell structure and dynamics. Opt Lett 2006; 31: 775–777.

    Not in our library10.1364/ol.31.000775

32. 32.

    Coppola G, Di Caprio G, Gioffré M, Puglisi R, Balduzzi D et al. Digital self-referencing quantitative phase microscopy by wavefront folding in holographic image reconstruction. Opt Lett 2010; 35: 3390–3392.

    Not in our library10.1364/ol.35.003390

33. 33.

    Wang Z, Millet L, Mir M, Ding HF, Unarunotai S et al. Spatial light interference microscopy (SLIM). Opt Express 2011; 19: 1016–1026.

    Not in our library10.1364/oe.19.001016

34. 34.

    Rivenson Y, Katz B, Kelner R, Rosen J. Single channel in-line multimodal digital holography. Opt Lett 2013; 38: 4719–4722.

    Not in our library10.1364/ol.38.004719

35. 35.

    Shechtman Y, Eldar YC, Cohen O, Chapman HN, Miao JW et al. Phase retrieval with application to optical imaging: a contemporary overview. IEEE Signal Process Mag 2015; 32: 87–109.

    Not in our library

36. 36.

    Kelner R, Rosen J. Methods of single-channel digital holography for three-dimensional imaging. IEEE Trans Ind Inform 2016; 12: 220–230.

    Not in our library10.1109/tii.2015.2462803

37. 37.

    Zuo JM, Vartanyants I, Gao M, Zhang R, Nagahara LA. Atomic resolution imaging of a carbon nanotube from diffraction intensities. Science 2003; 300: 1419–1421.

    Not in our library10.1126/science.1083887

38. 38.

    Song CY, Jiang HD, Mancuso A, Amirbekian B, Peng L et al. Quantitative imaging of single, unstained viruses with coherent X rays. Phys Rev Lett 2008; 101: 158101.

    Not in our library10.1103/physrevlett.101.158101

39. 39.

    Miao JW, Ishikawa T, Shen Q, Earnest T. Extending X-ray crystallography to allow the imaging of noncrystalline materials, cells, and single protein complexes. Annu Rev Phys Chem 2008; 59: 387–410.

    Not in our library10.1146/annurev.physchem.59.032607.093642

40. 40.

    Loh ND, Hampton CY, Martin AV, Starodub D, Sierra RG et al. Fractal morphology, imaging and mass spectrometry of single aerosol particles in flight. Nature 2012; 486: 513–517.

    Not in our library10.1038/nature11222

41. 41.
    Deep learning · 2015

    No notice
42. 42.

    Schmidhuber J. Deep learning in neural networks: An overview. Neural Netw 2015; 61: 85–117.

    Not in our library10.1016/j.neunet.2014.09.003

43. 43.

    Dong C, Loy CC, He KM, Tang XO. Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 2016; 38: 295–307.

    Not in our library10.1109/tpami.2015.2439281

44. 44.

    Rivenson Y, Gorocs Z, Gunaydin H, Zhang YB, Wang HD et al Deep learning microscopy. Optica 2017; 4: 1437–1443.

    Not in our library

45. 45.

    Jin KH, McCann MT, Froustey E, Unser M. Deep convolutional neural network for inverse problems in imaging. IEEE Trans Image Process 2017; 26: 4509–4522.

    Not in our library10.1109/tip.2017.2713099

46. 46.

    Wang SS, Su ZH, Ying L, Peng X, Zhu S et al. Accelerating magnetic resonance imaging via deep learning. Proceedings of the 13th International Symposium on Biomedical Imaging (ISBI); 13-16 April 2016; Prague, Czech Republic. IEEE: Prague, Czech Republic 2016 doi:10.1109/ISBI.2016.7493320.

    Not in our library10.1109/isbi.2016.7493320

47. 47.
    Deep learning for photoacoustic tomography from sparse data · 2018

    No notice
48. 48.

    Jo Y, Park S, Jung J, Yoon J, Joo H et al. Holographic deep learning for rapid optical screening of anthrax spores. Sci Adv 2017; 3: e1700606.

    Not in our library10.1126/sciadv.1700606

49. 49.

    Sinha A, Lee J, Li S, Barbastathis G. Lensless computational imaging through deep learning. arXiv: 1702.08516, 2017.

    Not in our library

50. 50.

    Bartels M, Krenkel M, Haber J, Wilke RN, Salditt T. X-ray holographic imaging of hydrated biological cells in solution. Phys Rev Lett 2015; 114: 048103.

    Not in our library10.1103/physrevlett.114.048103

51. 51.

    McNulty I, Kirz J, Jacobsen C, Anderson EH, Howells MR et al. High-resolution imaging by Fourier transform X-ray holography. Science 1992; 256: 1009–1012.

    Not in our library10.1126/science.256.5059.1009

52. 52.

    Greenbaum A, Zhang YB, Feizi A, Chung PL, Luo W et al. Wide-field computational imaging of pathology slides using lens-free on-chip microscopy. Sci Transl Med 2014; 6: 267ra175–267ra175.

    Not in our library10.1126/scitranslmed.3009850

53. 53.

    Nowlan SJ, Hinton GE. Simplifying neural networks by soft weight-sharing. Neural Comput 1992; 4: 473–493.

    Not in our library

54. 54.

    Wang Z, Bovik AC, Sheikh HR, Simoncelli EP. Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 2004; 13: 600–612.

    Not in our library10.1109/tip.2003.819861

55. 55.

    Lohmann A. Optische einseitenbandübertragung angewandt auf das Gabor-Mikroskop. Opt Acta 1956; 3: 97–99.

    Not in our library

56. 56.

    Leith EN, Upatnieks J. Reconstructed wavefronts and communication theory. J Opt Soc Am 1962; 52: 1123–1130.

    Not in our library

57. 57.

    Goodman JW. Introduction to Fourier Optics. 3rd edn.Roberts and Company Publishers: Greenwood Village, Colorado; 2005.

    Not in our library

58. 58.

    cuFFT. NVIDIA Developer 2012. Available at https://developer.nvidia.com/cufft (accessed on 9th April 2017) (The content in the link is not NVIDIA Developer).

    Not in our library

59. 59.

    Thrust-Parallel Algorithms Library. Available at https://thrust.github.io/ (accessed and 9th April 2017) (The content in the link is not NVIDIA Developer).

    Not in our library

Cited by papers we hold 5

• Deep learning in optical metrology: a review · 2022
  Corrected
• doi:10.37188/lam.2021.023 · 2021
  Not checked
• Phase imaging with an untrained neural network · 2020
  No notice
• PhaseStain: the digital staining of label-free quantitative phase microscopy images using deep learning · 2019
  No notice
• Deep-learning-based ghost imaging · 2017
  Corrected

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.