Deep learning for photoacoustic tomography from sparse data

Abstract

The development of fast and accurate image reconstruction algorithms is a central aspect of computed tomography. In this paper, we investigate this issue for the sparse data problem in photoacoustic tomography (PAT). We develop a direct and highly efficient reconstruction algorithm based on deep learning. In our approach, image reconstruction is performed with a deep convolutional neural network (CNN), whose weights are adjusted prior to the actual image reconstruction based on a set of training data. The proposed reconstruction approach can be interpreted as a network that uses the PAT filtered backprojection algorithm for the first layer, followed by the U-net architecture for the remaining layers. Actual image reconstruction with deep learning consists in one evaluation of the trained CNN, which does not require time-consuming solution of the forward and adjoint problems. At the same time, our numerical results demonstrate that the proposed deep learning approach reconstructs images with a quality comparable to state of the art iterative approaches for PAT from sparse data.

Full text

Its licence does not allow us to show the full text. Read it at PubMed Central:

References 80

1. 1.

   Goodfellow I, Bengio Y, Courville A. Deep learning. Cambridge (MA): MIT Press; 2016.

   Not in our library

2. 2.
   Deep learning · 2015

   No notice
3. 3.

   Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate. arXiv:1409.0473; 2014.

   Not in our library

4. 4.

   Collobert R, Weston J, Bottou L, et al Natural language processing (almost) from scratch. J Mach Learn Res. 2011;12:2493–2537.

   Not in our library

5. 5.

   Greenspan H, van Ginneken B, Summers RM. Guest editorial deep learning in medical imaging: overview and future promise of an exciting new technique. IEEE Trans Med Imaging. 2016;35(5):1153–1159

   Not in our library10.1109/tmi.2016.2553401

6. 6.

   Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv:1502.03167; 2015.

   Not in our library

7. 7.

   Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst. 2012;25:1097–1105.

   Not in our library

8. 8.

   Litjens G, Kooi T, Ehteshami Bejnordi B, et al A survey on deep learning in medical image analysis. Med Image Anal 2017;41:60–88.

   Not in our library10.1016/j.media.2017.07.005

9. 9.

   Ma J, Sheridan RP, Liaw A, et al Deep neural nets as a method for quantitative structure–activity relationships. J Chem Inf Model. 2015;55(2):263–274

   Not in our library10.1021/ci500747n

10. 10.

    Wu R, Yan S, Shan Y, et al Deep image: scaling up image recognition. arXiv:1501.02876; 2015.

    Not in our library

11. 11.

    Chen H, Zhang Y, Zhang W, et al Low-dose CT via convolutional neural network. Biomed Opt Express. 2017;8(2):679–694

    Not in our library10.1364/boe.8.000679

12. 12.

    Han Y, Yoo JJ, Ye JC. Deep residual learning for compressed sensing CT reconstruction via persistent homology analysis. Available from: arXiv:1611.06391; 2016.

    Not in our library

13. 13.

    Jin KH, McCann MT, Froustey E, et al Deep convolutional neural network for inverse problems in imaging. arXiv:1611.03679; 2016.

    Not in our library10.1109/tip.2017.2713099

14. 14.

    Wang G. A perspective on deep imaging. IEEE Access. 2016;4:8914–8924

    Not in our library10.1109/access.2016.2624938

15. 15.

    Wang S, Su Z, Ying L, et al Accelerating magnetic resonance imaging via deep learning. IEEE 13th international symposium on biomedical Imaging (ISBI); 2016. p. 514–517.

    Not in our library10.1109/isbi.2016.7493320

16. 16.

    Würfl T, Ghesu FC, Christlein V. Deep learning computed tomography. International conference on medical image computing and computer-assisted intervention; Springer: 2016. p. 432–440.

    Not in our library

17. 17.

    Zhang H, Li L, Qiao K, et al Image prediction for limited-angle tomography via deep learning with convolutional neural network. arXiv:1607.08707; 2016.

    Not in our library

18. 18.

    Beard P. Biomedical photoacoustic imaging. Interface Focus. 2011;1(4):602–631

    Not in our library10.1098/rsfs.2011.0028

19. 19.

    Kruger RA, Lui P, Fang YR, et al Photoacoustic ultrasound (paus) – reconstruction tomography. Med Phys. 1995;22(10):1605–1609

    Not in our library10.1118/1.597429

20. 20.

    Paltauf G, Nuster R, Haltmeier M, et al Photoacoustic tomography using a Mach-Zehnder interferometer as an acoustic line detector. Appl Opt. 2007;46(16):3352–3358

    Not in our library10.1364/ao.46.003352

21. 21.

    Wang LV. Multiscale photoacoustic microscopy and computed tomography. Nat Photon. 2009;3(9):503–509

    Not in our library10.1038/nphoton.2009.157

22. 22.

    Haltmeier M. Sampling conditions for the circular radon transform. IEEE Trans Image Process. 2016;25(6):2910–2919

    Not in our library10.1109/tip.2016.2551364

23. 23.

    Natterer F. The mathematics of computerized tomographytte, volume 32 of Classics in applied mathematics. Philadelphia: SIAM; 2001.

    Not in our library

24. 24.

    Arridge S, Beard P, Betcke M, et al Accelerated high-resolution photoacoustic tomography via compressed sensing. Phys Med Biol. 2016;61(24):8908

    Not in our library10.1088/1361-6560/61/24/8908

25. 25.

    Bauer-Marschallinger J, Felbermayer K, Bouchal K-D, et al Photoacoustic projection imaging using a 64-channel fiber optic detector array. Proc. San Francisco (CA): SPIE; 2015. p. 9323.

    Not in our library

26. 26.

    Gratt S, Nuster R, Wurzinger G, et al 64-line-sensor array: fast imaging system for photoacoustic tomography. Proc SPIE. 2014;8943:894365.

    Not in our library

27. 27.

    Guo Z, Li C, Song L, et al Compressed sensing in photoacoustic tomography in vivo. J Biomed Opt. 2010;15(2):021311–021311

    Not in our library10.1117/1.3381187

28. 28.

    Rosenthal A, Ntziachristos V, Razansky D. Acoustic inversion in optoacoustic tomography: a review. Curr Med Imaging Rev. 2013;9(4):318–336

    Not in our library10.2174/15734056113096660006

29. 29.

    Sandbichler M, Krahmer F, Berer T, et al A novel compressed sensing scheme for photoacoustic tomography. SIAM J Appl Math. 2015;75(6):2475–2494

    Not in our library10.1137/141001408

30. 30.

    Arridge SR, Betcke MM, Cox BT, et al On the adjoint operator in photoacoustic tomography. Inverse Problems. 2016;32(11):115012 (19pp)

    Not in our library10.1088/0266-5611/32/11/115012

31. 31.

    Belhachmi Z, Glatz T, Scherzer O. A direct method for photoacoustic tomography with inhomogeneous sound speed. Inverse Problems. 2016;32(4):045005

    Not in our library10.1088/0266-5611/32/4/045005

32. 32.

    Haltmeier M, Nguyen LV. Analysis of iterative methods in photoacoustic tomography with variable sound speed. SIAM J Imaging Sci. 2017;10(2):751–781

    Not in our library10.1137/16m1104822

33. 33.

    Huang C, Wang K, Nie L, et al Full-wave iterative image reconstruction in photoacoustic tomography with acoustically inhomogeneous media. IEEE Trans Med Imaging. 2013;32(6):1097–1110

    Not in our library10.1109/tmi.2013.2254496

34. 34.

    Javaherian A, Holman S. A multi-grid iterative method for photoacoustic tomography. IEEE Trans Med Imaging. 2017;36(3):696–706

    Not in our library10.1109/tmi.2016.2625272

35. 35.

    Schwab J, Pereverzyev Jr S, Haltmeier M. A Galerkin least squares approach for photoacoustic tomography. arXiv:1612.08094; 2016.

    Not in our library

36. 36.

    Wang K, Su R, Oraevsky AA, et al Investigation of iterative image reconstruction in three-dimensional optoacoustic tomography. Phys Med Biol. 2012;57(17):5399

    Not in our library10.1088/0031-9155/57/17/5399

37. 37.

    Acar R, Vogel CR. Analysis of bounded variation penalty methods for ill-posed problems. Inverse Problems. 1994;10(6):1217–1229

    Not in our library10.1088/0266-5611/10/6/003

38. 38.

    Daubechies I, Defrise M, De Mol C. An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Comm Pure Appl Math. 2004;57(11):1413–1457

    Not in our library10.1002/cpa.20042

39. 39.

    Frikel J, Haltmeier M. Efficient regularization with wavelet sparsity constraints in photoacoustic tomography. Inverse Prob. 2018; 34:024006(28pp).

    Not in our library

40. 40.

    Grasmair M, Haltmeier M, Scherzer O. Sparsity in inverse geophysical problems. In Freeden W, Nashed MZ, Sonar T, editors. Handbook of geomathematics. Berlin, Heidelberg: Springer; 2010. p. 763–784.

    Not in our library

41. 41.

    Provost J, Lesage F. The application of compressed sensing for photo-acoustic tomography. IEEE Trans Med Imaging. 2009;28(4):585–594

    Not in our library10.1109/tmi.2008.2007825

42. 42.

    Scherzer O, Grasmair M, Grossauer H, et al Variational methods in imaging, volume 167 of Applied mathematical sciences. New York: Springer; 2009.

    Not in our library

43. 43.

    Burgholzer P, Bauer-Marschallinger J, Grün H, et al Temporal back-projection algorithms for photoacoustic tomography with integrating line detectors. Inverse Problems. 2007;23(6):S65–S80

    Not in our library10.1088/0266-5611/23/6/s06

44. 44.

    Finch D, Haltmeier M, Rakesh Inversion of spherical means and the wave equation in even dimensions. SIAM J Appl Math. 2007;68(2):392–412

    Not in our library10.1137/070682137

45. 45.

    Finch D, Patch SK, Rakesh Determining a function from its mean values over a family of spheres. SIAM J Math Anal. 2004;35(5):1213–1240

    Not in our library10.1137/s0036141002417814

46. 46.

    Haltmeier M. Inversion of circular means and the wave equation on convex planar domains. Comput Math Appl. 2013;65(7):1025–1036

    Not in our library10.1016/j.camwa.2013.01.036

47. 47.

    Haltmeier M. Universal inversion formulas for recovering a function from spherical means. SIAM J Math Anal. 2014;46(1):214–232

    Not in our library10.1137/120881270

48. 48.

    Kunyansky LA. Explicit inversion formulae for the spherical mean Radon transform. Inverse Problems. 2007;23(1):373–383

    Not in our library10.1088/0266-5611/23/1/021

49. 49.

    Xu M, Wang LV. Universal back-projection algorithm for photoacoustic computed tomography. Phys Rev E. 2005;71(1):016706.

    Not in our library10.1103/physreve.71.016706

50. 50.

    Agranovsky M, Kuchment P. Uniqueness of reconstruction and an inversion procedure for thermoacoustic and photoacoustic tomography with variable sound speed. Inverse Problems. 2007;23(5):2089–2102

    Not in our library10.1088/0266-5611/23/5/016

51. 51.

    Haltmeier M, Scherzer O, Zangerl G. A reconstruction algorithm for photoacoustic imaging based on the nonuniform FFT. IEEE Trans Med Imaging. 2009;28(11):1727–1735

    Not in our library10.1109/tmi.2009.2022623

52. 52.

    Jaeger M, Schüpbach S, Gertsch A, et al Fourier reconstruction in optoacoustic imaging using truncated regularized inverse k-space interpolation. Inverse Problems. 2007;23:S51–S63

    Not in our library10.1088/0266-5611/23/6/s05

53. 53.

    Kunyansky LA. A series solution and a fast algorithm for the inversion of the spherical mean Radon transform. Inverse Problems. 2007;23(6):S11–S20

    Not in our library10.1088/0266-5611/23/6/s02

54. 54.

    Xu Y, Xu M, Wang LV. Exact frequency-domain reconstruction for thermoacoustic tomography–II: cylindrical geometry. IEEE Trans Med Imaging. 2002;21:829–833

    Not in our library10.1109/tmi.2002.801171

55. 55.

    Burgholzer P, Matt GJ, Haltmeier M, et al Exact and approximate imaging methods for photoacoustic tomography using an arbitrary detection surface. Phys Rev E. 2007;75(4):046706

    Not in our library10.1103/physreve.75.046706

56. 56.

    Hristova Y, Kuchment P, Nguyen L. Reconstruction and time reversal in thermoacoustic tomography in acoustically homogeneous and inhomogeneous media. Inverse Problems. 2008;24(5):055006(25pp)

    Not in our library10.1088/0266-5611/24/5/055006

57. 57.

    Stefanov P, Uhlmann G. Thermoacoustic tomography with variable sound speed. Inverse Problems. 2009;25(7):075011 16

    Not in our library10.1088/0266-5611/25/7/075011

58. 58.

    Treeby BE, Cox BT. k-wave: MATLAB toolbox for the simulation and reconstruction of photoacoustic wave-fields. J Biomed Opt. 2010;15:021314

    Not in our library10.1117/1.3360308

59. 59.

    Ronneberge O, Fischer P, Brox T. U-net: convolutional networks for biomedical image segmentation. CoRR; 2015.

    Not in our library

60. 60.

    Dreier F, Haltmeier M, Pereverzyev Jr S. Operator learning approach for the limited view problem in photoacoustic tomography. arXiv:1705.02698; 2017.

    Not in our library

61. 61.

    Kuchment P, Kunyansky L. Mathematics of photoacoustic and thermoacoustic tomography. In: Otmar S, editor. Handbook math methods imaging. New York: Springer; 2011. p. 817–865.

    Not in our library

62. 62.

    Xu M, Wang LV. Photoacoustic imaging in biomedicine. Rev Sci Instrum. 2006;77(4):041101(22pp)

    Not in our library10.1063/1.2195024

63. 63.

    Çiçek O, Abdulkadir A, Lienkamp SS, et al 3d u-net: learning dense volumetric segmentation from sparse annotation. arXiv:1606.06650; 2016.

    Not in our library

64. 64.

    Haltmeier M, Sandbichler M, Berer T, et al A sparsification and reconstruction strategy for compressed sensing photoacoustic tomography. J Acoust Soc Amer. 2018;141(6):3838(11pp).

    Not in our library10.1121/1.5042230

65. 65.

    Stefanov P, Uhlmann G. Thermoacoustic tomography with variable sound speed. Inverse Problems. 2009;25(7):075011 16

    Not in our library10.1088/0266-5611/25/7/075011

66. 66.

    Grün H, Berer T, Burgholzer P, et al Three-dimensional photoacoustic imaging using fiber-based line detectors. J Biomed Opt. 2010;15(2):021306–021308

    Not in our library10.1117/1.3381186

67. 67.

    Nuster R, Holotta M, Kremser C, et al Photoacoustic microtomography using optical interferometric detection. J Biomed Opt. 2010;15(2):021307–021307–6

    Not in our library10.1117/1.3333547

68. 68.

    Meng J, Wang LV, Liang D, et al In vivo optical-resolution photoacoustic computed tomography with compressed sensing. Opt Lett. 2012;37(22):4573–4575

    Not in our library10.1364/ol.37.004573

69. 69.

    Funahashi K. On the approximate realization of continuous mappings by neural networks. Neural Netw. 1989;2(3):183–192

    Not in our library10.1016/0893-6080(89)90003-8

70. 70.

    Hornik K. Approximation capabilities of multilayer feedforward networks. Neural Netw. 1991;4(2):251–257

    Not in our library10.1016/0893-6080(91)90009-t

71. 71.

    Bishop CM. Pattern recognition and machine learning. New York: Springer; 2006.

    Not in our library

72. 72.

    De Cezaro A, Haltmeier M, Leitão A, et al On steepest-descent-Kaczmarz methods for regularizing systems of nonlinear ill-posed equations. Appl Math Comput. 2008;202(2):596–607.

    Not in our library

73. 73.

    Gordon R, Bender R, Herman GT. Algebraic reconstruction techniques (ART) for three-dimensional electron microscopy and x-ray photography. J Theor Biol. 1970;29(3):471–481

    Not in our library10.1016/0022-5193(70)90109-8

74. 74.

    LeCun Y, Boser B, Denker JS, et al Backpropagation applied to handwritten zip code recognition. Neural Comput. 1989;1(4):541–551

    Not in our library10.1162/neco.1989.1.4.541

75. 75.

    Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. Proceedings of the IEEE conference on computer vision and pattern recognition; Boston, USA; 2015. p. 3431–3440.

    Not in our library

76. 76.

    Haltmeier M. A mollification approach for inverting the spherical mean Radon transform. SIAM J Appl Math. 2011;71(5):1637–1652

    Not in our library10.1137/110821561

77. 77.

    Glorot X, Bengio Y. Understanding the difficulty of training deep feedforward neural networks. Proceedings of the thirteenth international conference on artificial intelligence and statistics. Vol. 9. 2010; p. 249–256.

    Not in our library

78. 78.

    Vogel CR, Oman ME. Iterative methods for total variation denoising. SIAM J Sci Comput. 1996;17:227–238

    Not in our library10.1137/0917016

79. 79.
    Piezoelectric line detector array for photoacoustic tomography · 2017

    Not checked
80. 80.

    Schwab J, Antholzer S, Nuster R, et al Real-time photoacoustic projection imaging using deep learning. arXiv:1801.06693; 2018.

    Not in our library

Cited by papers we hold 5

• Fast iterative reconstruction for photoacoustic tomography using learned physical model: Theoretical validation · 2023
  Not checked
• Deep learning for biomedical photoacoustic imaging: A review · 2021
  No notice
• Review of cost reduction methods in photoacoustic computed tomography · 2019
  Not checked
• Model-Based Learning for Accelerated, Limited-View 3-D Photoacoustic Tomography · 2018
  No notice
• Phase recovery and holographic image reconstruction using deep learning in neural networks · 2017
  No notice

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.