DeepVesselNet: Vessel Segmentation, Centerline Prediction, and Bifurcation Detection in 3-D Angiographic Volumes

Abstract

We present DeepVesselNet, an architecture tailored to the challenges faced when extracting vessel trees and networks and corresponding features in 3-D angiographic volumes using deep learning. We discuss the problems of low execution speed and high memory requirements associated with full 3-D networks, high-class imbalance arising from the low percentage (<3%) of vessel voxels, and unavailability of accurately annotated 3-D training data—and offer solutions as the building blocks of DeepVesselNet. First, we formulate 2-D orthogonal cross-hair filters which make use of 3-D context information at a reduced computational burden. Second, we introduce a class balancing cross-entropy loss function with false-positive rate correction to handle the high-class imbalance and high false positive rate problems associated with existing loss functions. Finally, we generate a synthetic dataset using a computational angiogenesis model capable of simulating vascular tree growth under physiological constraints on local network structure and topology and use these data for transfer learning. We demonstrate the performance on a range of angiographic volumes at different spatial scales including clinical MRA data of the human brain, as well as CTA microscopy scans of the rat brain. Our results show that cross-hair filters achieve over 23% improvement in speed, lower memory footprint, lower network complexity which prevents overfitting and comparable accuracy that does not differ from full 3-D filters. Our class balancing metric is crucial for training the network, and transfer learning with synthetic data is an efficient, robust, and very generalizable approach leading to a network that excels in a variety of angiography segmentation tasks. We observe that sub-sampling and max pooling layers may lead to a drop in performance in tasks that involve voxel-sized structures. To this end, the DeepVesselNet architecture does not use any form of sub-sampling layer and works well for vessel segmentation, centerline prediction, and bifurcation detection. We make our synthetic training data publicly available, fostering future research, and serving as one of the first public datasets for brain vessel tree segmentation and analysis.

Full text

Its licence does not allow us to show the full text. Read it at PubMed Central:

References 56

1. 1.

   Bengio Y. Glorot X. (2010 ). Understanding the difficulty of training deep feedforward neuralnetworks, in Proceedings of the 13th International Conference on Artificial Intelligence and Statistics (AISTATS) (Sardinia), Vol. 9.

   Not in our library

2. 2.

   Chaichana T. Sun Z. Barrett-Baxendale M. Nagar A. (2017 ). Automatic location of blood vessel bifurcations in digital eye fundus images, in Proceedings of Sixth International Conference on Soft Computing for Problem Solving: SocProS 2016, Vol. 2 (Singapore: Springer Singapore), 332–342.

   Not in our library10.1007/978-981-10-3325-4_33

3. 3.

   Chen D. Cohen L. D. (2015 ). Piecewise geodesics for vessel centerline extraction and boundary delineation with application to retina segmentation, in Scale Space and Variational Methods in Computer Vision: 5th Int. Conf., SSVM 2015 (Lége-Cap Ferret: Springer International Publishing), 270–281.

   Not in our library10.1007/978-3-319-18461-6_22

4. 4.

   Christ P. F. Elshaer M. E. A. Ettlinger F. Tatavarty S. Bickel M. Bilic P. (2016 ). Automatic liver and lesion segmentation in CT using cascaded fully convolutional neural networks and 3d conditional random fields, in Proc. MICCAI 2016 (Cham: Springer International Publishing), 415–423.

   Not in our library10.1007/978-3-319-46723-8_48

5. 5.

   Chung A. C. S. Noble J. A. (1999 ). Statistical 3d vessel segmentation using a Rician distribution, in Proc. MICCAI 1999, eds Taylor C. Colchester A. (Berlin; Heidelberg: Springer), 82–89.

   Not in our library10.1007/10704282_9

6. 6.

   Çiçek Ö. Abdulkadir A. Lienkamp S. S. Brox T. Ronneberger O. (2016 ). 3D u-net: Learning dense volumetric segmentation from sparse annotation. CoRR abs/1606.06650

   Not in our library10.1007/978-3-319-46723-8_49

7. 7.

   Ciresan D. C. Giusti A. Gambardella L. M. Schmidhuber J. (2012 ). Deep neural networks segment neuronal membranes. Electron microscopy images, in NIPS (Nevada, NV), 2852–2860.

   Not in our library

8. 8.

   Dalca A. Danagoulian G. Kikinis R. Schmidt E. Golland P. (2011 ). Segmentation of nerve bundles and ganglia in spine MRI using particle filters, in Proc. MICCAI 2011 (Berlin; Heidelberg: Springer), 537–545.

   Not in our library10.1007/978-3-642-23626-6_66

9. 9.

   Florin C. Paragios N. Williams J. (2006 ). Globally optimal active contours, sequential Monte Carlo and on-line learning for vessel segmentation, in Computer Vision-ECCV 2006: 9th European Conf. on Computer Vision, Graz, Austria (Berlin; Heidelberg: Springer), 476–489.

   Not in our library

10. 10.

    Forkert N. D. Schmidt-Richberg A. Fiehler J. Illies T. Möller D. Handels H.. (2011 ). Fuzzy-based vascular structure enhancement in time-of-flight MRA images for improved segmentation. Methods Inform. Med. 50, 74–83.

    Not in our library10.3414/me10-02-0003

11. 11.

    Forkert N. D. Schmidt-Richberg A. Fiehler J. Illies T. Möller D. Säring D.. (2013 ). 3d cerebrovascular segmentation combining fuzzy vessel enhancement and level-sets with anisotropic energy weights. Magn. Reson. Imaging 31, 262–271.

    Not in our library10.1016/j.mri.2012.07.008

12. 12.

    Frangi A. F. Niessen W. J. Vincken K. L. Viergever M. A. (1998 ). Multiscale vessel enhancement filtering, in Proc. MICCAI 1998, eds Wells W. M. Colchester A. Delp S. (Berlin; Heidelberg: Springer), 130–137.

    Not in our library10.1007/bfb0056195

13. 13.

    Grzymala-Busse J. W. Goodwin L. K. Grzymala-Busse W. J. Zheng X. (2004 ). An approach to imbalanced data sets based on changing rule strength, in Rough-Neural Computing: Techniques for Computing With Words, eds Pal S. K. Polkowski L. Skowron A. (Berlin; Heidelberg: Springer), 543–553.

    Not in our library10.1007/978-3-642-18859-6_21

14. 14.

    Haixiang G. Yijing L. Shang J. Mingyun G. Yuanyue H. Bing G. (2017 ). Learning from class-imbalanced data: review of methods and applications. Expert Syst. Appl. 73, 220–239.

    Not in our library10.1016/j.eswa.2016.12.035

15. 15.

    Hwang J.-J. Liu T.-L. (2015 ). Pixel-wise deep learning for contour detection, in ICLR (San Diego, CA).

    Not in our library

16. 16.

    Kirbas C. Quek F. (2004 ). A review of vessel extraction techniques and algorithms. ACM Comput. Surv. 36, 81–121.

    Not in our library10.1145/1031120.1031121

17. 17.

    Koziński M. Mosinska A. Salzmann M. Fua P. (2018 ). Learning to segment 3d linear structures using only 2d annotations, in Medical Image Computing and Computer Assisted Intervention-MICCAI 2018, eds Frangi A. F. Schnabel J. A. Davatzikos C. Alberola-López C. Fichtinger G. (Cham: Springer International Publishing), 283–291.

    Not in our library10.1007/978-3-030-00934-2_32

18. 18.

    Law M. W. K. Chung A. C. S. (2008 ). Three dimensional curvilinear structure detection using optimally oriented flux, in Computer Vision-ECCV 2008, eds Forsyth D. Torr P. Zisserman A. (Berlin; Heidelberg: Springer), 368–382.

    Not in our library10.1007/978-3-540-88693-8_27

19. 19.

    Lesage D. Angelini E. Bloch I. Funka-Lea G. (2009 ). A review of 3d vessel lumen segmentation techniques: models, features and extraction schemes. Med. Image Anal. 13, 819–845.

    Not in our library10.1016/j.media.2009.07.011

20. 20.

    Liao W. Rohr K. Wörz S. (2013 ). Globally optimal curvature-regularized fast marching for vessel segmentation, in Proc. MICCAI 2013 (Berlin; Heidelberg: Springer), 550–557.

    Not in our library10.1007/978-3-642-40811-3_69

21. 21.

    Liu S. Zhang D. Song Y. Peng H. Cai W. (2017 ). Triple-crossing 2.5d convolutional neural network for detecting neuronal arbours in 3d microscopic images, in Mach. Learn. in Med. Imaging (Quebec, QC: Springer Int. Publishing), 185–193.

    Not in our library10.1007/978-3-319-67389-9_22

22. 22.

    Macedo M. M. G. Mekkaoui C. Jackowski M. P. (2010 ). Vessel centerline tracking in CTA and MRA images using hough transform, in Progress in Pattern Recognition, Image Anal., Computer Vision, and Applications: 15th Iberoamerican Congress on Pattern Recognition, CIARP 2010 (Berlin; Heidelberg: Springer), 295–302.

    Not in our library

23. 23.

    Maddah M. Afzali-khusha A. Soltanian H. (2003 ). Snake modeling and distance transform approach to vascular center line extraction and quantification. Computer. Med. Imag. Graph. 27, 503–512.

    Not in our library10.1016/s0895-6111(03)00040-5

24. 24.

    Maninis K.-K. Pont-Tuset J. Arbeláez P. Van Gool L. (2016 ). Deep retinal image understanding, in Proc. MICCAI 2016, Part II (Athens: Springer International Publishing), 140–148.

    Not in our library10.1007/978-3-319-46723-8_17

25. 25.

    Martínez-Pérez M. E. Hughes A. D. Stanton A. V. Thom S. A. Bharath A. A. Parker K. H. (1999 ). Retinal blood vessel segmentation by means of scale-space analysis and region growing, in Proc. MICCAI 1999 (Berlin; Heidelberg: Springer), 90–97.

    Not in our library10.1007/10704282_10

26. 26.

    Milletari F. Navab N. Ahmadi S. (2016 ). V-net: Fully convolutional neural networks for volumetric med. image segmentation, in Fourth Int. Conf. on 3D Vision (3DV) (Stanford), 565–571.

    Not in our library10.1109/3dv.2016.79

27. 27.

    Moreno R. Wang C. Smedby Ö. (2013 ). Vessel wall segmentation using implicit models and total curvature penalizers, in Image Anal.: 18th Scandinavian Conf., Proc. (Berlin; Heidelberg: Springer), 299–308.

    Not in our library10.1007/978-3-642-38886-6_29

28. 28.

    Nain D. Yezzi A. Turk G. (2004 ). Vessel segmentation using a shape driven flow, in Proc. MICCAI 2004 (Berlin; Heidelberg: Springer), 51–59.

    Not in our library10.1007/978-3-540-30135-6_7

29. 29.

    Nogues I. Lu L. Wang X. Roth H. Bertasius G. Lay N. (2016 ). Automatic lymph node cluster segmentation using holistically-nested neural networks and structured optimization in CT images, in Proc. MICCAI 2016, Part II (Athens: Springer International Publishing), 388–397.

    Not in our library10.1007/978-3-319-46723-8_45

30. 30.

    Phellan R. Forkert N. D. (2017 ). Comparison of vessel enhancement algorithms applied to time-of-flight MRA images for cerebrovascular segmentation. Med. Phys. 44, 5901–5915.

    Not in our library10.1002/mp.12560

31. 31.

    Phellan R. Peixinho A. Falc ao A. Forkert N. D. (2017 ). Vascular segmentation in TOF MRA images of the brain using a deep convolutional neural network, in Intravascular Imaging and Computer Assisted Stenting, and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis (Quebec: Springer International Publishing), 39–46.

    Not in our library10.1007/978-3-319-67534-3_5

32. 32.

    Piraud M. Sekuboyina A. Menze B. H. (2019 ). Multi-level activation for segmentation of hierarchically-nested classes, in Computer Vision-ECCV 2018 Workshops, eds Leal-Taixé L. Roth S. (Cham: Springer International Publishing), 345–353.

    Not in our library10.1007/978-3-030-11024-6_24

33. 33.

    Reichold J. Stampanoni M. Keller A. L. Buck A. Jenny P. Weber B. (2009 ). Vascular graph model to simulate the cerebral blood flow in realistic vascular networks. J. Cereb. Blood Flow Metab. 29, 1429–1443.

    Not in our library10.1038/jcbfm.2009.58

34. 34.

    Rempfler M. Schneider M. Ielacqua G. D. Xiao X. Stock S. R. Klohs J.. (2015 ). Reconstructing cerebrovascular networks under local physiological constraints by integer programming. Med. Image Anal. 86–94.

    Not in our library10.1016/j.media.2015.03.008

35. 35.

    Rigamonti R. Sironi A. Lepetit V. Fua P. (2013 ). Learning separable filters, in The IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) (Portland, OR).

    Not in our library10.1109/cvpr.2013.355

36. 36.

    Roth H. R. Lu L. Farag A. Sohn A. Summers R. M. (2016 ). Spatial aggregation of holistically-nested networks for automated pancreas segmentation, in Proc. MICCAI 2016, Part II (Athens: Springer International Publishing), 451–459.

    Not in our library10.1007/978-3-319-46723-8_52

37. 37.

    Roth H. R. Lu L. Seff A. Cherry K. M. Hoffman J. Wang S.. (2014 ). A new 2.5d representation for lymph node detection using random sets of deep convolutional neural network observations, in Proc. MICCAI 2014 (Boston, MA: Springer International Publishing), 520–527.

    Not in our library10.1007/978-3-319-10404-1_65

38. 38.

    Santamaría-Pang A. Colbert C. M. Saggau P. Kakadiaris I. A. (2007 ). Automatic centerline extraction of irregular tubular structures using probability volumes from multiphoton imaging, in Proc MICCAI 2007, Part II (Berlin; Heidelberg: Springer), 486–494.

    Not in our library10.1007/978-3-540-75759-7_59

39. 39.

    Schneider M. Hirsch S. Weber B. Székely G. Menze B. (2014 ). TGIF: topological gap in-fill for vascular networks-a generative physiological modeling approach, in International Conference on Medical Image Computing and Computer-Assisted Intervention, Vol. 17, 89–96.

    Not in our library10.1007/978-3-319-10470-6_12

40. 40.

    Schneider M. Hirsch S. Weber B. Székely G. Menze B. H. (2015 ). Joint 3-d vessel segmentation and centerline extraction using oblique hough forests with steerable filters. Med. Image Anal. 19, 220–249.

    Not in our library10.1016/j.media.2014.09.007

Cited by papers we hold 1

• A U-Net Deep Learning Framework for High Performance Vessel Segmentation in Patients With Cerebrovascular Disease · 2019
  No notice

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.