Deep Learning for Cardiac Image Segmentation: A Review

Abstract

Deep learning has become the most widely used approach for cardiac image segmentation in recent years. In this paper, we provide a review of over 100 cardiac image segmentation papers using deep learning, which covers common imaging modalities including magnetic resonance imaging (MRI), computed tomography (CT), and ultrasound and major anatomical structures of interest (ventricles, atria, and vessels). In addition, a summary of publicly available cardiac image datasets and code repositories are included to provide a base for encouraging reproducible research. Finally, we discuss the challenges and limitations with current deep learning-based approaches (scarcity of labels, model generalizability across different domains, interpretability) and suggest potential directions for future research.

Full text

Its licence does not allow us to show the full text. Read it at PubMed Central:

References 257

1. 1.

   Petitjean C Zuluaga MA Bai W Dacher JN Grosgeorge D Caudron J. Right ventricle segmentation from cardiac MRI: a collation study. Med Image Anal. (2015 ) 19:187–202.

   Not in our library10.1016/j.media.2014.10.004

2. 2.
   A review of heart chamber segmentation for structural and functional analysis using cardiac magnetic resonance imaging · 2016

   No notice
3. 3.

   Tavakoli V Amini AA A survey of shaped-based registration and segmentation techniques for cardiac images. Comput Vis Image Understand. (2013 ) 117:966–89.

   Not in our library10.1016/j.cviu.2012.11.017

4. 4.

   Lesage D Angelini ED Bloch I Funka-Lea G. A review of 3D vessel lumen segmentation techniques: models, features and extraction schemes. Med Image Anal. (2009 ) 13:819–45.

   Not in our library10.1016/j.media.2009.07.011

5. 5.

   Greenspan H Van Ginneken B Summers RM Guest editorial deep learning in medical imaging: overview and future promise of an exciting new technique. IEEE Trans Med Imaging. (2016 ) 35:1153–9.

   Not in our library10.1109/tmi.2016.2553401

6. 6.

   Shen D Wu G Suk HI. Deep learning in medical image analysis. Annu Rev Biomed Eng. (2017 ) 19:221–48.

   Not in our library10.1146/annurev-bioeng-071516-044442

7. 7.

   Litjens G Kooi T Bejnordi BE Setio AAA Ciompi F Ghafoorian M. A survey on deep learning in medical image analysis. Med Image Anal. (2017 ) 42:60–88.

   Not in our library10.1016/j.media.2017.07.005

8. 8.

   Gandhi S Mosleh W Shen J Chow CM. Automation, machine learning, and artificial intelligence in echocardiography: a brave new world. Echocardiography. (2018 ) 35:1402–18.

   Not in our library10.1111/echo.14086

9. 9.

   Mazurowski MA Buda M Saha A Bashir MR. Deep learning in radiology: an overview of the concepts and a survey of the state of the art with focus on MRI. J Magn Reson Imaging. (2019 ) 49:939–54.

   Not in our library10.1002/jmri.26534

10. 10.

    Andreopoulos A Tsotsos JK. Efficient and generalizable statistical models of shape and appearance for analysis of cardiac MRI. Med Image Anal. (2008 ) 12:335–57.. Data source: http://www.cse.yorku.ca/~mridataset/

    Not in our library10.1016/j.media.2007.12.003

11. 11.

    Radau P Lu Y Connelly K Paul G Dick AJ Wright GA Evaluation framework for algorithms segmenting short axis cardiac MRI. MIDAS J. (2009 ) 49 Available online at: http://hdl.handle.net/10380/3070. Data source: http://www.cardiacatlas.org/studies/sunnybrook-cardiac-data/

    Not in our library

12. 12.

    Suinesiaputra A Cowan BR Al-Agamy AO Elattar MA Ayache N Fahmy AS. A collaborative resource to build consensus for automated left ventricular segmentation of cardiac MR images. Med Image Anal. (2014 ) 18:50–62.. Data source: http://www.cardiacatlas.org/challenges/lv-segmentation-challenge/

    Not in our library10.1016/j.media.2013.09.001

13. 13.

    Karim R Housden RJ Balasubramaniam M Chen Z Perry D Uddin A. Evaluation of current algorithms for segmentation of scar tissue from late gadolinium enhancement cardiovascular magnetic resonance of the left atrium: an open-access grand challenge. J Cardiovasc Magn Reson. (2013 ) 15:105.. Data source: https://www.doc.ic.ac.uk/~rkarim/la_lv_framework/fibrosis

    Not in our library10.1186/1532-429x-15-105

14. 14.

    Karim R Bhagirath P Claus P James Housden R Chen Z Karimaghaloo Z. Evaluation of state-of-the-art segmentation algorithms for left ventricle infarct from late gadolinium enhancement MR images. Med Image Anal. (2016 ) 30:95–107. Data source: https://www.doc.ic.ac.uk/~rkarim/la_lv_framework/lv_infarct/

    Not in our library10.1016/j.media.2016.01.004

15. 15.

    Tobon-Gomez C Geers AJ Peters J Weese J Pinto K Karim R. Benchmark for algorithms segmenting the left atrium from 3D CT and MRI datasets. IEEE Trans Med Imaging. (2015 ) 34:1460–73.. Data source: www.cardiacatlas.org/challenges/left-atrium-segmentation-challenge/

    Not in our library10.1109/tmi.2015.2398818

16. 16.

    Pace DF Dalca AV Geva T Powell AJ Moghari MH Golland P. Interactive whole-heart segmentation in congenital heart disease. Med Image Comput Comput Assist Interv. (2015 ) 9351:80–8.. Data source: http://segchd.csail.mit.edu/

    Not in our library10.1007/978-3-319-24574-4_10

17. 17.

    Bernard O Lalande A Zotti C Cervenansky F Yang X Heng PA. Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: is the problem solved? IEEE Trans Med Imaging. (2018 ) 37:2514–25. Data source: https://acdc.creatis.insa-lyon.fr/

    Not in our library10.1109/tmi.2018.2837502

18. 18.

    Zhao J Xiong Z 2018 Left Atrial Segmentation Challenge Dataset (2018 ). available online at: http://atriaseg2018.cardiacatlas.org/. Data source: http://atriaseg2018.cardiacatlas.org/

    Not in our library

19. 19.
    Evaluation of algorithms for Multi-Modality Whole Heart Segmentation: An open-access grand challenge · 2019

    No notice
20. 20.

    Schaap M Metz CT van Walsum T van der Giessen AG Weustink AC Mollet NR. Standardized evaluation methodology and reference database for evaluating coronary artery centerline extraction algorithms. Med Image Anal. (2009 ) 13:701–14.. Data source: http://coronary.bigr.nl/centerlines/

    Not in our library10.1016/j.media.2009.06.003

21. 21.

    Kirişli HA Schaap M Metz CT Dharampal AS Meijboom WB Papadopoulou SL. Standardized evaluation framework for evaluating coronary artery stenosis detection, stenosis quantification and lumen segmentation algorithms in computed tomography angiography. Med Image Anal. (2013 ) 17:859–76.. Data source: http://coronary.bigr.nl/stenoses

    Not in our library10.1016/j.media.2013.05.007

22. 22.

    Bernard O Bosch JG Heyde B Alessandrini M Barbosa D Camarasu-Pop S. Standardized evaluation system for left ventricular segmentation algorithms in 3D echocardiography. IEEE Trans Med Imaging. (2016 ) 35:967–77. Data source: https://www.creatis.insa-lyon.fr/Challenge/CETUS

    Not in our library10.1109/tmi.2015.2503890

23. 23.

    Leclerc S Smistad E Pedrosa J Ostvik A Cervenansky F Espinosa F. Deep learning for segmentation using an open large-scale dataset in 2D echocardiography. IEEE Trans Med Imaging. (2019 ) 38:2198–210.

    Not in our library10.1109/tmi.2019.2900516

24. 24.

    Tran PV A fully convolutional neural network for cardiac segmentation in short-axis MRI. arxiv (2016 ) abs/1604.00494. Available online at: http://arxiv.org/abs/1604.00494 (accessed September 1, 2019).

    Not in our library

25. 25.

    Baumgartner CF Koch LM Pollefeys M Konukoglu E An exploration of 2D and 3D deep learning techniques for cardiac MR image segmentation. In: Pop M Sermesant M Jodoin P-M Lalande A Zhuang X Yang G Young AA Bernard O, editors. International Workshop on Statistical Atlases and Computational Models of the Heart, Vo. 10663. Cham: Springer (2017 ). p. 1–8.

    Not in our library

26. 26.

    Isensee F Jaeger PF Full PM Wolf I Engelhardt S Maier-Hein KH Automatic cardiac disease assessment on cine-MRI via time-series segmentation and domain specific features. In: Pop M Sermesant M Jodoin P-M Lalande A Zhuang X Yang G Young AA Bernard O, editors. Proceedings of the 8th International Workshop, STACOM 2017, Held in Conjunction with MICCAI 2017, Statistical Atlases and Computational Models of the Heart. ACDC and MMWHS Challenges. Quebec City, QC: Springer International Publishing (2017 ). p. 120–9.

    Not in our library

27. 27.

    Zheng Q Delingette H Duchateau N Ayache N. 3-D consistent and robust segmentation of cardiac images by deep learning with spatial propagation. IEEE Trans Med Imaging. (2018 ) 37:2137–48.

    Not in our library10.1109/tmi.2018.2820742

28. 28.

    Qin C Bai W Schlemper J Petersen SE Piechnik SK Neubauer S Joint learning of motion estimation and segmentation for cardiac MR image sequences. In: Frangi AF Schnabel JA Davatzikos C Alberola-López C Fichtinger G, editors. Proceedings, Part I of the 21st International Conference on Medical Image Computing and Computer Assisted Intervention—MICCAI, 2018. Granada: Springer International Publishing (2018 ). p. 472–80.

    Not in our library

29. 29.

    Khened M Kollerathu VA Krishnamurthi G. Fully convolutional multi-scale residual DenseNets for cardiac segmentation and automated cardiac diagnosis using ensemble of classifiers. Med Image Anal. (2019 ) 51:21–45.

    Not in our library10.1016/j.media.2018.10.004

30. 30.

    Yu L Cheng JZ Dou Q Yang X Chen H Qin J Automatic 3D cardiovascular MR segmentation with densely-connected volumetric ConvNets. In: Descoteaux M Maier-Hein L Franz AM Jannin P Collins DL Duchesne S, editors. Proceedings, Part II of the 20th International Conference on Medical Image Computing and Computer Assisted Intervention—MICCAI, 2017. Quebec City, QC: Springer International Publishing (2017 ). p. 287–95.

    Not in our library

31. 31.
    Automated cardiovascular magnetic resonance image analysis with fully convolutional networks · 2018

    No notice
32. 32.

    Bai W Suzuki H Qin C Tarroni G Oktay O Matthews PM Recurrent neural networks for aortic image sequence segmentation with sparse annotations. In: Frangi AF Schnabel JA Davatzikos C Alberola-López C Fichtinger G, editors. 21st International Conference on Medical Image Computing and Computer Assisted Intervention—MICCAI, 2018. Granada: Springer International Publishing (2018 ). p. 586–94.

    Not in our library

33. 33.

    Duan J Bello G Schlemper J Bai W Dawes TJW Biffi C Automatic 3D bi-ventricular segmentation of cardiac images by a shape-constrained multi-task deep learning approach. IEEE Transactions on Medical Imaging. (2019 ) PP:1

    Not in our library10.1109/tmi.2019.2894322

34. 34.

    Chen C Bai W Rueckert D Multi-task learning for left atrial segmentation on GE-MRI. In: Pop M Sermesant M Zhao J Li S McLeod K Young AA Rhode KS Mansi T, editors. Proceedings of the 9th International Workshop, STACOM 2018, Held in Conjunction With MICCAI 2018, Statistical Atlases and Computational Models of the Heart. Atrial Segmentation and LV Quantification Challenges. Granada: Springer International Publishing (2018 ). p. 292–301.

    Not in our library

35. 35.

    Yu L Wang S Li X Fu CW Heng PA Uncertainty-aware self-ensembling model for semi-supervised 3D left atrium segmentation. In: Shen D Liu T Peters TM Staib LH Essert C Zhou S Yap P-T Khan A, editors. Proceedings, Part II of the 22nd International Conference on Medical Image Computing and Computer Assisted Intervention—MICCAI, 2019. Shenzhen: Springer International Publishing (2019 ). p. 605–13.

    Not in our library

36. 36.

    Yang X Bian C Yu L Ni D Heng PA Hybrid loss guided convolutional networks for whole heart parsing. In: Pop M Sermesant M Jodoin P-M Lalande A Zhuang X Yang G Young AA Bernard O, editors. Proceedings of the 8th International Workshop, STACOM 2017, Held in Conjunction with MICCAI 2017, Statistical Atlases and Computational Models of the Heart. ACDC and MMWHS Challenges. Quebec City, QC: Springer (2017 ). p. 215–23.

    Not in our library

37. 37.

    Xu Z Wu Z Feng J CFUN: combining faster R-CNN and U-net network for efficient whole heart segmentation. arxiv (2018 ) abs/1812.04914. Available online at: http://arxiv.org/abs/1812.04914 (accessed September 1, 2019).

    Not in our library

38. 38.

    Merkow J Marsden A Kriegman D Tu Z Dense volume-to-volume vascular boundary detection. In: urselin S Joskowicz L Sabuncu MR Ünal GB Wells W, editors. Proceedings, Part III of the 19th International Conference on Medical Image Computing and Computer-Assisted Intervention—MICCAI, 2016. Athens: Springer (2016 ). p. 371–9.

    Not in our library

39. 39.

    Dou Q Ouyang C Chen C Chen H Heng PA Unsupervised cross-modality domain adaptation of ConvNets for biomedical image segmentations with adversarial loss. In: International Joint Conferences on Artificial Intelligence (2018 ). p. 691–7.

    Not in our library

40. 40.

    Dou Q Ouyang C Chen C Chen H Glocker B Zhuang X PnP-AdaNet: plug-and-play adversarial domain adaptation network at unpaired cross-modality cardiac segmentation. IEEE Access. (2019 ) 7:99065–76.

    Not in our library10.1109/access.2019.2929258

Cited by papers we hold 1

• Deep Neural Architectures for Medical Image Semantic Segmentation: Review · 2021
  Not checked

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.