Annotation-efficient deep learning for automatic medical image segmentation

Abstract

Automatic medical image segmentation plays a critical role in scientific research and medical care. Existing high-performance deep learning methods typically rely on large training datasets with high-quality manual annotations, which are difficult to obtain in many clinical applications. Here, we introduce Annotation-effIcient Deep lEarning (AIDE), an open-source framework to handle imperfect training datasets. Methodological analyses and empirical evaluations are conducted, and we demonstrate that AIDE surpasses conventional fully-supervised models by presenting better performance on open datasets possessing scarce or noisy annotations. We further test AIDE in a real-life case study for breast tumor segmentation. Three datasets containing 11,852 breast images from three medical centers are employed, and AIDE, utilizing 10% training annotations, consistently produces segmentation maps comparable to those generated by fully-supervised counterparts or provided by independent radiologists. The 10-fold enhanced efficiency in utilizing expert labels has the potential to promote a wide range of biomedical applications.

Full text

Its licence does not allow us to show the full text. Read it at PubMed Central:

References 66

1. 1.

   Hollon TC, et al. Near real-time intraoperative brain tumor diagnosis using stimulated Raman histology and deep neural networks. Nat. Med. 2020;26:52-58.

   Not in our library10.1038/s41591-019-0715-9

2. 2.

   Bai W, et al. A population-based phenome-wide association study of cardiac and aortic structure and function. Nat. Med. 2020;26:1654-1662.

   Not in our library10.1038/s41591-020-1009-y

3. 3.

   Mei X, et al. Artificial intelligence-enabled rapid diagnosis of patients with COVID-19. Nat. Med. 2020;26:1224-1228.

   Not in our library10.1038/s41591-020-0931-3

4. 4.

   Kickingereder P, et al. Automated quantitative tumour response assessment of MRI in neuro-oncology with artificial neural networks: a multicentre, retrospective study. Lancet Oncol. 2019;20:728-740.

   Not in our library10.1016/s1470-2045(19)30098-1

5. 5.

   Wang L, et al. Benchmark on automatic 6-month-old infant brain segmentation algorithms: the iSeg-2017 challenge. IEEE Trans. Med. Imaging. 2019;38:2219-2230.

   Not in our library10.1109/tmi.2019.2901712

6. 6.

   Tang H, et al. Clinically applicable deep learning framework for organs at risk delineation in CT images. Nat. Mach. Intell. 2019;1:480-491.

   Not in our library10.1038/s42256-019-0099-z

7. 7.

   Khened M, Kollerathu VA, Krishnamurthi G. Fully convolutional multi-scale residual DenseNets for cardiac segmentation and automated cardiac diagnosis using ensemble of classifiers. Med. Image Anal. 2019;51:21-45.

   Not in our library10.1016/j.media.2018.10.004

8. 8.

   Litjens G, et al. A survey on deep learning in medical image analysis. Med. Image Anal. 2017;42:60-88.

   Not in our library10.1016/j.media.2017.07.005

9. 9.

   Shen D, Wu G, Suk H-I. Deep learning in medical image analysis. Annu. Rev. Biomed. Eng. 2017;19:221-248.

   Not in our library10.1146/annurev-bioeng-071516-044442

10. 10.
    Deep Learning Techniques for Medical Image Segmentation: Achievements and Challenges · 2019

    No notice
11. 11.

    Li X, et al. H-DenseUNet: Hybrid densely connected UNet for liver and tumor segmentation from CT volumes. IEEE Trans. Med. Imaging. 2018;37:2663-2674.

    Not in our library10.1109/tmi.2018.2845918

12. 12.

    Dolz J, et al. HyperDense-Net: a hyper-densely connected CNN for multi-modal image segmentation. IEEE Trans. Med. Imaging. 2019;38:1116-1126.

    Not in our library10.1109/tmi.2018.2878669

13. 13.

    Haberl MG, et al. CDeep3M—Plug-and-Play cloud-based deep learning for image segmentation. Nat. Methods. 2018;15:677-680.

    Not in our library10.1038/s41592-018-0106-z

14. 14.
    Deep learning · 2015

    No notice
15. 15.

    Webb S. Deep learning for biology. Nature. 2018;554:555-557.

    Not in our library10.1038/d41586-018-02174-z

16. 16.

    Karimi D, Dou H, Warfield SK, Gholipour A. Deep learning with noisy labels: exploring techniques and remedies in medical image analysis. Med. Image Anal. 2020;65:101759.

    Not in our library10.1016/j.media.2020.101759

17. 17.

    Joskowicz L, Cohen D, Caplan N, Sosna J. Inter-observer variability of manual contour delineation of structures in CT. Eur. Radiol. 2019;29:1391-1399.

    Not in our library10.1007/s00330-018-5695-5

18. 18.

    Towards trustable machine learning. Nat. Biomed. Eng. 2, 709–710 (2018).

    Not in our library10.1038/s41551-018-0315-x

19. 19.

    Lutnick B, et al. An integrated iterative annotation technique for easing neural network training in medical image analysis. Nat. Mach. Intell. 2019;1:112-119.

    Not in our library10.1038/s42256-019-0018-3

20. 20.

    Esteva A, et al. A guide to deep learning in healthcare. Nat. Med. 2019;25:24-29.

    Not in our library10.1038/s41591-018-0316-z

21. 21.

    Holmberg OG, et al. Self-supervised retinal thickness prediction enables deep learning from unlabelled data to boost classification of diabetic retinopathy. Nat. Mach. Intell. 2020;2:719-726.

    Not in our library10.1038/s42256-020-00247-1

22. 22.

    Tajbakhsh N, et al. Embracing imperfect datasets: a review of deep learning solutions for medical image segmentation. Med. Image Anal. 2020;63:101693.

    Not in our library10.1016/j.media.2020.101693

23. 23.

    Blum, A. & Mitchell, T. Combining labeled and unlabeled data with co-training. In Conference on Computational Learning Theory (eds Bartlett, P. L. & Mansour, Y.) 92–100 (ACM, 1998).

    Not in our library

24. 24.

    Qiao, S., Shen, W., Zhang, Z., Wang, B. & Yuille, A. Deep co-training for semi-supervised image recognition. In European Conference on Computer Vision (ECCV) (eds Ferrari, V., Hebert, M., Sminchisescu, C. & Weiss, Y.) 135–152 (Springer, Cham, 2018).

    Not in our library

25. 25.

    Peng J, Estrada G, Pedersoli M, Desrosiers C. Deep co-training for semi-supervised image segmentation. Pattern Recogn. 2020;107:107269.

    Not in our library10.1016/j.patcog.2020.107269

26. 26.

    Xia Y, et al. Uncertainty-aware multi-view co-training for semi-supervised medical image segmentation and domain adaptation. Med. Image Anal. 2020;65:101766.

    Not in our library10.1016/j.media.2020.101766

27. 27.

    Han, B. et al. Co-teaching: robust training of deep neural networks with extremely noisy labels. In Conference on Neural Information Processing Systems (NeurIPS) (Montreal, Canada, 2018).

    Not in our library

28. 28.

    Yu, X. et al. How does disagreement help generalization against label corruption? In International Conference on Machine Learning (ICML) (Long Beach, California, USA, 2019).

    Not in our library

29. 29.

    Falk T, et al. U-Net: deep learning for cell counting, detection, and morphometry. Nat. Methods. 2019;16:67-70.

    Not in our library10.1038/s41592-018-0261-2

30. 30.

    Hazirbas, C. & Ma, L. FuseNet: incorporating depth into semantic segmentation via fusion-based CNN architecture. In Asian Conference on Computer Vision (ACCV) (Taipei, Taiwan, China, 2016).

    Not in our library

31. 31.

    Vohra, Y. et al. Data distillation: towards omni-supervised learning. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 4119–4128 (IEEE, 2018).

    Not in our library

32. 32.

    Lee, D.-H. Pseudo-label: the simple and efficient semi-supervised learning method for deep neural networks. ICML2013 Workshop on Challenges in Representational Learning, Vol. 3 (Atlanta, USA, 2013).

    Not in our library

33. 33.

    Kavur AE, et al. CHAOS Challenge—combined (CT-MR) healthy abdominal organ segmentation. Med. Image Anal. 2021;69:101950.

    Not in our library10.1016/j.media.2020.101950

34. 34.

    Litjens G, et al. Evaluation of prostate segmentation algorithms for MRI: the PROMISE12 challenge. Med. Image Anal. 2014;18:359-373.

    Not in our library10.1016/j.media.2013.12.002

35. 35.

    Zhang J, Saha A, Zhu Z, Mazurowski MA. Hierarchical convolutional neural networks for segmentation of breast tumors in MRI with application to radiogenomics. IEEE Trans. Med. Imaging. 2019;38:435-447.

    Not in our library10.1109/tmi.2018.2865671

36. 36.

    Qiao M, et al. Three-dimensional breast tumor segmentation on DCE-MRI with a multilabel attention-guided joint-phaselearning network. Comput. Med. Imaging Graph. 2021;90:101909.

    Not in our library10.1016/j.compmedimag.2021.101909

37. 37.

    Sun H, et al. AUNet: attention-guided dense-upsampling networks for breast mass segmentation in whole mammograms. Phys. Med. Biol. 2020;65:055005.

    Not in our library10.1088/1361-6560/ab5745

38. 38.

    Zhou Y, Huang W, Dong P, Xia Y, Wang S. D-UNet: a dimension-fusion U shape network for chronic stroke lesion segmentation. IEEE/ACM Trans. Comput. Biol. Bioinform. 2021;18:940-950.

    Not in our library10.1109/tcbb.2019.2939522

39. 39.

    Yang, H. et al. CLCI-Net: Cross-level fusion and context inference networks for lesion segmentation of chronic stroke. In International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) (eds Shen, D. et al.) Vol. 11766, 266–274 (Springer, Cham, 2019).

    Not in our library

40. 40.

    Qi, K. et al. X-Net: brain stroke lesion segmentation based on depthwise separable convolution and long-range dependencies. In International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) (eds Shen, D. et al.) Vol. 11766, 247–255 (Springer, Cham, 2019).

    Not in our library

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.