3D Deep Learning on Medical Images: A Review

Abstract

The rapid advancements in machine learning, graphics processing technologies and the availability of medical imaging data have led to a rapid increase in the use of deep learning models in the medical domain. This was exacerbated by the rapid advancements in convolutional neural network (CNN) based architectures, which were adopted by the medical imaging community to assist clinicians in disease diagnosis. Since the grand success of AlexNet in 2012, CNNs have been increasingly used in medical image analysis to improve the efficiency of human clinicians. In recent years, three-dimensional (3D) CNNs have been employed for the analysis of medical images. In this paper, we trace the history of how the 3D CNN was developed from its machine learning roots, we provide a brief mathematical description of 3D CNN and provide the preprocessing steps required for medical images before feeding them to 3D CNNs. We review the significant research in the field of 3D medical imaging analysis using 3D CNNs (and its variants) in different medical areas such as classification, segmentation, detection and localization. We conclude by discussing the challenges associated with the use of 3D CNNs in the medical imaging domain (and the use of deep learning models in general) and possible future trends in the field.

References 131

1. 1.

   Doi K. Computer-Aided Diagnosis in Medical Imaging: Historical Review, Current Status and Future Potential. Comput. Med. Imaging Graph. 2007;31:198-211.

   Not in our library10.1016/j.compmedimag.2007.02.002

2. 2.

   Miller A.S., Blott B.H., hames T.K. Review of neural network applications in medical imaging and signal processing. Med. Biol. Eng. Comput. 1992;30:449-464.

   Not in our library10.1007/bf02457822

3. 3.

   Siedband M.P. Medical imaging systems. Med. Instrum.-Appl. Des. 1998;518-576.

   Not in our library

4. 4.

   Prince J., Links J. Medical Imaging Signals and Systems. 2006;315-379.

   Not in our library

5. 5.

   Shapiro R.S., Wagreich J., Parsons R.B., Stancato-Pasik A., Yeh H.C., Lao R. Tissue harmonic imaging sonography: Evaluation of image quality compared with conventional sonography. Am. J. Roentgenol. 1998;171:1203-1206.

   Not in our library10.2214/ajr.171.5.9798848

6. 6.

   Matsumoto K., Jinzaki M., Tanami Y., Ueno A., Yamada M., Kuribayashi S. Virtual Monochromatic Spectral Imaging with Fast Kilovoltage Switching: Improved Image Quality as Compared with That Obtained with Conventional 120-kVp CT. Radiology. 2011;259:257-262.

   Not in our library10.1148/radiol.11100978

7. 7.

   Thibault J.-B., Sauer K.D., Bouman C.A., Hsieh J. A three-dimensional statistical approach to improved image quality for multislice helical CT. Med. Phys. 2007;34:4526-4544.

   Not in our library10.1118/1.2789499

8. 8.

   Marin D., Nelson R.C., Schindera S.T., Richard S., Youngblood R.S., Yoshizumi T.T., Samei E. Low-Tube-Voltage, High-Tube-Current Multidetector Abdominal CT: Improved Image Quality and Decreased Radiation Dose with Adaptive Statistical Iterative Reconstruction Algorithm—Initial Clinical Experience. Radiology. 2010;254:145-153.

   Not in our library10.1148/radiol.09090094

9. 9.

   Ker J., Singh S.P., Bai Y., Rao J., Lim T., Wang L. Image Thresholding Improves 3-Dimensional Convolutional Neural Network Diagnosis of Different Acute Brain Hemorrhages on Computed Tomography Scans. Sensors. 2019;19:2167.

   Not in our library10.3390/s19092167

10. 10.

    Parmar H.S., Nutter B., Long R., Antani S., Mitra S., Gimi B.S., Krol A. Deep learning of volumetric 3D CNN for fMRI in Alzheimer’s disease classification. Medical Imaging 2020: Biomedical Applications in Molecular, Structural, and Functional Imaging, Houston, TX, USA, 2020. 2020;Volume 11317:11.

    Not in our library

11. 11.

    Shen D., Wu G., Suk H.-I. Deep Learning in Medical Image Analysis. Annu. Rev. Biomed. Eng. 2017;19:221-248.

    Not in our library10.1146/annurev-bioeng-071516-044442

12. 12.

    Gruetzemacher R., Gupta A., Paradice D. 3D deep learning for detecting pulmonary nodules in CT scans. J. Am. Med. Inform. Assoc. 2018;25:1301-1310.

    Not in our library10.1093/jamia/ocy098

13. 13.

    Wang S.H., Phillips P., Sui Y., Liu B., Yang M., Cheng H. Classification of Alzheimer’s Disease Based on Eight-Layer Convolutional Neural Network with Leaky Rectified Linear Unit and Max Pooling. J. Med. Syst. 2018;42:85.

    Not in our library10.1007/s10916-018-0932-7

14. 14.

    Krizhevsky A., Sutskever I., Hinton G.E. ImageNet classification with deep convolutional neural networks. Commun. ACM. 2017;60:84-90.

    Not in our library10.1145/3065386

15. 15.

    Szegedy C., Liu W., Jia Y., Sermanet P., Reed S., Anguelov D., Erhan D., Vanhoucke V., Rabinovich A. Going deeper with convolutions. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

    Not in our library

16. 16.

    Hoi S.C.H., Jin R., Zhu J., Lyu M.R. Batch mode active learning and its application to medical image classification. Proceedings of the 23rd International Conference on Machine Learning. 417-424.

    Not in our library

17. 17.

    Rahman M.M., Bhattacharya P., Desai B.C. A Framework for Medical Image Retrieval Using Machine Learning and Statistical Similarity Matching Techniques With Relevance Feedback. IEEE Trans. Inf. Technol. Biomed. 2007;11:58-69.

    Not in our library10.1109/titb.2006.884364

18. 18.

    Wernick M., Yang Y., Brankov J., Yourganov G., Strother S. Machine Learning in Medical Imaging. IEEE Signal. Process. Mag. 2010;27:25-38.

    Not in our library10.1109/msp.2010.936730

19. 19.

    Criminisi A., Shotton J., Konukoglu E. Decision forests: A unified framework for classification, regression, density estimation, manifold learning and semi-supervised learning. Found. Trends® Comput. Graph. Vis. 2012;7:81-227.

    Not in our library10.1561/0600000035

20. 20.

    Singh S.P., Urooj S. An Improved CAD System for Breast Cancer Diagnosis Based on Generalized Pseudo-Zernike Moment and Ada-DEWNN Classifier. J. Med. Syst. 2016;40:105.

    Not in our library10.1007/s10916-016-0454-0

21. 21.

    Urooj S., Singh S.P. Rotation invariant detection of benign and malignant masses using PHT. Proceedings of the 2015 2nd International Conference on Computing for Sustainable Global Development (INDIACom).

    Not in our library

22. 22.

    Ji S., Xu W., Yang M., Yu K. 3D Convolutional Neural Networks for Human Action Recognition. IEEE Trans. Pattern Anal. Mach. Intell. 2013;35:221-231.

    Not in our library10.1109/tpami.2012.59

23. 23.
    Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement · 2009

    No notice
24. 24.

    Pezeshk A., Hamidian S., Petrick N., Sahiner B. 3-D Convolutional Neural Networks for Automatic Detection of Pulmonary Nodules in Chest CT. IEEE J. Biomed. Heal. Inform. 2019;23:2080-2090.

    Not in our library10.1109/jbhi.2018.2879449

25. 25.

    Springenberg J.T., Dosovitskiy A., Brox T., Riedmiller M. Striving for simplicity: The all convolutional net. Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015–Workshop Track Proceedings.

    Not in our library

26. 26.

    Kukačka J., Golkov V., Cremers D. Regularization for Deep Learning: A Taxonomy. arXiv. 2017.

    Not in our library

27. 27.

    Srivastava N., Hinton G., Krizhevsky A., Salakhutdinov R. Dropout: A Simple Way to Prevent Neural Networks from Overfitting. J. Mach. Learn. Res. 2014;15:1929-1958.

    Not in our library

28. 28.

    Ioffe S., Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, International Machine Learning Society (IMLS). Volume 1;448-456.

    Not in our library

29. 29.

    Simonyan K., Zisserman A. Very deep convolutional networks for large-scale image recognition. Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015, Conference Track Proceedings.

    Not in our library

30. 30.

    Szegedy C., Vanhoucke V., Ioffe S., Shlens J., Wojna Z. Rethinking the Inception Architecture for Computer Vision. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Volume 1;2818-2826.

    Not in our library

31. 31.

    Szegedy C., Ioffe S., Vanhoucke V., Alemi A.A. Inception-v4, inception-ResNet and the impact of residual connections on learning. Proceedings of the 31st AAAI Conference on Artificial Intelligence, AAAI 2017. 4278-4284.

    Not in our library

32. 32.

    He K., Zhang X., Ren S., Sun J. Deep residual learning for image recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Volume 1;770-778.

    Not in our library

33. 33.

    Ronneberger O., Fischer P., Brox T. U-net: Convolutional networks for biomedical image segmentation. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). 2015;Volume 9351:234-241.

    Not in our library

34. 34.

    Çiçek Ö., Abdulkadir A., Lienkamp S.S., Brox T., Ronneberger O. 3D U-net: Learning dense volumetric segmentation from sparse annotation. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). 2016;Volume 9901:424-432.

    Not in our library

35. 35.

    Ker J., Wang L., Rao J., Lim T. Deep Learning Applications in Medical Image Analysis. IEEE Access. 2018;6:9375-9389.

    Not in our library10.1109/access.2017.2788044

36. 36.

    Burt J. Volumetric Quantification of Cardiovascular Structures from Medical Imaging. U.S. Patent. 2018.

    Not in our library

37. 37.

    Esteban O., Markiewicz C.J., Blair R.W., Moodie C.A., Isik A.I., Erramuzpe A., Kent J.D., Goncalves M., DuPre E., Snyder M., et al. fmriprep: A Robust Preprocessing Pipeline for fMRI Data—Fmriprep version documentation. Nat. Methods. 2019;111-116.

    Not in our library10.1038/s41592-018-0235-4

38. 38.

    Alansary A., Kamnitsas K., Davidson A., Khlebnikov R., Rajchl M., Malamateniou C., Rutherford M., Hajnal J.V., Glocker B., Rueckert D., et al. Fast Fully Automatic Segmentation of the Human Placenta from Motion Corrupted MRI. International Conference on Medical Image Computing and Computer-Assisted Intervention. 2016;589-597.

    Not in our library

39. 39.

    Yang C., Rangarajan A., Ranka S. Visual Explanations from Deep 3D Convolutional Neural Networks for Alzheimer’s Disease Classification. AMIA Annu. Symp. Proc. AMIA Symp. 2018;2018:1571-1580.

    Not in our library

40. 40.

    Jones D.K., Griffin L.D., Alexander D.C., Catani M., Horsfield M.A., Howard R., Williams S.C.R. Spatial Normalization and Averaging of Diffusion Tensor MRI Data Sets. Neuroimage. 2002;17:592-617.

    Not in our library10.1006/nimg.2002.1148

Cited by papers we hold 1

• Deep learning-enabled medical computer vision · 2021
  No notice

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.