Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning

Abstract

Remarkable progress has been made in image recognition, primarily due to the availability of large-scale annotated datasets and deep convolutional neural networks (CNNs). CNNs enable learning data-driven, highly representative, hierarchical image features from sufficient training data. However, obtaining datasets as comprehensively annotated as ImageNet in the medical imaging domain remains a challenge. There are currently three major techniques that successfully employ CNNs to medical image classification: training the CNN from scratch, using off-the-shelf pre-trained CNN features, and conducting unsupervised CNN pre-training with supervised fine-tuning. Another effective method is transfer learning, i.e., fine-tuning CNN models pre-trained from natural image dataset to medical image tasks. In this paper, we exploit three important, but previously understudied factors of employing deep convolutional neural networks to computer-aided detection problems. We first explore and evaluate different CNN architectures. The studied models contain 5 thousand to 160 million parameters, and vary in numbers of layers. We then evaluate the influence of dataset scale and spatial image context on performance. Finally, we examine when and why transfer learning from pre-trained ImageNet (via fine-tuning) can be useful. We study two specific computer-aided detection (CADe) problems, namely thoraco-abdominal lymph node (LN) detection and interstitial lung disease (ILD) classification. We achieve the state-of-the-art performance on the mediastinal LN detection, and report the first five-fold cross-validation classification results on predicting axial CT slices with ILD categories. Our extensive empirical evaluation, CNN model analysis and valuable insights can be extended to the design of high performance CAD systems for other medical imaging tasks.

Shown for reading. Its publisher allows quoting it, not republishing it.

Full text

Its licence does not allow us to show the full text. Read it at PubMed Central:

References 73

1. 1.

   Deng J., “ImageNet: A large-scale hierarchical image database,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2009, pp. 248–255.

   Not in our library

2. 2.

   Russakovsky O., “ImageNet large scale visual recognition challenge,” ArXiv:1409.0575, 2014.

   Not in our library

3. 3.

   LeCun Y., Bottou L., Bengio Y., and Haffner P., “Gradient-based learning applied to document recognition,” Proc. IEEE, vol. 86, no. 11, pp. 2278–2324, Nov. 1998.

   Not in our library

4. 4.

   Krizhevsky A., Sutskever I., and Hinton G. E., “ImageNet classification with deep convolutional neural networks,” Proc. NIPS, pp. 1097–1105, 2012.

   Not in our library

5. 5.

   Krizhevsky A., Learning multiple layers of features from tiny images, M.S. thesisDept. Comp. Sci. Univ. Toronto, Toronto, Canada: 2009.

   Not in our library

6. 6.

   Girshick R., Donahue J., Darrell T., and Malik J., “Region-based convolutional networks for accurate object detection and semantic segmentation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 38, no. 1, pp. 142–158, Jan. 2016.

   Not in our library10.1109/tpami.2015.2437384

7. 7.

   He K., Zhang X., Ren S., and Sun J., “Spatial pyramid pooling in deep convolutional networks for visual recognition,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 37, no. 9, pp. 1904–1916, Sep. 2015.

   Not in our library10.1109/tpami.2015.2389824

8. 8.

   Everingham M., “The pascal visual object classes challenge: A retrospective,” Int. J. Comput. Vis., vol. 111, no. 1, pp. 98–136, 2015.

   Not in our library

9. 9.

   van Ginneken B., Setio A., Jacobs C., and Ciompi F., “Off-the-shelf convolutional neural network features for pulmonary nodule detection in computed tomography scans,” Proc. IEEE 12th Int. Symp. Biomed. Imag., pp. 286–289, Apr. 2015.

   Not in our library

10. 10.

    Bar Y., Diamant I., Greenspan H., and Wolf L., “Chest pathology detection using deep learning with non-medical training,” Proc. IEEE 12th Int. Symp. Biomed. Imag., pp. 294–297, 2015.

    Not in our library

11. 11.

    Shin H., “Interleaved text/image deep mining on a large-scale radiology image database,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2015, pp. 1090–1099.

    Not in our library

12. 12.

    Ciompi F., “Automatic classification of pulmonary peri-fissural nodules in computed tomography using an ensemble of 2d views and a convolutional neural network out-of-the-box,” Med. Image Anal., 2015.

    Not in our library10.1016/j.media.2015.08.001

13. 13.

    Menze B., Reyes M., and Van Leemput K., “The multimodal brain tumor image segmentation benchmark (BRATS),” IEEE Trans. Med. Imag., vol. 34, no. 10, pp. 1993–2024, Oct. 2015.

    Not in our library10.1109/tmi.2014.2377694

14. 14.

    Pan Y., “Brain tumor grading based on neural networks and convolutional neural networks,” Proc. 37th Annu. Int. Conf. IEEE Eng. Med. Biol. Soc., pp. 699–702, 2015.

    Not in our library10.1109/embc.2015.7318458

15. 15.

    Shen W., Zhou M., Yang F., Yang C., and Tian J., “Multi-scale convolutional neural networks for lung nodule classification,” in IPMI, 2015, pp. 588–599.

    Not in our library10.1007/978-3-319-19992-4_46

16. 16.

    Carneiro G., Nascimento J., and Bradley A. P., “Unregistered multiview mammogram analysis with pre-trained deep learning models,” Proc. MICCAI, pp. 652–660, 2015.

    Not in our library

17. 17.

    Wolterink J. M., Leiner T., Viergever M. A., and Išgum I., “Automatic coronary calcium scoring in cardiac CT angiography using convolutional neural networks,” Proc. MICCAI, pp. 589–596, 2015.

    Not in our library10.1016/j.media.2016.04.004

18. 18.

    Schlegl T., Ofner J., and Langs G. Unsupervised pre-training across image domains improves lung tissue classification Medical Comput. Vision: Algorithms for Big Data, New York: Springer, 2014, pp. 82–93.

    Not in our library

19. 19.

    Hofmanninger J. and Langs G., “Mapping visual features to semantic profiles for retrieval in medical imaging,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2015, pp. 457–465.

    Not in our library

20. 20.

    Carneiro G. and Nascimento J., “Combining multiple dynamic models and deep learning architectures for tracking the left ventricle endocardium in ultrasound data,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 11, pp. 2592–2607, Nov. 2013.

    Not in our library10.1109/tpami.2013.96

21. 21.

    Li R., “Deep learning based imaging data completion for improved brain disease diagnosis,” in Proc. MICCAI, 2014.

    Not in our library10.1007/978-3-319-10443-0_39

22. 22.

    Roth H., “Improving computer-aided detection using convolutional neural networks and random view aggregation,” IEEE Trans. Med. Imag..

    Not in our library10.1109/tmi.2015.2482920

23. 23.

    Barbu A., “Automatic detection and segmentation of lymph nodes from CT data,” IEEE Trans. Med. Imag., vol. 31, no. 2, pp. 240–250, Feb. 2012.

    Not in our library10.1109/tmi.2011.2168234

24. 24.

    Feulner J., Zhou S. K., Hammon M., Hornegger J., and Comaniciu D., “Lymph node detection and segmentation in chest CT data using discriminative learning and a spatial prior,” Med. Image Anal., vol. 17, no. 2, pp. 254–270, 2013.

    Not in our library10.1016/j.media.2012.11.001

25. 25.

    Feuerstein M., “Mediastinal atlas creation from 3-d chest computed tomography images: Application to automated detection and station mapping of lymph nodes,” Med. Image Anal., vol. 16, no. 1, pp. 63–74, 2012.

    Not in our library10.1016/j.media.2011.05.005

26. 26.

    Lu L., Devarakota P., Vikal S., Wu D., Zheng Y., and Wolf M. Computer aided diagnosis using multilevel image features on large-scale evaluation Medical Comput. Vision. Large Data in Medical Imaging, New York: Springer, 2014, pp. 161–174.

    Not in our library

27. 27.

    Lu L., Bi J., Wolf M., and Salganicoff M., “Effective 3d object detection and regression using probabilistic segmentation features in CT images,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2011, pp. 1049–1056.

    Not in our library

28. 28.

    Lu L., “Accurate polyp segmentation for 3d CT colonography using multi-staged probabilistic binary learning and compositional model,” in Proc. IEEE Comput. Vis. Pattern Recognit. Conf., 2008, pp. 1–8.

    Not in our library

29. 29.

    Tajbakhsh N., Gotway M. B., and Liang J., “Computer-aided pulmonary embolism detection using a novel vessel-aligned multi-planar image representation and convolutional neural networks,” in Proc. MICCAI, 2015.

    Not in our library

30. 30.

    Lowe D. G., “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis., vol. 60, no. 2, pp. 91–110, 2004.

    Not in our library

31. 31.

    Dalal N. and Triggs B., “Histograms of oriented gradients for human detection,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 1, pp. 886–893, 2005.

    Not in our library

32. 32.

    Torralba A., Fergus R., and Weiss Y., “Small codes and large image databases for recognition,” Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., pp. 1–8, 2008.

    Not in our library

33. 33.

    Szegedy C., “Going deeper with convolutions,” in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., 2015, pp. 1–9.

    Not in our library

34. 34.

    Chatfield K., Simonyan K., Vedaldi A., and Zisserman A., “Return of the devil in the details: Delving deep into convolutional nets,” Br. Mach. Vis. Conf., 2014.

    Not in our library

35. 35.

    Chatfield K., Lempitsky V. S., Vedaldi A., and Zisserman A., “The devil is in the details: An evaluation of recent feature encoding methods,” Br. Mach. Vis. Conf., 2011.

    Not in our library

36. 36.

    Seff A., Lu L., Barbu A., Roth H., Shin H.-C., and Summers R. M., “Leveraging mid-level semantic boundary cues for computer-aided lymph node detection,” Proc. MICCAI, 2015.

    Not in our library

37. 37.

    Depeursinge A., “Building a reference multimedia database for interstitial lung diseases,” Comput. Med. Imaging and. Grap., vol. 36, no. 3, pp. 227–238, 2012.

    Not in our library10.1016/j.compmedimag.2011.07.003

38. 38.

    Song Y., Cai W., Zhou Y., and Feng D. D., “Feature-based image patch approximation for lung tissue classification,” IEEE Trans. Med. Imag., vol. 32, no. 4, pp. 797–808, Apr. 2013.

    Not in our library10.1109/tmi.2013.2241448

39. 39.

    Song Y., “Large margin local estimate with applications to medical image classification,” IEEE Trans. Med. Imag., vol. 34, no. 6, pp. 1362–1377, Jun. 2015.

    Not in our library10.1109/tmi.2015.2393954

40. 40.

    Gao M., “Holistic classification of CT attenuation patterns for interstitial lung diseases via deep convolutional neural networks,” in Proc. MICCAI First Workshop Deep Learn. Med. Image Anal., 2015.

    Not in our library10.1080/21681163.2015.1124249

Cited by papers we hold 45

• Deep-Learning Integration of CNN–Transformer and U-Net for Bi-Temporal SAR Flash-Flood Detection · 2025
  Not checked
• Hybrid Attention-Enhanced Xception and Dynamic Chaotic Whale Optimization for Brain Tumor Diagnosis · 2025
  Not checked
• Flow Cytometry: The Next Revolution · 2023
  No notice
• Deep learning in veterinary medicine, an approach based on CNN to detect pulmonary abnormalities from lateral thoracic radiographs in cats · 2022
  Not checked
• Siamese Network-Based All-Purpose-Tracker, a Model-Free Deep Learning Tool for Animal Behavioral Tracking · 2022
  Not checked
• Investigating the power of eyes open resting state EEG for assisting in dementia diagnosis · 2022
  Not checked
• Deep Learning-Based Detection and Identification Method for Sports Health Video Dissemination · 2022
  Not checked
• Emerging Applications of Deep Learning in Bone Tumors: Current Advances and Challenges · 2022
  Not checked
• Data-Driven Framework for Understanding and Predicting Air Quality in Urban Areas · 2022
  Not checked
• Transfer learning for medical image classification: a literature review · 2022
  No notice
• A Review on Deep Learning Techniques for the Diagnosis of Novel Coronavirus (COVID-19) · 2021
  No notice
• Adaptive Local Ternary Pattern on Parameter Optimized-Faster Region Convolutional Neural Network for Pulmonary Emphysema Diagnosis · 2021
  Not checked
• Predicting Tyrosine Kinase Inhibitor Treatment Response in Stage IV Lung Adenocarcinoma Patients With EGFR Mutation Using Model-Based Deep Transfer Learning · 2021
  Not checked
• Generative adversarial network image synthesis method for skin lesion generation and classification · 2021
  Not checked
• Artificial Intelligence in Colorectal Cancer Screening, Diagnosis and Treatment. A New Era · 2021
  No notice
• Mapping single-cell data to reference atlases by transfer learning · 2021
  No notice
• Multimodal deep learning models for early detection of Alzheimer’s disease stage · 2021
  No notice
• A disease category feature database construction method of brain image based on deep convolutional neural network. · 2020
  Not checked
• LC-DNN: Local Connection Based Deep Neural Network for Indoor Localization With CSI · 2020
  Not checked
• Deep Regression via Multi-Channel Multi-Modal Learning for Pneumonia Screening · 2020
  Not checked

and 25 more

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.