Deep Neural Networks Improve Radiologists’ Performance in Breast Cancer Screening

Abstract

We present a deep convolutional neural network for breast cancer screening exam classification, trained, and evaluated on over 200000 exams (over 1000000 images). Our network achieves an AUC of 0.895 in predicting the presence of cancer in the breast, when tested on the screening population. We attribute the high accuracy to a few technical advances. 1) Our network's novel two-stage architecture and training procedure, which allows us to use a high-capacity patch-level network to learn from pixel-level labels alongside a network learning from macroscopic breast-level labels. 2) A custom ResNet-based network used as a building block of our model, whose balance of depth and width is optimized for high-resolution medical images. 3) Pretraining the network on screening BI-RADS classification, a related task with more noisy labels. 4) Combining multiple input views in an optimal way among a number of possible choices. To validate our model, we conducted a reader study with 14 readers, each reading 720 screening mammogram exams, and show that our model is as accurate as experienced radiologists when presented with the same data. We also show that a hybrid model, averaging the probability of malignancy predicted by a radiologist with a prediction of our neural network, is more accurate than either of the two separately. To further understand our results, we conduct a thorough analysis of our network's performance on different subpopulations of the screening population, the model's design, training procedure, errors, and properties of its internal representations. Our best models are publicly available at https://github.com/nyukat/breast_cancer_classifier.

Full text

Its licence does not allow us to show the full text. Read it at PubMed Central:

References 39

1. 1.

   Siegel RL, Miller KD, and Jemal A, “Cancer statistics, 2015,” CA, Cancer J. Clinicians, vol. 65, no. 1, pp. 5–29, 2015.

   Not in our library10.3322/caac.21254

2. 2.

   Duffy SW, “The impact of organized mammography service screening on breast carcinoma mortality in seven Swedish counties: A collaborative evaluation,” Cancer, vol. 95, no. 3, pp. 458–469, 2002.

   Not in our library10.1002/cncr.10765

3. 3.

   Kopans DB, “Beyond randomized controlled trials: Organized mammographic screening substantially reduces breast carcinoma mortality,” Cancer, vol. 94, no. 2, pp. 580–581, 2002.

   Not in our library10.1002/cncr.10220

4. 4.

   Duffy SW, Tabár L, and Smith RA, “The mammographic screening trials: Commentary on the recent work by Olsen and Gøtzsche,” CA, Cancer J. Clinicians, vol. 52, no. 2, pp. 68–71, 2002.

   Not in our library10.3322/canjclin.52.2.68

5. 5.

   Kopans DB, “An open letter to panels that are deciding guidelines for breast cancer screening,” Breast Cancer Res Treat, vol. 151, no. 1, pp. 19–25, 2015.

   Not in our library10.1007/s10549-015-3373-8

6. 6.

   Lehman CD, Wellman RD, Buist DSM, Kerlikowske K, Tosteson ANA, and Miglioretti DL, “Diagnostic accuracy of digital screening mammography with and without computer-aided detection,” JAMA Internal Med, vol. 175, no. 11, pp. 1828–1837, 2015.

   Not in our library10.1001/jamainternmed.2015.5231

7. 7.
   Deep learning · 2015

   No notice
8. 8.

   LeCun Y, “Handwritten digit recognition with a back-propagation network,” in Proc. NIPS, 1989, pp. 396–404.

   Not in our library

9. 9.

   Krizhevsky A, Sutskever I, and Hinton GE, “ImageNet classification with deep convolutional neural networks,” in Proc. NIPS, 2012, pp. 1097–1105.

   Not in our library

10. 10.

    Simonyan K and Zisserman A, “Very deep convolutional networks for large-scale image recognition,” in Proc. ICLR, 2014, pp. 1–14.

    Not in our library

11. 11.

    He K, Zhang X, Ren S, and Sun J, “Deep residual learning for image recognition,” in Proc. CVPR, 6 2016, pp. 770–778.

    Not in our library

12. 12.

    Huang G, Liu Z, van der Maaten L, and Weinberger KQ, “Densely connected convolutional networks,” in Proc. CVPR, 6 2017, pp. 4700–4708.

    Not in our library

13. 13.

    Wu N, “The NYU breast cancer screening dataset V1.0,” New York Univ., New York, NY, USA, Tech. Rep, 2019 [Online]. Available: https://cs.nyu.edu/~kgeras/reports/datav1.0.pdf

    Not in our library

14. 14.

    Geras KJ, “High-resolution breast cancer screening with multi-view deep convolutional neural networks,” 2017, arXiv:1703.07047. [Online]. Available: https://arxiv.org/abs/1703.07047

    Not in our library

15. 15.

    Wu N, “Breast density classification with deep convolutional neural networks,” in Proc. ICASSP, 4 2018, pp. 6682–6686.

    Not in our library

16. 16.

    Caruana R, “Multitask learning,” Mach. Learn, vol. 28, no. 1, pp. 41–75, 1997.

    Not in our library

17. 17.

    Kingma DP and Ba J, “Adam: A method for stochastic optimization,” in Proc. ICLR, 2015, pp. 1–15.

    Not in our library

18. 18.

    Li F, Neverova N, Wolf C, and Taylor G, “Modout: Learning multi-modal architectures by stochastic regularization,” in Proc. 12th IEEE Int. Conf. Autom. Face Gesture Recognit, May-Jun 2017, pp. 422–429.

    Not in our library

19. 19.

    Lotter W, Sorensen G, and Cox D, “A multi-scale CNN and curriculum learning strategy for mammogram classification,” in Proc. DLMIA, 2017, pp. 169–177.

    Not in our library

20. 20.
    Detecting and classifying lesions in mammograms with Deep Learning · 2018

    No notice
21. 21.

    Deng J, Dong W, Socher R, Li L-J, Li K, and Fei-Fei L, “ImageNet: A large-scale hierarchical image database,” in Proc. CVPR, 2009, pp. 1–8.

    Not in our library

22. 22.

    Krause J, “The unreasonable effectiveness of noisy data for fine-grained recognition,” 2015, arXiv:1511.06789. [Online]. Available: https://arxiv.org/abs/1511.06789

    Not in our library

23. 23.

    Sun C, Shrivastava A, Singh S, and Gupta A, “Revisiting unreasonable effectiveness of data in deep learning era,” 2017, arXiv:1707.02968. [Online]. Available: https://arxiv.org/abs/1707.02968

    Not in our library

24. 24.

    Dietterich TG, “Ensemble methods in machine learning,” in Multiple Classifier Systems. Berlin, Germany: Springer, 2000.

    Not in our library

25. 25.

    McInnes L, John H, and James M, “Umap: Uniform manifold approximation and projection for dimension reduction,” 2018, arXiv:1802.03426. [Online]. Available: https://arxiv.org/abs/1802.03426

    Not in our library

26. 26.

    Kyono T, Gilbert FJ, and van der Schaar M, “MAMMO: A deep learning solution for facilitating radiologist-machine collaboration in breast cancer diagnosis,” 2018, arXiv:1811.02661. [Online]. Available: https://arxiv.org/abs/1811.02661

    Not in our library

27. 27.

    Kim E-K, “Applying data-driven imaging biomarker in mammography for breast cancer screening: Preliminary study,” Sci. Rep, vol. 8, 2 2018, Art. no. 2762.

    Not in our library10.1038/s41598-018-21215-1

28. 28.

    Zhu W, Lou Q, Vang YS, and Xie X, “Deep multi-instance networks with sparse label assignment for whole mammogram classification,” in Proc. MICCAI, 2017, pp. 603–611.

    Not in our library

29. 29.

    Kooi T and Karssemeijer N, “Classifying symmetrical differences and temporal change for the detection of malignant masses in mammography using deep neural networks,” Proc. SPIE, vol. 4, no. 4, 2017, Art. no. 044501.

    Not in our library10.1117/1.jmi.4.4.044501

30. 30.

    Kooi T, “Large scale deep learning for computer aided detection of mammographic lesions,” Med. Image Anal, vol. 35, pp. 303–312, 1 2017.

    Not in our library10.1016/j.media.2016.07.007

31. 31.

    Shen L, “End-to-end training for whole image breast cancer diagnosis using an all convolutional design,” 2017, arXiv:1711.05775. [Online]. Available: https://arxiv.org/abs/1711.05775

    Not in our library

32. 32.

    Teare P, Fishman M, Benzaquen O, Toledano E, and Elnekave E, “Malignancy detection on mammography using dual deep convolutional neural networks and genetically discovered false color input enhancement,” J. Digit. Imag, vol. 30, no. 4, pp. 499–505, 2017.

    Not in our library10.1007/s10278-017-9993-2

33. 33.

    Gao Y, Geras KJ, Lewin AA, and Moy L, “New frontiers: An update on computer-aided diagnosis for breast imaging in the age of artificial intelligence,” Amer. J. Roentgenol, vol. 212, no. 2, pp. 300–307, 2018.

    Not in our library10.2214/ajr.18.20392

34. 34.

    Harvey H, “The role of deep learning in breast screening,” Current Breast Cancer Reports, vol. 11, no. 1, pp. 17–22, 2019.

    Not in our library

35. 35.

    Becker AS, Marcon M, Ghafoor S, Wurnig MC, Frauenfelder T, and Boss A, “Deep learning in mammography: Diagnostic accuracy of a multipurpose image analysis software in the detection of breast cancer,” Investigative Radiol, vol. 52, no. 7, pp. 434–440, 2017.

    Not in our library10.1097/rli.0000000000000358

36. 36.

    Rodriguez-Ruiz A, “Stand-alone artificial intelligence for breast cancer detection in mammography: Comparison with 101 radiologists,” J. Nat. Cancer Inst, vol. 111, no. 9, pp. 916–922, 2019.

    Not in our library10.1093/jnci/djy222

37. 37.

    Rodríguez-Ruiz A, “Detection of breast cancer with mammography: Effect of an artificial intelligence support system,” Radiology, vol. 290, no. 2, pp. 305–314, 2018.

    Not in our library10.1148/radiol.2018181371

38. 38.

    Moreira IC, Amaral I, Domingues I, Cardoso A, Cardoso MJ, and Cardoso JS, “INbreast: Toward a full-field digital mammographic database,” Acad. Radiol, vol. 19, no. 2, pp. 236–248, 2012.

    Not in our library10.1016/j.acra.2011.09.014

39. 39.

    Heath M, “Current status of the digital database for screening mammography,” in Digital Mammography. Dordrecht, The Netherlands: Springer, 1998.

    Not in our library

Cited by papers we hold 6

• Signaling pathways involved in colorectal cancer: pathogenesis and targeted therapy · 2024
  No notice
• Health system-scale language models are all-purpose prediction engines · 2023
  No notice
• Combining the strengths of radiologists and AI for breast cancer screening: a retrospective analysis · 2022
  No notice
• Diagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis · 2021
  No notice
• Evaluation of Combined Artificial Intelligence and Radiologist Assessment to Interpret Screening Mammograms · 2020
  No notice
• Key challenges for delivering clinical impact with artificial intelligence · 2019
  No notice

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.