Deep learning-based survival prediction for multiple cancer types using histopathology images

Abstract

Providing prognostic information at the time of cancer diagnosis has important implications for treatment and monitoring. Although cancer staging, histopathological assessment, molecular features, and clinical variables can provide useful prognostic insights, improving risk stratification remains an active research area. We developed a deep learning system (DLS) to predict disease specific survival across 10 cancer types from The Cancer Genome Atlas (TCGA). We used a weakly-supervised approach without pixel-level annotations, and tested three different survival loss functions. The DLS was developed using 9,086 slides from 3,664 cases and evaluated using 3,009 slides from 1,216 cases. In multivariable Cox regression analysis of the combined cohort including all 10 cancers, the DLS was significantly associated with disease specific survival (hazard ratio of 1.58, 95% CI 1.28-1.70, p<0.0001) after adjusting for cancer type, stage, age, and sex. In a per-cancer adjusted subanalysis, the DLS remained a significant predictor of survival in 5 of 10 cancer types. Compared to a baseline model including stage, age, and sex, the c-index of the model demonstrated an absolute 3.7% improvement (95% CI 1.0-6.5) in the combined cohort. Additionally, our models stratified patients within individual cancer stages, particularly stage II (p = 0.025) and stage III (p<0.001). By developing and evaluating prognostic models across multiple cancer types, this work represents one of the most comprehensive studies exploring the direct prediction of clinical outcomes using deep learning and histopathology images. Our analysis demonstrates the potential for this approach to provide significant prognostic information in multiple cancer types, and even within specific pathologic stages. However, given the relatively small number of cases and observed clinical events for a deep learning task of this type, we observed wide confidence intervals for model performance, thus highlighting that future work will benefit from larger datasets assembled for the purposes for survival modeling.

Full text

Its licence does not allow us to show the full text. Read it at PubMed Central:

References 32

1. 1.

   Weiser MR, Gönen M, Chou JF, Kattan MW, Schrag D. Predicting survival after curative colectomy for cancer: individualizing colon cancer staging. J Clin Oncol. 2011;29: 4796–4802.

   Not in our library10.1200/jco.2011.36.5080

2. 2.

   Cooperberg MR, Pasta DJ, Elkin EP, Litwin MS, Latini DM, Du CHANE J, et al The University of California, San Francisco Cancer of the Prostate Risk Assessment score: a straightforward and reliable preoperative predictor of disease recurrence after radical prostatectomy. Journal of Urology. 2005 pp. 1938–1942.

   Not in our library10.1097/01.ju.0000158155.33890.e7

3. 3.

   Sparano JA, Gray RJ, Ravdin PM, Makower DF, Pritchard KI, Albain KS, et al Clinical and Genomic Risk to Guide the Use of Adjuvant Therapy for Breast Cancer. N Engl J Med. 2019

   Not in our library10.1056/nejmoa1904819

4. 4.

   Sparano JA, Gray RJ, Makower DF, Pritchard KI, Albain KS, Hayes DF, et al Adjuvant Chemotherapy Guided by a 21-Gene Expression Assay in Breast Cancer. N Engl J Med. 2018;379: 111–121.

   Not in our library10.1056/nejmoa1804710

5. 5.

   Rakha EA, El-Sayed ME, Lee AHS, Elston CW, Grainge MJ, Hodi Z, et al Prognostic significance of Nottingham histologic grade in invasive breast carcinoma. J Clin Oncol. 2008;26: 3153–3158.

   Not in our library10.1200/jco.2007.15.5986

6. 6.
   Deep learning · 2015

   No notice
7. 7.

   Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, et al Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs. JAMA. 2016;316: 2402–2410.

   Not in our library10.1001/jama.2016.17216

8. 8.

   Ehteshami Bejnordi B, Veta M, Johannes van Diest P, van Ginneken B, Karssemeijer N, Litjens G, et al Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women With Breast Cancer. JAMA. 2017;318: 2199–2210.

   Not in our library10.1001/jama.2017.14585

9. 9.

   Liu Y, Kohlberger T, Norouzi M, Dahl GE, Smith JL, Mohtashamian A, et al Artificial Intelligence-Based Breast Cancer Nodal Metastasis Detection. Arch Pathol Lab Med. 2018

   Not in our library10.5858/arpa.2018-0147-oa

10. 10.
    Development and validation of a deep learning algorithm for improving Gleason scoring of prostate cancer · 2019

    Corrected
11. 11.

    Saltz J, Gupta R, Hou L, Kurc T, Singh P, Nguyen V, et al Spatial Organization and Molecular Correlation of Tumor-Infiltrating Lymphocytes Using Deep Learning on Pathology Images. Cell Rep. 2018;23: 181–193.e7.

    Not in our library10.1016/j.celrep.2018.03.086

12. 12.

    Mobadersany P, Yousefi S, Amgad M, Gutman DA, Barnholtz-Sloan JS, Velázquez Vega JE, et al Predicting cancer outcomes from histology and genomics using convolutional networks. Proc Natl Acad Sci U S A. 2018;115: E2970–E2979.

    Not in our library10.1073/pnas.1717139115

13. 13.
    Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features · 2016

    No notice
14. 14.

    Cooper LAD, Demicco EG, Saltz JH, Powell RT, Rao A, Lazar AJ. PanCancer insights from The Cancer Genome Atlas: the pathologist’s perspective. The Journal of Pathology. 2018 pp. 512–524.

    Not in our library10.1002/path.5028

15. 15.

    Wang C, Pécot T, Zynger DL, Machiraju R, Shapiro CL, Huang K. Identifying survival associated morphological features of triple negative breast cancer using multiple datasets. J Am Med Inform Assoc. 2013;20: 680–687.

    Not in our library10.1136/amiajnl-2012-001538

16. 16.

    Yu K-H, Berry GJ, Rubin DL, Ré C, Altman RB, Snyder M. Association of Omics Features with Histopathology Patterns in Lung Adenocarcinoma. Cell Syst. 2017;5: 620–627.e3.

    Not in our library10.1016/j.cels.2017.10.014

17. 17.
    Predicting survival from colorectal cancer histology slides using deep learning: A retrospective multicenter study · 2019

    No notice
18. 18.
    Deep learning based tissue analysis predicts outcome in colorectal cancer · 2018

    No notice
19. 19.

    Zhu X, Yao J, Zhu F, Huang J. WSISA: Making Survival Prediction from Whole Slide Histopathological Images. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2017.

    Not in our library10.1109/cvpr.2017.725

20. 20.

    Weinstein JN, The Cancer Genome Atlas Research Network, Collisson EA, Mills GB, Mills Shaw KR, Ozenberger BA, et al The Cancer Genome Atlas Pan-Cancer analysis project. Nat Genet. 2013;45: 1113–1120.

    Not in our library10.1038/ng.2764

21. 21.

    Liu J, Lichtenberg T, Hoadley KA, Poisson LM, Lazar AJ, Cherniack AD, et al An Integrated TCGA Pan-Cancer Clinical Data Resource to Drive High-Quality Survival Outcome Analytics. Cell. 2018;173: 400–416.e11.

    Not in our library10.1016/j.cell.2018.02.052

22. 22.

    The Cancer Genome Atlas Network. Genomic Classification of Cutaneous Melanoma. Cell. 2015;161: 1681

    Not in our library10.1016/j.cell.2015.05.044

23. 23.

    Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, et al MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. arXiv [cs.CV]. 2017 Available: http://arxiv.org/abs/1704.04861

    Not in our library

24. 24.

    Cox DR. Partial Likelihood. Biometrika. 1975 p. 269

    Not in our library10.2307/2335362

25. 25.

    Breslow N. Covariance Analysis of Censored Survival Data. Biometrics. 1974 p. 89

    Not in our library10.2307/2529620

26. 26.

    Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15: 361–387.

    Not in our library10.1002/(sici)1097-0258(19960229)15:4<361::aid-sim168>3.0.co;2-4

27. 27.

    Steck H, Krishnapuram B, Dehing-oberije C, Lambin P, Raykar VC. On Ranking in Survival Analysis: Bounds on the Concordance Index In: Platt JC, Koller D, Singer Y, Roweis ST, editors. Advances in Neural Information Processing Systems 20. Curran Associates, Inc.; 2008 pp. 1209–1216.

    Not in our library

28. 28.

    Tieleman T, Hinton G. Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural networks for machine learning. 2012;4: 26–31.

    Not in our library

29. 29.

    Cox DR. Regression Models and Life-Tables. J R Stat Soc Series B Stat Methodol. 1972;34: 187–220.

    Not in our library

30. 30.

    Harrell FE. Evaluating the yield of medical tests. JAMA: The Journal of the American Medical Association. 1982 pp. 2543–2546.

    Not in our library10.1001/jama.247.18.2543

31. 31.

    Davidson-Pilon C, Kalderstam J, Zivich P, Kuhn B, Fiore-Gartland A, Moneda L, et al CamDavidsonPilon/lifelines: v0.21.3. 2019

    Not in our library10.5281/zenodo.3240536

32. 32.

    Beck AH, Sangoi AR, Leung S, Marinelli RJ, Nielsen TO, van de Vijver MJ, et al Systematic analysis of breast cancer morphology uncovers stromal features associated with survival. Sci Transl Med. 2011;3: 108ra113.

    Not in our library10.1126/scitranslmed.3002564

Cited by papers we hold 6

• Artificial intelligence in diagnostic pathology · 2023
  No notice
• Interpretable survival prediction for colorectal cancer using deep learning · 2021
  No notice
• Deep learning-enabled medical computer vision · 2021
  No notice
• Review of deep learning: concepts, CNN architectures, challenges, applications, future directions · 2021
  No notice
• The impact of site-specific digital histology signatures on deep learning model accuracy and bias · 2021
  No notice
• Deep learning in cancer pathology: a new generation of clinical biomarkers · 2020
  No notice

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.