Interpretable survival prediction for colorectal cancer using deep learning

Abstract

Deriving interpretable prognostic features from deep-learning-based prognostic histopathology models remains a challenge. In this study, we developed a deep learning system (DLS) for predicting disease-specific survival for stage II and III colorectal cancer using 3652 cases (27,300 slides). When evaluated on two validation datasets containing 1239 cases (9340 slides) and 738 cases (7140 slides), respectively, the DLS achieved a 5-year disease-specific survival AUC of 0.70 (95% CI: 0.66–0.73) and 0.69 (95% CI: 0.64–0.72), and added significant predictive value to a set of nine clinicopathologic features. To interpret the DLS, we explored the ability of different human-interpretable features to explain the variance in DLS scores. We observed that clinicopathologic features such as T-category, N-category, and grade explained a small fraction of the variance in DLS scores ( R 2 = 18% in both validation sets). Next, we generated human-interpretable histologic features by clustering embeddings from a deep-learning-based image-similarity model and showed that they explained the majority of the variance ( R 2 of 73–80%). Furthermore, the clustering-derived feature most strongly associated with high DLS scores was also highly prognostic in isolation. With a distinct visual appearance (poorly differentiated tumor cell clusters adjacent to adipose tissue), this feature was identified by annotators with 87.0–95.5% accuracy. Our approach can be used to explain predictions from a prognostic deep learning model and uncover potentially-novel prognostic features that can be reliably identified by people for future validation studies.

Full text

Its licence does not allow us to show the full text. Read it at PubMed Central:

References 57

1. 1.

   Amin MB, et al. The Eighth Edition AJCC Cancer Staging Manual: Continuing to build a bridge from a population-based to a more “personalized’‘ approach to cancer staging. CA Cancer J. Clin. 2017;67:93-99.

   Not in our library10.3322/caac.21388

2. 2.

   Schneider NI, Langner C. Prognostic stratification of colorectal cancer patients: current perspectives. Cancer Manag. Res. 2014;6:291-300.

   Not in our library10.2147/cmar.s38827

3. 3.

   Weiser MR, et al. Individualized prediction of colon cancer recurrence using a nomogram. J. Clin. Oncol. 2008;26:380-385.

   Not in our library10.1200/jco.2007.14.1291

4. 4.

   Skrede O-J, et al. Deep learning for prediction of colorectal cancer outcome: a discovery and validation study. Lancet. 2020;395:350-360.

   Not in our library10.1016/s0140-6736(19)32998-8

5. 5.
   Predicting survival from colorectal cancer histology slides using deep learning: A retrospective multicenter study · 2019

   No notice
6. 6.
   Deep learning based tissue analysis predicts outcome in colorectal cancer · 2018

   No notice
7. 7.
   Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features · 2016

   No notice
8. 8.

   Beck AH, et al. Systematic analysis of breast cancer morphology uncovers stromal features associated with survival. Sci. Transl. Med. 2011;3:108ra113.

   Not in our library10.1126/scitranslmed.3002564

9. 9.

   Bray F, et al. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J. Clin. 2018;68:394-424.

   Not in our library10.3322/caac.21492

10. 10.

    QUASAR Collaborative Group. Adjuvant chemotherapy versus observation in patients with colorectal cancer: a randomised study. Lancet. 2007;370:2020-2029.

    Not in our library10.1016/s0140-6736(07)61866-2

11. 11.

    Kannarkatt J, Joseph J, Kurniali PC, Al-Janadi A, Hrinczenko B. Adjuvant chemotherapy for stage II colon cancer: a clinical dilemma. J. Oncol. Pract. 2017;13:233-241.

    Not in our library10.1200/jop.2016.017210

12. 12.

    Yothers G, et al. Oxaliplatin as adjuvant therapy for colon cancer: updated results of NSABP C-07 trial, including survival and subset analyses. J. Clin. Oncol. 2011;29:3768-3774.

    Not in our library10.1200/jco.2011.36.4539

13. 13.

    Puppa G, Sonzogni A, Colombari R, Pelosi G. TNM staging system of colorectal carcinoma: a critical appraisal of challenging issues. Arch. Pathol. Lab. Med. 2010;134:837-852.

    Not in our library10.5858/134.6.837

14. 14.

    Martin B, et al. Interobserver variability in the H&E-based assessment of tumor budding in pT3/4 colon cancer: does it affect the prognostic relevance? Virchows Arch. 2018;473:189-197.

    Not in our library10.1007/s00428-018-2341-1

15. 15.

    Harris EI, et al. Lymphovascular invasion in colorectal cancer: an interobserver variability study. Am. J. Surg. Pathol. 2008;32:1816-1821.

    Not in our library10.1097/pas.0b013e3181816083

16. 16.

    Bera K, Schalper KA, Rimm DL, Velcheti V, Madabhushi A. Artificial intelligence in digital pathology—new tools for diagnosis and precision oncology. Nat. Rev. Clin. Oncol. 2019;16:703-715.

    Not in our library10.1038/s41571-019-0252-y

17. 17.
    CellProfiler: image analysis software for identifying and quantifying cell phenotypes · 2006

    No notice
18. 18.

    Kamentsky L, et al. Improved structure, function and compatibility for CellProfiler: modular high-throughput image analysis software. Bioinformatics. 2011;27:1179-1180.

    Not in our library10.1093/bioinformatics/btr095

19. 19.

    Lu C, et al. Nuclear shape and orientation features from H&E images predict survival in early-stage estrogen receptor-positive breast cancers. Lab. Investig. 2018;98:1438-1448.

    Not in our library10.1038/s41374-018-0095-7

20. 20.

    Dimitriou N, Arandjelović O, Harrison DJ, Caie PD. A principled machine learning framework improves accuracy of stage II colorectal cancer prognosis. NPJ Digit. Med. 2018;1:52.

    Not in our library10.1038/s41746-018-0057-x

21. 21.

    Courtiol P, et al. Deep learning-based classification of mesothelioma improves prediction of patient outcome. Nat. Med. 2019;25:1519-1525.

    Not in our library10.1038/s41591-019-0583-3

22. 22.

    Mobadersany P, et al. Predicting cancer outcomes from histology and genomics using convolutional networks. Proc. Natl Acad. Sci. USA. 2018;115:E2970-E2979.

    Not in our library10.1073/pnas.1717139115

23. 23.
    Automated acquisition of explainable knowledge from unannotated histopathology images · 2019

    Not checked
24. 24.

    Simonyan, K., Vedaldi, A. & Zisserman, A. Deep inside convolutional networks: visualising image classification models and saliency maps. https://www.robots.ox.ac.uk/~vgg/publications/2014/Simonyan14a/ (2013).

    Not in our library

25. 25.

    Hegde N, et al. Similar image search for histopathology: SMILY. NPJ Digit. Med. 2019;2:56.

    Not in our library10.1038/s41746-019-0131-z

26. 26.

    Juan, D.-C. et al. Ultra Fine-Grained Image Semantic Embedding. In Proceedings of the 13th International Conference on Web Search and Data Mining. 10.1145/3336191.3371784 pp 277–285 (Association for Computing Machinery, New York, NY, USA, 2020).

    Not in our library

27. 27.

    O’Connell JB, Maggard MA, Ko CY. Colon cancer survival rates with the new American Joint Committee on Cancer sixth edition staging. J. Natl Cancer Inst. 2004;96:1420-1425.

    Not in our library10.1093/jnci/djh275

28. 28.

    Moertel CG, et al. Intergroup study of fluorouracil plus levamisole as adjuvant therapy for stage II/Dukes’ B2 colon cancer. J. Clin. Oncol. 1995;13:2936-2943.

    Not in our library10.1200/jco.1995.13.12.2936

29. 29.

    Liebig C, et al. Perineural invasion is an independent predictor of outcome in colorectal cancer. J. Clin. Oncol. 2009;27:5131-5137.

    Not in our library10.1200/jco.2009.22.4949

30. 30.

    Pagès F, et al. International validation of the consensus Immunoscore for the classification of colon cancer: a prognostic and accuracy study. Lancet. 2018;391:2128-2139.

    Not in our library10.1016/s0140-6736(18)30789-x

31. 31.

    Ueno H, et al. Desmoplastic pattern at the tumor front defines poor-prognosis subtypes of colorectal cancer. Am. J. Surg. Pathol. 2017;41:1506-1512.

    Not in our library10.1097/pas.0000000000000946

32. 32.

    Harisinghani MG, et al. Bowel wall fat halo sign in patients without intestinal disease. AJR Am. J. Roentgenol. 2003;181:781-784.

    Not in our library10.2214/ajr.181.3.1810781

33. 33.

    Horton KM, Corl FM, Fishman EK. CT evaluation of the colon: inflammatory disease. Radiographics. 2000;20:399-418.

    Not in our library10.1148/radiographics.20.2.g00mc15399

34. 34.

    Guiu B, et al. Visceral fat area is an independent predictive biomarker of outcome after first-line bevacizumab-based treatment in metastatic colorectal cancer. Gut. 2010;59:341-347.

    Not in our library10.1136/gut.2009.188946

35. 35.

    Dirat B, et al. Cancer-associated adipocytes exhibit an activated phenotype and contribute to breast cancer invasion. Cancer Res. 2011;71:2455-2465.

    Not in our library10.1158/0008-5472.can-10-3323

36. 36.

    Himbert C, et al. Signals from the adipose microenvironment and the obesity—cancer link-a systematic review. Cancer Prev. Res. 2017;10:494-506.

    Not in our library10.1158/1940-6207.capr-16-0322

37. 37.

    Koelzer VH, Lugli A. The tumor border configuration of colorectal cancer as a histomorphological prognostic indicator. Front. Oncol. 2014;4:29.

    Not in our library10.3389/fonc.2014.00029

38. 38.

    Cianchi F, et al. Character of the invasive margin in colorectal cancer: does it improve prognostic information of Dukes staging? Dis. Colon Rectum. 1997;40:1170-5.

    Not in our library10.1007/bf02055162

39. 39.

    Shia J, et al. Morphological characterization of colorectal cancers in The Cancer Genome Atlas reveals distinct morphology–molecular associations: clinical and biological implications. Mod. Pathol. 2017;30:599-609.

    Not in our library10.1038/modpathol.2016.198

40. 40.

    Cho S-J, Kakar S. Tumor budding in colorectal carcinoma: translating a morphologic score into clinically meaningful results. Arch. Pathol. Lab. Med. 2018;142:952-957.

    Not in our library10.5858/arpa.2018-0082-ra

Cited by papers we hold 3

• A whole-slide foundation model for digital pathology from real-world data · 2024
  No notice
• Artificial intelligence in digital pathology: a systematic review and meta-analysis of diagnostic test accuracy · 2024
  No notice
• Prostate cancer therapy personalization via multi-modal deep learning on randomized phase III clinical trials · 2022
  Corrected

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.