Layer-Wise Relevance Propagation for Explaining Deep Neural Network Decisions in MRI-Based Alzheimer's Disease Classification

Abstract

Deep neural networks have led to state-of-the-art results in many medical imaging tasks including Alzheimer's disease (AD) detection based on structural magnetic resonance imaging (MRI) data. However, the network decisions are often perceived as being highly non-transparent, making it difficult to apply these algorithms in clinical routine. In this study, we propose using layer-wise relevance propagation (LRP) to visualize convolutional neural network decisions for AD based on MRI data. Similarly to other visualization methods, LRP produces a heatmap in the input space indicating the importance/relevance of each voxel contributing to the final classification outcome. In contrast to susceptibility maps produced by guided backpropagation ("Which change in voxels would change the outcome most?"), the LRP method is able to directly highlight positive contributions to the network classification in the input space. In particular, we show that (1) the LRP method is very specific for individuals ("Why does this person have AD?") with high inter-patient variability, (2) there is very little relevance for AD in healthy controls and (3) areas that exhibit a lot of relevance correlate well with what is known from literature. To quantify the latter, we compute size-corrected metrics of the summed relevance per brain area, e.g., relevance density or relevance gain. Although these metrics produce very individual "fingerprints" of relevance patterns for AD patients, a lot of importance is put on areas in the temporal lobe including the hippocampus. After discussing several limitations such as sensitivity toward the underlying model and computation parameters, we conclude that LRP might have a high potential to assist clinicians in explaining neural network decisions for diagnosing AD (and potentially other diseases) based on structural MRI data.

Full text

Its licence does not allow us to show the full text. Read it at PubMed Central:

References 62

1. 1.

   Abdel-Hamid O. Mohamed A.-R. Jiang H. Deng L. Penn G. Yu D. (2014 ). Convolutional neural networks for speech recognition. IEEE/ACM Trans. Audio Speech. Language Proc. 22, 1533–1545.

   Not in our library10.1109/taslp.2014.2339736

2. 2.

   Adebayo J. Gilmer J. Muelly M. Goodfellow I. Hardt M. Kim B. (2018 ). Sanity checks for saliency maps, in Advances in Neural Information Processing Systems, eds Bengio S. Wallach H. Larochelle H. Grauman K. Cesa-Bianchi N. Garnett R. (Montréal, QC: Curran Associates, Inc.), 9505–9515.

   Not in our library

3. 3.
   On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation · 2015

   No notice
4. 4.
   The Scalable Brain Atlas: Instant Web-Based Access to Public Brain Atlases and Related Content · 2015

   No notice
5. 5.

   Binder A. Bach S. Montavon G. Müller K.-R. Samek W. (2016a ). Layer-wise relevance propagation for deep neural network architectures, in Information Science and Applications (ICISA) 2016, eds Kim K. J. Joukov N. (Singapore: Springer Singapore), 913–922.

   Not in our library

6. 6.

   Binder A. Montavon G. Lapuschkin S. Müller K.-R. Samek W. (2016b ). Layer-wise relevance propagation for neural networks with local renormalization layers, in International Conference on Artificial Neural Networks, eds Villa A. E. P. Masulli P. Rivero A. J. P. (Barcelona: Springer), 63–71.

   Not in our library

7. 7.

   Bondi M. W. Edmonds E. C. Salmon D. P. (2017 ). Alzheimer's disease: past, present, and future. J. Int. Neuropsychol. Soc. 23, 818–831.

   Not in our library10.1017/s135561771700100x

8. 8.

   Braak H. Braak E. (1991 ). Neuropathological stageing of Alzheimer-related changes. Acta Neuropathol. 82, 239–259.

   Not in our library10.1007/bf00308809

9. 9.
   High dimensional classification of structural MRI Alzheimer’s disease data based on large scale regularization · 2011

   Not checked
10. 10.

    Castelvecchi D. (2016 ). Can we open the black box of AI? Nature 538, 20–23.

    Not in our library10.1038/538020a

11. 11.

    Desikan R. S. Cabral H. J. Hess C. P. Dillon W. P. Glastonbury C. M. Weiner M. W.. (2009 ). Automated MRI measures identify individuals with mild cognitive impairment and Alzheimer's disease. Brain 132, 2048–2057.

    Not in our library10.1093/brain/awp123

12. 12.

    Ding Y. Sohn J. H. Kawczynski M. G. Trivedi H. Harnish R. Jenkins N. W. (2019 ). A deep learning model to predict a diagnosis of Alzheimer disease by using 18 F-FDG PET of the brain. Radiology 290, 456–464.

    Not in our library10.1148/radiol.2018180958

13. 13.

    Du A. T. Schuff N. Amend D. Laakso M. P. Hsu Y. Y. Jagust W. J.. (2001 ). Magnetic resonance imaging of the entorhinal cortex and hippocampus in mild cognitive impairment and Alzheimer's disease. J. Neurol. Neurosurg. Psychiat. 71, 441–447.

    Not in our library10.1136/jnnp.71.4.441

14. 14.

    Esmaeilzadeh S. Belivanis D. I. Pohl K. M. Adeli E. (2018 ). End to-end Alzheimer's disease diagnosis and biomarker identification, in International Workshop on Machine Learning in Medical Imaging, eds ShiHeung-Il Y. Liu S. (Granada: Springer), 337–345.

    Not in our library10.1007/978-3-030-00919-9_39

15. 15.
    Distinct subtypes of Alzheimer’s disease based on patterns of brain atrophy: longitudinal trajectories and clinical applications · 2017

    No notice
16. 16.

    Frisoni G. B. Fox N. C. Jack C. R. Jr Scheltens P. Thompson P. M. (2010 ). The clinical use of structural MRI in Alzheimer disease. Nat. Rev. Neurol. 6, 67–77.

    Not in our library10.1038/nrneurol.2009.215

17. 17.

    Gupta A. Ayhan M. Maida A. (2013 ). Natural image bases to represent neuroimaging data, in Proceedings of the 30th International Conference on Machine Learning (ICML-13), Vol 28, eds Dasgupta S. Mcallester D. (Atlanta, GA: JMLR Workshop and Conference Proceedings), 987–994.

    Not in our library

18. 18.

    Hidalgo-Muñoz A. R. Ramírez J. Górriz J. M. Padilla P. (2014 ). Regions of interest computed by SVM wrapped method for Alzheimer's disease examination from segmented MRI. Front. Aging Neurosci. 6:20.

    Not in our library10.3389/fnagi.2014.00020

19. 19.

    Kilimann I. Hausner L. Fellgiebel A. Filippi M. Würdemann T. J. Heinsen H.. (2017 ). Parallel atrophy of cortex and basal forebrain cholinergic system in mild cognitive impairment. Cereb. Cortex 27, 1841–1848.

    Not in our library10.1093/cercor/bhw019

20. 20.

    Kingma D. P. Ba J. (2015 ). Adam: A Method for Stochastic Optimization. San Diego, CA: ICLR.

    Not in our library

21. 21.

    Klein-Koerkamp Y. Heckemann R. Ramdeen K. Moreaud O. Keignart S. Krainik A.. (2014 ). Amygdalar atrophy in early Alzheimer's disease. Curr. Alzheimer Res. 11, 239–252.

    Not in our library10.2174/1567205011666140131123653

22. 22.

    Klöppel S. Stonnington C. M. Chu C. Draganski B. Scahill R. I. Rohrer J. D.. (2008 ). Automatic classification of MR scans in Alzheimer's disease. Brain 131, 681–689.

    Not in our library10.1093/brain/awm319

23. 23.

    Korolev S. Safiullin A. Belyaev M. Dodonova Y. (2017 ). Residual and plain convolutional neural networks for 3d brain mri classification, in 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017) (Melbourne, VIC: IEEE), 835–838.

    Not in our library

24. 24.

    Krizhevsky A. Sutskever I. Hinton G. E. (2012 ). ImageNet classification with deep convolutional neural networks, in Advances in Neural Information Processing Systems 25, eds Pereira F. Burges C. J. C. Bottou L. Weinberger K. Q. (Lake Tahoe, CA: Curran Associates, Inc), 1097–1105.

    Not in our library

25. 25.

    Lapuschkin S. Binder A. Montavon G. Muller K.-R. Samek W. (2016 ). Analyzing classifiers: fisher vectors and deep neural networks, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (Las Vegas, CA), 2912–2920.

    Not in our library

26. 26.

    LeCun Y. Bengio Y. (1995 ). Convolutional networks for images, speech, and time series, in The Handbook of Brain Theory and Neural Networks, ed Arbib M. A. (MIT Press), 3361.

    Not in our library

27. 27.
    Deep learning · 2015

    No notice
28. 28.

    Ledig C. Schuh A. Guerrero R. Heckemann R. A. Rueckert D. (2018 ). Structural brain imaging in Alzheimer's disease and mild cognitive impairment: biomarker analysis and shared morphometry database. Sci. Rep. 8:11258.

    Not in our library10.1038/s41598-018-29295-9

29. 29.

    Lipton Z. C. (2018 ). The mythos of model interpretability, in Proceedings of the 2016 ICML Workshop on Human Interpretability in Machine Learning, eds Kim B. Malioutov D. M. Varshney K. R. (New York, NY), 96–100.

    Not in our library

30. 30.

    Litjens G. Kooi T. Bejnordi B. E. Setio A. A. A. Ciompi F. Ghafoorian M.. (2017 ). A survey on deep learning in medical image analysis. Med. Image Analys. 42, 60–88.

    Not in our library10.1016/j.media.2017.07.005

31. 31.

    Liu M. Cheng D. Wang K. Wang Y. (2018 ). Multi-modality cascaded convolutional neural networks for Alzheimer's disease diagnosis. Neuroinformatics 16, 295–308.

    Not in our library10.1007/s12021-018-9370-4

32. 32.

    Long J. Shelhamer E. Darrell T. (2015 ). Fully convolutional networks for semantic segmentation, in Proceedings of the IEEE conference on computer vision and pattern recognition (Boston, MA), 3431–3440.

    Not in our library10.1109/tpami.2016.2572683

33. 33.

    Long X. Chen L. Jiang C. Zhang L. Initiative A. D. N.. (2017 ). Prediction and classification of Alzheimer disease based on quantification of MRI deformation. PLoS ONE 12:e0173372.

    Not in our library10.1371/journal.pone.0173372

34. 34.
    Multimodal and Multiscale Deep Neural Networks for the Early Diagnosis of Alzheimer’s Disease using structural MR and FDG-PET images · 2018

    No notice
35. 35.

    Montavon G. Samek W. Müller K. R. (2018 ). Methods for interpreting and understanding deep neural networks. Digital Signal Proc. 73, 1–15.

    Not in our library10.1016/j.dsp.2017.10.011

36. 36.

    Morris J. C. (1993 ). The Clinical Dementia Rating (CDR). Neurology 43, 2412–2412–a.

    Not in our library10.1212/wnl.43.11.2412-a

37. 37.

    Murray M. E. Graff-Radford N. R. Ross O. A. Petersen R. C. Duara R. Dickson D. W. (2011 ). Neuropathologically defined subtypes of Alzheimer's disease with distinct clinical characteristics: a retrospective study. Lancet Neurol. 10, 785–796.

    Not in our library10.1016/s1474-4422(11)70156-9

38. 38.

    Noh Y. Jeon S. Lee J. M. Seo S. W. Kim G. H. Cho H. Ye B. S.. (2014 ). Anatomical heterogeneity of Alzheimer disease based on cortical thickness on MRIs. Neurology 83, 1936–1944.

    Not in our library10.1212/wnl.0000000000001003

39. 39.

    Orrù G. Pettersson-Yeo W. Marquand A. F. Sartori G. Mechelli A. (2012 ). Using support vector machine to identify imaging biomarkers of neurological and psychiatric disease: a critical review. Neurosci. Biobehav. Rev. 36, 1140–1152.

    Not in our library10.1016/j.neubiorev.2012.01.004

40. 40.

    Park J.-Y. Na H. K. Kim S. Kim H. Kim H. J. Seo S. W.. (2017 ). Robust identification of Alzheimer's disease subtypes based on cortical atrophy patterns. Sci. Rep. 7:43270.

    Not in our library10.1038/srep43270

41. 41.

    Payan A. Montana G. (2015 ). Predicting Alzheimer's disease: a neuroimaging study with 3D convolutional neural networks. CoRR abs/1502.0.

    Not in our library

42. 42.

    Quiroz Y. T. Stern C. E. Reiman E. M. Brickhouse M. Ruiz A. Sperling R. A.. (2013 ). Cortical atrophy in presymptomatic Alzheimer's disease presenilin 1 mutation carriers. J. Neurol. Neurosurg. Psychiat. 84, 556–561.

    Not in our library10.1136/jnnp-2012-303299

43. 43.

    Rathore S. Habes M. Iftikhar M. A. Shacklett A. Davatzikos C. (2017 ). A review on neuroimaging-based classification studies and associated feature extraction methods for Alzheimer's disease and its prodromal stages. NeuroImage 155, 530–548.

    Not in our library10.1016/j.neuroimage.2017.03.057

44. 44.

    Rieke J. Eitel F. Weygandt M. Haynes J.-D. Ritter K. (2018 ). Visualizing convolutional networks for MRI-based diagnosis of Alzheimer's disease. in Understanding and Interpreting Machine Learning in Medical Image Computing Applications (Granada: Springer), 24–31.

    Not in our library

45. 45.

    Ritter K. Lange C. Weygandt M. Mäurer A. Roberts A. Estrella M.. (2016 ). Combination of structural MRI and FDG-PET of the brain improves diagnostic accuracy in newly manifested cognitive impairment in geriatric inpatients. J. Alzheimer's Dis. 54, 1319–1331.

    Not in our library10.3233/jad-160380

46. 46.

    Ritter K. Schumacher J. Weygandt M. Buchert R. Allefeld C. Haynes J.-D. (2015 ). Multimodal prediction of conversion to Alzheimer's disease based on incomplete biomarkers. Alzheimer's Dement. Diagnos. Assess. Dis. Monitor. 1, 206–215.

    Not in our library10.1016/j.dadm.2015.01.006

47. 47.

    Samek W. Binder A. Montavon G. Bach S. Müller K.-R. (2015 ). Evaluating the visualization of what a deep neural network has learned. arXiv arXiv:1509.06321.

    Not in our library10.1109/tnnls.2016.2599820

48. 48.

    Sarraf S. Tofighi G. (2016 ). DeepAD: Alzheimer's disease classification via deep convolutional neural networks using MRI and fMRI. bioRxiv.

    Not in our library

49. 49.

    Scheltens N. M. Galindo-Garre F. Pijnenburg Y. A. van der Vlies A. E. Smits L. L. Koene T.. (2016 ). The identification of cognitive subtypes in Alzheimer's disease dementia using latent class analysis. J. Neurol. Neurosurg. Psychiatry 87, 235–243.

    Not in our library10.1136/jnnp-2014-309582

50. 50.

    Simonyan K. Vedaldi A. Zisserman A. (2013 ). Deep inside convolutional networks: visualising image classification models and saliency maps. arXiv arXiv:1312.6034.

    Not in our library

51. 51.

    Springenberg J. T. Dosovitskiy A. Brox T. Riedmiller M. (2014 ). Striving for simplicity: the all convolutional net. arXiv arXiv:1412.6806.

    Not in our library

52. 52.

    Sturm I. Bach S. Samek W. Müller K.-R. (2016 ). Interpretable deep neural networks for single-trial EEG classification. CoRR abs/1604.0.

    Not in our library10.1016/j.jneumeth.2016.10.008

53. 53.

    Suk H.-I. Lee S.-W. Shen D. (2014 ). Hierarchical feature representation and multimodal fusion with deep learning for AD/MCI diagnosis. NeuroImage 101, 569–582.

    Not in our library10.1016/j.neuroimage.2014.06.077

54. 54.

    Thomas A. W. Heekeren H. R. Müller K.-R. Samek W. (2018 ). Interpretable LSTMs for whole-brain neuroimaging analyses. arXiv arXiv:1810.09945.

    Not in our library

55. 55.

    Velayudhan L. Proitsi P. Westman E. Muehlboeck J.-S. Mecocci P. Vellas B.. (2013 ). Entorhinal cortex thickness predicts cognitive decline in Alzheimer's disease. J. Alzheimer's Dis. 33, 755–766.

    Not in our library10.3233/jad-2012-121408

56. 56.

    Vieira S. Pinaya W. H. Mechelli A. (2017 ). Using deep learning to investigate the neuroimaging correlates of psychiatric and neurological disorders: methods and applications. Neurosci. Biobehav. Rev. 74, 58–75.

    Not in our library10.1016/j.neubiorev.2017.01.002

57. 57.

    Weiner M. W. Veitch D. P. Aisen P. S. Beckett L. A. Cairns N. J. Green R. C.. (2013 ). The Alzheimer's disease neuroimaging initiative: a review of papers published since its inception. Alzheimer's Dement. 9, e111–e194.

    Not in our library10.1016/j.jalz.2013.05.1769

58. 58.

    WHO (2017 ). Dementia. Available online at: https://www.who.int/news-room/fact-sheets/detail/dementia (accessed July 21, 2019).

    Not in our library

59. 59.

    Yang C. Rangarajan A. Ranka S. (2018 ). Visual explanations from deep 3D convolutional neural networks for Alzheimer's disease classification. arXiv arXiv:1803.02544.

    Not in our library

60. 60.

    Yosinski J. Clune J. Nguyen A. Fuchs T. Lipson H. (2015 ). Understanding neural networks through deep visualization. arXiv arXiv:1506.06579.

    Not in our library

61. 61.

    Zeiler M. Fergus R. (2014 ). Visualizing and understanding convolutional networks. in Computer Vision–ECCV 2014, volume 8689 of Lecture Notes in Computer Science, eds Fleet D. Pajdla T. Schiele B. Tuytelaars T. (Zürich: Springer International Publishing), 818–833.

    Not in our library

62. 62.

    Zhang X. Mormino E. C. Sun N. Sperling R. A. Sabuncu M. R. Yeo B. T. T.. (2016 ). Bayesian model reveals latent atrophy factors with dissociable cognitive trajectories in Alzheimer's disease. Proc. Natl. Acad. Sci. U. S. A. 113, E6535–E6544.

    Not in our library10.1073/pnas.1611073113

Cited by papers we hold 1

• DeepAtrophy: Teaching a neural network to detect progressive changes in longitudinal MRI of the hippocampal region in Alzheimer's disease · 2021
  Not checked

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.