Multimodal and Multiscale Deep Neural Networks for the Early Diagnosis of Alzheimer’s Disease using structural MR and FDG-PET images

Abstract

Alzheimer’s Disease (AD) is a progressive neurodegenerative disease where biomarkers for disease based on pathophysiology may be able to provide objective measures for disease diagnosis and staging. Neuroimaging scans acquired from MRI and metabolism images obtained by FDG-PET provide in-vivo measurements of structure and function (glucose metabolism) in a living brain. It is hypothesized that combining multiple different image modalities providing complementary information could help improve early diagnosis of AD. In this paper, we propose a novel deep-learning-based framework to discriminate individuals with AD utilizing a multimodal and multiscale deep neural network. Our method delivers 82.4% accuracy in identifying the individuals with mild cognitive impairment (MCI) who will convert to AD at 3 years prior to conversion (86.4% combined accuracy for conversion within 1–3 years), a 94.23% sensitivity in classifying individuals with clinical diagnosis of probable AD, and a 86.3% specificity in classifying non-demented controls improving upon results in published literature.

Full text

Its licence does not allow us to show the full text. Read it at PubMed Central:

References 47

1. 1.

   Association A, et al. Alzheimer’s disease facts and figures. Alzheimer’s & dementia: journal Alzheimer’s Assoc. 2011;7:208.

   Not in our library10.1016/j.jalz.2011.02.004 (opens in a new tab)

2. 2.

   Petersen RC, et al. Mild cognitive impairment: ten years later. Arch. neurology. 2009;66:1447-1455.

   Not in our library10.1001/archneurol.2009.266 (opens in a new tab)

3. 3.

   Brookmeyer R, Johnson E, Ziegler-Graham K, Arrighi HM. Forecasting the global burden of Alzheimer’s disease. Alzheimer’s & dementia. 2007;3:186-191.

   Not in our library10.1016/j.jalz.2007.04.381 (opens in a new tab)

4. 4.

   Davatzikos C, Bhatt P, Shaw LM, Batmanghelich KN, Trojanowski JQ. Prediction of MCI to AD conversion, via MRI, CSF biomarkers, and pattern classification. Neurobiol. aging. 2011;32:2322-e19.

   Not in our library10.1016/j.neurobiolaging.2010.05.023 (opens in a new tab)

5. 5.

   Landau SM, et al. Associations between cognitive, functional, and FDG-PET measures of decline in AD and MCI. Neurobiol. aging. 2011;32:1207-1218.

   Not in our library10.1016/j.neurobiolaging.2009.07.002 (opens in a new tab)

6. 6.

   Farhan, S., Fahiem, M. A. & Tauseef, H. An ensemble-of-classifiers based approach for early diagnosis of Alzheimer’s disease: Classification using structural features of brain images. Comput. and mathematical methods medicine2014 (2014).

   Not in our library10.1155/2014/862307 (opens in a new tab)

7. 7.

   Korolev, S., Safiullin, A., Belyaev, M. & Dodonova, Y. Residual and Plain Convolutional Neural Networks for 3D Brain MRI Classification. arXiv preprint arXiv:1701.06643 (2017).

   Not in our library

8. 8.

   Payan, A. & Montana, G. Predicting Alzheimer’s disease: a neuroimaging study with 3D convolutional neural networks. arXiv preprint arXiv:1502.02506 (2015).

   Not in our library

9. 9.

   Eskildsen SF, et al. Prediction of Alzheimer’s disease in subjects with mild cognitive impairment from the ADNI cohort using patterns of cortical thinning. Neuroimage. 2013;65:511-521.

   Not in our library10.1016/j.neuroimage.2012.09.058 (opens in a new tab)

10. 10.

    Misra C, Fan Y, Davatzikos C. Baseline and longitudinal patterns of brain atrophy in MCI patients, and their use in prediction of short-term conversion to AD: results from ADNI. Neuroimage. 2009;44:1415-1422.

    Not in our library10.1016/j.neuroimage.2008.10.031 (opens in a new tab)

11. 11.
    Multi-Method Analysis of MRI Images in Early Diagnostics of Alzheimer's Disease · 2011

    No notice
12. 12.

    Cuingnet R, et al. Automatic classification of patients with Alzheimer’s disease from structural MRI: a comparison of ten methods using the ADNI database. Neuroimage. 2011;56:766-781.

    Not in our library10.1016/j.neuroimage.2010.06.013 (opens in a new tab)

13. 13.

    Cho Y, et al. Individual subject classification for Alzheimer’s disease based on incremental learning using a spatial frequency representation of cortical thickness data. Neuroimage. 2012;59:2217-2230.

    Not in our library10.1016/j.neuroimage.2011.09.085 (opens in a new tab)

14. 14.

    Mosconi L, et al. Pre-clinical detection of Alzheimer’s disease using FDG-PET, with or without amyloid imaging. J. Alzheimer’s Dis. 2010;20:843-854.

    Not in our library10.3233/jad-2010-091504 (opens in a new tab)

15. 15.

    Gray KR, et al. Multi-region analysis of longitudinal FDG-PET for the classification of Alzheimer’s disease. NeuroImage. 2012;60:221-229.

    Not in our library10.1016/j.neuroimage.2011.12.071 (opens in a new tab)

16. 16.

    Toussaint P-J, et al. Resting state FDG-PET functional connectivity as an early biomarker of Alzheimer’s disease using conjoint univariate and independent component analyses. Neuroimage. 2012;63:936-946.

    Not in our library10.1016/j.neuroimage.2012.03.091 (opens in a new tab)

17. 17.

    Illán I, et al. 18 F-FDG PET imaging analysis for computer aided Alzheimer’s diagnosis. Inf. Sci. 2011;181:903-916.

    Not in our library10.1016/j.ins.2010.10.027 (opens in a new tab)

18. 18.

    Young J, et al. Accurate multimodal probabilistic prediction of conversion to Alzheimer’s disease in patients with mild cognitive impairment. NeuroImage: Clin. 2013;2:735-745.

    Not in our library10.1016/j.nicl.2013.05.004 (opens in a new tab)

19. 19.

    Zhang D, et al. Multimodal classification of Alzheimer’s disease and mild cognitive impairment. Neuroimage. 2011;55:856-867.

    Not in our library10.1016/j.neuroimage.2011.01.008 (opens in a new tab)

20. 20.

    Moradi E, et al. Machine learning framework for early MRI-based Alzheimer’s conversion prediction in MCI subjects. Neuroimage. 2015;104:398-412.

    Not in our library10.1016/j.neuroimage.2014.10.002 (opens in a new tab)

21. 21.
    Predicting Progression from Mild Cognitive Impairment to Alzheimer's Dementia Using Clinical, MRI, and Plasma Biomarkers via Probabilistic Pattern Classification · 2016

    No notice
22. 22.

    Ye J, et al. Sparse learning and stability selection for predicting MCI to AD conversion using baseline ADNI data. BMC Neurol. 2012;12:46.

    Not in our library10.1186/1471-2377-12-46 (opens in a new tab)

23. 23.
    BrainAGE in Mild Cognitive Impaired Patients: Predicting the Conversion to Alzheimer’s Disease · 2013

    No notice
24. 24.

    Zhang D, et al. Multi-modal multi-task learning for joint prediction of multiple regression and classification variables in Alzheimer’s disease. NeuroImage. 2012;59:895-907.

    Not in our library10.1016/j.neuroimage.2011.09.069 (opens in a new tab)

25. 25.

    Krizhevsky, A., Sutskever, I. & Hinton, G. E. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, 1097–1105 (2012).

    Not in our library

26. 26.

    Liu S, et al. Multimodal neuroimaging feature learning for multiclass diagnosis of Alzheimer’s disease. IEEE Transactions on Biomed. Eng. 2015;62:1132-1140.

    Not in our library10.1109/tbme.2014.2372011 (opens in a new tab)

27. 27.

    Liu, S. et al. Early diagnosis of Alzheimer’s disease with deep learning. In Biomedical Imaging (ISBI), 2014 IEEE 11th International Symposium on, 1015–1018 (IEEE, 2014).

    Not in our library

28. 28.

    Suk H-I, et al. Hierarchical feature representation and multimodal fusion with deep learning for AD/MCI diagnosis. NeuroImage. 2014;101:569-582.

    Not in our library10.1016/j.neuroimage.2014.06.077 (opens in a new tab)

29. 29.

    Zhang, W., Zelinsky, G. & Samaras, D. Real-time accurate object detection using multiple resolutions. In Computer Vision, 2007. ICCV 2007. IEEE 11th International Conference on, 1–8 (IEEE, 2007).

    Not in our library

30. 30.

    Lowe DG. Distinctive image features from scale-invariant keypoints. Int. journal computer vision. 2004;60:91-110.

    Not in our library10.1023/b:visi.0000029664.99615.94 (opens in a new tab)

31. 31.

    Tang, Y. & Mohamed, A.-R. Multiresolution Deep Belief Networks. In AISTATS, 1203–1211 (2012).

    Not in our library

32. 32.

    Dale AM, S. M, Fischl B. Cortical surface-based analysis. II: Inflation, flattening, and a surface-based coordinate system. Neuroimage. 1999;9(2):195-207.

    Not in our library10.1006/nimg.1998.0395 (opens in a new tab)

33. 33.

    Raamana PR, et al. Thickness network features for prognostic applications in dementia. Neurobiol. aging. 2015;36:S91-S102.

    Not in our library10.1016/j.neurobiolaging.2014.05.040 (opens in a new tab)

34. 34.

    Beg F, Miller M, Trouvé A, Younes L. Computing large deformation metric mappings via geodesic flows of diffeomorphisms. Int. journal computer vision. 2005;61(2):139-157.

    Not in our library10.1023/b:visi.0000043755.93987.aa (opens in a new tab)

35. 35.

    Jenkinson M, Bannister P, Brady M, Smith S. Improved optimization for the robust and accurate linear registration and motion correction of brain images. Neuroimage. 2002;17:825-841.

    Not in our library10.1006/nimg.2002.1132 (opens in a new tab)

36. 36.

    Bengio Y, et al. Greedy layer-wise training of deep networks. Adv. neural information processing systems. 2007;19:153.

    Not in our library

37. 37.

    Kingma, D. & Ba, J. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).

    Not in our library

38. 38.

    Bengio, Y. Practical recommendations for gradient-based training of deep architectures. In Neural networks: Tricks of the trade, 437–478 (Springer, 2012).

    Not in our library

39. 39.

    Srivastava N, Hinton GE, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 2014;15:1929-1958.

    Not in our library

40. 40.

    Liu, K., Chen, K., Yao, L. & Guo, X. Prediction of Mild Cognitive Impairment Conversion Using a Combination of Independent Component Analysis and the Cox Model. Front. human neuroscience11 (2017).

    Not in our library10.3389/fnhum.2017.00033 (opens in a new tab)

41. 41.

    Cheng B, Liu M, Zhang D, Munsell BC, Shen D. Domain transfer learning for MCI conversion prediction. IEEE Transactions on Biomed. Eng. 2015;62:1805-1817.

    Not in our library10.1109/tbme.2015.2404809 (opens in a new tab)

42. 42.

    Zhu, X. et al. A novel relational regularization feature selection method for joint regression and classification in AD diagnosis. Med. image analysis (2017).

    Not in our library10.1016/j.media.2015.10.008 (opens in a new tab)

43. 43.

    Xu L, Wu X, Chen K, Yao L. Multi-modality sparse representation-based classification for Alzheimer’s disease and mild cognitive impairment. Comput. methods programs biomedicine. 2015;122:182-190.

    Not in our library10.1016/j.cmpb.2015.08.004 (opens in a new tab)

44. 44.
    Predicting Future Clinical Changes of MCI Patients Using Longitudinal and Multimodal Biomarkers · 2012

    No notice
45. 45.

    An, L. et al. A Hierarchical Feature and Sample Selection Framework and Its Application for Alzheimer’s Disease Diagnosis. Sci. Reports7 (2017).

    Not in our library10.1038/srep45269 (opens in a new tab)

46. 46.

    Huang, M. et al. Longitudinal measurement and hierarchical classification framework for the prediction of Alzheimer’s disease. Sci. reports7 (2017).

    Not in our library10.1038/srep39880 (opens in a new tab)

47. 47.

    Abadi, M. et al. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. http://tensorflow.org/Software availablefromtensor flow.org (2015).

    Not in our library

Cited by papers we hold 12

• Multimodal deep learning for Alzheimer’s disease dementia assessment · 2022
  No notice
• Predict Alzheimer’s disease using hippocampus MRI data: a lightweight 3D deep convolutional network model with visual and global shape representations · 2021
  Not checked
• A multilayer multimodal detection and prediction model based on explainable artificial intelligence for Alzheimer’s disease · 2021
  No notice
• Machine learning methods for predicting progression from mild cognitive impairment to Alzheimer’s disease dementia: a systematic review · 2021
  No notice
• TorchIO: A Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning · 2021
  No notice
• Imaging biomarkers in neurodegeneration: current and future practices · 2020
  No notice
• Deep Learning in Alzheimer's Disease: Diagnostic Classification and Prognostic Prediction Using Neuroimaging Data · 2019
  No notice
• Interpretable classification of Alzheimer’s disease pathologies with a convolutional neural network pipeline · 2019
  No notice
• Layer-Wise Relevance Propagation for Explaining Deep Neural Network Decisions in MRI-Based Alzheimer's Disease Classification · 2019
  No notice
• Predicting Alzheimer’s disease progression using multi-modal deep learning approach · 2019
  Corrected
• Multivariate Deep Learning Classification of Alzheimer’s Disease Based on Hierarchical Partner Matching Independent Component Analysis · 2018
  Not checked
• Convolutional Neural Networks-Based MRI Image Analysis for the Alzheimer’s Disease Prediction From Mild Cognitive Impairment · 2018
  No notice

Questions about this paper

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.