FastSurfer - A fast and accurate deep learning based neuroimaging pipeline

Abstract

Traditional neuroimage analysis pipelines involve computationally intensive, time-consuming optimization steps, and thus, do not scale well to large cohort studies with thousands or tens of thousands of individuals. In this work we propose a fast and accurate deep learning based neuroimaging pipeline for the automated processing of structural human brain MRI scans, replicating FreeSurfer's anatomical segmentation including surface reconstruction and cortical parcellation. To this end, we introduce an advanced deep learning architecture capable of whole-brain segmentation into 95 classes. The network architecture incorporates local and global competition via competitive dense blocks and competitive skip pathways, as well as multi-slice information aggregation that specifically tailor network performance towards accurate segmentation of both cortical and subcortical structures. Further, we perform fast cortical surface reconstruction and thickness analysis by introducing a spectral spherical embedding and by directly mapping the cortical labels from the image to the surface. This approach provides a full FreeSurfer alternative for volumetric analysis (in under 1 min) and surface-based thickness analysis (within only around 1 h runtime). For sustainability of this approach we perform extensive validation: we assert high segmentation accuracy on several unseen datasets, measure generalizability and demonstrate increased test-retest reliability, and high sensitivity to group differences in dementia.

References 113

1. 1.
   Thalamic pathology and memory loss in early Alzheimer’s disease: moving the focus from the medial temporal lobe to Papez circuit · 2016

   No notice
2. 2.
   Deep Learning for Brain MRI Segmentation: State of the Art and Future Directions · 2017

   No notice
3. 3.

   Amin J, Sharif M, Yasmin M, Fernandes SL, 2018 Big data analysis for brain tumor detection: deep convolutional neural networks. Future Generat. Comput. Syst 87, 290–297.

   Not in our library

4. 4.

   Arunachalam M, Savarimuthu SR, 2017 An efficient and automatic glioblastoma brain tumor detection using shift-invariant shearlet transform and neural networks. Int. J. Imag. Syst. Technol 27 (3), 216–226.

   Not in our library

5. 5.

   Avants BB, Tustison N, Song G, 2009 Advanced normalization tools (ants). Insight Journal 2, 1–35.

   Not in our library

6. 6.

   Badrinarayanan V, Kendall A, Cipolla R, 2017 Segnet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell 39 (12), 2481–2495.

   Not in our library10.1109/tpami.2016.2644615

7. 7.

   Bank RE, 1990 PLTMG: a software package for solving elliptic partial differential equations: users’ guide 6.0 In: Frontiers in Applied Mathematics, vol. 7 SIAM.

   Not in our library

8. 8.

   Baron J, Chêtelat G, Desgranges B, Perchey G, Landeau B, de la Sayette V, Eustache F, 2001 In vivo mapping of gray matter loss with voxel-based morphometry in mild alzheimer’s disease. Neuroimage 14 (2), 298–309.

   Not in our library10.1006/nimg.2001.0848

9. 9.

   Biller A, Reuter M, Patenaude B, Homola G, Breuer F, Bendszus M, Bartsch A, 2015 Responses of the human brain to mild dehydration and rehydration explored in vivo by 1h-mr imaging and spectroscopy. AJNR. American journal of neuroradiology 36 (12), 2277–2284.

   Not in our library10.3174/ajnr.a4508

10. 10.

    Braak H, Braak E, 1995 Staging of alzheimer’s disease-related neurofibrillary changes. Neurobiol. Aging 16 (3), 271–278 (the Schmitt Symposium: The Cytoskeleton and Alzheimer’s Disease).

    Not in our library10.1016/0197-4580(95)00021-6

11. 11.

    Brainlesion Glioma, 2018 Multiple Sclerosis, Stroke and Traumatic Brain Injuries - 4th International Workshop BrainLes. Held in Conjunction with MICCAI 2018, Granada, Spain, September 16, 2018, Revised Selected Papers, Part I, Vol. 11383 of Lecture Notes in Computer Science, Springer, 2019.

    Not in our library

12. 12.

    Breteler MM, Stöcker T, Pracht E, Brenner D, Stirnberg R, 2014 Mri in the rhineland study: a novel protocol for population neuroimaging, Alzheimer’s & Dementia. J Alzheimer’s Assos. 10 (4), P92.

    Not in our library

13. 13.

    Buckner RL, Head D, Parker J, Fotenos AF, Marcus D, Morris JC, Snyder AZ, 2004 A unified approach for morphometric and functional data analysis in young, old, and demented adults using automated atlas-based head size normalization: reliability and validation against manual measurement of total intracranial volume. Neuroimage 23, 724–738.

    Not in our library10.1016/j.neuroimage.2004.06.018

14. 14.

    Chen X, Konukoglu E, 2018 Unsupervised detection of lesions in brain MRI using constrained adversarial auto-encoders. In: Proceedings of the 1st International Conference on Medical Imaging with Deep Learning MIDL 2018 Amsterdam, Netherlands, 4-6 July 2018.

    Not in our library

15. 15.

    Chen H, Dou Q, Yu L, Qin J, Heng P-A, 2018 Voxresnet: deep voxelwise residual networks for brain segmentation from 3d mr images. Neuroimage 170, 446–455.

    Not in our library10.1016/j.neuroimage.2017.04.041

16. 16.

    Çiçek Ö, Abdulkadir A, Lienkamp SS, Brox T, Ronneberger O, 2016 3d u-net: learning dense volumetric segmentation from sparse annotation. In: Medical Image Computing and Computer-Assisted Intervention - MICCAI 2016 - 19th International Conference, pp. 424–432. Athens, Greece, October 17-21, 2016, Proceedings, Part II, Vol. 9901 of Lecture Notes in Computer Science.

    Not in our library

17. 17.

    Coupé P, Mansencal B, Clément M, Giraud R, de Senneville BD, Ta V-T, Lepetit V, Manjon JV, 2019. Assemblynet: a novel deep decision-making process for whole brain mri segmentation In: Shen D, Liu T, Peters TM, Staib LH, Essert C, Zhou S, Yap P-T, Khan A (Eds.), Medical Image Computing and Computer Assisted Intervention - MICCAI 2019, pp. 466–474.

    Not in our library

18. 18.

    Dale AM, Fischl B, Sereno MI, 1999 Cortical surface-based analysis: I. segmentation and surface reconstruction. Neuroimage 9 (2), 179–194.

    Not in our library10.1006/nimg.1998.0395

19. 19.

    de Brêbisson A, Montana G, 2015 Deep neural networks for anatomical brain segmentation. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops. CVPRW, pp. 20–28.

    Not in our library

20. 20.

    de Jong LW, van der Hiele K, Veer IM, Houwing JJ, Westendorp RGJ, Bollen ELEM, de Bruin PW, Middelkoop HAM, van Buchem MA, van der Grond J, 2008 Strongly reduced volumes of putamen and thalamus in Alzheimer’s disease: an MRI study. Brain 131 (12), 3277–3285.

    Not in our library10.1093/brain/awn278

21. 21.

    Dedmari MA, Conjeti S, Estrada S, Ehses P, Stöcker T, Reuter M, 2018 Complex fully convolutional neural networks for mr image reconstruction In: Knoll F, Maier A, Rueckert D (Eds.), Machine Learning for Medical Image Reconstruction. Springer International Publishing, pp. 30–38.

    Not in our library

22. 22.

    Desikan RS, Sêgonne F, Fischl B, Quinn BT, Dickerson BC, Blacker D, Buckner RL, Dale AM, Maguire RP, Hyman BT, Albert MS, Killiany RJ, 2006 An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest. Neuroimage 31 (3), 968–980.

    Not in our library10.1016/j.neuroimage.2006.01.021

23. 23.
    Enhancing studies of the connectome in autism using the autism brain imaging data exchange II · 2017

    No notice
24. 24.

    Ding J, Li A, Hu Z, Wang L, 2017 Accurate pulmonary nodule detection in computed tomography images using deep convolutional neural networks. In: Medical Image Computing and Computer Assisted Intervention - MICCAI 2017 - 20th International Conference, Quebec City, QC, Canada, September 11-13, 2017, Proceedings, Part III, Vol. 10435 of Lecture Notes in Computer Science. Springer, pp. 559–567.

    Not in our library

25. 25.

    Dolz J, Desrosiers C, Ayed IB, 2018 3d fully convolutional networks for subcortical segmentation in mri: a large-scale study. Neuroimage 170, 456–470.

    Not in our library10.1016/j.neuroimage.2017.04.039

26. 26.

    Dong H, Yang G, Liu F, Mo Y, Guo Y, July. Automatic brain tumor detection and segmentation using u-net based fully convolutional networks. In: Medical Image Understanding and Analysis - 21st Annual Conference. MIUA 2017, Edinburgh, UK, pp. 506–517. Proceedings, 2017.

    Not in our library

27. 27.

    Estrada S, Conjeti S, Ahmad M, Navab N, Reuter M, 2018 Competition vs. concatenation in skip connections of fully convolutional networks In: Machine Learning in Medical Imaging. Springer International Publishing, pp. 214–222.

    Not in our library

28. 28.

    Estrada S, Lu R, Conjeti S, Orozco-Ruiz X, Panos-Willuhn J, Breteler MMB, Reuter M, 2019 Fatsegnet: a fully automated deep learning pipeline for adipose tissue segmentation on abdominal dixon MRI. Magn. Reson. Med 83, 1471–1483.

    Not in our library10.1002/mrm.28022

29. 29.

    Fedorov A, Johnson J, Damaraju E, Ozerin A, Calhoun VD, Plis SM, May. End-to-end learning of brain tissue segmentation from imperfect labeling. In: 2017 International Joint Conference on Neural Networks. IJCNN 2017, Anchorage, AK, USA, pp. 3785–3792. IEEE, 2017.

    Not in our library

30. 30.

    Fischl B, 2012 Freesurfer. Neuroimage 62 (2), 774–781.

    Not in our library10.1016/j.neuroimage.2012.01.021

31. 31.

    Fischl B, Dale A, 2000 Measuring the thickness of the human cerebral cortex from magnetic resonance images. Proc. Natl. Acad. Sci. U. S. A 97 (20), 11050–11055.

    Not in our library10.1073/pnas.200033797

32. 32.

    Fischl B, Sereno MI, Dale AM, 1999 Cortical surface-based analysis: II: Inflation, flattening, and a surface-based coordinate system. Neuroimage 9 (2), 195–207.

    Not in our library10.1006/nimg.1998.0396

33. 33.

    Fischl B, Sereno MI, Tootell R, Dale A, 1999 High-resolution intersubject averaging and a coordinate system for the cortical surface. Hum. Brain Mapp 8, 272–284.

    Not in our library10.1002/(sici)1097-0193(1999)8:4<272::aid-hbm10>3.0.co;2-4

34. 34.

    Fischl B, Liu AK, Dale AM, 2001 Automated manifold surgery: constructing geometrically accurate and topologically correct models of the human cerebral cortex. IEEE Trans. Med. Imag 20 (1), 70–80.

    Not in our library10.1109/42.906426

35. 35.

    Fischl B, Salat DH, Busa E, Albert M, Dieterich M, Haselgrove C, Van Der Kouwe A, Killiany R, Kennedy D, Klaveness S,, 2002 Whole brain segmentation: automated labeling of neuroanatomical structures in the human brain. Neuron 33 (3), 341–355.

    Not in our library10.1016/s0896-6273(02)00569-x

36. 36.

    Fischl B, van der Kouwe A, Destrieux C, Halgren E, Sêgonne F, Salat D, Busa E, Seidman L, Goldstein J, Kennedy D, Caviness V, Makris N, Rosen B, Dale A, 2004 Automatically parcellating the human cerebral cortex. Cerebral cortex (New York, N.Y: 1991 14, 11–22.

    Not in our library10.1093/cercor/bhg087

37. 37.

    Friston K, Ashburner J, Kiebel S, Nichols T, Penny W, 2007 Statistical Parametric Mapping: the Analysis of Functional Brain Images. Academic Press.

    Not in our library

38. 38.

    Ghavami N, Hu Y, Bonmati E, Rodell R, Gibson E, Moore C, Barratt D, 2018 Integration of spatial information in convolutional neural networks for automatic segmentation of intraoperative transrectal ultrasound images. J. Med. Imag 6 (1), 1–6.

    Not in our library10.1117/1.jmi.6.1.011003

39. 39.

    Goodfellow IJ, Warde-Farley D, Mirza M, Courville A, Bengio Y, 2013 Maxout networks. In: Proceedings of the 30th International Conference on International Conference on Machine Learning-Volume, vol. 28 JMLR. org, pp. III–1319.

    Not in our library

40. 40.

    Han X, Jovicich J, Salat D, van der Kouwe A, Quinn B, Czanner S, Busa E, Pacheco J, Albert M, Killiany R, Maguire P, Rosas D, Makris N, Dale A, Dickerson B, Fischl B, 2006 Reliability of mri-derived measurements of human cerebral cortical thickness: the effects of field strength, scanner upgrade and manufacturer. Neuroimage 32 (1), 180–194.

    Not in our library10.1016/j.neuroimage.2006.02.051

Cited by papers we hold 4

• Robust machine learning segmentation for large-scale analysis of heterogeneous clinical brain MRI datasets · 2023
  No notice
• The OpenNeuro resource for sharing of neuroscience data · 2021
  No notice
• Deep learning-based unlearning of dataset bias for MRI harmonisation and confound removal · 2021
  No notice
• The ANTsX ecosystem for quantitative biological and medical imaging · 2021
  No notice

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.