SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retraining

Abstract

Despite advances in data augmentation and transfer learning, convolutional neural networks (CNNs) difficultly generalise to unseen domains. When segmenting brain scans, CNNs are highly sensitive to changes in resolution and contrast: even within the same MRI modality, performance can decrease across datasets. Here we introduce SynthSeg, the first segmentation CNN robust against changes in contrast and resolution. SynthSeg is trained with synthetic data sampled from a generative model conditioned on segmentations. Crucially, we adopt a domain randomisation strategy where we fully randomise the contrast and resolution of the synthetic training data. Consequently, SynthSeg can segment real scans from a wide range of target domains without retraining or fine-tuning, which enables straightforward analysis of huge amounts of heterogeneous clinical data. Because SynthSeg only requires segmentations to be trained (no images), it can learn from labels obtained by automated methods on diverse populations (e.g., ageing and diseased), thus achieving robustness to a wide range of morphological variability. We demonstrate SynthSeg on 5,000 scans of six modalities (including CT) and ten resolutions, where it exhibits unparallelled generalisation compared with supervised CNNs, state-of-the-art domain adaptation, and Bayesian segmentation. Finally, we demonstrate the generalisability of SynthSeg by applying it to cardiac MRI and CT scans.

Full text

Its licence does not allow us to show the full text. Read it at PubMed Central:

References 76

1. 1.

   Abadi M, Barham P, Chen J, Chen Z, Davis A, 2016. Tensorflow: A system for large-scale machine learning. In: Symposium on Operating Systems Design and Implementation. pp. 265–283.

   Not in our library

2. 2.

   Arsigny V, Commowick O, Pennec X, Ayache N, 2006. A log-Euclidean framework for statistics on diffeomorphisms. In: Medical Image Computing and Computer Assisted Intervention. pp. 924–931.

   Not in our library10.1007/11866565_113

3. 3.

   Ashburner J, Friston K, 2005. Unified segmentation. NeuroImage 26 (3), 839–851.

   Not in our library10.1016/j.neuroimage.2005.02.018

4. 4.

   Bengio Y, Bastien F, Bergeron A, Boulanger-Lewandowski N, Breuel T, Chherawala Y, Cisse M,, 2011. Deep learners benefit more from out-of-distribution examples. In: International Conference on Artificial Intelligence and Statistics. pp. 164–172.

   Not in our library

5. 5.

   Billot B, Cerri S, Van Leemput K, Dalca A, Iglesias JE, 2021. Joint segmentation of multiple sclerosis lesions and brain anatomy in MRI scans of any contrast and resolution with CNNs. In: IEEE International Symposium on Biomedical Imaging. pp. 1971–1974.

   Not in our library10.1109/isbi48211.2021.9434127

6. 6.

   Billot B, Greve D, Van Leemput K, Fischl B, Iglesias JE, Dalca A, 2020a. A learning strategy for contrast-agnostic MRI segmentation. In: Medical Imaging with Deep Learning. pp. 75–93.

   Not in our library

7. 7.

   Billot B, Robinson E, Dalca A, Iglesias JE, 2020b. Partial volume segmentation of brain MRI scans of any resolution and contrast. In: Medical Image Computing and Computer Assisted Intervention. pp. 177–187.

   Not in our library

8. 8.

   Chaitanya K, Erdil E, Karani N, Konukoglu E, 2020. Contrastive learning of global and local features for medical image segmentation with limited annotations. In: Advances in Neural Information Processing Systems, vol. 33, pp. 12546–12558.

   Not in our library

9. 9.

   Chaitanya K, Karani N, Baumgartner C, Becker A, Donati O, Konukoglu E, 2019. Semi-supervised and task-driven data augmentation. In: Information Processing in Medical Imaging. pp. 29–41.

   Not in our library10.1016/j.media.2020.101934

10. 10.

    Chartsias A, Joyce T, Giuffrida M, Tsaftaris S, 2018. Multimodal MR synthesis via modality-invariant latent representation. IEEE Trans. Med. Imaging 37, 803–814.

    Not in our library10.1109/tmi.2017.2764326

11. 11.

    Chen C, Dou Q, Chen H, Qin J, Heng PA, 2019. Synergistic image and feature adaptation: Towards cross-modality domain adaptation for medical image segmentation. Proc. AAAI Conf. Artif. Intell. 33, 65–72.

    Not in our library

12. 12.

    Chen C, Qin C, Ouyang C, Li Z, Wang S, Qiu H, Chen L, Tarroni G, Bai W, Rueckert D, 2022. Enhancing MR image segmentation with realistic adversarial data augmentation. Med. Image Anal. 82.

    Not in our library10.1016/j.media.2022.102597

13. 13.

    Choi H, Haynor D, Kim Y, 1991. Partial volume tissue classification of multichannel magnetic resonance images-a mixel model. IEEE Trans. Med. Imaging 10, 395–407.

    Not in our library10.1109/42.97590

14. 14.

    Chollet F, 2015. Keras. https://keras.io.

    Not in our library

15. 15.

    Chupin M, Gérardin E, Cuingnet R, Boutet C, Lemieux L, Lehéricy S,, 2009. Fully automatic hippocampus segmentation and classification in Alzheimer’s disease and mild cognitive impairment applied on data from ADNI. Hippocampus 19 (6), 579–587.

    Not in our library10.1002/hipo.20626

16. 16.

    Clevert DA, Unterthiner T, Hochreiter S, 2016. Fast and accurate deep network learning by exponential linear units (ELUs). arXiv:1511.07289 [cs].

    Not in our library

17. 17.

    Cohen J, 1988. Statistical Power Analysis for the Behavioural Sciences. Routledge Academic.

    Not in our library

18. 18.

    Dagley A, LaPoint M, Huijbers W, Hedden T, McLaren D,, 2017. Harvard aging brain study: Dataset and accessibility. NeuroImage 144, 255–258.

    Not in our library10.1016/j.neuroimage.2015.03.069

19. 19.

    Dempster A, Laird N, Rubin DB, 1977. Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. 39 (1), 1–22.

    Not in our library

20. 20.

    Di Martino A, Yan CG, Li Q, Denio E, Castellanos F,, 2014. The autism brain imaging data exchange: Towards large-scale evaluation of the intrinsic brain architecture in autism. Mol. Psychiatry 19 (6), 659–667.

    Not in our library10.1038/mp.2013.78

21. 21.

    Dou Q, Ouyang C, Chen C, Chen H, Glocker B, Zhuang X, Heng PA, 2019. PnP-AdaNet: Plug-and-play adversarial domain adaptation network at unpaired cross-modality cardiac segmentation. IEEE Access 7.

    Not in our library

22. 22.

    Fischl B, 2012. FreeSurfer. NeuroImage 62, 774–781.

    Not in our library10.1016/j.neuroimage.2012.01.021

23. 23.

    Fischl B, Salat D, Busa E, Albert M,, 2002. Whole brain segmentation: Automated labeling of neuroanatomical structures in the human brain. Neuron 33, 41–55.

    Not in our library10.1016/s0896-6273(02)00569-x

24. 24.

    Fischl B, Salat D, van der Kouwe A, Makris N, Ségonne F, Quinn B, Dale A, 2004. Sequence-independent segmentation of magnetic resonance images. NeuroImage 23, 69–84.

    Not in our library10.1016/j.neuroimage.2004.07.016

25. 25.

    Frid-Adar M, Klang E, Amitai M, Goldberger J, Greenspan H, 2018. Synthetic data augmentation using GAN for improved liver lesion classification. In: IEEE International Symposium on Biomedical Imaging. pp. 289–293.

    Not in our library

26. 26.

    Ganin Y, Ustinova E, Ajakan H, Germain P, Larochelle H, Laviolette F, Marchand M, Lempitsky V, 2017. Domain-adversarial training of neural networks. In: Domain Adaptation in Computer Vision Applications. In: Advances in Computer Vision and Pattern Recognition, pp. 189–209.

    Not in our library

27. 27.

    Ghafoorian M, Mehrtash A, Kapur T, Karssemeijer N, Marchiori E, Pesteie M, Fedorov A, Abolmaesumi P, Platel B, Wells W, 2017. Transfer learning for domain adaptation in MRI: Application in brain lesion segmentation. In: Medical Image Computing and Computer Assisted Intervention. pp. 516–524.

    Not in our library

28. 28.

    Gollub R, Shoemaker J, King M, White T, Ehrlich S,, 2013. The MCIC collection: A shared repository of multi-modal, multi-site brain image data from a clinical investigation of schizophrenia. Neuroinformatics 11 (3), 367–388.

    Not in our library10.1007/s12021-013-9184-3

29. 29.

    Havaei M, Guizard N, Chapados N, Bengio Y, 2016. HeMIS: Hetero-modal image segmentation. In: Medical Image Computing and Computer-Assisted Intervention. pp. 469–477.

    Not in our library

30. 30.

    He Y, Carass A, Zuo L, Dewey B, Prince J, 2021. Autoencoder based self-supervised test-time adaptation for medical image analysis. Med. Image Anal. 102136.

    Not in our library10.1016/j.media.2021.102136

31. 31.

    Hoffman J, Tzeng E, Park T, Zhu JY, Isola P, Saenko K, Efros A, Darrell T, 2018. CyCADA: Cycle-consistent adversarial domain adaptation. In: International Conference on Machine Learning. pp. 1989–1998.

    Not in our library

32. 32.
    Brain Genomics Superstruct Project initial data release with structural, functional, and behavioral measures · 2015

    No notice
33. 33.

    Huo Y, Xu Z, Moon H, Bao S, Assad A, Moyo T, Savona M, Abramson R, Landman B, 2019. SynSeg-Net: Synthetic segmentation without target modality ground truth. IEEE Trans. Med. Imaging 38 (4), 1016–1025.

    Not in our library10.1109/tmi.2018.2876633

34. 34.

    Hynd G, Semrud-Clikeman M, Lorys A, Novey E, Eliopulos D, Lyytinen H, 1991. Corpus callosum morphology in attention deficit-hyperactivity disorder: Morphometric analysis of MRI. J. Learn. Disabil. 24 (3), 141–146.

    Not in our library10.1177/002221949102400302

35. 35.

    Iglesias JE, Billot B, Balbastre Y, Tabari A, Conklin J, Gilberto González R, Alexander DC, Golland P, Edlow BL, Fischl B, 2021. Joint super-resolution and synthesis of 1 mm isotropic MP-RAGE volumes from clinical MRI exams with scans of different orientation, resolution and contrast. NeuroImage 237.

    Not in our library10.1016/j.neuroimage.2021.118206

36. 36.
    A probabilistic atlas of the human thalamic nuclei combining ex vivo MRI and histology · 2018

    No notice
37. 37.

    Ioffe S, Szegedy C, 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning. pp. 448–456.

    Not in our library

38. 38.

    Isensee F, Jaeger P, Kohl S, Petersen J, Maier-Hein K, 2021. nnU-Net: A self-configuring method for deep learning-based biomedical image segmentation. Nature Methods 18 (2), 203–211.

    Not in our library10.1038/s41592-020-01008-z

39. 39.

    Isola P, Zhu J, Zhou T, Efros A, 2017. Image-to-image translation with conditional adversarial networks. In: IEEE Conference on Computer Vision and Pattern Recognition. pp. 1125–1134.

    Not in our library

40. 40.

    Jakobi N, Husbands P, Harvey I, 1995. Noise and the reality gap: The use of simulation in evolutionary robotics. In: Advances in Artificial Life. pp. 704–720.

    Not in our library

Cited by papers we hold 1

• A human brain network linked to restoration of consciousness after deep brain stimulation · 2025
  Not checked

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.