No retraction, correction or concern found when last checked on Oct 19, 2026.

NiftyNet: a deep-learning platform for medical imaging

Eli Gibson, Wenqi Li, Carole Sudre and 14 more

Computer Methods and Programs in Biomedicine · 2018 · CC BY

doi:10.1016/j.cmpb.2018.01.025

Abstract

Background and objectives. Medical image analysis and computer-assisted intervention problems are increasingly being addressed with deep-learning-based solutions. Established deep-learning platforms are flexible but do not provide specific functionality for medical image analysis and adapting them for this domain of application requires substantial implementation effort. Consequently, there has been substantial duplication of effort and incompatible infrastructure developed across many research groups. This work presents the open-source NiftyNet platform for deep learning in medical imaging. The ambition of NiftyNet is to accelerate and simplify the development of these solutions, and to provide a common mechanism for disseminating research outputs for the community to use, adapt and build upon.

Methods. The NiftyNet infrastructure provides a modular deep-learning pipeline for a range of medical imaging applications including segmentation, regression, image generation and representation learning applications. Components of the NiftyNet pipeline including data loading, data augmentation, network architectures, loss functions and evaluation metrics are tailored to, and take advantage of, the idiosyncracies of medical image analysis and computer-assisted intervention. NiftyNet is built on the TensorFlow framework and supports features such as TensorBoard visualization of 2D and 3D images and computational graphs by default.

Results. We present three illustrative medical image analysis applications built using NiftyNet infrastructure: (1) segmentation of multiple abdominal organs from computed tomography; (2) image regression to predict computed tomography attenuation maps from brain magnetic resonance images; and (3) generation of simulated ultrasound images for specified anatomical poses.

Conclusions. The NiftyNet infrastructure enables researchers to rapidly develop and distribute deep learning solutions for segmentation, regression, image generation and representation learning applications, or extend the platform to new applications.

Full text

Its licence does not allow us to show the full text. Read it at PubMed Central:

References 57

1.
Abadi M., Agarwal A., Barham P., Brevdo E., Chen Z., Citro C., Corrado G.S., Davis A., Dean J., Devin M., Ghemawat S., Goodfellow I., Harp A., Irving G., Isard M., Jia Y., Jozefowicz R., Kaiser L., Kudlur M., Levenberg J., Mane D., Monga R., Moore S., Murray D., Olah C., Schuster M., Shlens J., Steiner B., Sutskever I., Talwar K., Tucker P., Vanhoucke V., Vasudevan V., Viegas F., Vinyals O., Warden P., Wattenberg M., Wicke M., Yu Y., Zheng X. TensorFlow: large-scale machine learning on heterogeneous distributed systems. 2016.
Not in our library
2.
Avants B.B., Tustison N.J., Song G., Cook P.A., Klein A., Gee J.C. A reproducible evaluation of ANTs similarity metric performance in brain image registration. Neuroimage. 2011;54(3):2033-2044.
Not in our library10.1016/j.neuroimage.2010.09.025
3.
J.L. Ba, J.R. Kiros, G.E. Hinton, Layer normalization (2016). arXiv:1607.06450v1.
Not in our library
4.
Bastien F., Lamblin P., Pascanu R., Bergstra J., Goodfellow I.J., Bergeron A., Bouchard N., Bengio Y. Theano: new features and speed improvements. Proceedings of the Workshop on Deep Learning and Unsupervised Feature Learning NIPS. 2012.
Not in our library
5.
L. Berger, E. Hyde, M.J. Cardoso, S. Ourselin, An adaptive sampling scheme to efficiently train fully convolutional networks for semantic segmentation, (2017). ArXiv e-prints arXiv:1709.02764.
Not in our library
6.
M. Brett, M. Hanke, B. Cipollini, M.-A. Côté, C. Markiewicz, S. Gerhard, E. Larson, Nibabel, 2016, Online. doi:10.5281/zenodo.60808.
Not in our library
7.
Burgos N., Cardoso M., Thielemans K., Modat M., Pedemonte S., Dickson J., Barnes A., Ahmed R., Mahoney J., Schott J., Duncan J., Atkinson D., Arridge S., Hutton B., Ourselin S. Attenuation correction synthesis for hybrid PET-MR scanners: Application to brain studies. Med. Imaging IEEE Trans. 2014;33(12):2332-2341.
Not in our library10.1109/tmi.2014.2340135
8.
Cardoso M., Clarkson M., Modat M., Ourselin S. NiftySeg: open-source software for medical image segmentation, label fusion and cortical thickness estimation. Proceedings of the ISBI Workshop on Open Source Medical Image Analysis Software. 2012.
Not in our library
9.
S. Chetlur, C. Woolley, P. Vandermersch, J. Cohen, J. Tran, B. Catanzaro, E. Shelhamer, cuDNN: efficient primitives for deep learning, arXiv:1410.0759v3.
Not in our library
10.
F. Chollet, et al., Keras, 2015, https://github.com/fchollet/keras.
Not in our library
11.
Çiçek Ö., Abdulkadir A., Lienkamp S.S., Brox T., Ronneberger O. 3D U-net: learning dense volumetric segmentation from sparse annotation. Proceedings of the MICCAI. 2016;424-432.
Not in our library
12.
Clarkson M.J., Zombori G., Thompson S., Totz J., Song Y., Espak M., Johnsen S., Hawkes D., Ourselin S. The NifTK software platform for image-guided interventions: platform overview and NiftyLink messaging. Int. J. Comput. Assist. Radiol. Surg. 2015;10(3):301-316.
Not in our library10.1007/s11548-014-1124-7
13.
Collobert R., Kavukcuoglu K., Farabet C. Torch7: A MATLAB-like environment for machine learning. Proceedings of the NIPS Workshop on Algorithms, Systems, and Tools for Learning at Scale (Big Learning). 2011.
Not in our library
14.
Dale A.M., Fischl B., Sereno M.I. Cortical surface-based analysis:I. Segmentation and surface reconstruction. Neuroimage. 1999;9(2):179-194.
Not in our library10.1006/nimg.1998.0395
15.
S. Dieleman, J. Schlter, C. Raffel, E. Olson, S.K. Snderby, D. Nouri, D. Maturana, M. Thoma, E. Battenberg, J. Kelly, J.D. Fauw, M. Heilman, D.M. de Almeida, B. McFee, H. Weideman, G. TakÃ!‘cs, P. de Rivaz, J. Crall, G. Sanders, K. Rasul, C. Liu, G. French, J. Degrave, Lasagne: first release, 2015, 10.5281/zenodo.27878
Not in our library
16.
Doel T., Shakir D.I., Pratt R., Aertsen M., Moggridge J., Bellon E., David A.L., Deprest J., Vercauteren T., Ourselin S. GIFT-Cloud: A data sharing and collaboration platform for medical imaging research. Comput. Methods Programs Biomed. 2017;139:181-190.
Not in our library10.1016/j.cmpb.2016.11.004
17.
Dong H., Supratak A., Mai L., Liu F., Oehmichen A., Yu S., Guo Y. TensorLayer: a versatile library for efficient deep learning development. 2017.
Not in our library
18.
L. Fidon, W. Li, L.C. Garcia-Peraza-Herrera, J. Ekanayake, N. Kitchen, S. Ourselin, T. Vercauteren, Generalised Wasserstein Dice score for imbalanced multi-class segmentation using holistic convolutional networks, (2017). arXiv:1707.00478.
Not in our library
19.
Fischl B., Sereno M.I., Dale A.M. Cortical surface-based analysis: II: inflation, flattening, and a surface-based coordinate system. Neuroimage. 1999;9(2):195-207.
Not in our library10.1006/nimg.1998.0396
20.
Gamma E., Vlissides J., Johnson R., Helm R. Design Patterns: Elements of Reusable Object-Oriented Software. 1994.
Not in our library
21.
Garcia-Peraza-Herrera L.C., Li W., Fidon L., Gruijthuijsen C., Devreker A., Attilakos G., Deprest J., Poorten E.V., Stoyanov D., Vercauteren T., Ourselin S. ToolNet: holistically-nested real-time segmentation of robotic surgical tools. 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC. 2017;5717-5722.
Not in our library
22.
Gibson E., Giganti F., Hu Y., Bonmati E., Bandula S., Gurusamy K., Davidson B., Pereira S.P., Clarkson M.J., Barratt D.C. Automatic multi-organ segmentation on abdominal CT with dense v-networks. IEEE Trans. Med. Imaging. 2017.
Not in our library10.1109/tmi.2018.2806309
23.
Gibson E., Giganti F., Hu Y., Bonmati E., Bandula S., Gurusamy K., Davidson B.R., Pereira S.P., Clarkson M.J., Barratt D.C. Towards image-guided pancreas and biliary endoscopy: automatic multi-organ segmentation on abdominal CT with dense dilated networks. Proceedings of the 20th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI). 2017.
Not in our library
24.
Gibson E., Robu M.R., Thompson S., Edwards P.E., Schneider C., Gurusamy K., Davidson B., Hawkes D.J., Barratt D.C., Clarkson M.J. Deep residual networks for automatic segmentation of laparoscopic videos of the liver. Proceedings of the SPIE, Medical Imaging. 2017;vol. 10135.
Not in our library
25.
I. Goodfellow, NIPS 2016 tutorial: generative adversarial networks (2016). arXiv:1701.00160v4.
Not in our library
26.
Hu Y., Gibson E., Lee L.-L., Xie W., Barratt D.C., Vercauteren T., Noble J.A. Freehand ultrasound image simulation with spatially-conditioned generative adversarial networks. Proceedings of MICCAI Workshop on Reconstruction and Analysis of Moving Body Organs (RAMBO). 2017.
Not in our library
27.
Ioffe S., Guyon I., Luxburg U.V., Bengio S., Wallach H., Fergus R., Vishwanathan S., Garnett R. Batch renormalization: towards reducing minibatch dependence in batch-normalized models. Advances in Neural Information Processing Systems 30. 2017;1942-1950.
Not in our library
28.
Jia Y., Shelhamer E., Donahue J., Karayev S., Long J., Girshick R., Guadarrama S., Darrell T. Caffe: Convolutional architecture for fast feature embedding. Proceedings of the 22nd ACM International Conference on Multimedia (ACMMM). 2014;675-678.
Not in our library
29.
Johnsen S.F., Taylor Z.A., Clarkson M.J., Hipwell J., Modat M., Eiben B., Han L., Hu Y., Mertzanidou T., Hawkes D.J., Ourselin S. NiftySim: a GPU-based nonlinear finite element package for simulation of soft tissue biomechanics. Int. J. Comput. Assist. Radiol. Surg. 2015;10(7):1077-1095.
Not in our library10.1007/s11548-014-1118-5
30.
Kamnitsas K., Ledig C., Newcombe V.F., Simpson J.P., Kane A.D., Menon D.K., Rueckert D., Glocker B. Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med. Image Anal. 2017;36:61-78.
Not in our library10.1016/j.media.2016.10.004
31.
Klein S., Staring M., Murphy K., Viergever M.A., Pluim J.P. Elastix: a toolbox for intensity-based medical image registration. IEEE Trans. Med. Imaging. 2010;29(1):196-205.
Not in our library10.1109/tmi.2009.2035616
32.
B. Landman, Z. Xu, J.E. Igelsias, M. Styner, T.R. Langerak, A. Klein, Multi-atlas labeling beyond the cranial vault, 2015, URL: https://www.synapse.org/#!Synapse:syn3193805, accessed July 2017. 10.7303/syn3193805.
Not in our library
33.
Li W., Wang G., Fidon L., Ourselin S., Cardoso M.J., Vercauteren T. On the compactness, efficiency, and representation of 3D convolutional networks: Brain parcellation as a pretext task. Proceedings of Information Processing in Medical Imaging (IPMI). 2017;348-360.
Not in our library
34.
Litjens G., Kooi T., Bejnordi B.E., Setio A.A.A., Ciompi F., Ghafoorian M., van der Laak J.A.W.M., van Ginneken B., Sánchez C.I. A survey on deep learning in medical image analysis. Medical Image Analysis. 2017;42:60-88.
Not in our library10.1016/j.media.2017.07.005
35.
Lo S.-C., Lou S.-L., Lin J.-S., Freedman M.T., Chien M.V., Mun S.K. Artificial convolution neural network techniques and applications for lung nodule detection. IEEE Trans. Med. Imaging. 1995;14(4):711-718.
Not in our library10.1109/42.476112
36.
The Design of SimpleITK · 2013
No notice
37.
D. Mané, et al., TensorBoard: TensorFlow’s visualization toolkit, 2015, https://github.com/tensorflow/tensorboard.
Not in our library
38.
Mehrtash A., Pesteie M., Hetherington J., Behringer P.A., Kapur T., Wells III W.M., Rohling R., Fedorov A., Abolmaesumi P. DeepInfer: Open-source deep learning deployment toolkit for image-guided therapy. Proceedings of the SPIE, Medical Imaging. 2017;10135.
Not in our library10.1117/12.2256011
39.
Menze B.H., Jakab A., Bauer S., Kalpathy-Cramer J., Farahani K., Kirby J., Burren Y., Porz N., Slotboom J., Wiest R., Lanczi L., Gerstner E., Weber M.A., Arbel T., Avants B.B., Ayache N., Buendia P., Collins D.L., Cordier N., Corso J.J., Criminisi A., Das T., Delingette H., Ç. Demiralp, Durst C.R., Dojat M., Doyle S., Festa J., Forbes F., Geremia E., Glocker B., Golland P., Guo X., Hamamci A., Iftekharuddin K.M., Jena R., John N.M., Konukoglu E., Lashkari D., Mariz J.A., Meier R., Pereira S., Precup D., Price S.J., Raviv T.R., Reza S.M.S., Ryan M., Sarikaya D., Schwartz L., Shin H.C., Shotton J., Silva C.A., Sousa N., Subbanna N.K., Szekely G., Taylor T.J., Thomas O.M., Tustison N.J., Unal G., Vasseur F., Wintermark M., Ye D.H., Zhao L., Zhao B., Zikic D., Prastawa M., Reyes M., Leemput K.V. The multimodal brain tumor image segmentation benchmark (BraTS). IEEE Trans. Med. Imaging. 2015;34(10):1993-2024.
Not in our library10.1109/tmi.2014.2377694
40.
Milletari F., Navab N., Ahmadi S.-A. V-Net: Fully convolutional neural networks for volumetric medical image segmentation. Proceedings of the Fourth International Conference on 3D Vision (3DV). 2016;565-571.
Not in our library

All 57 references

Cited by papers we hold 8

Automated Lung Segmentation from Computed Tomography Images of Normal and COVID-19 Pneumonia Patients · 2022
Not checked
Real-world data: a brief review of the methods, applications, challenges and opportunities · 2022
Corrected
Deep learning—a first meta-survey of selected reviews across scientific disciplines, their commonalities, challenges and research impact · 2021
Not checked
TorchIO: A Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning · 2021
No notice
An automated framework for localization, segmentation and super-resolution reconstruction of fetal brain MRI · 2020
No notice
Data Augmentation for Brain-Tumor Segmentation: A Review · 2019
No notice
Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with convolutional neural networks · 2019
No notice
Weakly-supervised convolutional neural networks for multimodal image registration · 2018
No notice

Reads

Read by people on this page

None yet.

Checked or read by AI through naulon

None yet.

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.

Are you an author?

Is this your paper?

Claim it to add your note and be paid when agents cite it. Free.