Automatic Knee Osteoarthritis Diagnosis from Plain Radiographs: A Deep Learning-Based Approach

Abstract

Knee osteoarthritis (OA) is the most common musculoskeletal disorder. OA diagnosis is currently conducted by assessing symptoms and evaluating plain radiographs, but this process suffers from subjectivity. In this study, we present a new transparent computer-aided diagnosis method based on the Deep Siamese Convolutional Neural Network to automatically score knee OA severity according to the Kellgren-Lawrence grading scale. We trained our method using the data solely from the Multicenter Osteoarthritis Study and validated it on randomly selected 3,000 subjects (5,960 knees) from Osteoarthritis Initiative dataset. Our method yielded a quadratic Kappa coefficient of 0.83 and average multiclass accuracy of 66.71% compared to the annotations given by a committee of clinical experts. Here, we also report a radiological OA diagnosis area under the ROC curve of 0.93. Besides this, we present attention maps highlighting the radiological features affecting the network decision. Such information makes the decision process transparent for the practitioner, which builds better trust toward automatic methods. We believe that our model is useful for clinical decision making and for OA research; therefore, we openly release our training codes and the data set created in this study.

Full text

Its licence does not allow us to show the full text. Read it at PubMed Central:

References 33

1. 1.

   Cross, M. et al. The global burden of hip and knee osteoarthritis: estimates from the global burden of disease 2010 study. Annals rheumatic diseases annrheumdis–2013 (2014).

   Not in our library10.1136/annrheumdis-2013-204763

2. 2.

   Puig-Junoy, J. & Zamora, A. R. Socio-economic costs of osteoarthritis: A systematic review of cost-of-illness studies. In Seminars in arthritis and rheumatism, vol. 44, 531–541 (Elsevier, 2015).

   Not in our library10.1016/j.semarthrit.2014.10.012

3. 3.

   Karsdal M, et al. Disease-modifying treatments for osteoarthritis (dmoads) of the knee and hip: lessons learned from failures and opportunities for the future. Osteoarthr. cartilage. 2016;24:2013-2021.

   Not in our library10.1016/j.joca.2016.07.017

4. 4.

   Finnilä MA, et al. Association between subchondral bone structure and osteoarthritis histopathological grade. J. Orthop. Res. 2017;35:785-792.

   Not in our library10.1002/jor.23312

5. 5.

   Kellgren J, Lawrence J. Radiological assessment of osteo-arthrosis. Annals rheumatic diseases. 1957;16:494.

   Not in our library10.1136/ard.16.4.494

6. 6.

   Gossec L, et al. Comparative evaluation of three semi-quantitative radiographic grading techniques for knee osteoarthritis in terms of validity and reproducibility in 1759 x-rays: report of the oarsi–omeract task force: Extended report. Osteoarthr. cartilage. 2008;16:742-748.

   Not in our library10.1016/j.joca.2008.02.021

7. 7.

   Sheehy L, et al. Validity and sensitivity to change of three scales for the radiographic assessment of knee osteoarthritis using images from the multicenter osteoarthritis study (most). Osteoarthritis and Cartilage. 2015;23:1491-1498.

   Not in our library10.1016/j.joca.2015.05.003

8. 8.

   Culvenor AG, Engen CN, Øiestad BE, Engebretsen L, Risberg MA. Defining the presence of radiographic knee osteoarthritis: a comparison between the kellgren and lawrence system and oarsi atlas criteria. Knee Surgery, Sports Traumatol. Arthrosc. 2015;23:3532-3539.

   Not in our library10.1007/s00167-014-3205-0

9. 9.

   Drew T, Võ ML-H, Wolfe JM. The invisible gorilla strikes again: Sustained inattentional blindness in expert observers. Psychol. science. 2013;24:1848-1853.

   Not in our library10.1177/0956797613479386

10. 10.

    Dacree J, Huskisson E. The automatic assessment of knee radiographs in osteoarthritis using digital image analysis. Rheumatol. 1989;28:506-510.

    Not in our library10.1093/rheumatology/28.6.506

11. 11.

    Shamir L, Felson DT, Ferrucci L, Goldberg IG. Assessment of osteoarthritis initiative–kellgren and lawrence scoring projects quality using computer analysis. J. Musculoskelet. Res. 2010;13:197-201.

    Not in our library10.1142/s0218957710002612

12. 12.

    Woloszynski T, Podsiadlo P, Stachowiak G, Kurzynski M. A dissimilarity-based multiple classifier system for trabecular bone texture in detection and prediction of progression of knee osteoarthritis. Proc. Inst. Mech. Eng. Part H: J. Eng. Medicine. 2012;226:887-894.

    Not in our library10.1177/0954411912456650

13. 13.

    Shamir L, et al. Early detection of radiographic knee osteoarthritis using computer-aided analysis. Osteoarthritis and Cartilage. 2009;17:1307-1312.

    Not in our library10.1016/j.joca.2009.04.010

14. 14.

    Thomson, J., O’Neill, T., Felson, D. & Cootes, T. Automated shape and texture analysis for detection of osteoarthritis from radiographs of the knee. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 127–134 (Springer, 2015).

    Not in our library

15. 15.

    Antony, J., McGuinness, K., Connor, N. E. & Moran, K. Quantifying radiographic knee osteoarthritis severity using deep convolutional neural networks. arXiv preprint arXiv:1609.02469 (2016).

    Not in our library

16. 16.

    Antony, J., McGuinness, K., Moran, K. & O’Connor, N. E. Automatic detection of knee joints and quantification of knee osteoarthritis severity using convolutional neural networks. arXiv preprint arXiv:1703.09856 (2017).

    Not in our library

17. 17.

    Simonyan, K. & Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).

    Not in our library

18. 18.

    He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 770–778 (2016).

    Not in our library

19. 19.

    Esteva A, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017;542:115-118.

    Not in our library10.1038/nature21056

20. 20.
    Computer-Aided Diagnosis with Deep Learning Architecture: Applications to Breast Lesions in US Images and Pulmonary Nodules in CT Scans · 2016

    No notice
21. 21.

    Selvaraju, R. R. et al. Grad-cam: Why did you say that? Visual explanations from deep networks via gradient-based localization. arXiv preprint arXiv:1610.02391 (2016).

    Not in our library

22. 22.

    Montavon G, Lapuschkin S, Binder A, Samek W, Müller K-R. Explaining nonlinear classification decisions with deep taylor decomposition. Pattern Recognit. 2017;65:211-222.

    Not in our library10.1016/j.patcog.2016.11.008

23. 23.

    Lever J, Krzywinski M, Altman N. Points of significance: Model selection and overfitting. Nat. Methods. 2016;13:703-704.

    Not in our library10.1038/nmeth.3968

24. 24.

    Srivastava N, Hinton GE, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learni. Res. 2014;15:1929-1958.

    Not in our library

25. 25.

    Tiulpin, A., Thevenot, J., Rahtu, E. & Saarakkala, S. A novel method for automatic localization of joint area on knee plain radiographs. In Scandinavian Conference on Image Analysis, in press (Springer, 2017).

    Not in our library

26. 26.

    Chopra, S., Hadsell, R. & LeCun, Y. Learning a similarity metric discriminatively, with application to face verification. In Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, vol. 1, 539–546 (IEEE, 2005).

    Not in our library

27. 27.

    Ioffe, S. & Szegedy, C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 (2015).

    Not in our library

28. 28.

    Zhao, B., Wu, X., Feng, J., Peng, Q. & Yan, S. Diversified visual attention networks for fine-grained object classification. IEEE Transactions on Multimed (2017).

    Not in our library

29. 29.

    Mnih, V., Heess, N., Graves, A. et al. Recurrent models of visual attention. In Advances in neural information processing systems, 2204–2212 (2014).

    Not in our library

30. 30.

    Paszke, A., Gross, S., Chintala, S. & Chanan, G. Pytorch: Tensors and dynamic neural networks in python with strong gpu acceleration. https://github.com/pytorch/pytorch (2017).

    Not in our library

31. 31.

    Kingma, D. & Ba, J. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).

    Not in our library

32. 32.

    Minciullo, L. & Cootes, T. Fully automated shape analysis for detection of osteoarthritis from lateral knee radiographs. In Pattern Recognition (ICPR), 2016 23rd International Conference on, 3787–3791 (IEEE, 2016).

    Not in our library

33. 33.

    Landis, J. R. & Koch, G. G. The measurement of observer agreement for categorical data. biometrics 159–174 (1977).

    Not in our library

Cited by papers we hold 1

• Multimodal Machine Learning-based Knee Osteoarthritis Progression Prediction from Plain Radiographs and Clinical Data · 2019
  No notice

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.