Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition

Abstract

The primate visual system achieves remarkable visual object recognition performance even in brief presentations, and under changes to object exemplar, geometric transformations, and background variation (a.k.a. core visual object recognition). This remarkable performance is mediated by the representation formed in inferior temporal (IT) cortex. In parallel, recent advances in machine learning have led to ever higher performing models of object recognition using artificial deep neural networks (DNNs). It remains unclear, however, whether the representational performance of DNNs rivals that of the brain. To accurately produce such a comparison, a major difficulty has been a unifying metric that accounts for experimental limitations, such as the amount of noise, the number of neural recording sites, and the number of trials, and computational limitations, such as the complexity of the decoding classifier and the number of classifier training examples. In this work, we perform a direct comparison that corrects for these experimental limitations and computational considerations. As part of our methodology, we propose an extension of "kernel analysis" that measures the generalization accuracy as a function of representational complexity. Our evaluations show that, unlike previous bio-inspired models, the latest DNNs rival the representational performance of IT cortex on this visual object recognition task. Furthermore, we show that models that perform well on measures of representational performance also perform well on measures of representational similarity to IT, and on measures of predicting individual IT multi-unit responses. Whether these DNNs rely on computational mechanisms similar to the primate visual system is yet to be determined, but, unlike all previous bio-inspired models, that possibility cannot be ruled out merely on representational performance grounds.

Full text

Its licence does not allow us to show the full text. Read it at PubMed Central:

References 76

1. 1.

   This reference’s wording can’t be shown under the paper’s licence.

   Not in our library

2. 2.

   This reference’s wording can’t be shown under the paper’s licence.

   Not in our library

3. 3.

   This reference’s wording can’t be shown under the paper’s licence.

   Not in our library

4. 4.

   This reference’s wording can’t be shown under the paper’s licence.

   Not in our library

5. 5.

   This reference’s wording can’t be shown under the paper’s licence.

   Not in our library

6. 6.

   This reference’s wording can’t be shown under the paper’s licence.

   Not in our library

7. 7.

   This reference’s wording can’t be shown under the paper’s licence.

   Not in our library

8. 8.

   This reference’s wording can’t be shown under the paper’s licence.

   Not in our library

9. 9.

   This reference’s wording can’t be shown under the paper’s licence.

   Not in our library

10. 10.

    This reference’s wording can’t be shown under the paper’s licence.

    Not in our library

11. 11.

    This reference’s wording can’t be shown under the paper’s licence.

    Not in our library

12. 12.

    This reference’s wording can’t be shown under the paper’s licence.

    Not in our library

13. 13.

    This reference’s wording can’t be shown under the paper’s licence.

    Not in our library

14. 14.

    This reference’s wording can’t be shown under the paper’s licence.

    Not in our library

15. 15.

    This reference’s wording can’t be shown under the paper’s licence.

    Not in our library

16. 16.

    This reference’s wording can’t be shown under the paper’s licence.

    Not in our library

17. 17.

    This reference’s wording can’t be shown under the paper’s licence.

    Not in our library

18. 18.

    This reference’s wording can’t be shown under the paper’s licence.

    Not in our library

19. 19.

    This reference’s wording can’t be shown under the paper’s licence.

    Not in our library

20. 20.

    This reference’s wording can’t be shown under the paper’s licence.

    Not in our library

21. 21.

    This reference’s wording can’t be shown under the paper’s licence.

    Not in our library

22. 22.

    This reference’s wording can’t be shown under the paper’s licence.

    Not in our library

23. 23.

    This reference’s wording can’t be shown under the paper’s licence.

    Not in our library

24. 24.
    Why is Real-World Visual Object Recognition Hard? · 2008

    No notice
25. 25.

    This reference’s wording can’t be shown under the paper’s licence.

    Not in our library

26. 26.

    This reference’s wording can’t be shown under the paper’s licence.

    Not in our library

27. 27.

    This reference’s wording can’t be shown under the paper’s licence.

    Not in our library

28. 28.

    This reference’s wording can’t be shown under the paper’s licence.

    Not in our library

29. 29.

    This reference’s wording can’t be shown under the paper’s licence.

    Not in our library

30. 30.

    This reference’s wording can’t be shown under the paper’s licence.

    Not in our library

31. 31.

    This reference’s wording can’t be shown under the paper’s licence.

    Not in our library

32. 32.

    This reference’s wording can’t be shown under the paper’s licence.

    Not in our library

33. 33.

    This reference’s wording can’t be shown under the paper’s licence.

    Not in our library

34. 34.

    This reference’s wording can’t be shown under the paper’s licence.

    Not in our library

35. 35.

    This reference’s wording can’t be shown under the paper’s licence.

    Not in our library

36. 36.

    This reference’s wording can’t be shown under the paper’s licence.

    Not in our library

37. 37.

    This reference’s wording can’t be shown under the paper’s licence.

    Not in our library

38. 38.

    This reference’s wording can’t be shown under the paper’s licence.

    Not in our library

39. 39.

    This reference’s wording can’t be shown under the paper’s licence.

    Not in our library

40. 40.

    This reference’s wording can’t be shown under the paper’s licence.

    Not in our library

Cited by papers we hold 11

• Brains and algorithms partially converge in natural language processing · 2022
  Corrected
• Deep convolutional models improve predictions of macaque V1 responses to natural images · 2019
  No notice
• Distinct contributions of functional and deep neural network features to representational similarity of scenes in human brain and behavior · 2018
  No notice
• Deep convolutional networks do not classify based on global object shape · 2018
  No notice
• SiFit: inferring tumor trees from single-cell sequencing data under finite-sites models · 2017
  No notice
• Generic decoding of seen and imagined objects using hierarchical visual features · 2017
  No notice
• Reward-based training of recurrent neural networks for cognitive and value-based tasks · 2017
  No notice
• Towards deep learning with segregated dendrites · 2017
  No notice
• Random synaptic feedback weights support error backpropagation for deep learning · 2016
  No notice
• Comparison of deep neural networks to spatio-temporal cortical dynamics of human visual object recognition reveals hierarchical correspondence · 2016
  No notice
• Deep Supervised, but Not Unsupervised, Models May Explain IT Cortical Representation · 2014
  No notice

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.