Learnable latent embeddings for joint behavioural and neural analysis

Abstract

Mapping behavioural actions to neural activity is a fundamental goal of neuroscience. As our ability to record large neural and behavioural data increases, there is growing interest in modelling neural dynamics during adaptive behaviours to probe neural representations 1–3 . In particular, although neural latent embeddings can reveal underlying correlates of behaviour, we lack nonlinear techniques that can explicitly and flexibly leverage joint behaviour and neural data to uncover neural dynamics 3–5 . Here, we fill this gap with a new encoding method, CEBRA, that jointly uses behavioural and neural data in a (supervised) hypothesis- or (self-supervised) discovery-driven manner to produce both consistent and high-performance latent spaces. We show that consistency can be used as a metric for uncovering meaningful differences, and the inferred latents can be used for decoding. We validate its accuracy and demonstrate our tool’s utility for both calcium and electrophysiology datasets, across sensory and motor tasks and in simple or complex behaviours across species. It allows leverage of single- and multi-session datasets for hypothesis testing or can be used label free. Lastly, we show that CEBRA can be used for the mapping of space, uncovering complex kinematic features, for the production of consistent latent spaces across two-photon and Neuropixels data, and can provide rapid, high-accuracy decoding of natural videos from visual cortex.

Full text

Its licence does not allow us to show the full text. Read it at PubMed Central:

References 62

1. 1.

   Urai AE, Doiron B, Leifer AM, Churchland AK. Large-scale neural recordings call for new insights to link brain and behavior. Nat. Neurosci. 2021;25:11-19.

   Not in our library10.1038/s41593-021-00980-9

2. 2.

   Krakauer JW, Ghazanfar AA, Gomez-Marin A, MacIver MA, Poeppel D. Neuroscience needs behavior: correcting a reductionist bias. Neuron. 2017;93:480-490.

   Not in our library10.1016/j.neuron.2016.12.041

3. 3.

   Jazayeri M, Ostojic S. Interpreting neural computations by examining intrinsic and embedding dimensionality of neural activity. Curr. Opin. Neurobiol. 2021;70:113-120.

   Not in our library10.1016/j.conb.2021.08.002

4. 4.

   Humphries, M. D. Strong and weak principles of neural dimension reduction. Neuron. Behav. Data Anal. Theoryhttps://nbdt.scholasticahq.com/article/24619 (2020).

   Not in our library

5. 5.

   Zhou, D., & Wei, X. Learning identifiable and interpretable latent models of high-dimensional neural activity using pi-VAE. Adv. Neural Inf. Process. Syst.https://proceedings.neurips.cc//paper/2020/file/510f2318f324cf07fce24c3a4b89c771-Paper.pdf (2020).

   Not in our library

6. 6.

   Vargas-Irwin CE, et al. Decoding complete reach and grasp actions from local primary motor cortex populations. J. Neurosci. 2010;30:9659-9669.

   Not in our library10.1523/jneurosci.5443-09.2010

7. 7.

   Okorokova, E. V., Goodman, J. M., Hatsopoulos, N. G. & Bensmaia, S. J. Decoding hand kinematics from population responses in sensorimotor cortex during grasping. J. Neural Eng. 17, 046035 (2020).

   Not in our library10.1088/1741-2552/ab95ea

8. 8.

   Yu BM, et al. Gaussian-process factor analysis for low-dimensional single-trial analysis of neural population activity. J. Neurophysiol. 2008;102:614-635.

   Not in our library10.1152/jn.90941.2008

9. 9.

   Churchland M, et al. Neural population dynamics during reaching. Nature. 2012;487:51-56.

   Not in our library10.1038/nature11129

10. 10.
    Cortical population activity within a preserved neural manifold underlies multiple motor behaviors · 2018

    No notice
11. 11.

    McInnes L, Healy J, Saul N, Großberger L. UMAP: Uniform Manifold Approximation and Projection for dimension reduction. J. Open Source Softw. 2018;3:861.

    Not in our library10.21105/joss.00861

12. 12.

    Maaten LV, Postma EO, Herik JV. Dimensionality reduction: a comparative review. J. Mach. Learn. Res. 2009;10:13.

    Not in our library

13. 13.

    Roeder, G., Metz, L. & Kingma, D. P. On linear identifiability of learned representations. Proc. Mach. Learn. Res.139, 9030–9039 (2021).

    Not in our library

14. 14.

    Hyvärinen, A., Sasaki, H. & Turner, R. E. Nonlinear ICA using auxiliary variables and generalized contrastive learning. Proc. Mach. Learn. Res.89, 859–868 (2019).

    Not in our library

15. 15.

    Sani OG, Abbaspourazad H, Wong YT, Pesaran B, Shanechi MM. Modeling behaviorally relevant neural dynamics enabled by preferential subspace identification. Nat. Neurosci. 2020;24:140-149.

    Not in our library10.1038/s41593-020-00733-0

16. 16.

    Klindt, D. A. et al. Towards nonlinear disentanglement in natural data with temporal sparse coding. International Conference on Learning Representationshttps://openreview.net/forum?id=EbIDjBynYJ8 (2021).

    Not in our library

17. 17.

    Pandarinath C, et al. Inferring single-trial neural population dynamics using sequential auto-encoders. Nat. Methods. 2017;15:805-815.

    Not in our library10.1038/s41592-018-0109-9

18. 18.

    Prince, L. Y., Bakhtiari, S., Gillon, C. J., & Richards, B. A. Parallel inference of hierarchical latent dynamics in two-photon calcium imaging of neuronal populations. Preprint at https://www.biorxiv.org/content/10.1101/2021.03.05.434105v1 (2021).

    Not in our library

19. 19.

    Gutmann MU, Hyvärinen A. Noise-contrastive estimation of unnormalized statistical models, with applications to natural image statistics. J. Mach. Learn. Res. 2012;13:307-361.

    Not in our library

20. 20.

    Oord, A. V., Li, Y. & Vinyals, O. Representation learning with contrastive predictive coding. Preprint at 10.48550/arXiv.1807.03748 (2018).

    Not in our library

21. 21.

    Khosla P, et al. Supervised contrastive learning. Adv. Neural Inf. Process. Syst. 2020;33:18661-18673.

    Not in our library

22. 22.

    Chen, T., Kornblith, S., Norouzi, M. & Hinton, G. E. A simple framework for contrastive learning of visual representations. Proc. Mach. Learn. Res.119, 1597–1607 (2020).

    Not in our library

23. 23.

    Hälvä H, et al. Disentangling identifiable features from noisy data with structured nonlinear ICA. Adv. Neural Inf. Process. Syst. 2021;34:1624-1633.

    Not in our library

24. 24.

    Zimmermann, R. S., Sharma, Y., Schneider, S., Bethge, M. & Brendel, W. Contrastive learning inverts the data generating process. Proc. Mach. Learn. Res.139, 12979–12990 (2021).

    Not in our library

25. 25.

    Keshtkaran MR, et al. A large-scale neural network training framework for generalized estimation of single-trial population dynamics. Nat. Methods. 2022;19:1572-1577.

    Not in our library10.1038/s41592-022-01675-0

26. 26.

    Grosmark AD, Buzsáki G. Diversity in neural firing dynamics supports both rigid and learned hippocampal sequences. Science. 2016;351:1440-1443.

    Not in our library10.1126/science.aad1935

27. 27.

    Huxter JR, Burgess N, O’Keefe J. Independent rate and temporal coding in hippocampal pyramidal cells. Nature. 2003;425:828-832.

    Not in our library10.1038/nature02058

28. 28.

    Moser EI, Kropff E, Moser M. Place cells, grid cells, and the brain’s spatial representation system. Annu. Rev. Neurosci. 2008;31:69-89.

    Not in our library10.1146/annurev.neuro.31.061307.090723

29. 29.

    Dombeck DA, Harvey CD, Tian L, Looger LL, Tank DW. Functional imaging of hippocampal place cells at cellular resolution during virtual navigation. Nat. Neurosci. 2010;13:1433-1440.

    Not in our library10.1038/nn.2648

30. 30.

    Curto C. What can topology tell us about the neural code? Bull. Am. Math. Soc. 2016;54:63-78.

    Not in our library10.1090/bull/1554

31. 31.

    Chaudhuri R, Gerçek B, Pandey B, Peyrache A, Fiete IR. The intrinsic attractor manifold and population dynamics of a canonical cognitive circuit across waking and sleep. Nat. Neurosci. 2019;22:1512-1520.

    Not in our library10.1038/s41593-019-0460-x

32. 32.

    Silva VD, Morozov D, Vejdemo-Johansson M. Persistent cohomology and circular coordinates. Discrete Comput. Geom. 2009;45:737-759.

    Not in our library10.1007/s00454-011-9344-x

33. 33.
    Toroidal topology of population activity in grid cells · 2022

    No notice
34. 34.

    Chowdhury RH, Glaser JI, Miller LE. Area 2 of primary somatosensory cortex encodes kinematics of the whole arm. eLife. 2019;9:e48198.

    Not in our library10.7554/elife.48198

35. 35.

    Prud’homme MJ, Kalaska JF. Proprioceptive activity in primate primary somatosensory cortex during active arm reaching movements. J. Neurophysiol. 1994;72:2280-2301.

    Not in our library10.1152/jn.1994.72.5.2280

36. 36.

    London BM, Miller LE. Responses of somatosensory area 2 neurons to actively and passively generated limb movements. J. Neurophysiol. 2013;109:1505-1513.

    Not in our library10.1152/jn.00372.2012

37. 37.

    Pei, F. et al. Neural Latents Benchmark ‘21: Evaluating latent variable models of neural population activity. Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarkshttps://openreview.net/forum?id=KVMS3fl4Rsv (2021).

    Not in our library

38. 38.
    Community-based benchmarking improves spike rate inference from two-photon calcium imaging data · 2018

    No notice
39. 39.

    Hafting T, Fyhn M, Molden S, Moser M, Moser EI. Microstructure of a spatial map in the entorhinal cortex. Nature. 2005;436:801-806.

    Not in our library10.1038/nature03721

40. 40.

    Schultz W, Dayan P, Montague PR. A neural substrate of prediction and reward. Science. 1997;275:1593-1599.

    Not in our library10.1126/science.275.5306.1593

Cited by papers we hold 1

• Preserved neural dynamics across animals performing similar behaviour · 2023
  No notice

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.