A Tutorial for Information Theory in Neuroscience

Abstract

Understanding how neural systems integrate, encode, and compute information is central to understanding brain function. Frequently, data from neuroscience experiments are multivariate, the interactions between the variables are nonlinear, and the landscape of hypothesized or possible interactions between variables is extremely broad. Information theory is well suited to address these types of data, as it possesses multivariate analysis tools, it can be applied to many different types of data, it can capture nonlinear interactions, and it does not require assumptions about the structure of the underlying data (i.e., it is model independent). In this article, we walk through the mathematics of information theory along with common logistical problems associated with data type, data binning, data quantity requirements, bias, and significance testing. Next, we analyze models inspired by canonical neuroscience experiments to improve understanding and demonstrate the strengths of information theory analyses. To facilitate the use of information theory analyses, and an understanding of how these analyses are implemented, we also provide a free MATLAB software package that can be applied to a wide range of data from neuroscience experiments, as well as from other fields of study.

Shown for reading. Its publisher allows quoting it, not republishing it.

References 112

1. 1.

   Adriaans P (2012 ) Information In: Standford Encyclopedia of Philosophy (Zalta EN, ed.).

   Not in our library

2. 2.

   Asaad WF, Lauro PM, Perge JA, Eskandar EN (2017 ) Prefrontal neurons encode a solution to the credit-assignment problem. J Neurosci 37:6995–7007.

   Not in our library10.1523/jneurosci.3311-16.2017

3. 3.

   Bear MF, Connors BW, Paradiso MA (2007 ) Neuroscience: exploring the brain, Third Edition Baltimore, MD: Lippincott Williams and Wilkins.

   Not in our library

4. 4.

   Beer RD, Williams PL (2014 ) Information processing and dynamics in minimally cognitive agents. Cogn Sci 1–38.

   Not in our library10.1111/cogs.12142

5. 5.

   Benjamini Y, Hochberg Y (1995 ) Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc B 57:289–300.

   Not in our library

6. 6.

   Benjamini Y, Yekutieli D (2001 ) The control of the false discovery rate in multiple testing under dependency. Ann Stat 29:1165–1188.

   Not in our library10.1214/aos/1013699998

7. 7.

   Bertschinger N, Rauh J, Olbrich E, Jost J, Ay N (2014 ) Quantifying unique information. Entropy 16:2161–2183.

   Not in our library10.3390/e16042161

8. 8.

   Bettencourt LMA, Gintautas V, Ham MI (2008 ) Identification of functional information subgraphs in complex networks. Phys Rev Lett 100:238701.

   Not in our library10.1103/physrevlett.100.238701

9. 9.

   Bialek W, Rieke F, De Ruyter van Steveninck RR, Warland D (1991 ) Reading a neural code. Science 252:1854–1857.

   Not in our library10.1126/science.2063199

10. 10.

    Bonachela JA, Hinrichsen H, Muñoz MA (2008 ) Entropy estimates of small data sets. J Phys A 41:202001.

    Not in our library10.1088/1751-8113/41/20/202001

11. 11.

    Bonifazi P, Goldin M, Picardo MA, Jorquera I, Cattani A, Bianconi G, Represa A, Ben-Ari Y, Cossart R (2009 ) GABAergic hub neurons orchestrate synchrony in developing hippocampal networks. Science 326:1419–1424.

    Not in our library10.1126/science.1175509

12. 12.

    Borst A, Theunissen FE (1999 ) Information theory and neural coding. Nat Neurosci 2:947–957.

    Not in our library10.1038/14731

13. 13.

    Bossomaier T, Barnett L, Harre M, Lizier JT (2016 ) An introduction to transfer entropy. New York: Springer International.

    Not in our library

14. 14.

    Brenner N, Strong SP, Koberle R, Bialek W, de R, van Steveninck RR (2000 ) Synergy in a neural code. Neur Comput 12:1531–1532.

    Not in our library10.1162/089976600300015259

15. 15.

    Bullmore E, Sporns O (2009 ) Complex brain networks: graph theoretical analysis of structural and function systems. Nat Rev Neurosci 10:186–198.

    Not in our library10.1038/nrn2618

16. 16.

    Butts DA (2003 ) How much information is associated with a particular stimulus? Netw Comput Neur Syst 14:177–187.

    Not in our library10.1088/0954-898x_14_2_301

17. 17.

    Butts DA, Weng C, Jin J, Yeh C, Lesica NA, Alonso J, Stanley GB (2007 ) Temporal precision in the neural code and the timescales of natural vision. Nat Lett 449:92–95.

    Not in our library10.1038/nature06105

18. 18.

    Castellucci V, Pinsker H, Kupfermann I, Kandel ER (1970 ) Neuronal mechanisms of habituation and dishabituation of the gill-withdrawal reflex in aplysia. Science 167:1745–1748.

    Not in our library10.1126/science.167.3926.1745

19. 19.

    Cover TM, Thomas JA (2006 ) Elements of Information Theory, 2nd Edition New York: Wiley-Interscience.

    Not in our library

20. 20.

    Cunningham JP, Yu BM (2014 ) Dimensionality reduction for large-scale neural recordings. Nat Neurosci 17:1500–1509.

    Not in our library10.1038/nn.3776

21. 21.

    Daw CS, Finney CEA, Tracy ER (2003 ) A review of symbolic analysis of experimental data. Rev Scientif Instr 74:915–930.

    Not in our library10.1063/1.1531823

22. 22.

    DeWeese MR, Meister M (1999 ) How to measure the information gained from one symbol. Netw Comput Neur Syst 10:325–340.

    Not in our library10.1088/0954-898x_10_4_303

23. 23.

    Dimitrov AG, Lazar AA, Victor JD (2011 ) Information theory in neuroscience. J Comput Neurosci 30:1–5.

    Not in our library10.1007/s10827-011-0314-3

24. 24.

    Ding M, Chen Y, Bressler SL (2006 ) Granger causality: basic theory and application to neuroscience In: Handbook of Time Series Analysis: Recent Theoretical Developments and Applications (Schelter B, Winterhalder M, Timmer J, eds), p 437 Weinheim, Germany: Wiley-VCH.

    Not in our library

25. 25.

    Dit-Contributors (2018 ) Dit: Discrete information theory. Available at https://dit.readthedocs.io/en/latest/.

    Not in our library

26. 26.

    Finn C, Lizier JT (2018 ) Pointwise information decomposition using the specificity and ambiguity lattices. arXiv doi:10.3390/ecea-4-05024.

    Not in our library10.3390/e20040297

27. 27.

    Friston KJ (2011 ) Functional and effective connectivity: a review. Brain Connect 1:13–36.

    Not in our library10.1089/brain.2011.0008

28. 28.

    Friston KJ, Harrison L, Penny W (2003 ) Dynamic causal modelling. Neuroimage 19:1273–1302.

    Not in our library10.1016/s1053-8119(03)00202-7

29. 29.

    Georgopoulos AP, Kalaska JF, Caminiti R, Massey JT (1982 ) On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex. J Neurosci 2:1527–1537.

    Not in our library10.1523/jneurosci.02-11-01527.1982

30. 30.

    Goldberg DH, Victor JD, Gardner EP, Gardner D (2009 ) Spike train analysis toolkit: enabling wider application of information-theoretic techniques in neurophysiology. Neuroinformatics 7:165–178.

    Not in our library10.1007/s12021-009-9049-y

31. 31.

    Gomez-Herraro G, Wu W, Rutanen K, Soriano MC, Pipa G, Vicente R (2015 ) Assessing coupling dynamics from an ensemble of time series. Entropy 17:1958–1970.

    Not in our library10.3390/e17041958

32. 32.

    Granger CWJ (1969 ) Investigating causal relations by econometric models and cross-spectral methods. Econometrica 37:424–438.

    Not in our library10.2307/1912791

33. 33.

    Griffith V, Chong EKP, James RG, Ellison CJ, Crutchfield JP (2014 ) Intersection information based on common randomness. Entropy 16:1985–2000.

    Not in our library10.3390/e16041985

34. 34.

    Harder M, Salge C, Polani D (2013 ) Bivariate measure of redundant information. Phys Rev E 87:012130.

    Not in our library10.1103/physreve.87.012130

35. 35.

    Honey CJ, Kotter R, Breakspear M, Sporns O (2007 ) Network structure of cerebral cortex shapes functional connectivity on multiple time scales. Proc Natl Acad Sci U S A 104:10240–10245.

    Not in our library10.1073/pnas.0701519104

36. 36.

    Hramov AE, Koronovskii AA, Makarov VA, Pavlov AN, Sitnikova E (2015 ) Wavelets in neuroscience. Berlin: Springer.

    Not in our library

37. 37.

    Ignac TM, Skupin A, Sakhanenko NA, Galas DJ (2014 ) Discovering pair-wise genetic interactions: an information theory-based approach. PLOS One 9:e92310.

    Not in our library10.1371/journal.pone.0092310

38. 38.

    Ince RAA (2017 ) Measuring multivariate redundant information with pointwise common change in surprisal. Entropy 19:318–344.

    Not in our library10.3390/e19070318

39. 39.

    Ince RAA, Petersen RS, Swan DC, Panzeri S (2009 ) Python for information theoretic analysis of neural data. Front Neuroinform 3:4.

    Not in our library10.3389/neuro.11.004.2009

40. 40.

    Ince RAA, Mazzoni A, Petersen RS, Panzeri S (2010 ) Open source tools for the information theoretic analysis of neural data. Front Neurosci 4:62–70.

    Not in our library10.3389/neuro.01.011.2010

Cited by papers we hold 1

• PLD3 affects axonal spheroids and network defects in Alzheimer’s disease · 2022
  No notice

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.