Supervised learning in spiking neural networks with FORCE training

Abstract

Populations of neurons display an extraordinary diversity in the behaviors they affect and display. Machine learning techniques have recently emerged that allow us to create networks of model neurons that display behaviors of similar complexity. Here we demonstrate the direct applicability of one such technique, the FORCE method, to spiking neural networks. We train these networks to mimic dynamical systems, classify inputs, and store discrete sequences that correspond to the notes of a song. Finally, we use FORCE training to create two biologically motivated model circuits. One is inspired by the zebra finch and successfully reproduces songbird singing. The second network is motivated by the hippocampus and is trained to store and replay a movie scene. FORCE trained networks reproduce behaviors comparable in complexity to their inspired circuits and yield information not easily obtainable with other techniques, such as behavioral responses to pharmacological manipulations and spike timing statistics.

Full text

Its licence does not allow us to show the full text. Read it at PubMed Central:

References 77

1. 1.

   Sussillo D, Abbott LF. Generating coherent patterns of activity from chaotic neural networks. Neuron. 2009;63:544-557.

   Not in our library10.1016/j.neuron.2009.07.018

2. 2.

   DePasquale, B., Churchland, M. M. & Abbott, L. Using firing-rate dynamics to train recurrent networks of spiking model neurons. Preprint at https://arxiv.org/abs/1601.07620 (2016).

   Not in our library

3. 3.

   Abbott L, DePasquale B, Memmesheimer RM. Building functional networks of spiking model neurons. Nat. Neurosci. 2016;19:350-355.

   Not in our library10.1038/nn.4241

4. 4.

   Thalmeier D, Uhlmann M, Kappen HJ, Memmesheimer R-M. Learning universal computations with spikes. PLoS. Comput. Biol. 2016;12:e1004895.

   Not in our library10.1371/journal.pcbi.1004895

5. 5.
   Predictive Coding of Dynamical Variables in Balanced Spiking Networks · 2013

   No notice
6. 6.

   Schwemmer MA, Fairhall AL, Denéve S, Shea-Brown ET. Constructing precisely computing networks with biophysical spiking neurons. J. Neurosci. 2015;35:10112-10134.

   Not in our library10.1523/jneurosci.4951-14.2015

7. 7.

   Bourdoukan, R. & Denéve, S. Enforcing balance allows local supervised learning in spiking recurrent networks. In Advances in Neural Information Processing Systems, 982–990 (2015).

   Not in our library

8. 8.

   Eliasmith C, et al. A large-scale model of the functioning brain. Science. 2012;338:1202-1205.

   Not in our library10.1126/science.1225266

9. 9.

   Eliasmith C, Anderson CH. Neural engineering: Computation, representation, and dynamics in neurobiological systems. 2002.

   Not in our library

10. 10.

    Gilra, A. & Gerstner, W. Predicting non-linear dynamics: a stable local learning scheme for recurrent spiking neural networks. arXiv:1702.06463 (2017).

    Not in our library10.7554/elife.28295

11. 11.

    Alemi, A., Machens, C., Denéve, S. & Slotine, J.-J. Learning arbitrary dynamics in efficient, balanced spiking networks using local plasticity rules. Preprint at https://arxiv.org/abs/1705.08026 (2017).

    Not in our library

12. 12.

    Brendel, W., Bourdoukan, R., Vertechi, P., Machens, C. K. & Denéve, S. Learning to represent signals spike by spike. arXiv:1703.03777 (2017).

    Not in our library10.1371/journal.pcbi.1007692

13. 13.

    Huh, D. & Sejnowski, T. J. Gradient descent for spiking neural networks. arXiv:1706.04698 (2017).

    Not in our library

14. 14.

    Lukoševičius M, Jaeger H, Schrauwen B. Reservoir computing trends. KI-Künstliche Intelligenz. 2012;26:365-371.

    Not in our library10.1007/s13218-012-0204-5

15. 15.

    Lukoševičius M, Jaeger H. Reservoir computing approaches to recurrent neural network training. Comput. Sci. Rev. 2009;3:127-149.

    Not in our library10.1016/j.cosrev.2009.03.005

16. 16.

    Schrauwen, B., Verstraeten, D. & Van Campenhout, J. An overview of reservoir computing: theory, applications and implementations. In Proceedings of the 15th European Symposium on Articial Neural Networks, 471–482 (2007).

    Not in our library

17. 17.

    Dominey PF. Complex sensory-motor sequence learning based on recurrent state representation and reinforcement learning. Biol. Cybern. 1995;73:265-274.

    Not in our library10.1007/bf00201428

18. 18.

    Maass W, Natschläger T, Markram H. Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural. Comput. 2002;14:2531-2560.

    Not in our library10.1162/089976602760407955

19. 19.

    Jaeger H. The “echo state” approach to analysing and training recurrent neural networks-with an erratum note. Bonn, Ger. 2001;148:34.

    Not in our library

20. 20.

    Schliebs, S., Mohemmed, A. & Kasabov, N. Are probabilistic spiking neural networks suitable for reservoir computing? In Neural Networks (IJCNN), The 2011 International Joint Conference on Neural Networks, 3156–3163 (2011).

    Not in our library

21. 21.

    Ozturk MC, Principe JC. Computing with transiently stable states in Proceedings. IEEE International Joint Conference on Neural Networks. 2005;3:1467-1472.

    Not in our library

22. 22.

    Maass, W. Liquid state machines: motivation, theory, and applications. In: B. Cooper, A. Sorbi (eds.) Computability in Context: Computation and Logic in the Real World, 275–296 (Imperial College Press, 2011).

    Not in our library

23. 23.

    Maass W, Natschläger T, Markram H. Fading memory and kernel properties of generic cortical microcircuit models. J. Physiol. 2004;98:315-330.

    Not in our library10.1016/j.jphysparis.2005.09.020

24. 24.

    Wojcik GM, Kaminski WA. Liquid state machine built of Hodgkin–Huxley neurons and pattern recognition. Neurocomputing. 2004;58:245-251.

    Not in our library10.1016/j.neucom.2004.01.051

25. 25.

    Buonomano DV, Merzenich MM. Temporal information transformed into a spatial code by a neural network with realistic properties. Science. 1995;267:1028.

    Not in our library10.1126/science.7863330

26. 26.

    Van Vreeswijk C, Sompolinsky H. Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science. 1996;274:1724.

    Not in our library10.1126/science.274.5293.1724

27. 27.

    Ostojic S. Two types of asynchronous activity in networks of excitatory and inhibitory spiking neurons. Nat. Neurosci. 2014;17:594-600.

    Not in our library10.1038/nn.3658

28. 28.

    Harish O, Hansel D. Asynchronous rate chaos in spiking neuronal circuits. PLoS. Comput. Biol. 2015;11:e1004266.

    Not in our library10.1371/journal.pcbi.1004266

29. 29.

    Haykin, S. S. Neural networks and learning machines (Pearson Upper Saddle River, NJ, 2009).

    Not in our library

30. 30.

    Haykin, S. in Neural Networks and Learning Machines 3rd edn, 246-249 (Upper Saddle River, NJ, USA; Pearson, 2009).

    Not in our library

31. 31.

    Hines ML, Morse T, Migliore M, Carnevale NT, Shepherd GM. ModelDB: a database to support computational neuroscience. J. Comput. Neurosci. 2004;17:7-11.

    Not in our library10.1023/b:jcns.0000023869.22017.2e

32. 32.

    Rajan K, Abbott L. Eigenvalue spectra of random matrices for neural networks. Phys. Rev. Lett. 2006;97:188104.

    Not in our library10.1103/physrevlett.97.188104

33. 33.

    London M, Roth A, Beeren L, Häusser M, Latham PE. Sensitivity to perturbations in vivo implies high noise and suggests rate coding in cortex. Nature. 2010;466:123-127.

    Not in our library10.1038/nature09086

34. 34.

    Monteforte M, Wolf F. Dynamic flux tubes form reservoirs of stability in neuronal circuits. Phys. Rev. X. 2012;2:041007.

    Not in our library

35. 35.

    Hahnloser RH, Kozhevnikov AA, Fee MS. An ultra-sparse code underlies the generation of neural sequences in a songbird. Nature. 2002;419:65-70.

    Not in our library10.1038/nature00974

36. 36.

    Long MA, Jin DZ, Fee MS. Support for a synaptic chain model of neuronal sequence generation. Nature. 2010;468:394-399.

    Not in our library10.1038/nature09514

37. 37.

    Leonardo A, Fee MS. Ensemble coding of vocal control in birdsong. J. Neurosci. 2005;25:652-661.

    Not in our library10.1523/jneurosci.3036-04.2005

38. 38.

    Vicario DS, Raksin JN. Possible roles for GABAergic inhibition in the vocal control system of the zebra finch. Neuroreport. 2000;11:3631-3635.

    Not in our library10.1097/00001756-200011090-00046

39. 39.

    Nottebohm F, Stokes TM, Leonard CM. Central control of song in the canary, Serinus canarius. J. Comp. Neurol. 1976;165:457-486.

    Not in our library10.1002/cne.901650405

40. 40.

    Euston DR, Tatsuno M, McNaughton BL. Fast-forward playback of recent memory sequences in prefrontal cortex during sleep. Science. 2007;318:1147-1150.

    Not in our library10.1126/science.1148979

41. 41.

    Diba K, Buzsáki G. Forward and reverse hippocampal place-cell sequences during ripples. Nat. Neurosci. 2007;10:1241-1242.

    Not in our library10.1038/nn1961

42. 42.

    Mello GB, Soares S, Paton JJ. A scalable population code for time in the striatum. Curr. Biol. 2015;25:1113-1122.

    Not in our library10.1016/j.cub.2015.02.036

43. 43.

    Nádasdy Z, Hirase H, Czurkó A, Csicsvari J, Buzsáki G. Replay and time compression of recurring spike sequences in the hippocampus. J. Neurosci. 1999;19:9497-9507.

    Not in our library10.1523/jneurosci.19-21-09497.1999

44. 44.

    Motanis H, Buonomano DV. Neural coding: time contraction and dilation in the striatum. Curr. Biol. 2015;25:R374-R376.

    Not in our library10.1016/j.cub.2015.02.057

45. 45.

    Churchland MM, et al. Neural population dynamics during reaching. Nature. 2012;487:51-56.

    Not in our library10.1038/nature11129

46. 46.

    Konishi M. Birdsong: from behavior to neuron. Ann. Rev. Neurosci. 1985;125-170.

    Not in our library10.1146/annurev.ne.08.030185.001013

47. 47.

    Mooney R. Different subthreshold mechanisms underlie song selectivity in identied HVc neurons of the zebra nch. J. Neurosci. 2000;20:5420-5436.

    Not in our library10.1523/jneurosci.20-14-05420.2000

48. 48.

    Bouchard KE, Brainard MS. Auditory-induced neural dynamics in sensory-motor circuitry predict learned temporal and sequential statistics of birdsong. Proc. Natl Acad. Sci. 2016;113:9641-9646.

    Not in our library10.1073/pnas.1606725113

49. 49.

    Kozhevnikov AA, Fee MS. Singing-related activity of identified HVC neurons in the zebra finch. J. Neurophysiol. 2007;97:4271-4283.

    Not in our library10.1152/jn.00952.2006

50. 50.

    Buzsáki G. Theta oscillations in the hippocampus. Neuron. 2002;33:325-340.

    Not in our library10.1016/s0896-6273(02)00586-x

51. 51.

    Klimesch W. EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis. Brain Res. Rev. 1999;29:169-195.

    Not in our library10.1016/s0165-0173(98)00056-3

52. 52.

    Buzsáki G. Theta rhythm of navigation: link between path integration and landmark navigation, episodic and semantic memory. Hippocampus. 2005;15:827-840.

    Not in our library10.1002/hipo.20113

53. 53.

    Lengyel M, Huhn Z, Érdi P. Computational theories on the function of theta oscillations. Biol. Cybern. 2005;92:393-408.

    Not in our library10.1007/s00422-005-0567-x

54. 54.

    Lubenov EV, Siapas AG. Hippocampal theta oscillations are travelling waves. Nature. 2009;459:534-539.

    Not in our library10.1038/nature08010

55. 55.

    Lisman JE, Jensen O. The theta-gamma neural code. Neuron. 2013;77:1002-1016.

    Not in our library10.1016/j.neuron.2013.03.007

56. 56.

    Givens B, Olton DS. Bidirectional modulation of scopolamine-induced working memory impairments by muscarinic activation of the medial septal area. Neurobiol. Learn. Mem. 1995;63:269-276.

    Not in our library10.1006/nlme.1995.1031

57. 57.

    Boyce R, Glasgow SD, Williams S, Adamantidis A. Causal evidence for the role of REM sleep theta rhythm in contextual memory consolidation. Science. 2016;352:812-816.

    Not in our library10.1126/science.aad5252

58. 58.

    Pastalkova E, Itskov V, Amarasingham A, Buzsáki G. Internally generated cell assembly sequences in the rat hippocampus. Science. 2008;321:1322-1327.

    Not in our library10.1126/science.1159775

59. 59.

    MacDonald CJ, Lepage KQ, Eden UT, Eichenbaum H. Hippocampal “time cells” bridge the gap in memory for discontiguous events. Neuron. 2011;71:737-749.

    Not in our library10.1016/j.neuron.2011.07.012

60. 60.

    Eichenbaum H. Time cells in the hippocampus: a new dimension for mapping memories. Nat. Rev. Neurosci. 2014;15:732-744.

    Not in our library10.1038/nrn3827

61. 61.

    Wang Y, Romani S, Lustig B, Leonardo A, Pastalkova E. Theta sequences are essential for internally generated hippocampal firing fields. Nat. Neurosci. 2015;18:282-288.

    Not in our library10.1038/nn.3904

62. 62.

    Salz DM, et al. Time cells in hippocampal area CA3. J. Neurosci. 2016;36:7476-7484.

    Not in our library10.1523/jneurosci.0087-16.2016

63. 63.

    Robinson NT, et al. Medial entorhinal cortex selectively supports temporal coding by hippocampal. Neurons. Neuron. 2017;94:677-688.

    Not in our library10.1016/j.neuron.2017.04.003

64. 64.

    Mizuseki K, Sirota A, Pastalkova E, Buzsáki G. Theta oscillations provide temporal windows for local circuit computation in the entorhinal-hippocampal loop. Neuron. 2009;64:267-280.

    Not in our library10.1016/j.neuron.2009.08.037

65. 65.

    Rivkind A, Barak O. Local dynamics in trained recurrent neural networks. Phys. Rev. Lett. 2017;118:258101.

    Not in our library10.1103/physrevlett.118.258101

66. 66.

    Tsodyks M, Pawelzik K, Markram H. Neural networks with dynamic synapses. Neural. Comput. 1998;10:821-835.

    Not in our library10.1162/089976698300017502

67. 67.

    Maass W, Markram H. On the computational power of circuits of spiking neurons. J. Comput. Syst. Sci. 2004;69:593-616.

    Not in our library10.1016/j.jcss.2004.04.001

68. 68.

    Maass, W., Natschläger, T. & Markram, H. A model for real-time computation in generic neural microcircuits. Adv. Neural Inform. Process. Syst. 15, NIPS 2002, MIT Press, Cambridge, MA 213–220 (2002).

    Not in our library

69. 69.

    Rajan K, Harvey CD, Tank DW. Recurrent network models of sequence generation and memory. Neuron. 2016;90:128-142.

    Not in our library10.1016/j.neuron.2016.02.009

70. 70.

    Li N, Daie K, Svoboda K, Druckmann S. Robust neuronal dynamics in premotor cortex during motor planning. Nature. 2016;532:459-464.

    Not in our library10.1038/nature17643

71. 71.

    Enel P, Procyk E, Quilodran R, Dominey PF. Reservoir computing properties of neural dynamics in prefrontal cortex. PLoS Comput. Biol. 2016;12:e1004967.

    Not in our library10.1371/journal.pcbi.1004967

72. 72.

    Laje R, Buonomano DV. Robust timing and motor patterns by taming chaos in recurrent neural networks. Nat. Neurosci. 2013;16:925-933.

    Not in our library10.1038/nn.3405

73. 73.

    Dur-e-Ahmad M, Nicola W, Campbell SA, Skinner FK. Network bursting using experimentally constrained single compartment CA3 hippocampal neuron models with adaptation. J. Comput. Neurosci. 2012;33:21-40.

    Not in our library10.1007/s10827-011-0372-6

74. 74.

    Van Vreeswijk C, Sompolinsky H. Chaotic balanced state in a model of cortical circuits. Neural. Comput. 1998;10:1321-1371.

    Not in our library10.1162/089976698300017214

75. 75.

    Crandall, S. R. & Nick, T. A. Neural population spiking activity during singing: adult and longitudinal developmental recordings in the zebra finch. CRCNS.org 10.6080/K0NP22C8 (2014).

    Not in our library

76. 76.

    Sawer, C. & Chan, D. Mutopia (Mutopia Project, 2008).

    Not in our library

77. 77.
    Supervised learning in spiking neural networks with FORCE training · 2017

    No notice

Cited by papers we hold 6

• Intrinsic bursts facilitate learning of Lévy flight movements in recurrent neural network models · 2022
  Not checked
• Neural heterogeneity promotes robust learning · 2021
  No notice
• Optimal learning with excitatory and inhibitory synapses. · 2020
  Not checked
• A solution to the learning dilemma for recurrent networks of spiking neurons · 2020
  No notice
• Contrasting the effects of adaptation and synaptic filtering on the timescales of dynamics in recurrent networks. · 2019
  Not checked
• How single neuron properties shape chaotic dynamics and signal transmission in random neural networks. · 2019
  Not checked

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.