Equilibrium Propagation: Bridging the Gap between Energy-Based Models and Backpropagation

Abstract

We introduce Equilibrium Propagation, a learning framework for energy-based models. It involves only one kind of neural computation, performed in both the first phase (when the prediction is made) and the second phase of training (after the target or prediction error is revealed). Although this algorithm computes the gradient of an objective function just like Backpropagation, it does not need a special computation or circuit for the second phase, where errors are implicitly propagated. Equilibrium Propagation shares similarities with Contrastive Hebbian Learning and Contrastive Divergence while solving the theoretical issues of both algorithms: our algorithm computes the gradient of a well-defined objective function. Because the objective function is defined in terms of local perturbations, the second phase of Equilibrium Propagation corresponds to only nudging the prediction (fixed point or stationary distribution) toward a configuration that reduces prediction error. In the case of a recurrent multi-layer supervised network, the output units are slightly nudged toward their target in the second phase, and the perturbation introduced at the output layer propagates backward in the hidden layers. We show that the signal "back-propagated" during this second phase corresponds to the propagation of error derivatives and encodes the gradient of the objective function, when the synaptic update corresponds to a standard form of spike-timing dependent plasticity. This work makes it more plausible that a mechanism similar to Backpropagation could be implemented by brains, since leaky integrator neural computation performs both inference and error back-propagation in our model. The only local difference between the two phases is whether synaptic changes are allowed or not. We also show experimentally that multi-layer recurrently connected networks with 1, 2, and 3 hidden layers can be trained by Equilibrium Propagation on the permutation-invariant MNIST task.

Full text

Its licence does not allow us to show the full text. Read it at PubMed Central:

References 37

1. 1.

   Almeida L. B. (1987 ). A learning rule for asynchronous perceptrons with feedback in a combinatorial environment, in Proceedings of the IEEE First International Conference on Neural Networks, Vol. 2 (San Diego, CA; New York, NY: IEEE), 609–618.

   Not in our library

2. 2.

   Arora S. Liang Y. Ma T. (2015 ). Why Are Deep Nets Reversible: A Simple Theory, with Implications for Training. Technical Report, arXiv:1511.05653, Princeton University.

   Not in our library

3. 3.

   Bengio Y. Fischer A. (2015 ). Early Inference in Energy-Based Models Approximates Back-Propagation. Technical Report, arXiv:1510.02777, Universite de Montreal.

   Not in our library

4. 4.

   Bengio Y. Lee D.-H. Bornschein J. Lin Z. (2015a ). Towards biologically plausible deep learning. arXiv:1502.04156.

   Not in our library

5. 5.

   Bengio Y. Mesnard T. Fischer A. Zhang S. Wu Y. (2015b ). STDP as presynaptic activity times rate of change of postsynaptic activity. arXiv:1509.05936.

   Not in our library

6. 6.

   Bengio Y. Mesnard T. Fischer A. Zhang S. Wu Y. (2017 ). STDP as presynaptic activity times rate of change of postsynaptic activity approximates backpropagation. Neural Comput. 29, 1–23.

   Not in our library10.1162/neco_a_00934

7. 7.

   Bengio Y. Scellier B. Bilaniuk O. Sacramento J. Senn W. (2016 ). Feedforward initialization for fast inference of deep generative networks is biologically plausible. arXiv:1606.01651.

   Not in our library

8. 8.

   Berkes P. Orban G. Lengyel M. Fiser J. (2011 ). Spontaneous cortical activity reveals hallmarks of an optimal internal model of the environment. Science 331, 83–87.

   Not in our library10.1126/science.1195870

9. 9.

   Bi G. Poo M. (2001 ). Synaptic modification by correlated activity: Hebb's postulate revisited. Annu. Rev. Neurosci. 24, 139–166.

   Not in our library10.1146/annurev.neuro.24.1.139

10. 10.

    Bienenstock E. L. Cooper L. N. Munro P. W. (1982 ). Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex. J. Neurosci. 2, 32–48.

    Not in our library10.1523/jneurosci.02-01-00032.1982

11. 11.

    Feldman D. E. (2012 ). The spike timing dependence of plasticity. Neuron 75, 556–571.

    Not in our library10.1016/j.neuron.2012.08.001

12. 12.

    Friston K. J. Stephan K. E. (2007 ). Free-energy and the brain. Synthese 159, 417–458.

    Not in our library10.1007/s11229-007-9237-y

13. 13.

    Froemke R. C. Dan Y. (2002 ). Spike-timing-dependent synaptic modification induced by natural spike trains. Nature 416, 433–438.

    Not in our library10.1038/416433a

14. 14.

    Gerstner W. Kempter R. van Hemmen J. Wagner H. (1996 ). A neuronal learning rule for sub-millisecond temporal coding. Nature 386, 76–78.

    Not in our library10.1038/383076a0

15. 15.

    Gjorgjievaa J. Clopathb C. Audetc J. Pfister J.-P. (2011 ). A triplet spike-timing dependent plasticity model generalizes the bienenstock cooper munro rule to higher-order spatiotemporal correlations. Proc. Natl. Acad. Sci. U.S.A. 108, 19383–19388.

    Not in our library10.1073/pnas.1105933108

16. 16.

    Glorot X. Bengio Y. (2010 ). Understanding the difficulty of training deep feedforward neural networks, in Proceedings of AISTATS' 2010 (Montréal, QC), 249–256.

    Not in our library

17. 17.

    Hertz J. A. Krogh A. Lautrup B. Lehmann T. (1997 ). Nonlinear backpropagation: doing backpropagation without derivatives of the activation function. IEEE Trans. Neural Netw. 8, 1321–1327.

    Not in our library10.1109/72.641455

18. 18.

    Hinton G. E. (2002 ). Training products of experts by minimizing contrastive divergence. Neural Comput. 14, 1771–1800.

    Not in our library10.1162/089976602760128018

19. 19.

    Hinton G. E. Sejnowski T. J. (1986 ). Learning and releaming in boltzmann machines, in Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol. 1, eds Rumelhart D. E. McClelland J. L. (Cambridge, MA: MIT Press), 282–317.

    Not in our library

20. 20.

    Hopfield J. J. (1984 ). Neurons with graded responses have collective computational properties like those of two-state neurons. Proc. Natl. Acad. Sci. U.S.A. 81, 3088–3092.

    Not in our library10.1073/pnas.81.10.3088

21. 21.

    Intrator N. Cooper L. N. (1992 ). Objective function formulation of the BCM theory of visual cortical plasticity: statistical connections, stability conditions. Neural Netw. 5, 3–17.

    Not in our library10.1016/s0893-6080(05)80003-6

22. 22.

    LeCun Y. Cortes C. (1998 ). The MNIST database of handwritten digits. Available online at: http://yann.lecun.com/exdb/mnist/

    Not in our library

23. 23.
    Random synaptic feedback weights support error backpropagation for deep learning · 2016

    No notice
24. 24.
    Spike-Timing-Dependent Plasticity: A Comprehensive Overview · 2012

    No notice
25. 25.

    Markram H. Sakmann B. (1995 ). Action potentials propagating back into dendrites triggers changes in efficacy. Soc. Neurosci. Abs. 21.

    Not in our library

26. 26.

    Mesnard T. Gerstner W. Brea J. (2016 ). Towards deep learning with spiking neurons in energy based models with contrastive hebbian plasticity. arXiv:1612.03214.

    Not in our library

27. 27.

    Movellan J. R. (1990 ). Contrastive Hebbian learning in the continuous Hopfield model, in Proceedings of the 1990 Connectionist Models Summer School (San Mateo, CA).

    Not in our library

28. 28.

    Ollivier Y. Tallec C. Charpiat G. (2015 ). Training Recurrent Networks Online without Backtracking. Technical report, arXiv:1507.07680, Centre National de la Recherche Scientifique.

    Not in our library

29. 29.

    O'Reilly R. C. (1996 ). Biologically plausible error-driven learning using local activation differences: the generalized recirculation algorithm. Neural Comput. 8, 895–938.

    Not in our library

30. 30.

    Pineda F. J. (1987 ). Generalization of back-propagation to recurrent neural networks. 59, 2229–2232.

    Not in our library10.1103/physrevlett.59.2229

31. 31.

    Salakhutdinov R. Hinton G. E. (2009 ). Deep Boltzmann machines, in International Conference on Artificial Intelligence and Statistic (AISTATS'2009) (Toronto, ON), 448–455.

    Not in our library

32. 32.

    Sutskever I. Tieleman T. (2010 ). On the convergence properties of contrastive divergence, in Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTATS), Vol. 9, eds Teh Y. W. Titterington M. (Toronto, ON), 789–795.

    Not in our library

33. 33.

    Tallec C. Ollivier Y. (2017 ). Unbiased online recurrent optimization. arXiv:1702.05043.

    Not in our library

34. 34.

    Tieleman T. (2008 ). Training restricted boltzmann machines using approximations to the likelihood gradient, in Proceedings of the 25th International Conference on Machine Learning (New York, NY: ACM), 1064–1071.

    Not in our library

35. 35.

    Vincent P. Larochelle H. Lajoie I. Bengio Y. Manzagol P.-A. (2010 ). Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res. 11, 3371–3408.

    Not in our library

36. 36.

    Xie X. Seung H. S. (2000 ). Spike-based learning rules and stabilization of persistent neural activity, in Advances in Neural Information Processing Systems 12, eds Solla S. Leen T. Müller K. (Boston, MA: MIT Press), 199–208.

    Not in our library

37. 37.

    Xie X. Seung H. S. (2003 ). Equivalence of backpropagation and contrastive Hebbian learning in a layered network. Neural Comput. 15, 441–454.

    Not in our library10.1162/089976603762552988

Cited by papers we hold 3

• Deep physical neural networks trained with backpropagation · 2022
  No notice
• Theories of Error Back-Propagation in the Brain · 2019
  No notice
• Computational Foundations of Natural Intelligence · 2017
  Not checked

Reads

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.