A Measure of Information Available for Inference

The mutual information between the state of a neural network and the state of the external world represents the amount of information stored in the neural network that is associated with the external world. In contrast, the surprise of the sensory input indicates the unpredictability of the current input. In other words, this is a measure of inference ability, and an upper bound of the surprise is known as the variational free energy. According to the free-energy principle (FEP), a neural network continuously minimizes the free energy to perceive the external world. For the survival of animals, inference ability is considered to be more important than simply memorized information. In this study, the free energy is shown to represent the gap between the amount of information stored in the neural network and that available for inference. This concept involves both the FEP and the infomax principle, and will be a useful measure for quantifying the amount of information available for inference.

Abstract licence: CC BY

1. 10.

   Hierarchical Models in the Brain · 2008

2. 12.

   Active inference, communication and hermeneutics · 2015

3. 13.

   Towards a Mathematical Theory of Cortical Micro-circuits · 2009

4. 16.

   A Duet for one · 2015

5. 21.

   Life as we know it · 2013

6. 24.

   Information and Efficiency in the Nervous System—A Synthesis · 2013

7. 45.

   Reinforcement Learning or Active Inference? · 2009

8. 60.

   Neuromodulated Spike-Timing-Dependent Plasticity, and Theory of Three-Factor Learning Rules · 2016

9. 69.

   Deep learning · 2015

70 more references, whose wording the paper’s licence keeps private.

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.