Generalizations of Fano’s Inequality for Conditional Information Measures via Majorization Theory

Fano&#8217;s inequality is one of the most elementary, ubiquitous, and important tools in information theory. Using majorization theory, Fano&#8217;s inequality is generalized to a broad class of information measures, which contains those of Shannon and R&#233;nyi. When specialized to these measures, it recovers and generalizes the classical inequalities. Key to the derivation is the construction of an appropriate conditional distribution inducing a desired marginal distribution on a countably infinite alphabet. The construction is based on the infinite-dimensional version of Birkhoff&#8217;s theorem proven by R&#233;v&#233;sz [<i>Acta Math. Hungar.</i> <b>1962</b>, <i>3</i>, 188&#8722;198], and the constraint of maintaining a desired marginal distribution is similar to coupling in probability theory. Using our Fano-type inequalities for Shannon&#8217;s and R&#233;nyi&#8217;s information measures, we also investigate the asymptotic behavior of the sequence of Shannon&#8217;s and R&#233;nyi&#8217;s equivocations when the error probabilities vanish. This asymptotic behavior provides a novel characterization of the asymptotic equipartition property (AEP) via Fano&#8217;s inequality.

Abstract licence: CC BY

40 more references, whose wording the paper’s licence keeps private.

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.