Conditional Rényi Divergence Saddlepoint and the Maximization of α -Mutual Information

R&#233;nyi-type generalizations of entropy, relative entropy and mutual information have found numerous applications throughout information theory and beyond. While there is consensus that the ways A. R&#233;nyi generalized entropy and relative entropy in 1961 are the &#8220;right&#8221; ones, several candidates have been put forth as possible mutual informations of order <inline-formula> <math display="inline"> <semantics> <mi>&#945;</mi> </semantics> </math> </inline-formula>. In this paper we lend further evidence to the notion that a Bayesian measure of statistical distinctness introduced by R. Sibson in 1969 (closely related to Gallager&#8217;s <inline-formula> <math display="inline"> <semantics> <msub> <mi>E</mi> <mn>0</mn> </msub> </semantics> </math> </inline-formula> function) is the most natural generalization, lending itself to explicit computation and maximization, as well as closed-form formulas. This paper considers general (not necessarily discrete) alphabets and extends the major analytical results on the saddle-point and saddle-level of the conditional relative entropy to the conditional R&#233;nyi divergence. Several examples illustrate the main application of these results, namely, the maximization of <inline-formula> <math display="inline"> <semantics> <mi>&#945;</mi> </semantics> </math> </inline-formula>-mutual information with and without constraints.

Abstract licence: CC BY

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.