Convergence of Non-Convex Non-Concave GANs Using Sinkhorn Divergence

Sinkhorn divergence is a symmetric normalization of entropic regularized optimal transport. It is a smooth and continuous metrized weak-convergence with excellent geometric properties. We use it as an alternative for the minimax objective function in formulating generative adversarial networks. The optimization is defined with Sinkhorn divergence as the objective, under the non-convex and non-concave condition. This work focuses on the optimization&#x2019;s convergence and stability. We propose a first order sequential stochastic gradient descent ascent (SeqSGDA) algorithm. Under some mild approximations, the learning converges to local minimax points. Using the structural similarity index measure (SSIM), we supply a non-asymptotic analysis of the algorithm&#x2019;s convergence rate. Empirical evidences show a convergence rate, which is inversely proportional to the number of iterations, when tested on tiny colour datasets <monospace>Cats</monospace> and <monospace>CelebA</monospace> on the deep convolutional generative adversarial networks and ResNet neural architectures. The entropy regularization parameter <inline-formula> <tex-math notation="LaTeX">$\varepsilon $ </tex-math></inline-formula> is approximated to the SSIM tolerance <inline-formula> <tex-math notation="LaTeX">$\epsilon $ </tex-math></inline-formula>. We determine that the iteration complexity to return to an <inline-formula> <tex-math notation="LaTeX">$\epsilon $ </tex-math></inline-formula>-stationary point to be <inline-formula> <tex-math notation="LaTeX">$\mathcal {O}\left ({\kappa \, \log (\epsilon ^{-1})}\right)$ </tex-math></inline-formula>, where <inline-formula> <tex-math notation="LaTeX">$\kappa $ </tex-math></inline-formula> is a value that depends on the Sinkhorn divergence&#x2019;s convexity and the minimax step ratio in the SeqSGDA algorithm.

Abstract licence: CC BY-NC-ND

37 more references, whose wording the paper’s licence keeps private.

A person counts once a day per paper. We keep no cookie and no address: a reader is a hash that changes every day. Any month under five reads shows no number.