Auto-encoding Variational Bayes 阅读笔记

Notation

• pθ(z|x)$p_{\theta }(z|x)$: intractable posterior
• pθ(x|z)$p_{\theta }(x|z)$: probabilistic decoder
• qϕ(z|x)$q_{\varphi }(z|x)$: recognition model, variational approximation to pθ(z|x)$p_{\theta }(z|x)$, also regarded as a probabilistic encoder
• pθ(z)pθ(x|z)$p_{\theta }(z)p_{\theta }(x|z)$: generative model
• ϕ$\varphi$: variational parameters
• θ$\theta$: generative parameters

Abbreviation

• SGVB: Stochastic Gradient Variational Bayes
• AEVB: auto-encoding VB
• ML: maximum likelihood
• MAP: maximum a posteriori

---

Motivation

• Problem

  • How to perform efficient inference and learning in directed probabilistic models, in the presence of continuous latent variables with intractable posterior distribution pθ(z|x)$p_{\theta }(z|x)$ and large datasets?

• Existing Solution and Difficulty

  • VB: involves the optimization of an approximation to the intractable posterior
  • mean-field: requires analytical solutions of expectations w.r.t. the approximate posterior, which are also intractable in the general case

---

Contribution of this paper

• (1) SGVB estimator: an estimator of the variational lower bound
  
  • yielded by a reparameterization of the variational lower bound
  • simple & differentiable & unbiased
  • straightforwad to optimize using standard SG ascent techniques
• (2) AEVB algorithm
  
  • using SGVB to optimize a recognition model that allows us to perform very efficient approximate posterior inference using simple ancestral sampling, which in turn allows us to efficiently learn the model parameters, without the need of expensive iterative inference schemes (such as MCMC) per datapoint.
  • condition: i.i.d. datasets X={x(i)}Ni=1$X=\{x^{(i)}{\}}_{i=1}^N$ & continuous latent variable z$z$ per datapoint

---

Methodology

assumption

• directed graphical models with continuous latent variables
• i.i.d. dataset with latent variables per datapoint
  
  • where we like to perform
    
    • ML or MAP inference on the (global) paramters θ$\theta$
    • variational inference on the latent variable z$z$
• pθ(z)$p_{\theta }(z)$ and pθ(x|z)$p_{\theta }(x|z)$: both PDFs are differentiable almost everywhere w.r.t. both θ$\theta$ and z$z$

target case

• intractability
  
  • pθ(x)=∫pθ(z)pθ(x|z)dz$p_{\theta }(x)=\int p_{\theta }(z)p_{\theta }(x|z)dz$: so we cannot evaluate or differentiate it.
  • pθ(z|x)=pθ(x|z)pθ(z)pθ(x)$p_{\theta }(z|x)=\frac{p_{\theta }(x|z)p_{\theta }(z)}{p_{\theta }(x)}$: so the EM algorithm cannot be used.
  • the required integrals for any reasonable mean-field VB algorithm: so the VB algorithm cannot be used.
  • in cases of moderately complicated likelihood function, e.g. in a neural network with a nonlinear hidden layer
• a large dataset
  
  • batch optimization is too costly => minibatch or single datapoints
  • sampling bases solutions are too slow, e.g. Monte Carlo EM, since it involves a typically expensive sampling loop per datapoint.

solution and application

• efficient approximate ML or MAP estimation for θ$\theta$ (Full): Appendix F
  
  • allow us to mimic the hidden random process and generate artificial data that resemble the real data
• efficient approximate posterior inference pθ(z|x)$p_{\theta }(z|x)$ for a choice of θ$\theta$
  
  • useful for coding or data representation tasks
• efficient approximate marginal inference of x$x$: Appendix D
  
  • allow us to perform all kinds of inference tasks where p(x)$p(x)$ is required, such as image denoising, inpainting, and super-resolution.

1. derivation of the variational bound

Here we use xi$x^i$ to represent x(i)$x^{(i)}$

Becuase DKL≥0$D_{KL}\ge 0$, so LB$L_B$ is called the (variational) lower bound, then we have

• −DKL[qϕ(z|xi)∥pθ(z)]$-D_{KL}[q_{\varphi }(z|x^i)\parallel p_{\theta }(z)]$: act as a regularizer
• Eqϕ(z|xi)[logpθ(xi|z)]$E_{q_{\varphi }(z|x^i)}[\log p_{\theta }(x^i|z)]$: a an expected negative reconstruction error
• TARGET: defferentiate and optimize LB$L_B$ w.r.t. both ϕ$\varphi$ and θ$\theta$. However, ∇ϕLB${\mathrm{\nabla }}_{\varphi }{L}_B$ is problematic.

2. Solution-1 Naive Monte Carlo gradient estimator

• disadvantages: high variance & impractical for this purpose, 原文中出现的公式如下：
  
  ∇ϕEqϕ(z)[f(z)]=Eqϕ(z)[f(z)∇qϕ(z)logqϕ(z)]≃1L∑l=1Lf(z)∇qϕ(zl)logqϕ(zl)where zl∼qϕ(z|xi)$$\begin{gathered} {\mathrm{\nabla }}_{\varphi }{E}_{q_{\varphi }(z)}[f(z)]=E_{q_{\varphi }(z)}[f(z){\mathrm{\nabla }}_{q_{\varphi }(z)}\log q_{\varphi }(z)]\simeq \frac{1}{L}\sum _{l=1}^{L}f(z){\mathrm{\nabla }}_{q_{\varphi }(z^l)}\log q_{\varphi }(z^l) \\ \text{where}{z}^l\sim q_{\varphi }(z|x^i) \end{gathered}$$

但由于我还没来得及学Monte Carlo Gradient Estimator的理论，根据VAE这边论文后面的公式，个人觉得…上面的公式应该是（数学功底不够深厚，不确定二者是否等价）：

3. Solution-2 SGVB estimator

• reparamterize z̃ ∼qϕ(z|x)$\tilde{z}\sim q_{\varphi }(z|x)$ using a differentiable transformation gϕ(ϵ,x)$g_{\varphi }(ϵ,x)$ of an (auxiliary) noise variable ϵ$ϵ$
  
  • under certain mild conditions for a chosen approximate posterior qϕ(z|x)$q_{\varphi }(z|x)$
    
    z̃ =gϕ(ϵ,x) with ϵ∼p(ϵ)$$\tilde{z}=g_{\varphi }(ϵ,x)withϵ\sim p(ϵ)$$
• form Monte Carlo estimates as follows:
  
  qϕ(z|x)∏idzi=p(ϵ)∏idϵi∫qϕ(z|x)f(z)dz=∫p(ϵ)f(z)dϵ=∫p(ϵ)f(gϕ(ϵ,x))dϵEqϕ(z|xi)[f(z)]=Ep(ϵ)[f(gϕ(ϵ,xi))]≃1L∑l=1Lf(gϕ(ϵl,xi))where ϵl∼p(ϵ)$$\begin{gathered} q_{\varphi }(z|x)\prod _{i}d{z}_i=p(ϵ)\prod _{i}d{ϵ}_i \\ \int q_{\varphi }(z|x)f(z)dz=\int p(ϵ)f(z)dϵ=\int p(ϵ)f(g_{\varphi }(ϵ,x))dϵ \\ {E}_{q_{\varphi }(z|x^i)}[f(z)]=E_{p(ϵ)}[f(g_{\varphi }(ϵ,x^i))]\simeq \frac{1}{L}\sum _{l=1}^{L}f(g_{\varphi }({ϵ}^l,x^i)) \\ where{ϵ}^l\sim p(ϵ) \end{gathered}$$
• apply the MC estimator technique to LB(θ,ϕ;xi)$L_B(\theta ,\varphi ;x^i)$, yielding 2 SGVB estimator L̃ A${\tilde{L}}^A$ and L̃ B${\tilde{L}}^B$:
  
  L̃ A(θ,ϕ;xi)=1L∑l=1Llogpθ(xi,zi,l)−logqϕ(zi,l|xi)L̃ B(θ,ϕ;xi)=−DKL(qϕ(z|xi)∥pθ(z))+1L∑l=1Llogpθ(xi|zi,l)where zi,l=gϕ(ϵi,l,xi) and ϵl∼p(ϵ)$$\begin{gathered} {\tilde{L}}^A(\theta ,\varphi ;x^i)=\frac{1}{L}\sum _{l=1}^L\log p_{\theta }(x^i,z^{i,l})-\log q_{\varphi }(z^{i,l}|x^i) \\ {\tilde{L}}^B(\theta ,\varphi ;x^i)=-D_{KL}(q_{\varphi }(z|x^i)\parallel p_{\theta }(z))+\frac{1}{L}\sum _{l=1}^L\log p_{\theta }(x^i|z^{i,l}) \\ \text{where }{z}^{i,l}=g_{\varphi }({ϵ}^{i,l},x^i)\text{and}{ϵ}^l\sim p(ϵ) \end{gathered}$$
• minibatch with size M$M$
  
  LB(θ,ϕ;X)≃L̃ M(θ,ϕ;XM)=NM∑i=1ML̃ (θ,ϕ;xi)$$L_B(\theta ,\varphi ;X)\simeq {\tilde{L}}^M(\theta ,\varphi ;X^M)=\frac{N}{M}\sum {i=1}^M\tilde{L}(\theta ,\varphi ;x^i)$$
  
  
  • the number of samples L per datapoint can be set to 1 as long as the minibatch size M$M$ was large enough, e.g. M=100$M=100$

3.1 Minibatch version of AEVB algorithm

• θ,ϕ←$\theta ,\varphi \leftarrow$ Initialize parameters
• repeate
  
  • XM←$X^M\leftarrow$ Random minibatch of M$M$ datapoints
  • ϵ←$ϵ\leftarrow$ Random samples from noise distribution p(ϵ)$p(ϵ)$
  • g←∇θ,ϕL̃ M(θ,ϕ;XM,ϵ)$g\leftarrow {\mathrm{\nabla }}_{\theta ,\varphi }{\tilde{L}}^M(\theta ,\varphi ;X^M,ϵ)$
  • θ,ϕ←$\theta ,\varphi \leftarrow$ Update parameters using gradients g$g$ (e.g. SGD or Adagrad)
• until convergence of parameters θ,ϕ$\theta ,\varphi$
• return θ,ϕ$\theta ,\varphi$

chosen of qϕ(z|x)$q_{\varphi }(z|x)$, p(ϵ)$p(ϵ)$, and gϕ(ϵ,x)$g_{\varphi }(ϵ,x)$

There are three basic approaches:

• Tractable inverse CDF. Let ϵ∼U(0,I)$ϵ\sim U(0,I)$, gϕ(ϵ,x)$g_{\varphi }(ϵ,x)$ be the inverse CDF of qϕ(z|x)$q_{\varphi }(z|x)$.
  
  • Examples: Exponential, Cauchy, Logistic, Rayleigh, Pareto, Weibull, Reciprocal, Gompertz, Gumbel and Erlang distributions.
• ”location-scale” family of distributions: choose the standard distribution (with location = 0, scale = 1) as the auxiliary variable ϵ$ϵ$, and let g(⋅)=location+scale⋅ϵ$g(\cdot )=location+scale\cdot ϵ$
  
  • Examples: Laplace, Elliptical, Student’s t, Logistic, Uniform, Triangular and Gaussian distributions.
  • 下文介绍的VAE即适用于此种情况
• Composition: It is often possible to express random variables as different transformations of auxiliary variables.
  
  • Examples: Log-Normal (exponentiation of normally distributed variable), Gamma (a sum over exponentially distributed variables), Dirichlet (weighted sum of Gamma variates), Beta, Chi-Squared, and F distributions.

---

Variational Auto-Encoder

• let pθ(z)=N(z;0,I)$p_{\theta }(z)=N(z;0,I)$
• pθ(z|x)$p_{\theta }(z|x)$ is intractable
• use a neural network for qϕ(z|x)$q_{\varphi }(z|x)$
• ϕ$\varphi$ and θ$\theta$ are optimized jointly with the AEVB algorithm
• params of pθ(x|z)$p_{\theta }(x|z)$ are computed from z$z$ with a MLP (multi-layered perceptrons, a fully-connected neural network with a hidden layer)
  
  • multivariae Gaussian: in case of real-valued data
    
    logp(x|z)=logN(x;μ,σ2I)where μ=Wμh+bμlogσ2=Wσh+bσh=tanh(Whz+bh)$$\begin{gathered} \log p(x|z)=\log N(x;\mu ,{\sigma }^{2}I) \\ \text{where}\mu =W_{\mu }h+b_{\mu } \\ \log {\sigma }^2=W_{\sigma }h+b_{\sigma } \\ h=\tanh (W_{h}z+b_h) \end{gathered}$$
  • Bernouli: incase of binary data
    
    logp(x|z)=∑i=1Dxilogyi+(1−xi)⋅log(1−yi)where y=fσ(Wytanh(Wh+bh)+by)fσ(⋅):elementwise sigmoid activation function$$\begin{gathered} \log p(x|z)=\sum _{i=1}^D{x}_i\log y_i+(1-x_i)\cdot \log (1-y_i) \\ \text{where}y=f_{\sigma }(W_y\tanh (W_h+b_h)+b_y) \\ {f}_{\sigma }(\cdot ):\text{elementwise sigmoid activation function} \end{gathered}$$
    
    
    

From what mentioned above, we have:

where μi${\mu }^i$ and σi${\sigma }^i$ are outputs of the encoding MLP.

We sample form zi,l∼qϕ(z|xi)$z^{i,l}\sim q_{\varphi }(z|x^i)$ using zi,l=gϕ(xi,ϵl)=μi+σi⊙ϵl$z^{i,l}=g_{\varphi }(x^i,{ϵ}^l)={\mu }^i+{\sigma }^i\odot {ϵ}^l$, where ϵl∼N(0,I)${ϵ}^l\sim N(0,I)$ and ⊙$\odot$ denotes element-wize product.

Here both pθ(z)$p_{\theta }(z)$ and qϕ(z|x)$q_{\varphi }(z|x)$ are Gaussian, so we can use the estimator L̃ B${\tilde{L}}^B$, since the KL divergence item is analytical. Then we have:

Solution of −DKL(qϕ(z)∥pθ(z))$-D_{KL}(q_{\varphi }(z)\parallel p_{\theta }(z))$, Gaussian case

Let J$J$ be the dimensionality of z$z$, then we have:

And:

Therefore:

此处附上相关证明：

Visulization

Since the prior of the latent space is Gaussian, linearly spaced coordinates on the unit square were transformed through the inverse CDF of the Gaussian to produce values of the latent variables z$z$.