Understanding disentangling in β-VAE

We present new intuitions and theoretical assessments of the emergence of representation in variational autoencoders. Taking a-distortion theory perspective, we show the circumstances under which aligned with the underlying generative factors of variation of emerge when optimising the modified ELBO bound in \β-VAE, as training. From these insights, we propose a modification to the training of \β-VAE, that progressively increases the information capacity of latent code during training. This modification facilitates the robust of disentangled representations in \β-VAE, without the previous-off in reconstruction accuracy.