arXiv:2303.14430

Beta-VAE has 2 Behaviors: PCA or ICA?

Zhouzheng Li, Hao liu

제출 2023-03-25

원문 초록

Beta-VAE is a very classical model for disentangled representation learning, the use of an expanding bottleneck that allow information into the decoder gradually is key to representation disentanglement as well as high-quality reconstruction. During recent experiments on such fascinating structure, we discovered that the total amount of latent variables can affect the representation learnt by the network: with very few latent variables, the network tend to learn the most important or principal variables, acting like a PCA; with very large numbers of latent variables, the variables tend to be more disentangled, and act like an ICA. Our assumption is that the competition between latent variables while trying to gain the most information bandwidth can lead to this phenomenon.

함께 살펴볼 논문

공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.

arXiv:2506.19133공통 태그 Decoder, Representation Learning공식 코드 · Papers with Code

Riemannian generative decoder

토론

이 논문과 연결된 포럼 글이 아직 없습니다.