Learning to Drop Out: An Adversarial Approach to Training Sequence VAEs
원문 초록
In principle, applying variational autoencoders (VAEs) to sequential data offers a method for controlled sequence generation, manipulation, and structured representation learning. However, training sequence VAEs is challenging: autoregressive decoders can often explain the data without utilizing the latent space, known as posterior collapse. To mitigate this, state-of-the-art models weaken the powerful decoder by applying uniformly random dropout to the decoder input. We show theoretically that this removes pointwise mutual information provided by the decoder input, which is compensated for by utilizing the latent space. We then propose an adversarial training strategy to achieve information-based stochastic dropout. Compared to uniform dropout on standard text benchmark datasets, our targeted approach increases both sequence modeling performance and the information captured in the latent space.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Riemannian generative decoder
Boosting Team Modeling through Tempo-Relational Representation Learning
Representation Learning through Multimodal Attention and Time-Sync Comments for Affective Video Content Analysis
SAR Image Despeckling Using Continuous Attention Module
Learning to Represent the Evolution of Dynamic Graphs with Recurrent Models
Making Language Model a Hierarchical Classifier and Generator
토론
이 논문과 연결된 포럼 글이 아직 없습니다.