Latent Space Characterization of Autoencoder Variants
원문 초록
Understanding the latent spaces learned by deep learning models is crucial in exploring how they represent and generate complex data. Autoencoders (AEs) have played a key role in the area of representation learning, with numerous regularization techniques and training principles developed not only to enhance their ability to learn compact and robust representations, but also to reveal how different architectures influence the structure and smoothness of the lower-dimensional non-linear manifold. We strive to characterize the structure of the latent spaces learned by different autoencoders including convolutional autoencoders (CAEs), denoising autoencoders (DAEs), and variational autoencoders (VAEs) and how they change with the perturbations in the input. By characterizing the matrix manifolds corresponding to the latent spaces, we provide an explanation for the well-known observation that the latent spaces of CAE and DAE form non-smooth manifolds, while that of VAE forms a smooth manifold. We also map the points of the matrix manifold to a Hilbert space using distance preserving transforms and provide an alternate view in terms of the subspaces generated in the Hilbert space as a function of the distortion in the input. The results show that the latent manifolds of CAE and DAE are stratified with each stratum being a smooth product manifold, while the manifold of VAE is a smooth product manifold of two symmetric positive definite matrices and a symmetric positive semi-definite matrix.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Expressive Score-Based Priors for Distribution Matching with Geometry-Preserving Regularization
Similarity-Guided Diffusion for Contrastive Sequential Recommendation
GeoRecon: Graph-Level Representation Learning for 3D Molecules via Reconstruction-Based Pretraining
SAR Image Despeckling Using Continuous Attention Module
Dual Dimensions Geometric Representation Learning Based Document Dewarping
ScoreAdv: Score-based Targeted Generation of Natural Adversarial Examples via Diffusion Models
토론
이 논문과 연결된 포럼 글이 아직 없습니다.