PyTorchKR Papers

최신 논문

총 4편
태그·기간 등 상세 필터

카테고리 탐색

로그인 후 이 태그 구독

arXiv:2607.10926v1cs.LG

The Spectral Structure of Latent Treatment Effects

Identifying heterogeneous treatment effects under unobserved confounding is central in observational causal inference. In proxy models with a discrete latent confounder, prior Synthetic Potential Outcomes (SPO) [Mazaheri-Squires-Uhler '25] recover the mixture of treatment effects through recursively constructed scalar moments. We show that this sequence is one projection of a more fundamental object. Under the same population factorization assumptions, there is an exact compressed observable operator: after projecting onto the shared proxy signal subspace, the difference of two treatment-arm quotient operators is similar to the diagonal matrix of latent treatment effects. Its eigenvalues are the latent effects; its lifted left eigenvectors, after anchor normalization, recover the target-proxy feature matrix and then the latent mixture proportions. Every scalar SPO moment is a bilinear functional of a power of this operator. The resulting estimator handles overcomplete proxy systems, replaces high-order scalar inversion with finite-dimensional spectral analysis, and admits high-probability first-order perturbation bounds for treatment effects, feature rows, and simplex-projected mixture weights.

arXiv:2607.05531v1cs.LG

$\mathbf{λ}$-VAE: 사후 붕괴 해소를 위한 분산 균등화

$\mathbfλ$-VAE: Variance Equalization for Posterior Collapse

변분 오토인코더(VAE)의 사후 붕괴(posterior collapse)를 야기하는 두 독립 원인인 그래디언트 불균형과 정보 격차를 수식으로 규명하고, 재매개변수화 단계에서 샘플링 노이즈를 차원별 지수로 스케일링하는 λ-VAE를 제안해 분산 균등화(variance equalization)를 달성한다. Binary MNIST, CIFAR-10 등 표준 벤치마크에서 붕괴 차원을 대폭 감소시키고 정보 용량을 최대 2.8× 향상시키며 재구성 품질을 최대 +0.33 BPD 개선하는 성과를 보인다.

arXiv:2602.15029v3cs.LG

Symmetry in language statistics shapes the geometry of model representations

The internal representations learned by language models consistently exhibit striking geometric structure: calendar months organize into a circle, historical years form a smooth one-dimensional manifold, and cities' latitudes and longitudes can be decoded using a linear probe. To explain this neural code, we first show that language statistics exhibit translation symmetry (for example, the frequency with which any two months co-occur in text depends only on the time interval between them). We prove that this symmetry governs these geometric structures in high-dimensional word embedding models, and we analytically derive the manifold geometry of word representations. These predictions empirically match large text embedding models and large language models. Moreover, the representational geometry persists at moderate embedding dimension even when the relevant statistics are perturbed (e.g., by removing all sentences in which two months co-occur). We prove that this robustness emerges naturally when the co-occurrence statistics are controlled by an underlying latent variable. Our results indicate that these representational manifolds originate in the statistical symmetries of natural language.

arXiv:2606.18509v1cs.LG

컨셉 변조 모델: 식별 가능성과 외삽을 위한 통합 프레임워크

Concept Modulation Models: A Unified Framework for Identifiability and Extrapolation

조건부 잠재 변수 모델에서 식별 가능성(identifiability)과 외삽(extrapolation) 보장이 모델별로 분리되어 있던 문제를, 속성이 변조기를 거쳐 잠재 컨셉을 생성하는 컨셉 변조 모델(CMMs)로 통합하고 속성 포텐셜(attribute potentials)을 통해 전이 기반 식별 가능성을 조건부 설정으로 확장한다. 관측 속성에서의 특징 일치가 관측되지 않은 속성까지 이어질 조건을 대수적으로 제시하며, nonlinear ICA, 인과 표현 학습(causal representation learning), 섭동 모델링 등 기존 결과의 공통 증명 구조를 드러내 모델별 강성 논증과 결합해 기존 보장을 회복한다.