Exploring the Limits of KV Cache Compression in Visual Autoregressive Transformers
원문 초록
A fundamental challenge in Visual Autoregressive models is the substantial memory overhead required during inference to store previously generated representations. Despite various attempts to mitigate this issue through compression techniques, prior works have not explicitly formalized the problem of KV-cache compression in this context. In this work, we take the first step in formally defining the KV-cache compression problem for Visual Autoregressive transformers. We then establish a fundamental negative result, proving that any mechanism for sequential visual token generation under attention-based architectures must use at least $\Omega(n^2 d)$ memory, when $d = \Omega(\log n)$, where $n$ is the number of tokens generated and $d$ is the embedding dimensionality. This result demonstrates that achieving truly sub-quadratic memory usage is impossible without additional structural constraints. Our proof is constructed via a reduction from a computational lower bound problem, leveraging randomized embedding techniques inspired by dimensionality reduction principles. Finally, we discuss how sparsity priors on visual representations can influence memory efficiency, presenting both impossibility results and potential directions for mitigating memory overhead.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Active Learning for Manifold Gaussian Process Regression
Distributed Lyapunov Functions for Nonlinear Networks
Deep Symmetric Autoencoders from the Eckart-Young-Schmidt Perspective
Enabling stratified sampling in high dimensions via nonlinear dimensionality reduction
Assessing parameter identifiability of a hemodynamics PDE model using spectral surrogates and dimension reduction
Biological Pathway Guided Gene Selection Through Collaborative Reinforcement Learning
토론
이 논문과 연결된 포럼 글이 아직 없습니다.