AudioLens: A Closer Look at Auditory Attribute Perception of Large Audio-Language Models
원문 초록
Understanding the internal mechanisms of large audio-language models (LALMs) is crucial for interpreting their behavior and improving performance. This work presents the first in-depth analysis of how LALMs internally perceive and recognize auditory attributes. By applying vocabulary projection on three state-of-the-art LALMs, we track how attribute information evolves across layers and token positions. We find that attribute information generally decreases with layer depth when recognition fails, and that resolving attributes at earlier layers correlates with better accuracy. Moreover, LALMs heavily rely on querying auditory inputs for predicting attributes instead of aggregating necessary information in hidden states at attribute-mentioning positions. Based on our findings, we demonstrate a method to enhance LALMs. Our results offer insights into auditory attribute processing, paving the way for future improvements.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Attributes Shape the Embedding Space of Face Recognition Models
Ref-Long: Benchmarking the Long-context Referencing Capability of Long-context Language Models
Emergent Semantics Beyond Token Embeddings: Transformer LMs with Frozen Visual Unicode Representations
MambaFusion: Height-Fidelity Dense Global Fusion for Multi-modal 3D Object Detection
The Trilemma of Truth in Large Language Models
Style-Aligned Image Composition for Robust Detection of Abnormal Cells in Cytopathology
토론
이 논문과 연결된 포럼 글이 아직 없습니다.