The SuperActivator Mechanism: Transformers Concentrate Reliable Concept Signals in the Tail

원문 초록

Concept vectors aim to enhance model interpretability by linking learned internal representations with human-understandable semantics, but their practical utility is often limited by noisy and inconsistent activations. In this work, we discover the SuperActivator Mechanism: a transformer dynamic that amplifies concept activation gaps, concentrating the most reliable concept evidence into a small set of highly activated tokens, which we call SuperActivators. We theoretically characterize this mechanism, proving that concept-aligned attention produces two competing effects: a tail amplification effect, where gaps among already-high activations grow disproportionately fast, and an equalization effect that eventually dominates as attention concentrates on extreme tokens. We validate the predicted tail amplification empirically, finding that in-concept activation distributions develop well-separated upper tails consistently across models, image and text modalities, and concept types. Moreover, we demonstrate the practical implications of SuperActivators for concept detection and localization. First, using only high-activation tokens as concept indicators improves detection by up to 14 F1 points over competing baselines. Second, restricting detection to the extreme activation tail is not merely sufficient but essential: including activations beyond the top 5-10% of the in-concept distribution introduces noise that degrades performance. Finally, using SuperActivators as instance-specific attribution targets yields more accurate and faithful concept localization heatmaps than using global concept vectors.