Tangent Space Sensitivity and Distribution of Linear Regions in ReLU Networks
원문 초록
Recent articles indicate that deep neural networks are efficient models for various learning problems. However they are often highly sensitive to various changes that cannot be detected by an independent observer. As our understanding of deep neural networks with traditional generalization bounds still remains incomplete, there are several measures which capture the behaviour of the model in case of small changes at a specific state. In this paper we consider adversarial stability in the tangent space and suggest tangent sensitivity in order to characterize stability. We focus on a particular kind of stability with respect to changes in parameters that are induced by individual examples without known labels. We derive several easily computable bounds and empirical measures for feed-forward fully connected ReLU (Rectified Linear Unit) networks and connect tangent sensitivity to the distribution of the activation regions in the input space realized by the network. Our experiments suggest that even simple bounds and measures are associated with the empirical generalization gap.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
AI Wizards at CheckThat! 2025: Enhancing Transformer-Based Embeddings with Sentiment for Subjectivity Detection in News Articles
Generative Latent Kernel Modeling for Blind Motion Deblurring
Remember Past, Anticipate Future: Learning Continual Multimodal Misinformation Detectors
Narrative Shift Detection: A Hybrid Approach of Dynamic Topic Models and Large Language Models
Characterizing Linguistic Shifts in Croatian News via Diachronic Word Embeddings
How Grounded is Wikipedia? A Study on Structured Evidential Support
토론
이 논문과 연결된 포럼 글이 아직 없습니다.