On the Value of Labeled Data and Symbolic Methods for Hidden Neuron Activation Analysis
원문 초록
A major challenge in Explainable AI is in correctly interpreting activations of hidden neurons: accurate interpretations would help answer the question of what a deep learning system internally detects as relevant in the input, demystifying the otherwise black-box nature of deep learning systems. The state of the art indicates that hidden node activations can, in some cases, be interpretable in a way that makes sense to humans, but systematic automated methods that would be able to hypothesize and verify interpretations of hidden neuron activations are underexplored. This is particularly the case for approaches that can both draw explanations from substantial background knowledge, and that are based on inherently explainable (symbolic) methods. In this paper, we introduce a novel model-agnostic post-hoc Explainable AI method demonstrating that it provides meaningful interpretations. Our approach is based on using a Wikipedia-derived concept hierarchy with approximately 2 million classes as background knowledge, and utilizes OWL-reasoning-based Concept Induction for explanation generation. Additionally, we explore and compare the capabilities of off-the-shelf pre-trained multimodal-based explainable methods. Our results indicate that our approach can automatically attach meaningful class expressions as explanations to individual neurons in the dense layer of a Convolutional Neural Network. Evaluation through statistical analysis and degree of concept activation in the hidden layer show that our method provides a competitive edge in both quantitative and qualitative aspects compared to prior work.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
RealFactBench: A Benchmark for Evaluating Large Language Models in Real-World Fact-Checking
LiTEx: A Linguistic Taxonomy of Explanations for Understanding Within-Label Variation in Natural Language Inference
Does Rationale Quality Matter? Enhancing Mental Disorder Detection via Selective Reasoning Distillation
SNAPE-PM: Building and Utilizing Dynamic Partner Models for Adaptive Explanation Generation
Rethinking Vision-Language Model in Face Forensics: Multi-Modal Interpretable Forged Face Detector
Target-Augmented Shared Fusion-based Multimodal Sarcasm Explanation Generation
토론
이 논문과 연결된 포럼 글이 아직 없습니다.