Knowledge Distillation of Convolutional Neural Networks through Feature Map Transformation using Decision Trees
원문 초록
The interpretation of reasoning by Deep Neural Networks (DNN) is still challenging due to their perceived black-box nature. Therefore, deploying DNNs in several real-world tasks is restricted by the lack of transparency of these models. We propose a distillation approach by extracting features from the final layer of the convolutional neural network (CNN) to address insights to its reasoning. The feature maps in the final layer of a CNN are transformed into a one-dimensional feature vector using a fully connected layer. Subsequently, the extracted features are used to train a decision tree to achieve the best accuracy under constraints of depth and nodes. We use the medical images of dermaMNIST, octMNIST, and pneumoniaMNIST from the medical MNIST datasets to demonstrate our proposed work. We observed that performance of the decision tree is as good as a CNN with minimum complexity. The results encourage interpreting decisions made by the CNNs using decision trees.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
DVFL-Net: A Lightweight Distilled Video Focal Modulation Network for Spatio-Temporal Action Recognition
The Trilemma of Truth in Large Language Models
G$^{2}$D: Boosting Multimodal Learning with Gradient-Guided Distillation
FedBKD: Distilled Federated Learning to Embrace Gerneralization and Personalization on Non-IID Data
Tackling Data Heterogeneity in Federated Learning through Knowledge Distillation with Inequitable Aggregation
GNN's Uncertainty Quantification using Self-Distillation
토론
이 논문과 연결된 포럼 글이 아직 없습니다.