Learning Deep Representations with Probabilistic Knowledge Transfer
원문 초록
Knowledge Transfer (KT) techniques tackle the problem of transferring the knowledge from a large and complex neural network into a smaller and faster one. However, existing KT methods are tailored towards classification tasks and they cannot be used efficiently for other representation learning tasks. In this paper a novel knowledge transfer technique, that is capable of training a student model that maintains the same amount of mutual information between the learned representation and a set of (possible unknown) labels as the teacher model, is proposed. Apart from outperforming existing KT techniques, the proposed method allows for overcoming several limitations of existing methods providing new insight into KT as well as novel KT applications, ranging from knowledge transfer from handcrafted feature extractors to {cross-modal} KT from the textual modality into the representation extracted from the visual modality of the data.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
These are Not All the Features You are Looking For: A Fundamental Bottleneck In Supervised Pretraining
Facial Beauty Analysis Using Distribution Prediction and CNN Ensembles
Multi-modal Siamese Network for Entity Alignment
In-Car driver response classification using Deep Learning (CNN) based Computer Vision
FrImCla: A Framework for Image Classification Using Traditional and Transfer Learning Techniques
RaMen: Multi-Strategy Multi-Modal Learning for Bundle Construction
토론
이 논문과 연결된 포럼 글이 아직 없습니다.