Beyond Classification: Knowledge Distillation using Multi-Object Impressions
원문 초록
Knowledge Distillation (KD) utilizes training data as a transfer set to transfer knowledge from a complex network (Teacher) to a smaller network (Student). Several works have recently identified many scenarios where the training data may not be available due to data privacy or sensitivity concerns and have proposed solutions under this restrictive constraint for the classification task. Unlike existing works, we, for the first time, solve a much more challenging problem, i.e., "KD for object detection with zero knowledge about the training data and its statistics". Our proposed approach prepares pseudo-targets and synthesizes corresponding samples (termed as "Multi-Object Impressions"), using only the pretrained Faster RCNN Teacher network. We use this pseudo-dataset as a transfer set to conduct zero-shot KD for object detection. We demonstrate the efficacy of our proposed method through several ablations and extensive experiments on benchmark datasets like KITTI, Pascal and COCO. Our approach with no training samples, achieves a respectable mAP of 64.2% and 55.5% on the student with same and half capacity while performing distillation from a Resnet-18 Teacher of 73.3% mAP on KITTI.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Towards Reliable Detection of Empty Space: Conditional Marked Point Processes for Object Detection
A Real-Time System for Egocentric Hand-Object Interaction Detection in Industrial Domains
ECORE: Energy-Conscious Optimized Routing for Deep Learning Models at the Edge
DuET: Dual Incremental Object Detection via Exemplar-Free Task Arithmetic
Universal Fine-grained Visual Categorization by Concept Guided Learning
Integrity Assessment of Maritime Object Detection Impacted by Partial Camera Obstruction
토론
이 논문과 연결된 포럼 글이 아직 없습니다.