Do the Frankenstein, or how to achieve better out-of-distribution performance with manifold mixing model soup
원문 초록
The standard recipe applied in transfer learning is to finetune a pretrained model on the task-specific dataset with different hyperparameter settings and pick the model with the highest accuracy on the validation dataset. Unfortunately, this leads to models which do not perform well under distribution shifts, e.g. when the model is given graphical sketches of the object as input instead of photos. In order to address this, we propose the manifold mixing model soup, an algorithm which mixes together the latent space manifolds of multiple finetuned models in an optimal way in order to generate a fused model. We show that the fused model gives significantly better out-of-distribution performance (+3.5 % compared to best individual model) when finetuning a CLIP model for image classification. In addition, it provides also better accuracy on the original dataset where the finetuning has been done.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
DocXClassifier: High Performance Explainable Deep Network for Document Image Classification
A Vision-based Solution for Track Misalignment Detection
EPRNet: Efficient Pyramid Representation Network for Real-Time Street Scene Segmentation
Transfer learning for automatic brain tumor classification Using MRI Images.
FrImCla: A Framework for Image Classification Using Traditional and Transfer Learning Techniques
Hashed Watermark as a Filter: Defeating Forging and Overwriting Attacks in Weight-based Neural Network Watermarking
토론
이 논문과 연결된 포럼 글이 아직 없습니다.