Revisit Multinomial Logistic Regression in Deep Learning: Data Dependent Model Initialization for Image Recognition
원문 초록
We study in this paper how to initialize the parameters of multinomial logistic regression (a fully connected layer followed with softmax and cross entropy loss), which is widely used in deep neural network (DNN) models for classification problems. As logistic regression is widely known not having a closed-form solution, it is usually randomly initialized, leading to several deficiencies especially in transfer learning where all the layers except for the last task-specific layer are initialized using a pre-trained model. The deficiencies include slow convergence speed, possibility of stuck in local minimum, and the risk of over-fitting. To address those deficiencies, we first study the properties of logistic regression and propose a closed-form approximate solution named regularized Gaussian classifier (RGC). Then we adopt this approximate solution to initialize the task-specific linear layer and demonstrate superior performance over random initialization in terms of both accuracy and convergence speed on various tasks and datasets. For example, for image classification, our approach can reduce the training time by 10 times and achieve 3.2% gain in accuracy for Flickr-style classification. For object detection, our approach can also be 10 times faster in training for the same accuracy, or 5% better in terms of mAP for VOC 2007 with slightly longer training.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
FrImCla: A Framework for Image Classification Using Traditional and Transfer Learning Techniques
A Robust Feature Downsampling Module for Remote Sensing Visual Tasks
DocXClassifier: High Performance Explainable Deep Network for Document Image Classification
A Vision-based Solution for Track Misalignment Detection
EPRNet: Efficient Pyramid Representation Network for Real-Time Street Scene Segmentation
Transfer learning for automatic brain tumor classification Using MRI Images.
토론
이 논문과 연결된 포럼 글이 아직 없습니다.