Improving Predictor Reliability with Selective Recalibration
원문 초록
A reliable deep learning system should be able to accurately express its confidence with respect to its predictions, a quality known as calibration. One of the most effective ways to produce reliable confidence estimates with a pre-trained model is by applying a post-hoc recalibration method. Popular recalibration methods like temperature scaling are typically fit on a small amount of data and work in the model's output space, as opposed to the more expressive feature embedding space, and thus usually have only one or a handful of parameters. However, the target distribution to which they are applied is often complex and difficult to fit well with such a function. To this end we propose \textit{selective recalibration}, where a selection model learns to reject some user-chosen proportion of the data in order to allow the recalibrator to focus on regions of the input space that can be well-captured by such a model. We provide theoretical analysis to motivate our algorithm, and test our method through comprehensive experiments on difficult medical imaging and zero-shot classification tasks. Our results show that selective recalibration consistently leads to significantly lower calibration error than a wide range of selection and recalibration baselines.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
DEARLi: Decoupled Enhancement of Recognition and Localization for Semi-supervised Panoptic Segmentation
Efficient Medical Vision-Language Alignment Through Adapting Masked Vision Models
GeoVision Labeler: Zero-Shot Geospatial Classification with Vision and Language Models
AmorLIP: Efficient Language-Image Pretraining via Amortization
Uniformity First: Uniformity-aware Test-time Adaptation of Vision-language Models against Image Corruption
StarFT: Robust Fine-tuning of Zero-shot Models via Spuriosity Alignment
토론
이 논문과 연결된 포럼 글이 아직 없습니다.