Identifying Training Stop Point with Noisy Labeled Data
원문 초록
Training deep neural networks (DNNs) with noisy labels is a challenging problem due to over-parameterization. DNNs tend to essentially fit on clean samples at a higher rate in the initial stages, and later fit on the noisy samples at a relatively lower rate. Thus, with a noisy dataset, the test accuracy increases initially and drops in the later stages. To find an early stopping point at the maximum obtainable test accuracy (MOTA), recent studies assume either that i) a clean validation set is available or ii) the noise ratio is known, or, both. However, often a clean validation set is unavailable, and the noise estimation can be inaccurate. To overcome these issues, we provide a novel training solution, free of these conditions. We analyze the rate of change of the training accuracy for different noise ratios under different conditions to identify a training stop region. We further develop a heuristic algorithm based on a small-learning assumption to find a training stop point (TSP) at or close to MOTA. To the best of our knowledge, our method is the first to rely solely on the \textit{training behavior}, while utilizing the entire training set, to automatically find a TSP. We validated the robustness of our algorithm (AutoTSP) through several experiments on CIFAR-10, CIFAR-100, and a real-world noisy dataset for different noise ratios, noise types, and architectures.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Detect and Correct: A Selective Noise Correction Method for Learning with Noisy Labels
DGSolver: Diffusion Generalist Solver with Universal Posterior Sampling for Image Restoration
D$^2$-DPM: Dual Denoising for Quantized Diffusion Probabilistic Models
BiDM: Pushing the Limit of Quantization for Diffusion Models
How to Continually Adapt Text-to-Image Diffusion Models for Flexible Customization?
One Homography is All You Need: IMM-based Joint Homography and Multiple Object State Estimation
토론
이 논문과 연결된 포럼 글이 아직 없습니다.