Non-Determinism in TensorFlow ResNets
원문 초록
We show that the stochasticity in training ResNets for image classification on GPUs in TensorFlow is dominated by the non-determinism from GPUs, rather than by the initialisation of the weights and biases of the network or by the sequence of minibatches given. The standard deviation of test set accuracy is 0.02 with fixed seeds, compared to 0.027 with different seeds---nearly 74\% of the standard deviation of a ResNet model is non-deterministic. For test set loss the ratio of standard deviations is more than 80\%. These results call for more robust evaluation strategies of deep learning models, as a significant amount of the variation in results across runs can arise simply from GPU randomness.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Evaluating the Performance of TAAF for image classification models
FewVS: A Vision-Semantics Integration Framework for Few-Shot Image Classification
Attention Feature Fusion Network via Knowledge Propagation for Automated Respiratory Sound Classification
Evolving Efficient CNN Based Model for Image Classification
IPCL: Iterative Pseudo-Supervised Contrastive Learning to Improve Self-Supervised Feature Representation
Trainable Fractional Fourier Transform
토론
이 논문과 연결된 포럼 글이 아직 없습니다.