Evaluating Deep Learning in SystemML using Layer-wise Adaptive Rate Scaling(LARS) Optimizer
원문 초록
Increasing the batch size of a deep learning model is a challenging task. Although it might help in utilizing full available system memory during training phase of a model, it results in significant loss of test accuracy most often. LARS solved this issue by introducing an adaptive learning rate for each layer of a deep learning model. However, there are doubts on how popular distributed machine learning systems such as SystemML or MLlib will perform with this optimizer. In this work, we apply LARS optimizer to a deep learning model implemented using SystemML.We perform experiments with various batch sizes and compare the performance of LARS optimizer with \textit{Stochastic Gradient Descent}. Our experimental results show that LARS optimizer performs significantly better than Stochastic Gradient Descent for large batch sizes even with the distributed machine learning framework, SystemML.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
AnalogVNN: A fully modular framework for modeling and optimizing photonic neural networks
Evolutionary bagging for ensemble learning
Neural network accelerator for quantum control
Deep Reinforcement Learning for Multi-Agent Interaction
Disparate Censorship & Undertesting: A Source of Label Bias in Clinical Machine Learning
Unsupervised machine learning framework for discriminating major variants of concern during COVID-19
토론
이 논문과 연결된 포럼 글이 아직 없습니다.