Recurrent neural network training with preconditioned stochastic gradient descent
원문 초록
This paper studies the performance of a recently proposed preconditioned stochastic gradient descent (PSGD) algorithm on recurrent neural network (RNN) training. PSGD adaptively estimates a preconditioner to accelerate gradient descent, and is designed to be simple, general and easy to use, as stochastic gradient descent (SGD). RNNs, especially the ones requiring extremely long term memories, are difficult to train. We have tested PSGD on a set of synthetic pathological RNN learning problems and the real world MNIST handwritten digit recognition task. Experimental results suggest that PSGD is able to achieve highly competitive performance without using any trick like preprocessing, pretraining or parameter tweaking.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Compact and Efficient Neural Networks for Image Recognition Based on Learned 2D Separable Transform
Integrated Gradient Correlation: a Dataset-wise Attribution Method
PolyLUT: Learning Piecewise Polynomials for Ultra-Low Latency FPGA LUT-based Inference
A CNN Based Framework for Unistroke Numeral Recognition in Air-Writing
Cloud-Based Deep Learning: End-To-End Full-Stack Handwritten Digit Recognition
Learning the Precise Feature for Cluster Assignment
토론
이 논문과 연결된 포럼 글이 아직 없습니다.