Efficient SGD Neural Network Training via Sublinear Activated Neuron Identification
원문 초록
Deep learning has been widely used in many fields, but the model training process usually consumes massive computational resources and time. Therefore, designing an efficient neural network training method with a provable convergence guarantee is a fundamental and important research question. In this paper, we present a static half-space report data structure that consists of a fully connected two-layer neural network for shifted ReLU activation to enable activated neuron identification in sublinear time via geometric search. We also prove that our algorithm can converge in $O(M^2/\epsilon^2)$ time with network size quadratic in the coefficient norm upper bound $M$ and error term $\epsilon$.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
LightSNN: Lightweight Architecture Search for Sparse and Accurate Spiking Neural Networks
Automatic Joint Structured Pruning and Quantization for Efficient Neural Network Training and Compression
When do they StOP?: A First Step Towards Automatically Identifying Team Communication in the Operating Room
HALO: Hadamard-Assisted Lower-Precision Optimization for LLMs
Learn2Mix: Training Neural Networks Using Adaptive Data Integration
Efficient Neural Network Encoding for 3D Color Lookup Tables
토론
이 논문과 연결된 포럼 글이 아직 없습니다.