L2 Regularization
태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.
- 하위 태스크
- 0
- 평가표
- 0
- 보존된 평가 기록
- 0
See Weight Decay.
$L_{2}$ Regularization or Weight Decay, is a regularization technique applied to the weights of a neural network. We minimize a loss function compromising both the primary loss function and a penalty on the $L\_{2}$ Norm of the weights:
$$L\_{new}\left(w\right) = L\_{original}\left(w\right) + \lambda{w^{T}w}$$
where $\lambda$ is a value determining the strength of the penalty (encouraging smaller weights).
Weight decay can be incorporated directly into the weight update rule, rather than just implicitly by defining it through to objective function. Often weight decay refers to the implementation where we specify it directly in the weight update rule (whereas L2 regularization is usually the implementation which is specified in the objective function).
논문
이 태스크와 하위 분야의 관련 논문입니다. 실제 성능 비교는 평가 결과에서 확인하세요.
Rotational Equilibrium: How Weight Decay Balances Learning Across Neural Networks
It's Enough: Relaxing Diagonal Constraints in Linear Autoencoders for Recommendation
Electromyography Signal Classification Using Deep Learning
Maximum margin learning of t-SPNs for cell classification with filtered input
Emphasizing Unseen Words: New Vocabulary Acquisition for End-to-End Speech Recognition
Planting and Mitigating Memorized Content in Predictive-Text Language Models
Globally Gated Deep Linear Networks
Linking Neural Collapse and L2 Normalization with Improved Out-of-Distribution Detection in Deep Neural Networks
On the utility and protection of optimization with differential privacy and classic regularization techniques
Perturbation of Deep Autoencoder Weights for Model Compression and Classification of Tabular Data
Guidelines for the Regularization of Gammas in Batch Normalization for Deep Residual Networks
A Note on the Regularity of Images Generated by Convolutional Neural Networks
A Closer Look at Rehearsal-Free Continual Learning
Motion Correction and Volumetric Reconstruction for Fetal Functional Magnetic Resonance Imaging Data
How Infinitely Wide Neural Networks Can Benefit from Multi-task Learning -- an Exact Macroscopic Characterization
Probabilistic fine-tuning of pruning masks and PAC-Bayes self-bounded learning
Disturbing Target Values for Neural Network Regularization
Regularized Training of Nearest Neighbor Language Models
Sequence Length is a Domain: Length-based Overfitting in Transformer Models
Saddle-to-Saddle Dynamics in Deep Linear Networks: Small Initialization Training, Symmetry, and Sparsity
Guiding Teacher Forcing with Seer Forcing for Neural Machine Translation
The Limitations of Large Width in Neural Networks: A Deep Gaussian Process Perspective
Learning with Hyperspherical Uniformity
Towards Unsupervised Deep Image Enhancement with Generative Adversarial Network
Neural Pruning via Growing Regularization
Effect of the regularization hyperparameter on deep learning-based segmentation in LGE-MRI
Gram Regularization for Multi-view 3D Shape Retrieval
Label-Only Membership Inference Attacks
Exponentially Weighted l_2 Regularization Strategy in Constructing Reinforced Second-order Fuzzy Rule-based Model
An FPGA-Based On-Device Reinforcement Learning Approach using Online Sequential Learning
A Bayesian traction force microscopy method with automated denoising in a user-friendly software package
Correlated Initialization for Correlated Data
Tighter Bound Estimation of Sensitivity Analysis for Incremental and Decremental Data Modification
Regularisation Can Mitigate Poisoning Attacks: A Novel Analysis Based on Multiobjective Bilevel Optimisation
Data and Model Dependencies of Membership Inference Attack
출처: Papers with Code · CC BY-SA 4.0. 아카이브 자료와 새로 수집한 논문은 별도로 관리합니다.