Papers with Code 아카이브

L2 Regularization

태스크의 하위 분류와 논문·코드·평가·기법·데이터셋을 탐색하세요.

하위 태스크: 0
평가표: 0
보존된 평가 기록: 0

$L_{2}$ Regularization or Weight Decay, is a regularization technique applied to the weights of a neural network. We minimize a loss function compromising both the primary loss function and a penalty on the $L\_{2}$ Norm of the weights:

$$L\_{new}\left(w\right) = L\_{original}\left(w\right) + \lambda{w^{T}w}$$

where $\lambda$ is a value determining the strength of the penalty (encouraging smaller weights).

Weight decay can be incorporated directly into the weight update rule, rather than just implicitly by defining it through to objective function. Often weight decay refers to the implementation where we specify it directly in the weight update rule (whereas L2 regularization is usually the implementation which is specified in the objective function).

논문

이 태스크와 하위 분야의 관련 논문입니다. 실제 성능 비교는 평가 결과에서 확인하세요.

L2 Regularization

논문

Overcoming catastrophic forgetting in neural networks

From large-eddy simulations to deep learning: A U-net model for fast urban canopy flow predictions

DACN: Dual-Attention Convolutional Network for Hyperspectral Image Super-Resolution

Geometry of Learning -- L2 Phase Transitions in Deep and Shallow Neural Networks

Deep Learning in Renewable Energy Forecasting: A Cross-Dataset Evaluation of Temporal and Spatial Models

Understand the Effect of Importance Weighting in Deep Learning on Dataset Shift

Semantic segmentation for building houses from wooden cubes

GPT Meets Graphs and KAN Splines: Testing Novel Frameworks on Multitask Fine-Tuned GPT-2 with LoRA

CtrTab: Tabular Data Synthesis with High-Dimensional and Limited Data

Low-rank bias, weight decay, and model merging in neural networks

Multimodal Bearing Fault Classification Under Variable Conditions: A 1D CNN with Transfer Learning

Renewable Energy Prediction: A Comparative Study of Deep Learning Models for Complex Dataset Analysis

Learning in Log-Domain: Subthreshold Analog AI Accelerator Based on Stochastic Gradient Descent

Super-Resolution for Remote Sensing Imagery via the Coupling of a Variational Model and Deep Learning

Parkinson's Disease Diagnosis Through Deep Learning: A Novel LSTM-Based Approach for Freezing of Gait Detection

Effectiveness of L2 Regularization in Privacy-Preserving Machine Learning

Analysis of High-dimensional Gaussian Labeled-unlabeled Mixture Model via Message-passing Algorithm

Recurrent Stochastic Configuration Networks with Hybrid Regularization for Nonlinear Dynamics Modelling

Carbon price fluctuation prediction using blockchain information A new hybrid machine learning approach

Weight decay induces low-rank attention layers

WALINET: A water and lipid identification convolutional Neural Network for nuisance signal removal in 1H MR Spectroscopic Imaging

Rethinking Conventional Wisdom in Machine Learning: From Generalization to Scaling

Training Dynamics of Nonlinear Contrastive Learning Model in the High Dimensional Limit

Comparative Study of Bitcoin Price Prediction

Derivative-based regularization for regression

Monkeypox disease recognition model based on improved SE-InceptionV3

Convergence of a L2 regularized Policy Gradient Algorithm for the Multi Armed Bandit

An Experiment on Feature Selection using Logistic Regression

Prevalidated ridge regression is a highly-efficient drop-in replacement for logistic regression for high-dimensional data

Reverse Engineering Deep ReLU Networks An Optimization-based Algorithm

Gradient-based bilevel optimization for multi-penalty Ridge regression through matrix differential calculus

The Transient Nature of Emergent In-Context Learning in Transformers

On sparse regression, Lp-regularization, and automated model discovery

Maintaining Plasticity in Continual Learning via Regenerative Regularization

Less is More -- Towards parsimonious multi-task models using structured sparsity

Maintaining Plasticity in Deep Continual Learning