The column measure and Gradient-Free Gradient Boosting
원문 초록
Sparse model selection by structural risk minimization leads to a set of a few predictors, ideally a subset of the true predictors. This selection clearly depends on the underlying loss function $\tilde L$. For linear regression with square loss, the particular (functional) Gradient Boosting variant $L_2-$Boosting excels for its computational efficiency even for very large predictor sets, while still providing suitable estimation consistency. For more general loss functions, functional gradients are not always easily accessible or, like in the case of continuous ranking, need not even exist. To close this gap, starting from column selection frequencies obtained from $L_2-$Boosting, we introduce a loss-dependent ''column measure'' $\nu^{(\tilde L)}$ which mathematically describes variable selection. The fact that certain variables relevant for a particular loss $\tilde L$ never get selected by $L_2-$Boosting is reflected by a respective singular part of $\nu^{(\tilde L)}$ w.r.t. $\nu^{(L_2)}$. With this concept at hand, it amounts to a suitable change of measure (accounting for singular parts) to make $L_2-$Boosting select variables according to a different loss $\tilde L$. As a consequence, this opens the bridge to applications of simulational techniques such as various resampling techniques, or rejection sampling, to achieve this change of measure in an algorithmic way.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Gradient Boosting for Spatial Regression Models with Autoregressive Disturbances
A Unified Framework for Variable Selection in Model-Based Clustering with Missing Not at Random
Learning Latent Variable Models via Jarzynski-adjusted Langevin Algorithm
From Roots to Rewards: Dynamic Tree Reasoning with RL
DCR: Quantifying Data Contamination in LLMs Evaluation
U-RWKV: Lightweight medical image segmentation with direction-adaptive RWKV
토론
이 논문과 연결된 포럼 글이 아직 없습니다.