Reward-Relevance-Filtered Linear Offline Reinforcement Learning
원문 초록
This paper studies offline reinforcement learning with linear function approximation in a setting with decision-theoretic, but not estimation sparsity. The structural restrictions of the data-generating process presume that the transitions factor into a sparse component that affects the reward and could affect additional exogenous dynamics that do not affect the reward. Although the minimally sufficient adjustment set for estimation of full-state transition properties depends on the whole state, the optimal policy and therefore state-action value function depends only on the sparse component: we call this causal/decision-theoretic sparsity. We develop a method for reward-filtering the estimation of the state-action value function to the sparse component by a modification of thresholded lasso in least-squares policy evaluation. We provide theoretical guarantees for our reward-filtered linear fitted-Q-iteration, with sample complexity depending only on the size of the sparse component.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Reasoning on a Budget: Miniaturizing DeepSeek R1 with SFT-GRPO Alignment for Instruction-Tuned LLMs
Meta Reinforcement Learning for Multi-Task Offloading in Vehicular Edge Computing
Investigating the Performance and Reliability, of the Q-Learning Algorithm in Various Unknown Environments
Margin Trader: A Reinforcement Learning Framework for Portfolio Management with Margin and Constraints
Catch: Collaborative Feature Set Search for Automated Feature Engineering
Solving Dynamic Traveling Salesman Problems With Deep Reinforcement Learning
토론
이 논문과 연결된 포럼 글이 아직 없습니다.