Offline Action-Free Learning of Ex-BMDPs by Comparing Diverse Datasets
원문 초록
While sequential decision-making environments often involve high-dimensional observations, not all features of these observations are relevant for control. In particular, the observation space may capture factors of the environment which are not controllable by the agent, but which add complexity to the observation space. The need to ignore these "noise" features in order to operate in a tractably-small state space poses a challenge for efficient policy learning. Due to the abundance of video data available in many such environments, task-independent representation learning from action-free offline data offers an attractive solution. However, recent work has highlighted theoretical limitations in action-free learning under the Exogenous Block MDP (Ex-BMDP) model, where temporally-correlated noise features are present in the observations. To address these limitations, we identify a realistic setting where representation learning in Ex-BMDPs becomes tractable: when action-free video data from multiple agents with differing policies are available. Concretely, this paper introduces CRAFT (Comparison-based Representations from Action-Free Trajectories), a sample-efficient algorithm leveraging differences in controllable feature dynamics across agents to learn representations. We provide theoretical guarantees for CRAFT's performance and demonstrate its feasibility on a toy example, offering a foundation for practical methods in similar settings.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
TextAtari: 100K Frames Game Playing with Language Agents
On Designing Diffusion Autoencoders for Efficient Generation and Representation Learning
Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis
LLINBO: Trustworthy LLM-in-the-Loop Bayesian Optimization
Dual Dimensions Geometric Representation Learning Based Document Dewarping
Feed-Forward SceneDINO for Unsupervised Semantic Scene Completion
토론
이 논문과 연결된 포럼 글이 아직 없습니다.