Improve the Training Efficiency of DRL for Wireless Communication Resource Allocation: The Role of Generative Diffusion Models
원문 초록
Dynamic resource allocation in mobile wireless networks involves complex, time-varying optimization problems, motivating the adoption of deep reinforcement learning (DRL). However, most existing works rely on pre-trained policies, overlooking dynamic environmental changes that rapidly invalidate the policies. Periodic retraining becomes inevitable but incurs prohibitive computational costs and energy consumption-critical concerns for resource-constrained wireless systems. We identify three root causes of inefficient retraining: high-dimensional state spaces, suboptimal action spaces exploration-exploitation trade-offs, and reward design limitations. To overcome these limitations, we propose Diffusion-based Deep Reinforcement Learning (D2RL), which leverages generative diffusion models (GDMs) to holistically enhance all three DRL components. Iterative refinement process and distribution modelling of GDMs enable (1) the generation of diverse state samples to improve environmental understanding, (2) balanced action space exploration to escape local optima, and (3) the design of discriminative reward functions that better evaluate action quality. Our framework operates in two modes: Mode I leverages GDMs to explore reward spaces and design discriminative reward functions that rigorously evaluate action quality, while Mode II synthesizes diverse state samples to enhance environmental understanding and generalization. Extensive experiments demonstrate that D2RL achieves faster convergence and reduced computational costs over conventional DRL methods for resource allocation in wireless communications while maintaining competitive policy performance. This work underscores the transformative potential of GDMs in overcoming fundamental DRL training bottlenecks for wireless networks, paving the way for practical, real-time deployments.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Adaptive Reparametrized Time for Score-Based Diffusion Sampling
심리학 기반 추론과 역할 인식 정책 최적화를 통한 일반 역할 연기 에이전트 개선
Improving General Role-Playing Agents via Psychology-Grounded Reasoning and Role-Aware Policy Optimization
When Can Learned Uncertainty Be Trusted? An Audit of Distributional Reinforcement Learning
보상 인플레이션: 강화학습을 위한 건강한 자극
Reward Inflation: A Healthy Stimulus for Reinforcement Learning
Learning Dexterous Manipulation Using Contact Wrench Guidance From Human Demonstration
로봇 행동을 위한 fNIRS 기반 오프라인 강화학습 접근법
An offline approach to fNIRS-guided reinforcement learning for robot behavior
토론
이 논문과 연결된 포럼 글이 아직 없습니다.