Deep Reinforcement Learning: A Convex Optimization Approach
원문 초록
In this paper, we consider reinforcement learning of nonlinear systems with continuous state and action spaces. We present an episodic learning algorithm, where we for each episode use convex optimization to find a two-layer neural network approximation of the optimal $Q$-function. The convex optimization approach guarantees that the weights calculated at each episode are optimal, with respect to the given sampled states and actions of the current episode. For stable nonlinear systems, we show that the algorithm converges and that the converging parameters of the trained neural network can be made arbitrarily close to the optimal neural network parameters. In particular, if the regularization parameter in the training phase is given by $\rho$, then the parameters of the trained neural network converge to $w$, where the distance between $w$ and the optimal parameters $w^\star$ is bounded by $\mathcal{O}(\rho)$. That is, when the number of episodes goes to infinity, there exists a constant $C$ such that \[ \|w-w^\star\| \le C\rho. \] In particular, our algorithm converges arbitrarily close to the optimal neural network parameters as the regularization parameter goes to zero. As a consequence, our algorithm converges fast due to the polynomial-time convergence of convex optimization algorithms.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Reasoning on a Budget: Miniaturizing DeepSeek R1 with SFT-GRPO Alignment for Instruction-Tuned LLMs
Margin Trader: A Reinforcement Learning Framework for Portfolio Management with Margin and Constraints
Solving Dynamic Traveling Salesman Problems With Deep Reinforcement Learning
D3PG: Dirichlet DDPG for Task Partitioning and Offloading With Constrained Hybrid Action Space in Mobile-Edge Computing
Resource allocation algorithm for MEC based on Deep Reinforcement Learning
Task Independent Capsule-Based Agents for Deep Q-Learning
토론
이 논문과 연결된 포럼 글이 아직 없습니다.