图 书 馆
近端强化学习：学习在原始-对偶空间中行动Proximal Reinforcement Learning: Learning to Act in Primal Dual Spaces |
|
| 课程网址: | http://videolectures.net/rldm2015_mahadevan_dual_spaces/ |
| 主讲教师: | Sridhar Mahadevan |
| 开课单位: | 马萨诸塞大学阿默斯特分校 |
| 开课时间: | 2015-07-28 |
| 课程语种: | 英语 |
| 中文简介: |  |
| 课程简介: | In this talk, we set forth a new framework for reinforcement learning developed by us over the past few years, one that yields mathematically rigorous solutions to longstanding fundamental questions that have remained unresolved over the past three decades: (i) how to design “safe” reinforcement learning algorithms that remain in a stable region of the parameter space (ii) how to design true stochastic gradient temporal-difference learning algorithms and give finite-sample bounds characterizing their convergence? (iii) more broadly, how to specify a flexible algorithmic framework that simplifies the design of reinforcement learning algorithms for various objective functions? The most important idea that emerges as a motif throughout the solution of these three problems is the use of primal dual spaces connected through the use of “mirror maps”: Legendre transforms that elegantly unify and generalize a myriad past algorithms for solving reinforcement learning problems, from natural gradient actor-critic methods and exponentiated-gradient methods to gradient TD and sparse RL methods. We introduce mirror-descent RL, a powerful family of RL methods that uses mirror maps through different Legendre transforms to achieve reliability, scalability, and sparsity. Our work builds extensively on the past 50 years of advances in stochastic optimization, from the study of proximal mappings, monotone operators, and operator splitting began in the mid-1950s to recent advances in first-order optimization and saddle-point extragradient methods for solving variational inequalities. |
| 关 键 词: | 强化学习新框架; 对偶空间; 指数梯度法 |
| 课程来源: | 视频讲座网 |
| 数据采集: | 2021-11-20：zkj |
| 最后编审: | 2021-11-20：zkj |
| 阅读次数: | 67 |