图 书 馆
线性函数逼近的时差学习快速梯度下降法Fast Gradient-Descent Methods for Temporal-Difference Learning with Linear Function Approximation |
|
| 课程网址: | http://videolectures.net/icml09_sutton_fgdm/ |
| 主讲教师: | Richard S. Sutton |
| 开课单位: | 阿尔伯塔大学 |
| 开课时间: | 2009-09-17 |
| 课程语种: | 英语 |
| 中文简介: |  |
| 课程简介: | Sutton, Szepesvari and Maei (2009) recently introduced the first temporal-difference learning algorithm compatible with both linear function approximation and off-policy training, and whose complexity scales only linearly in the size of the function approximator. Although their gradient temporal difference (GTD) algorithm converges reliably, it can be very slow compared to conventional linear TD (on on-policy problems where TD is convergent), calling into question its practical utility. In this paper we introduce two new related algorithms with better convergence rates. The first algorithm, GTD2, is derived and proved convergent just as GTD was, but uses a different objective function and converges significantly faster (but still not as fast as conventional TD). The second new algorithm, linear TD with gradient correction, or TDC, uses the same update rule as conventional TD except for an additional term which is initially zero. In our experiments on small test problems and in a Computer Go application with a million features, the learning rate of this algorithm was comparable to that of conventional TD. This algorithm appears to extend linear TD to off-policy learning with no penalty in performance while only doubling computational requirements. |
| 关 键 词: | 线性函数; 时间差分学习算法; 收敛速度 |
| 课程来源: | 视频讲座网 |
| 最后编审: | 2019-04-24：lxf |
| 阅读次数: | 83 |