图 书 馆
基于样本的学习的搜索算法是永久学习和瞬态记忆分离Sample-Based Learning and Search with Permanent and Transient Memories |
|
| 课程网址: | http://videolectures.net/icml08_silver_sbl/ |
| 主讲教师: | David Silver |
| 开课单位: | 伦敦大学学院 |
| 开课时间: | 2008-08-12 |
| 课程语种: | 英语 |
| 中文简介: |  |
| 课程简介: | We present a reinforcement learning architecture, Dyna-2, that encompasses both sample-based learning and sample-based search, and that generalises across states during both learning and search. We apply Dyna-2 to high performance Computer Go. In this domain the most successful planning methods are based on sample-based search algorithms, such as UCT, in which states are treated individually, and the most successful learning methods are based on temporal-difference learning algorithms, such as Sarsa, in which linear function approximation is used. In both cases, an estimate of the value function is formed, but in the first case it is transient, computed and then discarded after each move, whereas in the second case it is more permanent, slowly accumulating over many moves and games. The idea of Dyna-2 is for the transient planning memory and the permanent learning memory to remain separate, but for both to be based on linear function approximation and both to be updated by Sarsa. To apply Dyna-2 to 9x9 Computer Go, we use a million binary features in the function approximator, based on templates matching small fragments of the board. Using only the transient memory, Dyna-2 performed at least as well as UCT. Using both memories combined, it significantly outperformed UCT. Our program based on Dyna-2 achieved a higher rating on the Computer Go Online Server than any handcrafted or traditional search based program. |
| 关 键 词: | 强化学习架构; 计算机; 搜索算法 |
| 课程来源: | 视频讲座网 |
| 最后编审: | 2019-04-21：lxf |
| 阅读次数: | 57 |