|
使用快速权重提高持久对比度发散性
Using Fast Weights to Improve Persistent Contrastive Divergence |
|
| |
| 课程网址: |
http://videolectures.net/icml09_tieleman_ufw/
|
| 主讲教师: |
Tijmen Tieleman
|
| 开课单位: |
多伦多大学
|
| 开课时间: |
2009-08-26
|
| 课程语种: |
英语
|
| 中文简介: |
受限制的玻尔兹曼机最常用的学习算法是对比发散，它在数据点启动马尔可夫链，并运行该链仅几次迭代，以获得模型下足够统计的廉价、低方差估计。Tieleman（2008）表明，通过使用一小组持久的“幻想粒子”来估计模型的统计数据，可以实现更好的学习，这些粒子在每次权重更新后都不会重新初始化为数据点。在足够小的权重更新的情况下，幻想粒子准确地表示平衡分布，但为了解释为什么该方法适用于更大的权重更新，有必要考虑权重更新和马尔可夫链之间的相互作用。我们表明，权重更新迫使马尔可夫链快速混合，利用这一见解，我们开发了一个更快的混合链，该混合链使用一组辅助的“快速权重”来实现对能源景观的临时覆盖。快速权重学习迅速，但也会迅速衰减，并且不会对定义模型的正常能量格局做出贡献。
|
| 课程简介: |
The most commonly used learning algorithm for restricted Boltzmann machines is contrastive divergence which starts a Markov chain at a data point and runs the chain for only a few iterations to get a cheap, low variance estimate of the sufficient statistics under the model. Tieleman (2008) showed that better learning can be achieved by estimating the model’s statistics using a small set of persistent ”fantasy particles” that are not reinitialized to data points after each weight update. With sufficiently small weight updates, the fantasy particles represent the equilibrium distribution accurately but to explain why the method works with much larger weight updates it is necessary to consider the interaction between the weight updates and the Markov chain. We show that the weight updates force the Markov chain to mix fast, and using this insight we develop an even faster mixing chain that uses an auxiliary set of ”fast weights” to implement a temporary overlay on the energy landscape. The fast weights learn rapidly but also decay rapidly and do not contribute to the normal energy landscape that defines the model.
|
| 关 键 词: |
玻尔兹曼机; 对比发散; 平衡分布
|
| 课程来源: |
视频讲座网
|
| 数据采集: |
2023-05-04：chenjy
|
| 最后编审: |
2023-05-04：chenjy
|
| 阅读次数: |
5
|
|
图 书 馆