图 书 馆
随机梯度下降并行化的无锁方法Lock-Free Approaches to Parallelizing Stochastic Gradient Descent |
|
| 课程网址: | http://videolectures.net/nipsworkshops2011_recht_lockfree/ |
| 主讲教师: | Benjamin Recht |
| 开课单位: | 加州大学伯克利分校 |
| 开课时间: | 2012-01-25 |
| 课程语种: | 英语 |
| 中文简介: |  |
| 课程简介: | Stochastic Gradient Descent (SGD) is a very popular optimization algorithm for solving data-driven machine learning problems. SGD is well suited to processing large amounts of data due to its robustness against noise, rapid convergence rates, and predictable memory footprint. Nevertheless, SGD seems to be impeded by many of the classical barriers to scalability: (1) SGD appears to be inherently sequential, (2) SGD assumes uniform sampling from the underlying data set resulting in poor locality, and (3) current approaches to parallelize SGD require performance-destroying, fine-grained communication. This talk aims to refute the conventional wisdom that SGD inherently suffers from these impediments. Specifically, I will show that SGD can be implemented in parallel with minimal communication, with no locking or synchronization, and with strong spatial locality. I will provide both theoretical and experimental evidence demonstrating the achievement of linear speedups on multicore workstations on several benchmark optimization problems. Finally, I will close with a discussion of a challenging problem raised by our implementations relating arithmetic and geometric means of matrices. |
| 关 键 词: | 随机梯度下降; 优化算法; 机器学习 |
| 课程来源: | 视频讲座网 |
| 最后编审: | 2020-05-31：王勇彬(课程编辑志愿者) |
| 阅读次数: | 108 |