图 书 馆
从平均到加速，只有一个步长From Averaging to Acceleration, There is Only a Step-size |
|
| 课程网址: | https://videolectures.net/videos/colt2015_flammarion_averaging_ac... |
| 主讲教师: | Nicolas Flammarion |
| 开课单位: | 信息不详。欢迎您在右侧留言补充。 |
| 开课时间: | 2025-02-04 |
| 课程语种: | 英语 |
| 中文简介: |  |
| 课程简介: | We show that accelerated gradient descent, averaged gradient descent and the heavy-ball method for non-strongly convex problems may be reformulated as constant parameter second-order difference equation algorithms, where stability of the system is equivalent to convergence at rate O(1/n^2) where n is the number of iterations. We provide a detailed analysis of the eigenvalues of the corresponding linear dynamical system, showing various oscillatory and non-oscillatory behaviors, together with a sharp stability result with explicit constants. We also consider the situation where noisy gradients are available, where we extend our general convergence result, which suggest an alternative algorithm (i.e., with different step sizes) that exhibits the good aspects of both averaging and acceleration. |
| 关 键 词: | 非强凸问题; 平均梯度下降法; 重球法 |
| 课程来源: | 视频讲座网 |
| 数据采集: | 2025-03-30：zsp |
| 最后编审: | 2025-03-30：zsp |
| 阅读次数: | 3 |