图 书 馆
机器学习的随机逼近算法的非渐近分析Non-Asymptotic Analysis of Stochastic Approximation Algorithms for Machine Learning |
|
| 课程网址: | http://videolectures.net/nips2011_bach_machine/ |
| 主讲教师: | Francis R. Bach |
| 开课单位: | INRIA研究机构 |
| 开课时间: | 2012-01-19 |
| 课程语种: | 英语 |
| 中文简介: |  |
| 课程简介: | We consider the minimization of a convex objective function defined on a Hilbert space, which is only available through unbiased estimates of its gradients. This problem includes standard machine learning algorithms such as kernel logistic regression and least-squares regression, and is commonly referred to as a stochastic approximation problem in the operations research community. We provide a non-asymptotic analysis of the convergence of two well-known algorithms, stochastic gradient descent (a.k.a.~Robbins-Monro algorithm) as well as a simple modification where iterates are averaged (a.k.a.~Polyak-Ruppert averaging). Our analysis suggests that a learning rate proportional to the inverse of the number of iterations, while leading to the optimal convergence rate in the strongly convex case, is not robust to the lack of strong convexity or the setting of the proportionality constant. This situation is remedied when using slower decays together with averaging, robustly leading to the optimal rate of convergence. We illustrate our theoretical results with simulations on synthetic and standard datasets. |
| 关 键 词: | 优化方法; 随机优化; 计算机科学; 机器学习 |
| 课程来源: | 视频讲座网 |
| 最后编审: | 2020-07-29：yumf |
| 阅读次数: | 84 |