图 书 馆
逃离鞍点——张量分解的在线随机梯度Escaping From Saddle Points --- Online Stochastic Gradient for Tensor Decomposition |
|
| 课程网址: | http://videolectures.net/colt2015_huang_tensor_decomposition/ |
| 主讲教师: | Furong Huang |
| 开课单位: | 加州大学欧文分校 |
| 开课时间: | 2015-08-20 |
| 课程语种: | 英语 |
| 中文简介: |  |
| 课程简介: | We analyze stochastic gradient descent for optimizing non-convex functions. For non-convex functions often it is good to find a reasonable local minimum, and the main concern is that gradient updates are trapped in saddle points. In this paper we identify strict saddle property for non-convex problem that allows for efficient optimization, and show that stochastic gradient descent converges to a local minimum in a polynomial number of iterations. To the best of our knowledge this is the first work that gives global convergence guarantees for stochastic gradient descent on non-convex functions with exponentially many local minima and saddle points. Our analysis can be applied to orthogonal tensor decomposition, which is widely used in learning a rich class of latent variable models. We propose a new optimization formulation for the tensor decomposition problem that has strict saddle property. As a result we get the first online algorithm for orthogonal tensor decomposition with convergence guarantee. |
| 关 键 词: | 张量分解; 随机梯度; 非凸函数 |
| 课程来源: | 视频讲座网 |
| 数据采集: | 2023-07-24：chenxin01 |
| 最后编审: | 2023-07-24：chenxin01 |
| 阅读次数: | 18 |