|
LaRank, SGD-QN -线性支持向量机的快速优化器
LaRank, SGD-QN - Fast Optimizers for Linear SVM |
|
| |
| 课程网址: |
http://videolectures.net/icml08_bordes_lrsq/
|
| 主讲教师: |
Antoine Bordes
|
| 开课单位: |
Facebook公司
|
| 开课时间: |
2008-09-01
|
| 课程语种: |
英语
|
| 中文简介: |
LaRank算法最初是为求解多类支持向量机而提出的，是一种双坐标上升算法，依赖于受感知器算法启发的随机探索[Bordes05, Bordes07]。这种方法与基于梯度的优化器在简单的二元和多类问题上具有竞争力。此外，很少有LaRank通过的训练示例提供的测试错误率接近最终解决方案的错误率。对于这个条目，我们运行了几代LaRank算法，直到达到收敛准则。
SGD-QN算法使用一种有效的方法修正随机梯度下降来估计逆Hessian的对角线。估计方法的灵感来自oLBFGS [Schraudolph, 07]。由于在每次迭代时几乎不需要更新这个估计矩阵，这种近似二阶随机梯度方法的迭代速度几乎与经典的随机梯度下降方法一样快[Bottou98, Bottou07]，但需要的迭代次数更少。
|
| 课程简介: |
Originally proposed for solving multiclass SVM, the LaRank algorithm is a dual coordinate ascent algorithm relying on a randomized exploration inspired by the perceptron algorithm [Bordes05, Bordes07]. This approach is competitive with gradient based optimizers on simple binary and multiclass problems. Furthermore, very few LaRank passes over the training examples delivers test error rates that are nearly as good as those of the final solution. For this entry we ran several epochs of the LaRank algorithm until reaching the convergence criterion.
The SGD-QN algorithm uses stochastic gradient descent modified using an efficient method to estimate the diagonal of the inverse Hessian. The estimation method is inspired oLBFGS [Schraudolph, 07]. Since there is a little need to update this estimated matrix at each iteration, this approximate second-order stochastic gradient method iterates nearly as fast than a classical stochastic gradient descent [Bottou98, Bottou07] but requires less iterations.
|
| 关 键 词: |
双坐标上升算法; 收敛准则; 二阶随机梯度
|
| 课程来源: |
视频讲座网
|
| 数据采集: |
2022-11-08：chenjy
|
| 最后编审: |
2022-11-08：chenjy
|
| 阅读次数: |
38
|
|
图 书 馆