图 书 馆
通信——非线性核机的高效分布式块最小化Communication-Efficient Distributed Block Minimization for Nonlinear Kernel Machines |
|
| 课程网址: | http://videolectures.net/kdd2017_hsieh_kernel_machines/ |
| 主讲教师: | Cho-Jui Hsieh |
| 开课单位: | 视频讲座网 |
| 开课时间: | 2017-10-09 |
| 课程语种: | 英语 |
| 中文简介: |  |
| 课程简介: | Kernel machines often yield superior predictive performance on various tasks; however, they suffer from severe computational challenges. In this paper, we show how to overcome the important challenge of speeding up kernel machines. In particular, we develop a parallel block minimization framework for solving kernel machines, including kernel SVM and kernel logistic regression. Our framework proceeds by dividing the problem into smaller subproblems by forming a block-diagonal approximation of the Hessian matrix. The subproblems are then solved approximately in parallel. After that, a communication efficient line search procedure is developed to ensure sufficient reduction of the objective function value at each iteration. We prove global linear convergence rate of the proposed method with a wide class of subproblem solvers, and our analysis covers strongly convex and some non-strongly convex functions. We apply our algorithm to solve large-scale kernel SVM problems on distributed systems, and show a significant improvement over existing parallel solvers. As an example, on the covtype dataset with half-a-million samples, our algorithm can obtain an approximate solution with 96% accuracy in 20 seconds using 32 machines, while all the other parallel kernel SVM solvers require more than 2000 seconds to achieve a solution with 95% accuracy. Moreover, our algorithm can scale to very large data sets, such as the kdd algebra dataset with 8 million samples and 20 million features. |
| 关 键 词: | 预测性能; 计算挑战; 求解核机; 直线搜索 |
| 课程来源: | 视频讲座网 |
| 数据采集: | 2022-11-20：chenxin01 |
| 最后编审: | 2022-11-20：chenxin01 |
| 阅读次数: | 26 |