图 书 馆
Apache Spark中扩展ML的基础Foundations for Scaling ML in Apache Spark |
|
| 课程网址: | https://videolectures.net/videos/kdd2016_bradley_apache_spark |
| 主讲教师: | Joseph K. Bradley |
| 开课单位: | KDD 2016研讨会 |
| 开课时间: | 2016-10-12 |
| 课程语种: | 英语 |
| 中文简介: |  |
| 课程简介: | Apache Spark has become the most active open source Big Data project, and its Machine Learning library MLlib has seen rapid growth in usage. A critical aspect of MLlib and Spark is the ability to scale: the same code used on a laptop can scale to 100’s or 1000’s of machines. This talk will describe ongoing and future efforts to make MLlib even faster and more scalable by integrating with two key initiatives in Spark. The first is Catalyst, the query optimizer underlying DataFrames and Datasets. The second is Tungsten, the project for approaching bare-metal speeds in Spark via memory management, cache-awareness, and code generation. This talk will discuss the goals, the challenges, and the benefits for MLlib users and developers. More generally, we will reflect on the importance of integrating ML with the many other aspects of big data analysis. About MLlib: MLlib is a general Machine Learning library providing many ML algorithms, feature transformers, and tools for model tuning and building workflows. The library benefits from integration with the rest of Apache Spark (SQL, streaming, Graph, core), which facilitates ETL, streaming, and deployment. It is used in both ad hoc analysis and production deployments throughout academia and industry. |
| 关 键 词: | 大数据项目; 机器学习库; 模型调优 |
| 课程来源: | 视频讲座网 |
| 数据采集: | 2024-12-30：liyq |
| 最后编审: | 2024-12-30：liyq |
| 阅读次数: | 8 |