图 书 馆
通过随机哈希进行可扩展和可持续的深度学习Scalable and Sustainable Deep Learning via Randomized Hashing |
|
| 课程网址: | http://videolectures.net/kdd2017_spring_deep_learning/ |
| 主讲教师: | Ryan Spring |
| 开课单位: | 莱斯大学 |
| 开课时间: | 2017-10-09 |
| 课程语种: | 英语 |
| 中文简介: |  |
| 课程简介: | Current deep learning architectures are growing larger in order to learn from complex datasets. These architectures require giant matrix multiplication operations to train millions of parameters. Conversely, there is another growing trend to bring deep learning to low-power, embedded devices. The matrix operations, associated with both training and testing of deep networks, are very expensive from a computational and energy standpoint. We present a novel hashing based technique to drastically reduce the amount of computation needed to train and test deep networks. Our approach combines recent ideas from adaptive dropouts and randomized hashing for maximum inner product search to select the nodes with the highest activation efficiently. Our new algorithm for deep learning reduces the overall computational cost of forward and back-propagation by operating on significantly fewer (sparse) nodes. As a consequence, our algorithm uses only 5% of the total multiplications, while keeping on average within 1% of the accuracy of the original model. A unique property of the proposed hashing based back-propagation is that the updates are always sparse. Due to the sparse gradient updates, our algorithm is ideally suited for asynchronous and parallel training leading to near linear speedup with increasing number of cores. We demonstrate the scalability and sustainability (energy efficiency) of our proposed algorithm via rigorous experimental evaluations on several real datasets. |
| 关 键 词: | 深度学习; 能源效率; 数据科学 |
| 课程来源: | 视频讲座网 |
| 数据采集: | 2023-12-26：wujk |
| 最后编审: | 2023-12-26：wujk |
| 阅读次数: | 14 |