图 书 馆
利用聚光灯机制从结构图像中转录内容Transcribing Content from Structural Images with Spotlight Mechanism |
|
| 课程网址: | http://videolectures.net/kdd2018_yin_transcribing_content/ |
| 主讲教师: | Yu Yin |
| 开课单位: | 中国科学技术大学 |
| 开课时间: | 2018-11-23 |
| 课程语种: | 英语 |
| 中文简介: |  |
| 课程简介: | Transcribing content from structural images, e.g., writing notes from music scores, is a challenging task as not only the content objects should be recognized, but the internal structure should also be preserved. Existing image recognition methods mainly work on images with simple content (e.g., text lines with characters), but are not capable to identify ones with more complex content (e.g., structured code), which often follow a fine-grained grammar. To this end, in this paper, we propose a hierarchical Spotlight Transcribing Network (STN) framework followed by a two-stage “where-to-what’’ solution. Specifically, we first decide “where-to-look’’ through a novel spotlight mechanism to focus on different areas of the original image following its structure. Then, we decide “what-to-write’’ by developing a GRU based network with the spotlight areas for transcribing the content accordingly. Moreover, we propose two implementations on the basis of STN, i.e., STNM and STNR, where the spotlight movement follows the Markov property and Recurrent modeling, respectively. We also design a reinforcement method to refine our STN framework by self-improving the spotlight mechanism. We conduct extensive experiments on many structural image datasets, where the results clearly demonstrate the effectiveness of STN framework. |
| 关 键 词: | 聚光灯转录网络; 从结构图像中转录内容; STN框架机械装置; 遵循马尔可夫属性和递归建模 |
| 课程来源: | 视频讲座网 |
| 数据采集: | 2023-03-23：cyh |
| 最后编审: | 2023-03-23：cyh |
| 阅读次数: | 39 |