图 书 馆
重复元组索引效率Efficient Indexing of Repeated n-Grams |
|
| 课程网址: | http://videolectures.net/wsdm2011_huston_eir/ |
| 主讲教师: | Samuel Huston |
| 开课单位: | 马萨诸塞大学 |
| 开课时间: | 2011-08-09 |
| 课程语种: | 英语 |
| 中文简介: |  |
| 课程简介: | The identification of repeated n-gram phrases in text has many practical applications, including authorship attribution, text reuse identification, and plagiarism detection. We consider methods for finding the repeated n-grams in text corpora, with emphasis on techniques that can be effectively scaled across a cluster of processors to handle very large amounts of text. We compare our proposed method to existing techniques using the 1.5 TB TREC ClueWeb-B text collection, using both single-processor and multi-processor approaches. The experiments show that our method offers an important tradeoff between speed and temporary storage space, and provides an alternative to previous approaches that scales almost linearly in the length of the sequence, is largely independent of n, and provides a uniform workload balance across the set of available processors. |
| 关 键 词: | 计算机科学; Web搜索; 多处理器; 文字识别 |
| 课程来源: | 视频讲座网 |
| 最后编审: | 2020-07-13：yumf |
| 阅读次数: | 32 |