图 书 馆
稀疏性分析的权重计算方案和文本分类中的应用Sparsity analsysis of term weighting schemes and application to text classification |
|
| 课程网址: | http://videolectures.net/slsfs05_brank_satws/ |
| 主讲教师: | Janez Brank |
| 开课单位: | 约瑟夫·斯特凡学院 |
| 开课时间: | 2007-02-25 |
| 课程语种: | 英语 |
| 中文简介: |  |
| 课程简介: | We revisit the common practice of feature selection for dimensionality and noise reduction. This typically involves scoring and ranking features based on some weighting scheme and selecting top ranked features for further processing. Experiments show that the performance of text classification methods is sensitive to characteristics of the used feature sets. For example, the size of the feature sets that yield the same performance level for a given classification method can be very different, depending on the feature scoring method used. We expand this exploration by considering representations of individual document vectors that result from a particular feature set. In particular, we observe the average number of features per document vector, i.e., the vector sparsity, or density and introduce sparsity curves to illustrate how the vector density increases with the feature set for different weighting schemes. We show that selecting feature by specifying the vector density parameter, instead of a feature set size, yields comparable results to the commonly adopted practice. However, it has the added benefit of understanding the effect of feature selection on document vector representation and system parameters, such as memory consumption of the classification operations. Furthermore, the corresponding classification performance curves link the sparsity and performance measures and provide further insight on how the feature specificity or distribution of the feature across documents in the corpus, is accounted for by the classification method. |
| 关 键 词: | 计算机科学; 文本挖掘; 稀疏曲线 |
| 课程来源: | 视频讲座网 |
| 最后编审: | 2020-07-28：yumf |
| 阅读次数: | 24 |