图 书 馆
使用分级相关度量评价社区问答Using Graded-Relevance Metrics for Evaluating Community QA Answer Selection |
|
| 课程网址: | http://videolectures.net/wsdm2011_sakai_ugr/ |
| 主讲教师: | Tetsuya Sakai |
| 开课单位: | 微软公司 |
| 开课时间: | 2011-08-09 |
| 课程语种: | 英语 |
| 中文简介: |  |
| 课程简介: | Community Question Answering (CQA) sites such as Yahoo! Answers have emerged as rich knowledge resources for information seekers. However, answers posted to CQA sites can be irrelevant, incomplete, redundant, incorrect, biased, ill-formed or even abusive. Hence, automatic selection of "good" answers for a given posted question is a practical research problem that will help us manage the quality of accumulated knowledge. One way to evaluate answer selection systems for CQA would be to use the Best Answers (BAs) that are readily available from the CQA sites. However, BAs may be biased, and even if they are not, there may be other good answers besides BAs. To remedy these two problems, we propose system evaluation methods that involve multiple answer assessors and graded-relevance information retrieval metrics. Our main findings from experiments using the NTCIR-8 CQA task data are that, using our evaluation methods, (a) we can detect many substantial differences between systems that would have been overlooked by BA-based evaluation; and (b) we can better identify hard questions (i.e. those that are handled poorly by many systems and therefore require focussed investigation) compared to BAbased evaluation. We therefore argue that our approach is useful for building effective CQA answer selection systems despite the cost of manual answer assessments. |
| 关 键 词: | 社区问答; 质量评估; 系统评价 |
| 课程来源: | 视频讲座网 |
| 最后编审: | 2020-09-21：heyf |
| 阅读次数: | 138 |