图 书 馆
分类数据集中异常记录的检测Detecting Anomalous Records in Categorical Datasets |
|
| 课程网址: | http://videolectures.net/kdd07_das_dar/ |
| 主讲教师: | Kaustav Das |
| 开课单位: | 卡内基梅隆大学 |
| 开课时间: | 2007-09-14 |
| 课程语种: | 英语 |
| 中文简介: |  |
| 课程简介: | We consider the problem of detecting anomalies in high arity categorical datasets. In most applications, anomalies are defined as data points that are ’abnormal’. Quite often we have access to data which consists mostly of normal records, along with a small percentage of unlabelled anomalous records. We are interested in the problem of unsupervised anomaly detection, where we use the unlabelled data for training, and detect records that do not follow the definition of normality. A standard approach is to create a model of normal data, and compare test records against it. A probabilistic approach builds a likelihood model from the training data. Records are tested for anomalousness based on the complete record likelihood given the probability model. For categorical attributes, bayes nets give a standard representation of the likelihood. While this approach is good at finding outliers in the dataset, it often tends to detect records with attribute values that are rare. Sometimes, just detecting rare values of an attribute is not desired and such outliers are not considered as anomalies in that context. We present an alternative definition of anomalies, and propose an approach of comparing against marginal distributions of attribute subsets. We show that this is a more meaningful way of detecting anomalies, and has a better performance over semi-synthetic as well as real world datasets. |
| 关 键 词: | 分类数据; 正态定义; 概率模型 |
| 课程来源: | 视频讲座网 |
| 最后编审: | 2019-05-08：lxf |
| 阅读次数: | 79 |