图 书 馆
在线控制实验中的错误发现率控制的异质处理效果检测False Discovery Rate Controlled Heterogeneous Treatment Effect Detection for Online Controlled Experiments |
|
| 课程网址: | http://videolectures.net/kdd2018_xie_false_heterogeneous/ |
| 主讲教师: | Yuxiang Xie |
| 开课单位: | 华盛顿大学 |
| 开课时间: | 2018-11-23 |
| 课程语种: | 英语 |
| 中文简介: |  |
| 课程简介: | Online controlled experiments (a.k.a. A/B testing) have been used as the mantra for data-driven decision making on feature changing and product shipping in many Internet companies. However, it is still a great challenge to systematically measure how every code or feature change impacts millions of users with great heterogeneity (e.g. countries, ages, devices). The most commonly used A/B testing framework in many companies is based on Average Treatment Effect (ATE), which cannot detect the heterogeneity of treatment effect on users with different characteristics. In this paper, we propose statistical methods that can systematically and accurately identify Heterogeneous Treatment Effect (HTE) of any user cohort of interest (e.g. mobile device type, country), and determine which factors (e.g. age, gender) of users contribute to the heterogeneity of the treatment effect in an A/B test. By applying these methods on both simulation data and real-world experimentation data, we show how they work robustly with controlled low False Discover Rate (FDR), and at the same time, provides us with useful insights about the heterogeneity of identified user groups. We have deployed a toolkit based on these methods, and have used it to measure the Heterogeneous Treatment Effect of many A/B tests at Snap. |
| 关 键 词: | 在线控制实验; 数据驱动决策; 治疗效果的异质性; 治疗效果的异质性 |
| 课程来源: | 视频讲座网 |
| 数据采集: | 2023-01-24：cyh |
| 最后编审: | 2023-01-24：cyh |
| 阅读次数: | 22 |