Statistical challenges in online controlled experiments: A review of a/b testing methodology

N Larsen, J Stallrich, S Sengupta, A Deng… - The American …, 2024 - Taylor & Francis
The rise of internet-based services and products in the late 1990s brought about an
unprecedented opportunity for online businesses to engage in large scale data-driven …

Optimal treatment allocation for efficient policy evaluation in sequential decision making

T Li, C Shi, J Wang, F Zhou - Advances in Neural …, 2024 - proceedings.neurips.cc
A/B testing is critical for modern technological companies to evaluate the effectiveness of
newly developed products against standard baselines. This paper studies optimal designs …

Disentangling and operationalizing AI fairness at linkedin

J Quiñonero Candela, Y Wu, B Hsu, S Jain… - Proceedings of the …, 2023 - dl.acm.org
Operationalizing AI fairness at LinkedIn's scale is challenging not only because there are
multiple mutually incompatible definitions of fairness but also because determining what is …

A/b testing for recommender systems in a two-sided marketplace

P Nandy, D Venugopalan, C Lo… - Advances in Neural …, 2021 - proceedings.neurips.cc
Two-sided marketplaces are standard business models of many online platforms (eg,
Amazon, Facebook, LinkedIn), wherein the platforms have consumers, buyers or content …

A/B testing in network data with covariate-adaptive randomization

J Wang, P Li, F Hu - International Conference on Machine …, 2023 - proceedings.mlr.press
Users linked together through a network often tend to have similar behaviors. This
phenomenon is usually known as network interaction. Users' characteristics, the covariates …

A bias correction approach for interference in ranking experiments

A Goli, A Lambrecht… - Marketing …, 2024 - pubsonline.informs.org
Online marketplaces use ranking algorithms to determine the rank-ordering of items sold on
their websites. The standard practice is to determine the optimal algorithm using A/B tests …

Causal inference from network data

E Zheleva, D Arbour - Proceedings of the 27th ACM SIGKDD Conference …, 2021 - dl.acm.org
This tutorial presents state-of-the-art research on causal inference from network data in the
presence of interference. We start by motivating research in this area with real-world …

Optimized covariance design for AB test on social network under interference

Q Chen, B Li, L Deng, Y Wang - Advances in Neural …, 2024 - proceedings.neurips.cc
Online A/B tests have become increasingly popular and important for social platforms.
However, accurately estimating the global average treatment effect (GATE) has proven to be …

Locally optimal design for a/b tests in the presence of covariates and network dependence

Q Zhang, L Kang - Technometrics, 2022 - Taylor & Francis
A/B test, a simple type of controlled experiment, refers to the statistical procedure of
experimenting to compare two treatments applied to test subjects. For example, many IT …

Promoting inactive members in edge-building marketplace

A Acharya, S Gao, B Ocejo, K Basu, A Saha… - … Proceedings of the …, 2023 - dl.acm.org
Social networks are platforms where content creators and consumers share and consume
content. The edge recommendation system, which determines who a member should …