Statistical challenges in online controlled experiments: A review of a/b testing methodology
The rise of internet-based services and products in the late 1990s brought about an
unprecedented opportunity for online businesses to engage in large scale data-driven …
unprecedented opportunity for online businesses to engage in large scale data-driven …
Optimal treatment allocation for efficient policy evaluation in sequential decision making
A/B testing is critical for modern technological companies to evaluate the effectiveness of
newly developed products against standard baselines. This paper studies optimal designs …
newly developed products against standard baselines. This paper studies optimal designs …
Disentangling and operationalizing AI fairness at linkedin
Operationalizing AI fairness at LinkedIn's scale is challenging not only because there are
multiple mutually incompatible definitions of fairness but also because determining what is …
multiple mutually incompatible definitions of fairness but also because determining what is …
A/b testing for recommender systems in a two-sided marketplace
Two-sided marketplaces are standard business models of many online platforms (eg,
Amazon, Facebook, LinkedIn), wherein the platforms have consumers, buyers or content …
Amazon, Facebook, LinkedIn), wherein the platforms have consumers, buyers or content …
A/B testing in network data with covariate-adaptive randomization
Users linked together through a network often tend to have similar behaviors. This
phenomenon is usually known as network interaction. Users' characteristics, the covariates …
phenomenon is usually known as network interaction. Users' characteristics, the covariates …
A bias correction approach for interference in ranking experiments
Online marketplaces use ranking algorithms to determine the rank-ordering of items sold on
their websites. The standard practice is to determine the optimal algorithm using A/B tests …
their websites. The standard practice is to determine the optimal algorithm using A/B tests …
Causal inference from network data
This tutorial presents state-of-the-art research on causal inference from network data in the
presence of interference. We start by motivating research in this area with real-world …
presence of interference. We start by motivating research in this area with real-world …
Optimized covariance design for AB test on social network under interference
Online A/B tests have become increasingly popular and important for social platforms.
However, accurately estimating the global average treatment effect (GATE) has proven to be …
However, accurately estimating the global average treatment effect (GATE) has proven to be …
Locally optimal design for a/b tests in the presence of covariates and network dependence
A/B test, a simple type of controlled experiment, refers to the statistical procedure of
experimenting to compare two treatments applied to test subjects. For example, many IT …
experimenting to compare two treatments applied to test subjects. For example, many IT …
Promoting inactive members in edge-building marketplace
Social networks are platforms where content creators and consumers share and consume
content. The edge recommendation system, which determines who a member should …
content. The edge recommendation system, which determines who a member should …