A Bayesian Model for Online Activity Sample Sizes
Thomas S. Richardson, Yu Liu, James Mcqueen, Doug Hains
Abstract
In many contexts it is useful to predict the number of individuals in some population who will initiate a particular activity during a given period. For example, the number of users who will install a software update, the number of customers who will use a new feature on a website or who will participate in an A/B test. In practical settings, there is heterogeneity amongst individuals with regard to the distribution of time until they will initiate. For these reasons it is inappropriate to assume that the number of new individuals observed on successive days will be identically distributed. Given observations on the number of unique users participating in an initial period, we present a simple but novel Bayesian method for predicting the number of additional individuals who will participate during a subsequent period. We illustrate the performance of the method in predicting sample size in online experimentation.
BibTeX
@InProceedings{pmlr-v151-richardson22a,
title = { A Bayesian Model for Online Activity Sample Sizes },
author = {Richardson, Thomas S. and Liu, Yu and Mcqueen, James and Hains, Doug},
booktitle = {Proceedings of The 25th International Conference on Artificial Intelligence and Statistics},
pages = {1775--1785},
year = {2022},
editor = {Camps-Valls, Gustau and Ruiz, Francisco J. R. and Valera, Isabel},
volume = {151},
series = {Proceedings of Machine Learning Research},
month = {28--30 Mar},
publisher = {PMLR},
pdf = {https://proceedings.mlr.press/v151/richardson22a/richardson22a.pdf},
url = {https://proceedings.mlr.press/v151/richardson22a.html},
abstract = { In many contexts it is useful to predict the number of individuals in some population who will initiate a particular activity during a given period. For example, the number of users who will install a software update, the number of customers who will use a new feature on a website or who will participate in an A/B test. In practical settings, there is heterogeneity amongst individuals with regard to the distribution of time until they will initiate. For these reasons it is inappropriate to assume that the number of new individuals observed on successive days will be identically distributed. Given observations on the number of unique users participating in an initial period, we present a simple but novel Bayesian method for predicting the number of additional individuals who will participate during a subsequent period. We illustrate the performance of the method in predicting sample size in online experimentation. }
}