Counterfactual Learning with General Data-Generating Policies
Yusuke Narita, Kyohei Okumura, Akihiro Shimizu, Kohei Yata
Abstract
Off-policy evaluation (OPE) attempts to predict the performance of counterfactual policies using log data from a different policy. We extend its applicability by developing an OPE method for a class of both full support and deficient support logging policies in contextual-bandit settings. This class includes deterministic bandit (such as Upper Confidence Bound) as well as deterministic decision-making based on supervised and unsupervised learning. We prove that our method's prediction converges in probability to the true performance of a counterfactual policy as the sample size increases. We validate our method with experiments on partly and entirely deterministic logging policies. Finally, we apply it to evaluate coupon targeting policies by a major online platform and show how to improve the existing policy.
BibTeX
@article{Narita_Okumura_Shimizu_Yata_2023, title={Counterfactual Learning with General Data-Generating Policies}, volume={37}, url={https://ojs.aaai.org/index.php/AAAI/article/view/26113}, DOI={10.1609/aaai.v37i8.26113}, abstractNote={Off-policy evaluation (OPE) attempts to predict the performance of counterfactual policies using log data from a different policy. We extend its applicability by developing an OPE method for a class of both full support and deficient support logging policies in contextual-bandit settings. This class includes deterministic bandit (such as Upper Confidence Bound) as well as deterministic decision-making based on supervised and unsupervised learning. We prove that our method’s prediction converges in probability to the true performance of a counterfactual policy as the sample size increases. We validate our method with experiments on partly and entirely deterministic logging policies. Finally, we apply it to evaluate coupon targeting policies by a major online platform and show how to improve the existing policy.}, number={8}, journal={Proceedings of the AAAI Conference on Artificial Intelligence}, author={Narita, Yusuke and Okumura, Kyohei and Shimizu, Akihiro and Yata, Kohei}, year={2023}, month={Jun.}, pages={9286-9293} }