Multi-Armed Bandit Algorithms and Their Applications in Online Recommendation Systems

Recommendation systems are used in various online services like streaming media services, e-commerce sites and news apps to enable users to find content of their choice quickly. This often involves sequential decision-making without knowing the rewards or pay-offs that the recommendations yield. A classic example of such decision-making is the multi-armed bandit (MAB) framework. This paper gives an overview of the basic MAB algorithms and their use cases for online recommendation systems. The paper covered some of the popular bandit algorithms, such as Upper Confidence Bound (UCB), Thompson Sampling, contextual bandits, off-policy evaluation, etc. Bandit algorithms are instrumental in solving the exploration-exploitation dilemma in the recommendation problem. They can learn the user preference and meanwhile, keep optimizing the quality of the item list. However, there are still a few practical problems, such as noisy feedback, large action space, and non-stationarity of user behavior, that remain to be solved.