The Bandit Learning beyond Stationarity under the Game Theory

It is a universal truth that classical multi-armed bandit (MAB) theory relies on the stationary assumption of reward distributions. However, this assumption is not viable in systems with multiple adaptive interacting agents where interactions endogenously redefine rewards. This paper explores bandit learning in non-stationary systems from a game-theoretic perspective, where learning in game theory, modern bandit theory, and recent advances in multi-player bandits, cooperative learning, federated decision systems, incentive design, adversarial robustness, and teamwork modeling are leveraged to propose a unifying theory of endogenous non-stationarity in bandit learning. The paper provides a conceptual framework and synthesis of strategic regret in interacting systems, stability regimes of learning systems, and how exploration-exploitation trade-offs are transformed in interactive systems.