Minimax policies for adversarial and stochastic bandits

Jean-Yves Audibert (1, 2) , Sébastien Bubeck (3)

1 LIGM - Laboratoire d'Informatique Gaspard-Monge
2 imagine [Marne-la-Vallée]
3 SEQUEL - Sequential Learning

We fill in a long open gap in the characterization of the minimax rate for the multi-armed bandit prob- lem. Concretely, we remove an extraneous loga- rithmic factor in the previously known upper bound and propose a new family of randomized algorithms based on an implicit normalization, as well as a new analysis. We also consider the stochastic case, and prove that an appropriate modification of the upper confidence bound policy UCB1 (Auer et al., 2002) achieves the distribution-free optimal rate while still having a distribution-dependent rate log- arithmic in the number of plays.

Domains: Computer Vision and Pattern Recognition [cs.CV]

Jean-Yves Audibert, Sébastien Bubeck. Minimax policies for adversarial and stochastic bandits. COLT, Jun 2009, Montreal, Canada. pp.217-226. ⟨hal-00834882⟩