Empirical Bernstein stopping

Volodymyr Mnih (1) , Csaba Szepesvari (1) , Jean-Yves Audibert (2, 3, 4, 5)

1 Department of Computing Science [Edmonton]
2 CERTIS - Centre d'Enseignement et de Recherche en Technologies de l'Information et Systèmes
3 WILLOW - Models of visual object recognition and scene understanding
4 imagine [Marne-la-Vallée]
5 LIGM - Laboratoire d'Informatique Gaspard-Monge

Sampling is a popular way of scaling up machine learning algorithms to large datasets. The question often is how many samples are needed. Adaptive stopping algorithms monitor the performance in an online fashion and they can stop early, saving valuable resources. We consider problems where probabilistic guarantees are desired and demonstrate how recently-introduced empirical Bernstein bounds can be used to design stopping rules that are efficient. We provide upper bounds on the sample complexity of the new rules, as well as empirical results on model selection and boosting in the filtering setting.

Domains: Computer Vision and Pattern Recognition [cs.CV]

Volodymyr Mnih, Csaba Szepesvari, Jean-Yves Audibert. Empirical Bernstein stopping. ICML '08 Proceedings of the 25th international conference on Machine learning, Jul 2008, Helsinki, Finland. pp.672-679, ⟨10.1145/1390156.1390241⟩. ⟨hal-00834983⟩