Header logo is

Worst-Case Bounds for Gaussian Process Models


Conference Paper


We present a competitive analysis of some non-parametric Bayesian algorithms in a worst-case online learning setting, where no probabilistic assumptions about the generation of the data are made. We consider models which use a Gaussian process prior (over the space of all functions) and provide bounds on the regret (under the log loss) for commonly used non-parametric Bayesian algorithms - including Gaussian regression and logistic regression - which show how these algorithms can perform favorably under rather general conditions. These bounds explicitly handle the infinite dimensionality of these non-parametric classes in a natural way. We also make formal connections to the minimax and emph{minimum description length} (MDL) framework. Here, we show precisely how Bayesian Gaussian regression is a minimax strategy.

Author(s): Kakade, S. and Seeger, M. and Foster, D.
Book Title: Advances in neural information processing systems 18
Journal: Advances in Neural Information Processing Systems 18: Proceedings of the 2005 Conference
Pages: 619-626
Year: 2006
Month: May
Day: 0
Editors: Weiss, Y. , B. Sch{\"o}lkopf, J. Platt
Publisher: MIT Press

Department(s): Empirical Inference
Bibtex Type: Conference Paper (inproceedings)

Event Name: Nineteenth Annual Conference on Neural Information Processing Systems (NIPS 2005)
Event Place: Vancouver, BC, Canada

Address: Cambridge, MA, USA
Digital: 0
ISBN: 0-262-23253-7
Language: en
Organization: Max-Planck-Gesellschaft
School: Biologische Kybernetik

Links: PDF


  title = {Worst-Case Bounds for Gaussian Process Models},
  author = {Kakade, S. and Seeger, M. and Foster, D.},
  journal = {Advances in Neural Information Processing Systems 18: Proceedings of the 2005 Conference},
  booktitle = {Advances in neural information processing systems 18},
  pages = {619-626},
  editors = {Weiss, Y. , B. Sch{\"o}lkopf, J. Platt},
  publisher = {MIT Press},
  organization = {Max-Planck-Gesellschaft},
  school = {Biologische Kybernetik},
  address = {Cambridge, MA, USA},
  month = may,
  year = {2006},
  month_numeric = {5}