A strongly quasiconvex PAC-Bayesian bound

Publikation: Bidrag til bog/antologi/rapportKonferencebidrag i proceedingsForskningfagfællebedømt

Standard

A strongly quasiconvex PAC-Bayesian bound. / Thiemann, Niklas; Igel, Christian; Wintenberger, Olivier; Seldin, Yevgeny.

Proceedings of International Conference on Algorithmic Learning Theory, 15-17 October 2017, Kyoto University, Kyoto, Japan . red. / Steve Hanneke; Lev Reyzin. Proceedings of Machine Learning Research, 2017. s. 466-492 (Proceedings of Machine Learning Research, Bind 76).

Publikation: Bidrag til bog/antologi/rapportKonferencebidrag i proceedingsForskningfagfællebedømt

Harvard

Thiemann, N, Igel, C, Wintenberger, O & Seldin, Y 2017, A strongly quasiconvex PAC-Bayesian bound. i S Hanneke & L Reyzin (red), Proceedings of International Conference on Algorithmic Learning Theory, 15-17 October 2017, Kyoto University, Kyoto, Japan . Proceedings of Machine Learning Research, Proceedings of Machine Learning Research, bind 76, s. 466-492 , The 28th International Conference on Algorithmic Learning Theory (ALT), Kyoto, Japan, 15/10/2017.

APA

Thiemann, N., Igel, C., Wintenberger, O., & Seldin, Y. (2017). A strongly quasiconvex PAC-Bayesian bound. I S. Hanneke, & L. Reyzin (red.), Proceedings of International Conference on Algorithmic Learning Theory, 15-17 October 2017, Kyoto University, Kyoto, Japan (s. 466-492 ). Proceedings of Machine Learning Research. Proceedings of Machine Learning Research, Bind. 76

Vancouver

Thiemann N, Igel C, Wintenberger O, Seldin Y. A strongly quasiconvex PAC-Bayesian bound. I Hanneke S, Reyzin L, red., Proceedings of International Conference on Algorithmic Learning Theory, 15-17 October 2017, Kyoto University, Kyoto, Japan . Proceedings of Machine Learning Research. 2017. s. 466-492 . (Proceedings of Machine Learning Research, Bind 76).

Author

Thiemann, Niklas ; Igel, Christian ; Wintenberger, Olivier ; Seldin, Yevgeny. / A strongly quasiconvex PAC-Bayesian bound. Proceedings of International Conference on Algorithmic Learning Theory, 15-17 October 2017, Kyoto University, Kyoto, Japan . red. / Steve Hanneke ; Lev Reyzin. Proceedings of Machine Learning Research, 2017. s. 466-492 (Proceedings of Machine Learning Research, Bind 76).

Bibtex

@inproceedings{55e139e32c0240c8acf265c5952644e2,
title = "A strongly quasiconvex PAC-Bayesian bound",
abstract = "We propose a new PAC-Bayesian bound and a way of constructing a hypothesis space, so that the bound is convex in the posterior distribution and also convex in a trade-off parameter between empirical performance of the posterior distribution and its complexity. The complexity is measured by the Kullback-Leibler divergence to a prior. We derive an alternating procedure for minimizing the bound. We show that the bound can be rewritten as a one-dimensional function of the trade-off parameter and provide sufficient conditions under which the function has a single global minimum. When the conditions are satisfied the alternating minimization is guaranteed to converge to the global minimum of the bound. We provide experimental results demonstrating that rigorous minimization of the bound is competitive with cross-validation in tuning the trade-off between complexity and empirical performance. In all our experiments the trade-off turned to be quasiconvex even when the sufficient conditions were violated.",
author = "Niklas Thiemann and Christian Igel and Olivier Wintenberger and Yevgeny Seldin",
year = "2017",
language = "English",
series = "Proceedings of Machine Learning Research",
pages = "466--492",
editor = "Steve Hanneke and Lev Reyzin",
booktitle = "Proceedings of International Conference on Algorithmic Learning Theory, 15-17 October 2017, Kyoto University, Kyoto, Japan",
publisher = "Proceedings of Machine Learning Research",
note = "The 28th International Conference on Algorithmic Learning Theory (ALT), ALT ; Conference date: 15-10-2017 Through 17-10-2017",
url = "http://www.comp.nus.edu.sg/~fstephan/alt/alt2017/",

}

RIS

TY - GEN

T1 - A strongly quasiconvex PAC-Bayesian bound

AU - Thiemann, Niklas

AU - Igel, Christian

AU - Wintenberger, Olivier

AU - Seldin, Yevgeny

PY - 2017

Y1 - 2017

N2 - We propose a new PAC-Bayesian bound and a way of constructing a hypothesis space, so that the bound is convex in the posterior distribution and also convex in a trade-off parameter between empirical performance of the posterior distribution and its complexity. The complexity is measured by the Kullback-Leibler divergence to a prior. We derive an alternating procedure for minimizing the bound. We show that the bound can be rewritten as a one-dimensional function of the trade-off parameter and provide sufficient conditions under which the function has a single global minimum. When the conditions are satisfied the alternating minimization is guaranteed to converge to the global minimum of the bound. We provide experimental results demonstrating that rigorous minimization of the bound is competitive with cross-validation in tuning the trade-off between complexity and empirical performance. In all our experiments the trade-off turned to be quasiconvex even when the sufficient conditions were violated.

AB - We propose a new PAC-Bayesian bound and a way of constructing a hypothesis space, so that the bound is convex in the posterior distribution and also convex in a trade-off parameter between empirical performance of the posterior distribution and its complexity. The complexity is measured by the Kullback-Leibler divergence to a prior. We derive an alternating procedure for minimizing the bound. We show that the bound can be rewritten as a one-dimensional function of the trade-off parameter and provide sufficient conditions under which the function has a single global minimum. When the conditions are satisfied the alternating minimization is guaranteed to converge to the global minimum of the bound. We provide experimental results demonstrating that rigorous minimization of the bound is competitive with cross-validation in tuning the trade-off between complexity and empirical performance. In all our experiments the trade-off turned to be quasiconvex even when the sufficient conditions were violated.

M3 - Article in proceedings

T3 - Proceedings of Machine Learning Research

SP - 466

EP - 492

BT - Proceedings of International Conference on Algorithmic Learning Theory, 15-17 October 2017, Kyoto University, Kyoto, Japan

A2 - Hanneke, Steve

A2 - Reyzin, Lev

PB - Proceedings of Machine Learning Research

T2 - The 28th International Conference on Algorithmic Learning Theory (ALT)

Y2 - 15 October 2017 through 17 October 2017

ER -

ID: 197764786