DeLTA seminar by Yi-Shan Wu
On 11 October, the DeLTA Lab from the Department of Computer Science, University of Copenhagen, holds a seminar titled 'Chebyshev-Cantelli PAC-Bayes-Bennett Inequality for the Weighted Majority Vote' by Yi-Shan Wu.
Speaker
Yi-Shan Wu, University of Copenhagen
Title
Chebyshev-Cantelli PAC-Bayes-Bennett Inequality for the Weighted Majority Vote
Abstract
We present a new second-order oracle bound for the expected risk of a weighted majority vote. The bound is based on a novel parametric form of the Chebyshev-Cantelli inequality (a.k.a. one-sided Chebyshev’s), which is amenable to efficient minimization. The new form resolves the optimization challenge faced by prior oracle bounds based on the Chebyshev-Cantelli inequality, the C-bounds [Germain et al., 2015], and, at the same time, it improves on the oracle bound based on second order Markov’s inequality introduced by Masegosa et al. [2020]. We also derive the PAC-Bayes-Bennett inequality, which we use for empirical estimation of tthe oracle bound. The PAC-Bayes-Bennett inequality improves on the PAC-Bayes-Bernstein inequality by Seldin et al. [2012]. We provide an empirical evaluation demonstrating that the new bounds can improve on the work by Masegosa et al. [2020]. Both the parametric form of the Chebyshev-Cantelli inequality and the PAC-Bayes-Bennett inequality may be of independent interest for the study of concentration of measure in other domains.
Joint work with Andrés R. Masegosa, Stephan S. Lorenzen, Christian Igel and Yevgeny Seldin
https://arxiv.org/pdf/2106.13624.pdf (accepted to NeurIPS 2021)
-------------------
Other Upcoming DeLTA seminars:
25 October 2021 @ 10:00. Niklas Andreas Pfister.
15 November 2021 @ 10:00. Sorawit Saengkyongam. Invariant Policy Learning: A Causal Perspective.
-------------------
You can subscribe to DeLTA Seminar mailing list by sending an empty email to delta-seminar-join@list.ku.dk.
Information about DeLTA Seminars is available at DeLTA Lab page: https://sites.google.com/diku.edu/delta
DeLTA is a research group affiliated with the Department of Computer Science at the University of Copenhagen studying diverse aspects of Machine Learning Theory and its applications, including, but not limited to Reinforcement Learning, Online Learning and Bandits, PAC-Bayesian analysis