Algorithms for estimating the partition function of restricted Boltzmann machines: (Extended Abstract)

Publikation: Bidrag til bog/antologi/rapportKonferencebidrag i proceedingsForskningfagfællebedømt

Standard

Algorithms for estimating the partition function of restricted Boltzmann machines : (Extended Abstract). / Krause, Oswin; Fischer, Asja; Igel, Christian.

Proceedings of the 29th International Joint Conference on Artificial Intelligence, IJCAI 2020. red. / Christian Bessiere. International Joint Conferences on Artificial Intelligence, 2020. s. 5045-5049 (IJCAI International Joint Conference on Artificial Intelligence, Bind 2021-January).

Publikation: Bidrag til bog/antologi/rapportKonferencebidrag i proceedingsForskningfagfællebedømt

Harvard

Krause, O, Fischer, A & Igel, C 2020, Algorithms for estimating the partition function of restricted Boltzmann machines: (Extended Abstract). i C Bessiere (red.), Proceedings of the 29th International Joint Conference on Artificial Intelligence, IJCAI 2020. International Joint Conferences on Artificial Intelligence, IJCAI International Joint Conference on Artificial Intelligence, bind 2021-January, s. 5045-5049, 29th International Joint Conference on Artificial Intelligence, IJCAI 2020, Yokohama, Japan, 01/01/2021. https://doi.org/10.24963/ijcai.2020/704

APA

Krause, O., Fischer, A., & Igel, C. (2020). Algorithms for estimating the partition function of restricted Boltzmann machines: (Extended Abstract). I C. Bessiere (red.), Proceedings of the 29th International Joint Conference on Artificial Intelligence, IJCAI 2020 (s. 5045-5049). International Joint Conferences on Artificial Intelligence. IJCAI International Joint Conference on Artificial Intelligence Bind 2021-January https://doi.org/10.24963/ijcai.2020/704

Vancouver

Krause O, Fischer A, Igel C. Algorithms for estimating the partition function of restricted Boltzmann machines: (Extended Abstract). I Bessiere C, red., Proceedings of the 29th International Joint Conference on Artificial Intelligence, IJCAI 2020. International Joint Conferences on Artificial Intelligence. 2020. s. 5045-5049. (IJCAI International Joint Conference on Artificial Intelligence, Bind 2021-January). https://doi.org/10.24963/ijcai.2020/704

Author

Krause, Oswin ; Fischer, Asja ; Igel, Christian. / Algorithms for estimating the partition function of restricted Boltzmann machines : (Extended Abstract). Proceedings of the 29th International Joint Conference on Artificial Intelligence, IJCAI 2020. red. / Christian Bessiere. International Joint Conferences on Artificial Intelligence, 2020. s. 5045-5049 (IJCAI International Joint Conference on Artificial Intelligence, Bind 2021-January).

Bibtex

@inproceedings{e12574443f0f4bdaa4161c5089013511,
title = "Algorithms for estimating the partition function of restricted Boltzmann machines: (Extended Abstract)",
abstract = "Estimating the normalization constants (partition functions) of energy-based probabilistic models (Markov random fields) with a high accuracy is required for measuring performance, monitoring the training progress of adaptive models, and conducting likelihood ratio tests. We devised a unifying theoretical framework for algorithms for estimating the partition function, including Annealed Importance Sampling (AIS) and Bennett's Acceptance Ratio method (BAR). The unification reveals conceptual similarities of and differences between different approaches and suggests new algorithms. The framework is based on a generalized form of Crooks' equality, which links the expectation over a distribution of samples generated by a transition operator to the expectation over the distribution induced by the reversed operator. Different ways of sampling, such as parallel tempering and path sampling, are covered by the framework. We performed experiments in which we estimated the partition function of restricted Boltzmann machines (RBMs) and Ising models. We found that BAR using parallel tempering worked well with a small number of bridging distributions, while path sampling based AIS performed best with many bridging distributions. The normalization constant is measured w.r.t. a reference distribution, and the choice of this distribution turned out to be very important in our experiments. Overall, BAR gave the best empirical results, outperforming AIS.",
author = "Oswin Krause and Asja Fischer and Christian Igel",
year = "2020",
doi = "10.24963/ijcai.2020/704",
language = "English",
series = "IJCAI International Joint Conference on Artificial Intelligence",
pages = "5045--5049",
editor = "Christian Bessiere",
booktitle = "Proceedings of the 29th International Joint Conference on Artificial Intelligence, IJCAI 2020",
publisher = "International Joint Conferences on Artificial Intelligence",
note = "29th International Joint Conference on Artificial Intelligence, IJCAI 2020 ; Conference date: 01-01-2021",

}

RIS

TY - GEN

T1 - Algorithms for estimating the partition function of restricted Boltzmann machines

T2 - 29th International Joint Conference on Artificial Intelligence, IJCAI 2020

AU - Krause, Oswin

AU - Fischer, Asja

AU - Igel, Christian

PY - 2020

Y1 - 2020

N2 - Estimating the normalization constants (partition functions) of energy-based probabilistic models (Markov random fields) with a high accuracy is required for measuring performance, monitoring the training progress of adaptive models, and conducting likelihood ratio tests. We devised a unifying theoretical framework for algorithms for estimating the partition function, including Annealed Importance Sampling (AIS) and Bennett's Acceptance Ratio method (BAR). The unification reveals conceptual similarities of and differences between different approaches and suggests new algorithms. The framework is based on a generalized form of Crooks' equality, which links the expectation over a distribution of samples generated by a transition operator to the expectation over the distribution induced by the reversed operator. Different ways of sampling, such as parallel tempering and path sampling, are covered by the framework. We performed experiments in which we estimated the partition function of restricted Boltzmann machines (RBMs) and Ising models. We found that BAR using parallel tempering worked well with a small number of bridging distributions, while path sampling based AIS performed best with many bridging distributions. The normalization constant is measured w.r.t. a reference distribution, and the choice of this distribution turned out to be very important in our experiments. Overall, BAR gave the best empirical results, outperforming AIS.

AB - Estimating the normalization constants (partition functions) of energy-based probabilistic models (Markov random fields) with a high accuracy is required for measuring performance, monitoring the training progress of adaptive models, and conducting likelihood ratio tests. We devised a unifying theoretical framework for algorithms for estimating the partition function, including Annealed Importance Sampling (AIS) and Bennett's Acceptance Ratio method (BAR). The unification reveals conceptual similarities of and differences between different approaches and suggests new algorithms. The framework is based on a generalized form of Crooks' equality, which links the expectation over a distribution of samples generated by a transition operator to the expectation over the distribution induced by the reversed operator. Different ways of sampling, such as parallel tempering and path sampling, are covered by the framework. We performed experiments in which we estimated the partition function of restricted Boltzmann machines (RBMs) and Ising models. We found that BAR using parallel tempering worked well with a small number of bridging distributions, while path sampling based AIS performed best with many bridging distributions. The normalization constant is measured w.r.t. a reference distribution, and the choice of this distribution turned out to be very important in our experiments. Overall, BAR gave the best empirical results, outperforming AIS.

U2 - 10.24963/ijcai.2020/704

DO - 10.24963/ijcai.2020/704

M3 - Article in proceedings

AN - SCOPUS:85097346766

T3 - IJCAI International Joint Conference on Artificial Intelligence

SP - 5045

EP - 5049

BT - Proceedings of the 29th International Joint Conference on Artificial Intelligence, IJCAI 2020

A2 - Bessiere, Christian

PB - International Joint Conferences on Artificial Intelligence

Y2 - 1 January 2021

ER -

ID: 256580661