Training Restricted Boltzmann Machines
Research output: Book/Report › Ph.D. thesis › Research
Standard
Training Restricted Boltzmann Machines. / Fischer, Asja.
Department of Computer Science, Faculty of Science, University of Copenhagen, 2014. 212 p.Research output: Book/Report › Ph.D. thesis › Research
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - BOOK
T1 - Training Restricted Boltzmann Machines
AU - Fischer, Asja
PY - 2014
Y1 - 2014
N2 - Restricted Boltzmann machines (RBMs) are probabilistic graphical models that can also be interpreted as stochastic neural networks. Training RBMs is known to be challenging. Computing the likelihood of the model parameters or its gradient is in general computationally intensive. Thus, training relies on sampling based approximations of the log-likelihood gradient. I will present an empirical and theoretical analysis of the bias of these approximations and show that the approximation error can lead to a distortion of the learning process. The bias decreases with increasing mixing rate of the applied sampling procedure and I will introduce a transition operator that leads to faster mixing. Finally, a different parametrisation of RBMs will be discussed that leads to better learning results and more robustness against changes in the data representation.
AB - Restricted Boltzmann machines (RBMs) are probabilistic graphical models that can also be interpreted as stochastic neural networks. Training RBMs is known to be challenging. Computing the likelihood of the model parameters or its gradient is in general computationally intensive. Thus, training relies on sampling based approximations of the log-likelihood gradient. I will present an empirical and theoretical analysis of the bias of these approximations and show that the approximation error can lead to a distortion of the learning process. The bias decreases with increasing mixing rate of the applied sampling procedure and I will introduce a transition operator that leads to faster mixing. Finally, a different parametrisation of RBMs will be discussed that leads to better learning results and more robustness against changes in the data representation.
UR - https://soeg.kb.dk/permalink/45KBDK_KGL/fbp0ps/alma99122112735905763
M3 - Ph.D. thesis
BT - Training Restricted Boltzmann Machines
PB - Department of Computer Science, Faculty of Science, University of Copenhagen
ER -
ID: 124435713