Qualitative and quantitative assessment of step size adaptation rules
Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review
Standard
Qualitative and quantitative assessment of step size adaptation rules. / Krause, Oswin; Glasmachers, Tobias; Igel, Christian.
Proceedings of the 14th ACM/SIGEVO Conference on Foundations of Genetic Algorithms. Association for Computing Machinery, 2017. p. 139-148.Research output: Chapter in Book/Report/Conference proceeding › Article in proceedings › Research › peer-review
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - GEN
T1 - Qualitative and quantitative assessment of step size adaptation rules
AU - Krause, Oswin
AU - Glasmachers, Tobias
AU - Igel, Christian
N1 - Conference code: 14
PY - 2017
Y1 - 2017
N2 - We present a comparison of step size adaptation methods for evolution strategies, covering recent developments in the field. Following recent work by Hansen et al. we formulate a concise list of performance criteria: a) fast convergence of the mean, b) near-optimal fixed point of the normalized step size dynamics, and c) invariance to adding constant dimensions of the objective function. Our results show that algorithms violating these principles tend to underestimate the step size or are unreliable when the function does not fit to the algorithm's tuned hyperparameters. In contrast, we find that cumulative step size adaptation (CSA) and twopoint adaptation (TPA) provide reliable estimates of the optimal step size. We further find that removing the evolution path of CSA still leads to a reliable algorithm without the computational requirements of CSA.
AB - We present a comparison of step size adaptation methods for evolution strategies, covering recent developments in the field. Following recent work by Hansen et al. we formulate a concise list of performance criteria: a) fast convergence of the mean, b) near-optimal fixed point of the normalized step size dynamics, and c) invariance to adding constant dimensions of the objective function. Our results show that algorithms violating these principles tend to underestimate the step size or are unreliable when the function does not fit to the algorithm's tuned hyperparameters. In contrast, we find that cumulative step size adaptation (CSA) and twopoint adaptation (TPA) provide reliable estimates of the optimal step size. We further find that removing the evolution path of CSA still leads to a reliable algorithm without the computational requirements of CSA.
KW - Comparison
KW - Evolution strategies
KW - Step size adaptation
UR - http://www.scopus.com/inward/record.url?scp=85018963553&partnerID=8YFLogxK
U2 - 10.1145/3040718.3040725
DO - 10.1145/3040718.3040725
M3 - Article in proceedings
AN - SCOPUS:85018963553
SP - 139
EP - 148
BT - Proceedings of the 14th ACM/SIGEVO Conference on Foundations of Genetic Algorithms
PB - Association for Computing Machinery
Y2 - 12 January 2017 through 15 January 2017
ER -
ID: 179557726