Reinforcement learning (RL) is a biological inspired learning
paradigm based on trial-and-error learning. A successful RL algorithm
has to balance exploration of new behavioral strategies and exploitation
of already obtained knowledge. In the initial learning phase exploration
is the dominant process. Exploration is realized by stochastic perturbations,
which can be applied at different levels. When considering direct
policy search in the space of neural network policies, exploration can be
applied on the synaptic level or on the level of neuronal activity. We
propose neuroevolution strategies (NeuroESs) for direct policy search in
RL. Learning using NeuroESs can be interpreted as modelling of extrinsic
perturbations on the level of synaptic weights. In contrast, policy
gradient methods (PGMs) can be regarded as intrinsic perturbation of
neuronal activity. We compare these two approaches conceptually and
experimentally.