Discrete Natural Evolution Strategies
Ahmad Ayaz Amin
TL;DR
This work extends Natural Evolution Strategies to discrete parameter spaces, enabling optimization when gradients are unavailable for discrete variables. It derives Monte Carlo gradient estimators for discrete search distributions (Bernoulli and categorical) and discusses natural-gradient updates without heavy Fisher information computations in practice. Through a program-induction sketching experiment, discrete NES with separate discrete and continuous holes achieves outputs close to ground-truth and shows improved stability over a VO baseline, while revealing that incorporating the Fisher information matrix can harm performance in discrete settings. The results suggest discrete NES as a practical tool for discrete-parameter optimization and a scalable complement to continuous NES, with future work directed at applying it to more complex problems such as discrete policies in reinforcement learning.
Abstract
Natural evolution strategies are a class of approximate-gradient black-box optimizers that have been successfully used for continuous parameter spaces. In this paper, we derive NES algorithms for discrete parameter spaces and demonstrate their effectiveness in tasks involving discrete parameters.