Procedural Generation of Algorithm Discovery Tasks in Machine Learning

Abstract

Automating the development of machine learning algorithms has the potential to unlock new breakthroughs. However, our ability to improve and evaluate algorithm discovery systems has thus far been limited by existing task suites. They suffer from many issues, such as: poor evaluation methodologies; data contamination; and containing saturated or very similar problems. Here, we introduce DiscoGen, a procedural generator of algorithm discovery tasks for machine learning, such as developing optimisers for reinforcement learning or loss functions for image classification. Motivated by the success of procedural generation in reinforcement learning, DiscoGen spans millions of tasks of varying difficulty and complexity from a range of machine learning fields. These tasks are specified by a small number of configuration parameters and can be used to optimise algorithm discovery agents (ADAs). We present DiscoBench, a benchmark consisting of a fixed, small subset of DiscoGen tasks for principled evaluation of ADAs. Finally, we propose a number of ambitious, impactful research directions enabled by DiscoGen, in addition to experiments demonstrating its use for prompt optimisation of an ADA. DiscoGen is released open-source at https://github.com/AlexGoldie/discogen.

Figures (25)

• Figure 1: A typical DiscoGen setup. DiscoGen procedurally generates new algorithm discovery tasks. For every generated task, an algorithm discovery agent iteratively develops new algorithms (the meta-loop) for training in the task's meta-train datasets (the inner-loops). The developed algorithm is evaluated on meta-test datasets, with the evaluation score used to optimise the agent (the ADA optimisation-loop). Datasets that are available in a task domain can also be excluded from the task. After each step, DiscoGen can be sampled for a new task. After ADA optimisation has completed, the agent is evaluated on DiscoBench, a set of ADA test tasks.
• Figure 2: Clustered average rank correlation over all datasets (meta-train and meta-test) for each task.
• Figure 3: Rank correlations for Meta-Train (left) and Meta-Test (right). The Meta-Test plot inherits the clustering order from Meta-Train to visualize consistency across splits.
• Figure 4: Visualisation of performance for the four environments that are used in the $K_{tasks}=1$ prompt optimisation experiment. We plot mean and standard error of the evaluation return achieved by the final policy for each environment over 8 inner-loop seeds, as defined by the On-Policy RL task domain implementation.
• Figure 5: DiscoBench Single results on Meta-Train tasks. (Part 1/4)