The Effect of Training Schedules on Morphological Robustness and Generalization

TL;DR

This paper defines various training schedules to specify how variations are introduced during an evolutionary learning process to improve generalization as a reinforcement learning problem, and performs an extensive analysis of the effect of these training schedules on morphological generalization.

Abstract

Robustness and generalizability are the key properties of artificial neural network (ANN)-based controllers for maintaining a reliable performance in case of changes. It is demonstrated that exposing the ANNs to variations during training processes can improve their robustness and generalization capabilities. However, the way in which this variation is introduced can have a significant impact. In this paper, we define various training schedules to specify how these variations are introduced during an evolutionary learning process. In particular, we focus on morphological robustness and generalizability concerned with finding an ANN-based controller that can provide sufficient performance on a range of physical variations. Then, we perform an extensive analysis of the effect of these training schedules on morphological generalization. Furthermore, we formalize the process of training sample selection (i.e., morphological variations) to improve generalization as a reinforcement learning problem. Overall, our results provide deeper insights into the role of variability and the ways of enhancing the generalization property of evolved ANN-based controllers.

Figures (5)

• Figure 1: Performance comparison of training schedules on the testing morphologies on Bipedal Walker. * indicates $p < 0.05$, ** indicates $p < 0.01$, denoting statistical significance levels.
• Figure 2: Performance comparison on the training morphologies on Bipedal Walker. * indicates $p < 0.05$, ** indicates $p < 0.01$, denoting statistical significance levels.
• Figure 3: Performance comparison of Random schedule and the Multi-Armed Bandit approach on the testing morphologies on Bipedal Walker, Walker2D, and Ant. * indicates $p < 0.05$, ** indicates $p < 0.01$, denoting statistical significance levels.
• Figure 4: Heatmap illustrating the frequency of selection for each morphology during training on Walker2D. Each cell in the heatmap represents a unique morphology, and the numerical value within the cell indicates the frequency with which that morphology was selected during training. This frequency is computed as the average occurrence across 30 independent training runs.
• Figure 5: Heatmap illustrating the average performance achieved on all morphological variations on Walker2D using the Random and MAB training schedules. The cells that are within the square with red borders show the training morphologies. The morphological parameters that are beyond these borders are used for the testing morphologies.