HyperDynamics: Meta-Learning Object and Agent Dynamics with Hypernetworks

TL;DR

HyperDynamics introduces a dynamics meta-learning framework that generates system-conditioned forward dynamics models on the fly by encoding interaction history and optional visual observations. A hypernetwork maps latent system properties to the weights of a dedicated forward model, enabling per-environment experts without maintaining a large ensemble or online gradient-based adaptation. The approach is demonstrated on object pushing and locomotion, where the generated experts rival standalone ensembles on seen environments and generalize to unseen properties with few-shot information, outperforming fixed-global models and gradient-based meta-learning baselines. The work highlights a multiplicative interaction between inferred system properties and low-dimensional state representations as a key driver of effective adaptation, with implications for scalable, transferable model-based RL.

Abstract

We propose HyperDynamics, a dynamics meta-learning framework that conditions on an agent's interactions with the environment and optionally its visual observations, and generates the parameters of neural dynamics models based on inferred properties of the dynamical system. Physical and visual properties of the environment that are not part of the low-dimensional state yet affect its temporal dynamics are inferred from the interaction history and visual observations, and are implicitly captured in the generated parameters. We test HyperDynamics on a set of object pushing and locomotion tasks. It outperforms existing dynamics models in the literature that adapt to environment variations by learning dynamics over high dimensional visual observations, capturing the interactions of the agent in recurrent state representations, or using gradient-based meta-optimization. We also show our method matches the performance of an ensemble of separately trained experts, while also being able to generalize well to unseen environment variations at test time. We attribute its good performance to the multiplicative interactions between the inferred system properties -- captured in the generated parameters -- and the low-dimensional state representation of the dynamical system.

Figures (8)

• Figure 1: HyperDynamics encodes the visual observations and a set of agent-environment interactions and generates the parameters of a dynamics model dedicated to the current environment and timestep using a hypernetwork. HyperDynamics for pushing follows the general formulation, with a visual encoder for detecting the object in 3D and encoding its shape, and an interaction encoder for encoding a small history of interactions of the agent with the environment.
• Figure 1: Motion prediction error (in centimeters).
• Figure 2: We consider two types of robots and two environments for locomotion. Top: Ant-Pier. Down: Cheetah-Slope.
• Figure 3: Comparison of average total return for locomotion tasks.
• Figure 3: Top: 24 objects used for training. Bottom: 7 objects used for testing. Objects selected from the ShapeNet Dataset shapenet2015 are realistic objects seen during daily life, and objects from the MIT Push dataset mitpush consist of basic geometries such as rectangle, triangle, and ellipse. Note that the shape of the testing objects differ from those used for training. (The third mug object used for testing is handleless.)