Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Survey of Imitation Learning Methods, Environments and Metrics

Nathan Gavenski, Felipe Meneguzzi, Michael Luck, Odinaldo Rodrigues

TL;DR

This survey addresses the fragmentation of imitation learning by introducing taxonomies for methods, environments, and metrics, and by advocating standardized evaluation protocols. It classifies IL approaches into behavioural cloning, dynamics-model-based, adversarial, hybrid, and online methods, highlighting the rise of hybrid and observation-based strategies. The work also proposes a threefold environment taxonomy (validation, precision, sequential) and a threefold metrics taxonomy (behaviour, domain, model), and critiques the field’s lack of standard benchmarks and evaluation procedures. It discusses safety, efficiency, and robustness as core concerns and outlines future directions to foster more human-like, transferable, and reliably evaluated IL systems. Overall, the paper provides a structured framework to compare IL methods comprehensively and to guide future research toward more consistent and meaningful evaluations.

Abstract

Imitation learning is an approach in which an agent learns how to execute a task by trying to mimic how one or more teachers perform it. This learning approach offers a compromise between the time it takes to learn a new task and the effort needed to collect teacher samples for the agent. It achieves this by balancing learning from the teacher, who has some information on how to perform the task, and deviating from their examples when necessary, such as states not present in the teacher samples. Consequently, the field of imitation learning has received much attention from researchers in recent years, resulting in many new methods and applications. However, with this increase in published work and past surveys focusing mainly on methodology, a lack of standardisation became more prominent in the field. This non-standardisation is evident in the use of environments, which appear in no more than two works, and evaluation processes, such as qualitative analysis, that have become rare in current literature. In this survey, we systematically review current imitation learning literature and present our findings by (i) classifying imitation learning techniques, environments and metrics by introducing novel taxonomies; (ii) reflecting on main problems from the literature; and (iii) presenting challenges and future directions for researchers.

A Survey of Imitation Learning Methods, Environments and Metrics

TL;DR

This survey addresses the fragmentation of imitation learning by introducing taxonomies for methods, environments, and metrics, and by advocating standardized evaluation protocols. It classifies IL approaches into behavioural cloning, dynamics-model-based, adversarial, hybrid, and online methods, highlighting the rise of hybrid and observation-based strategies. The work also proposes a threefold environment taxonomy (validation, precision, sequential) and a threefold metrics taxonomy (behaviour, domain, model), and critiques the field’s lack of standard benchmarks and evaluation procedures. It discusses safety, efficiency, and robustness as core concerns and outlines future directions to foster more human-like, transferable, and reliably evaluated IL systems. Overall, the paper provides a structured framework to compare IL methods comprehensively and to guide future research toward more consistent and meaningful evaluations.

Abstract

Imitation learning is an approach in which an agent learns how to execute a task by trying to mimic how one or more teachers perform it. This learning approach offers a compromise between the time it takes to learn a new task and the effort needed to collect teacher samples for the agent. It achieves this by balancing learning from the teacher, who has some information on how to perform the task, and deviating from their examples when necessary, such as states not present in the teacher samples. Consequently, the field of imitation learning has received much attention from researchers in recent years, resulting in many new methods and applications. However, with this increase in published work and past surveys focusing mainly on methodology, a lack of standardisation became more prominent in the field. This non-standardisation is evident in the use of environments, which appear in no more than two works, and evaluation processes, such as qualitative analysis, that have become rare in current literature. In this survey, we systematically review current imitation learning literature and present our findings by (i) classifying imitation learning techniques, environments and metrics by introducing novel taxonomies; (ii) reflecting on main problems from the literature; and (iii) presenting challenges and future directions for researchers.
Paper Structure (32 sections, 11 equations, 13 figures, 3 tables, 1 algorithm)

This paper contains 32 sections, 11 equations, 13 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Review preliminaries
  3. Background
  4. Imitation Learning Methods
  5. Behavioural Cloning
  6. Dynamics Model methods
  7. Inverse Dynamics Models
  8. Forward Dynamics Models
  9. Adversarial learning methods
  10. Hybrid approaches
  11. Online Imitation Learning
  12. Learning Methods Discussion
  13. Imitation Learning Environments
  14. Validation environments
  15. Precision environments
  16. ...and 17 more sections

Figures (13)

  • Figure 1: The Markovian Decision Process.
  • Figure 2: A general approach to imitation learning.
  • Figure 3: A new taxonomy for imitation learning methods.
  • Figure 4: Comparison between taxonomies. We remove online methods since both taxonomies are equal in that matter.
  • Figure 5: Imitation learning environments taxonomy.
  • ...and 8 more figures

Theorems & Definitions (8)

  • Definition 1
  • Definition 2
  • Definition 3
  • Definition 4
  • Definition 5
  • Definition 6
  • Definition 7
  • Definition 8