Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Survey on Machine Learning Approaches for Modelling Intuitive Physics

Jiafei Duan, Arijit Dasgupta, Jason Fischer, Cheston Tan

TL;DR

The paper addresses organizing recent ML work on intuitive physics for machine cognition, focusing on three facets (prediction, inference, causal reasoning) and six evaluation tasks. It proposes three high-level approaches (inverse rendering, inverse physics, inverse dynamics) to frame the methods and surveys datasets and metrics used in the field. It identifies core challenges, including the lack of unified evaluation benchmarks, scalability to real-world complexity, and generalization across object counts and contexts, and suggests directions for future research. Overall, the survey provides a structured map of the field to guide researchers and practitioners.

Abstract

Research in cognitive science has provided extensive evidence of human cognitive ability in performing physical reasoning of objects from noisy perceptual inputs. Such a cognitive ability is commonly known as intuitive physics. With advancements in deep learning, there is an increasing interest in building intelligent systems that are capable of performing physical reasoning from a given scene for the purpose of building better AI systems. As a result, many contemporary approaches in modelling intuitive physics for machine cognition have been inspired by literature from cognitive science. Despite the wide range of work in physical reasoning for machine cognition, there is a scarcity of reviews that organize and group these deep learning approaches. Especially at the intersection of intuitive physics and artificial intelligence, there is a need to make sense of the diverse range of ideas and approaches. Therefore, this paper presents a comprehensive survey of recent advances and techniques in intuitive physics-inspired deep learning approaches for physical reasoning. The survey will first categorize existing deep learning approaches into three facets of physical reasoning before organizing them into three general technical approaches and propose six categorical tasks of the field. Finally, we highlight the challenges of the current field and present some future research directions.

A Survey on Machine Learning Approaches for Modelling Intuitive Physics

TL;DR

The paper addresses organizing recent ML work on intuitive physics for machine cognition, focusing on three facets (prediction, inference, causal reasoning) and six evaluation tasks. It proposes three high-level approaches (inverse rendering, inverse physics, inverse dynamics) to frame the methods and surveys datasets and metrics used in the field. It identifies core challenges, including the lack of unified evaluation benchmarks, scalability to real-world complexity, and generalization across object counts and contexts, and suggests directions for future research. Overall, the survey provides a structured map of the field to guide researchers and practitioners.

Abstract

Research in cognitive science has provided extensive evidence of human cognitive ability in performing physical reasoning of objects from noisy perceptual inputs. Such a cognitive ability is commonly known as intuitive physics. With advancements in deep learning, there is an increasing interest in building intelligent systems that are capable of performing physical reasoning from a given scene for the purpose of building better AI systems. As a result, many contemporary approaches in modelling intuitive physics for machine cognition have been inspired by literature from cognitive science. Despite the wide range of work in physical reasoning for machine cognition, there is a scarcity of reviews that organize and group these deep learning approaches. Especially at the intersection of intuitive physics and artificial intelligence, there is a need to make sense of the diverse range of ideas and approaches. Therefore, this paper presents a comprehensive survey of recent advances and techniques in intuitive physics-inspired deep learning approaches for physical reasoning. The survey will first categorize existing deep learning approaches into three facets of physical reasoning before organizing them into three general technical approaches and propose six categorical tasks of the field. Finally, we highlight the challenges of the current field and present some future research directions.
Paper Structure (10 sections, 2 figures, 1 table)

This paper contains 10 sections, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Background
  3. Motivation
  4. Survey Organization
  5. Facets of Physical Reasoning
  6. Prediction
  7. Inference
  8. Causal Reasoning
  9. Challenges and Open Questions
  10. Conclusion

Figures (2)

  • Figure 1: Examples of everyday scenes that requires us to employ intuitive physics. (A) Predicting the trajectories of the billiard balls. (B) Balancing the stacking blocks. (C) Shooting the basketball with a parabola trajectory. (D) Balancing one's body during yoga. (E) Estimating the velocity of the car travelling ahead. (F) A gym with equipment that exhibits various degree of physical properties.
  • Figure 2: Summary of the six physical reasoning tasks. (1) PIO, to predict the different states or outcome of physical interactions (e.g., if objects within the dynamic scene is stable, contained, or contacted). (2) PTD, to predict the possible physical trajectories given only a few dynamics scenes. (3) PPI, to infer both the observed(e.g., size, color, and shape) and latent physical properties (e.g., mass, friction, velocity, and displacement) of the objects within the dynamic scenes. (4) VSG, to generate the unseen future frames of a long roll-out sequence given only the initial few dynamics frames. (5) VoE, to classify if there is any violation-of-expectation in a given dynamics scenes. (6) Others, other intuitive physics-inspired AI tasks such as curriculum learning, counterfactual prediction, physical equations prediction etc.