Digital Gene: Learning about the Physical World through Analytic Concepts

TL;DR

This work identifies a key limitation of current AI—semantic-level understanding from internet data—when dealing with physical world tasks, especially in safety-critical settings. It proposes analytic concepts, explicit, programmatic representations of physical concepts encoded as Class templates, to ground neural networks in physical laws and enable forward (grounding, reasoning, creating) and inverse (generating, simulating, planning) processes. By enabling bidirectional alignment between data and physical entities and supporting close-loop learning through active and self-supervised approaches, the framework aims to improve generalization, controllability, and reliability in robotics, autonomous systems, and physical simulation. An accompanying infrastructure—concept libraries, annotation platforms, and procedural-generation pipelines—facilitates data preparation and practical deployment of physics-informed AI systems.

Abstract

Reviewing the progress in artificial intelligence over the past decade, various significant advances (e.g. object detection, image generation, large language models) have enabled AI systems to produce more semantically meaningful outputs and achieve widespread adoption in internet scenarios. Nevertheless, AI systems still struggle when it comes to understanding and interacting with the physical world. This reveals an important issue: relying solely on semantic-level concepts learned from internet data (e.g. texts, images) to understand the physical world is far from sufficient -- machine intelligence currently lacks an effective way to learn about the physical world. This research introduces the idea of analytic concept -- representing the concepts related to the physical world through programs of mathematical procedures, providing machine intelligence a portal to perceive, reason about, and interact with the physical world. Except for detailing the design philosophy and providing guidelines for the application of analytic concepts, this research also introduce about the infrastructure that has been built around analytic concepts. I aim for my research to contribute to addressing these questions: What is a proper abstraction of general concepts in the physical world for machine intelligence? How to systematically integrate structured priors with neural networks to constrain AI systems to comply with physical laws?

Figures (9)

• Figure 1: Examples of applications with high (a,b) and low (c) failure tolerance. The former typically involves tasks related to semantic understanding and generation, while the latter primarily includes tasks involving interaction with the physical world.
• Figure 2: Concepts represented at semantic level vs. physical level.
• Figure 3: Common categories of physical world concepts.
• Figure 4: Illustrations of the significance of analytic concept's benefits.
• Figure 5: The concept cuboid expresses the shared commonalities of the four different boxes and reflect how such commonalities vary across them through parameter $\theta=(l,w,h)$.