Table of Contents
Fetching ...
Paper
Video models are zero-shot learners and reasoners
TL;DR
This work investigates whether large-scale video models can function as general-purpose vision foundation models akin to LLMs in NLP. By prompting Veo 3 without task-specific fine-tuning, the authors demonstrate zero-shot capabilities across perception, intuitive physics, object manipulation, and temporal-spatial reasoning (cof-based). They analyze tens of thousands of generated videos across dozens of tasks, showing consistent improvements from Veo 2 to Veo 3 and highlighting both the promise and current limits of zero-shot visual reasoning. The findings suggest a path toward unified, generalist vision models, facilitated by prompting strategies and scaling, with important implications for future research and application. The work also discusses costs, benchmarking caveats, and the potential trajectory toward a GPT-3-like paradigm for computer vision.
Abstract
The remarkable zero-shot capabilities of Large Language Models (LLMs) have
propelled natural language processing from task-specific models to unified,
generalist foundation models. This transformation emerged from simple
primitives: large, generative models trained on web-scale data. Curiously, the
same primitives apply to today's generative video models. Could video models be
on a trajectory towards general-purpose vision understanding, much like LLMs
developed general-purpose language understanding? We demonstrate that Veo 3 can
solve a broad variety of tasks it wasn't explicitly trained for: segmenting
objects, detecting edges, editing images, understanding physical properties,
recognizing object affordances, simulating tool use, and more. These abilities
to perceive, model, and manipulate the visual world enable early forms of
visual reasoning like maze and symmetry solving. Veo's emergent zero-shot
capabilities indicate that video models are on a path to becoming unified,
generalist vision foundation models.