Preliminary Explorations with GPT-4o(mni) Native Image Generation

TL;DR

This work provides a qualitative, taxonomy-driven evaluation of GPT-4o’s native image generation across traditional and multimodal tasks. It demonstrates strong general synthesis, grounding in many text- and multimodal-driven tasks, and notable capabilities in style transfer and context-aware generation, while revealing persistent gaps in precise spatial alignment, fine-grained control, and domain-specific factual accuracy. The study highlights that GPT-4o is not yet a world model, showing limited physical, temporal, and causal grounding, as well as weaknesses in instruction-grounding and multi-modal confinement. These findings point to essential directions for future work, including incorporating structured spatial priors, memory or grounding modules, and task-specific alignment to enable reliable, domain-critical generation. Together, the results illuminate both the substantial progress and the remaining hurdles toward robust, controllable, and knowledge-grounded multimodal generation systems.

Abstract

Recently, the visual generation ability by GPT-4o(mni) has been unlocked by OpenAI. It demonstrates a very remarkable generation capability with excellent multimodal condition understanding and varied task instructions. In this paper, we aim to explore the capabilities of GPT-4o across various tasks. Inspired by previous study, we constructed a task taxonomy along with a carefully curated set of test samples to conduct a comprehensive qualitative test. Benefiting from GPT-4o's powerful multimodal comprehension, its image-generation process demonstrates abilities surpassing those of traditional image-generation tasks. Thus, regarding the dimensions of model capabilities, we evaluate its performance across six task categories: traditional image generation tasks, discriminative tasks, knowledge-based generation, commonsense-based generation, spatially-aware image generation, and temporally-aware image generation. These tasks not only assess the quality and conditional alignment of the model's outputs but also probe deeper into GPT-4o's understanding of real-world concepts. Our results reveal that GPT-4o performs impressively well in general-purpose synthesis tasks, showing strong capabilities in text-to-image generation, visual stylization, and low-level image processing. However, significant limitations remain in its ability to perform precise spatial reasoning, instruction-grounded generation, and consistent temporal prediction. Furthermore, when faced with knowledge-intensive or domain-specific scenarios, such as scientific illustrations or mathematical plots, the model often exhibits hallucinations, factual errors, or structural inconsistencies. These findings suggest that while GPT-4o marks a substantial advancement in unified multimodal generation, there is still a long way to go before it can be reliably applied to professional or safety-critical domains.

Figures (150)

• Figure 1: Despite resolution-specific prompts, GPT-4o consistently outputs images at $1024\times 1024$. In cases of extremely high-resolution requests (i.e., $16384\times 16384$), the model refuses to generate any result, indicating a limitation in resolution controllability.
• Figure 2: GPT-4o is unable to strictly follow aspect ratio prompts and instead selects from a limited set of resolutions. For instance, it outputs $1024\times 1024$ for a 4:3 prompt, and $1536\times 1024$ for wider ratios such as 2:1 or 3:1, approximating the requested aspect ratios using the closest available resolution.
• Figure 3:
• Figure 4: Examples of text-to-image generation results by GPT-4o.
• Figure 5: Examples of abstract text-to-image generation results by GPT-4o.