ICE: Interactive 3D Game Character Editing via Dialogue

TL;DR

ICE introduces a multi-round, dialogue-based framework for interactive 3D game character editing that overcomes single-round limitations of prior methods. It combines an Instruction Parsing Module (IPM) that maps dialogue to actionable prompts with a Semantic-guided Low-dimension Parameter Solver (SLPS) that optimizes character parameters in a reduced space using a differentiable rendering imitator and CLIP guidance. A character attribute memory bank supports robust, continuous refinement across rounds, while a Fine-grained Parameter Editing module localizes edits to relevant channels for precise changes. Experimental results demonstrate improved semantic consistency, faster interactive feedback, and strong robustness across ablations, with practical integration into existing game pipelines. The work enables iterative, artist-friendly character customization and sets a path for faster, constraint-aware, in-game character editing.

Abstract

ost recent popular Role-Playing Games (RPGs) allow players to create in-game characters with hundreds of adjustable parameters, including bone positions and various makeup options. Although text-driven auto-customization systems have been developed to simplify the complex process of adjusting these intricate character parameters, they are limited by their single-round generation and lack the capability for further editing and fine-tuning. In this paper, we propose an Interactive Character Editing framework (ICE) to achieve a multi-round dialogue-based refinement process. In a nutshell, our ICE offers a more user-friendly way to enable players to convey creative ideas iteratively while ensuring that created characters align with the expectations of players. Specifically, we propose an Instruction Parsing Module (IPM) that utilizes large language models (LLMs) to parse multi-round dialogues into clear editing instruction prompts in each round. To reliably and swiftly modify character control parameters at a fine-grained level, we propose a Semantic-guided Low-dimension Parameter Solver (SLPS) that edits character control parameters according to prompts in a zero-shot manner. Our SLPS first localizes the character control parameters related to the fine-grained modification, and then optimizes the corresponding parameters in a low-dimension space to avoid unrealistic results. Extensive experimental results demonstrate the effectiveness of our proposed ICE for in-game character creation and the superior editing performance of ICE.

Figures (12)

• Figure 1: An example of the process of our ICE is shown on the left: A character, expected as a "secret agent", is created initially and then sequentially refined in a fine-grained and interactive manner, according to the editing instructions provided by users and suggestions of the system. Screenshots of the final generated character driven with various animations and expressions in the game are shown on the right.
• Figure 2: Inference Framework. User input is first parsed by IPM as actionable editing text prompt, editing strength, and feedback text. Sequentially, SLPS localizes the character control parameters related to the specified fine-grained modification, and then optimizes them in a low-dimension space in a differentiable manner. Finally, the parameters are applied to the game engine to render the edited character.
• Figure 3: Illustration of editing strength iteratively refining. The character attribute memory bank enables IPM to accurately understand multi-round dialogue and precisely control the editing intensity.
• Figure 4: Comparison of our method with state-of-the-art in the single-round creation. In the traditional single-round creation task, our method generates more high-quality results, avoiding abnormal faces, while maintaining strong semantic consistency.
• Figure 5: Visualization of final character under long text description. Some error attributes are labeled in red. When confronted with extensive text prompts, our full ICE method further maintains fidelity to each detail.