Can AI Prompt Humans? Multimodal Agents Prompt Players' Game Actions and Show Consequences to Raise Sustainability Awareness

TL;DR

EcoEcho investigates whether AI-driven multimodal agents can prompt players to take actions that reveal visible environmental consequences, thereby raising sustainability awareness. The authors design a narrative-driven, GenAI-powered game where in-game dialogue turns into actions via an intent-detection pipeline, and outcomes are shown through dynamic environmental feedback. In a mixed-methods study with 23 participants, the game significantly increased intended sustainable behaviors post-play, while attitudes toward sustainability changed only modestly, suggesting behavioral shifts may precede attitudinal change. The work demonstrates how combining prompt engineering, intent-to-action mapping, and action-consequence mechanics can motivate real-world sustainable behaviors and offers design implications for scalable AI-assisted serious games and related interventions.

Abstract

Unsustainable behaviors are challenging to prevent due to their long-term, often unclear consequences. Games offer a promising solution by creating artificial environments where players can immediately experience the outcomes of their actions. To explore this potential, we developed EcoEcho, a GenAI-powered game leveraging multimodal agents to raise sustainability awareness. These agents engage players in natural conversations, prompting them to take in-game actions that lead to visible environmental impacts. We evaluated EcoEcho using a mixed-methods approach with 23 participants. Results show a significant increase in intended sustainable behaviors post-game, although attitudes towards sustainability only slightly improved. This finding highlights the potential of multimodal agents and action-consequence mechanics to effectively motivate real-world behavioral changes such as raising environmental sustainability awareness.

Figures (11)

• Figure 1: Sequential visualization of the game's narrative structure across seven key stages: (1) game initiation, (2) opening sequence introducing the protagonist's world, (3) NPC interactions, (4) time travel mechanics, (5) player assessment phases, (6) critical decision point, and (7) branching endings (Alternative or Bad) based on player choices regarding environmental sustainability.
• Figure 2: Overview of NPC Roles and Task Sequence Hierarchy: Protagonist Ki (a young scientist responsible for advancing the main storyline), Narrator Kane (an astronaut scientist and Ki's father, providing story background and key emotional elements), Evaluator Emilia (a sustainability scientist and functional NPC, assessing player behavior and attitudes), Journalist Lisa (a reporter, advancing the story after completing tasks, Level 1), Security Guard (guiding players to the next storyline after completing specific tasks, Level 2), Union Leader Bob (driving key plot tasks, Level 3), and Minister of Energy Johnathan (the highest-level trigger NPC, final executor of key tasks, Level 4).
• Figure 3: Parallel views of three potential futures: (A) a world transformed by clean energy adoption, (B) a landscape dominated by traditional energy infrastructure, and (C) an environment rendered uninhabitable by environmental degradation.
• Figure 4: Architectural overview of a three-stage prompt engineering framework for role-playing dialogues: initial character profile compilation, prompt synthesis and structuring, and large language model (Llama-70B) dialogue generation, illustrated through an investigative journalist character scenario.
• Figure 5: The AI-driven dialogue system achieves intelligent, dynamic NPC-player interactions through multi-stage processing. Initially, the AI-Driven NPC initiates dialogue based on pre-defined NPC character profiles. After player input, an intent detection agent analyzes user intentions, incorporating conversation context stored in a database. The system then evaluates the detected intent in a core decision phase, selecting the most appropriate response strategy based on specific circumstances: (1 Generating a new response, (2) Returning a predefined answer, or (3) Providing a specific in-game item.