Colin: A Multimodal Human-AI Co-Creation Storytelling System To Support Children's Multi-Level Narrative Skills

TL;DR

Colin tackles the challenge of developing children's multi-level narrative skills by merging voice, drawing, and AI-generated content into a cohesive co-creation workflow. The system enables facilitator preview, adaptive question scaffolding, and a building phase that translates drawing into extended narratives, evaluated through formative design work and a 20-pair user study. Findings show improvements in higher-order narrative understanding and engagement, with nuanced effects across individual differences and stakeholder perspectives. The work offers a practical, multimodal approach and a foundation for personalized, AI-assisted narrative learning in both home and classroom settings.

Abstract

Children develop narrative skills by understanding and actively building connections between elements, image text matching, and consequences. However, it is challenging for children to clearly grasp these multi level links only through explanations of text or the facilitator's speech. To address this, we developed Colin, an interactive storytelling tool that supports children's multi level narrative skills through both voice and visual modalities. In the generation stage, Colin supports the facilitator to define and review the generated text and image content freely. In the understanding stage, a question feedback model helps children understand multi level connections while co creating stories with Colin. In the building phase, Colin actively encourages children to create connections between elements through drawing and speaking. A user study with 20 participants evaluated Colin by measuring children's engagement, understanding of cause and effect relationships, and the quality of their new story creations. Our results demonstrate that Colin significantly enhances the development of children's narrative skills across multiple levels.

Figures (9)

• Figure 1: Overview of the AI-assisted storytelling co-creation process. (A) Customize Library: Parents set educational goals and keywords. (B) Understanding Stage: Children engage through speaking and drawing to interpret the story. (C) Building Stage: Children reuse created elements to build new stories expressing the same core value.
• Figure 2: The overall workflow of Colin, an AI-assisted storytelling system that leverages GPT-4 to co-create stories with children, guide understanding of key story values, and support knowledge transfer.
• Figure 3: Dynamic prompt strategy for AI-assisted storytelling. The system adapts story generation and questioning to children’s responses while maintaining core educational goals.
• Figure 4: Comparison of human and GPT-4 performance in content and question generation. GPT-4 outperforms humans across most dimensions, including logical rationality, attraction, and inspiration.
• Figure 5: Three-level framework for evaluating children's narrative skills, including knowledge, key idea understanding, and transfer ability.