Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Archiving Body Movements: Collective Generation of Chinese Calligraphy

Aven Le Zhou, Jiayi Ye, Tianchen Liu, Kang Zhang

TL;DR

This work addresses how to archive body movements by transforming real-time motion into Chinese calligraphy through an interactive generative system named Wushu. The approach combines motion capture, bilingual image-to-text captioning, and calligraphy-inspired layout algorithms to produce a growing compendium and digital archive of audience movements. Its key contributions include an end-to-end workflow from motion input to calligraphic output, a participatory installation that treats audiences as co-creators, and a scalable archive that invites discussion of Chinese characters and calligraphy. The work demonstrates a novel intersection of embodied interaction, AI-generated painterly content, and traditional calligraphic aesthetics with potential for public engagement and cultural discourse.

Abstract

As a communication channel, body movements have been widely explored in behavioral studies and kinesics. Performing and visual arts share the same interests but focus on documenting and representing human body movements, such as for dance notation and visual work creation. This paper investigates body movements in oriental calligraphy and how to apply calligraphy principles to stimulate and archive body movements. Through an artwork (Wushu), the authors experiment with an interactive and generative approach to engage the audience's bodily participation and archive the body movements as a compendium of generated calligraphy. The audience assumes the role of both writers and readers; creating ("writing") and appreciating ("reading") the generated calligraphy becomes a cyclical process within this infinite "Book," which can motivate further attention and discussions concerning Chinese characters and calligraphy.

Archiving Body Movements: Collective Generation of Chinese Calligraphy

TL;DR

This work addresses how to archive body movements by transforming real-time motion into Chinese calligraphy through an interactive generative system named Wushu. The approach combines motion capture, bilingual image-to-text captioning, and calligraphy-inspired layout algorithms to produce a growing compendium and digital archive of audience movements. Its key contributions include an end-to-end workflow from motion input to calligraphic output, a participatory installation that treats audiences as co-creators, and a scalable archive that invites discussion of Chinese characters and calligraphy. The work demonstrates a novel intersection of embodied interaction, AI-generated painterly content, and traditional calligraphic aesthetics with potential for public engagement and cultural discourse.

Abstract

As a communication channel, body movements have been widely explored in behavioral studies and kinesics. Performing and visual arts share the same interests but focus on documenting and representing human body movements, such as for dance notation and visual work creation. This paper investigates body movements in oriental calligraphy and how to apply calligraphy principles to stimulate and archive body movements. Through an artwork (Wushu), the authors experiment with an interactive and generative approach to engage the audience's bodily participation and archive the body movements as a compendium of generated calligraphy. The audience assumes the role of both writers and readers; creating ("writing") and appreciating ("reading") the generated calligraphy becomes a cyclical process within this infinite "Book," which can motivate further attention and discussions concerning Chinese characters and calligraphy.
Paper Structure (30 sections, 9 equations, 7 figures)

This paper contains 30 sections, 9 equations, 7 figures.

Table of Contents

  1. Introduction
  2. Contextualization
  3. Documenting Body Movement in Performing Arts
  4. Representating Body Movement in Visual Arts
  5. Body Movement in Oriental Calligraphy
  6. Method
  7. Participant vs. Viewer
  8. Essential Principles in Calligraphy
  9. Wushu: the Infinite Book
  10. Audience Engagement and Interaction
  11. Body Pose and Trajectory Estimation
  12. Ink Effects and Stroke Simulation
  13. Interacion and Input
  14. Participatory Generation of Calligraphy
  15. Motion Caption to Generate Characters
  16. ...and 15 more sections

Figures (7)

  • Figure 1: An Example of Zhang Xu's Calligraphy Entitled "Four Poems." © National Palace Museum (CC BY 4.0.)
  • Figure 2: Participant vs. Viewer: the Creative Process. © Authors.
  • Figure 3: Wushu Installed with a Curved Screen at Computational Media and Arts Lab, HKUST(Guangzhou). © Authors.
  • Figure 4: From Body Movements (Left) to Ink Effects (Middle) and Strokes (Right). © Authors.
  • Figure 5: Ink Diffusion Effects (on Left) and Stroke Simulation (on Right) in Wushu. © Authors.
  • ...and 2 more figures