AACessTalk: Fostering Communication between Minimally Verbal Autistic Children and Parents with Contextual Guidance and Card Recommendation

TL;DR

AACessTalk addresses the challenge of fostering reciprocal communication between minimally verbal autistic children and their parents by deploying an AI-mediated tablet system that provides contextual parental guidance and child-oriented vocabulary cards. Built on formative expert/parent insights, the system uses LLM-driven pipelines to generate real-time parental prompts and context-relevant card decks, supporting balanced turn-taking. A two-week deployment with 11 parent–child dyads in Korea showed increased conversation frequency, longer interactions, and higher child engagement, while reducing parental pressure and enriching parent–child meaning-making. The work demonstrates the viability of AI-driven mediation to augment naturalistic parent–child interactions and outlines design considerations for sustaining long-term, neurodiversity-aware communication tools.

Abstract

As minimally verbal autistic (MVA) children communicate with parents through few words and nonverbal cues, parents often struggle to encourage their children to express subtle emotions and needs and to grasp their nuanced signals. We present AACessTalk, a tablet-based, AI-mediated communication system that facilitates meaningful exchanges between an MVA child and a parent. AACessTalk provides real-time guides to the parent to engage the child in conversation and, in turn, recommends contextual vocabulary cards to the child. Through a two-week deployment study with 11 MVA child-parent dyads, we examine how AACessTalk fosters everyday conversation practice and mutual engagement. Our findings show high engagement from all dyads, leading to increased frequency of conversation and turn-taking. AACessTalk also encouraged parents to explore their own interaction strategies and empowered the children to have more agency in communication. We discuss the implications of designing technologies for balanced communication dynamics in parent-MVA child interaction.

Figures (11)

• Figure 1: Photos taken from the parent interviews from formative study: (a) A parent writing down conversations they had with their child on the comic strip, and (b) parental guidance materials received from the therapy sessions and coated paper-based AAC cards used at home.
• Figure 2: Main screens and usage flow of AACessTalk: ① The home screen, where users select conversation topics. Selecting 'Plan' (Blue) or 'Recall' (Green) topics immediately begins the conversation, while 'Interest' (Orange) presents a pop-up ⓐ with the child’s pre-uploaded interests for further selection. ② The parent's turn screen, which provides ⓑ conversation guides and ⓒ example phrases for the parent. ③ The parent speaks to the child and then ④ presses the turn-pass button to end their turn. ⑤ The child's turn screen, which offers vocabulary cards for the child. ⑥ The child presses the turn-pass button to end their turn. ⑦ If a negative conversational pattern is detected in the parent’s previous turn, AACessTalk shows ⓓ feedback during the next parent turn ⑧. After several iterations of turn-taking like ②-⑥, either the parent or the child ends the conversation ⑨ via menu ⓔ.
• Figure 3: Generative pipeline for parental guidelines. The pipeline analyzes the current dialogue Ⓐ and generates parental guidelines with example messages Ⓗ.
• Figure 4: Generative pipeline for curating card decks for children. The pipeline analyzes the current dialogue and card information ⓐ and generates a deck of cards in four categories ⓘ along with symbol illustration retrieved from the KAAC database ⓖ.
• Figure 5: Parents and MVA child participants are engaging in conversation using the AACessTalk during the introductory session.