Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Foundation Models in Augmentative and Alternative Communication: Opportunities and Challenges

Ambra Di Paola, Serena Muraro, Roberto Marinelli, Christian Pilato

TL;DR

The paper investigates how foundation models can enhance augmentative and alternative communication (AAC) by enabling scalable, personalized content generation. It introduces AMBRA, an open platform that blends cloud and edge computing with federated learning and generative AI to produce tailored messages and symbols for AAC users, including a content tokenization and symbol-generation pipeline. The authors discuss opportunities such as highly personalized content and open collaboration, alongside challenges like privacy, standardization, and symbol simplicity, proposing a roadmap that emphasizes openness and cross-institutional sharing. The work aims to democratize AAC, reduce educator workload, and foster inclusive communication through AI-enabled, context-aware content creation.

Abstract

Augmentative and Alternative Communication (AAC) are essential techniques that help people with communication disabilities. AAC demonstrates its transformative power by replacing spoken language with symbol sequences. However, to unlock its full potential, AAC materials must adhere to specific characteristics, placing the onus on educators to create custom-tailored materials and symbols. This paper introduces AMBRA (Pervasive and Personalized Augmentative and Alternative Communication based on Federated Learning and Generative AI), an open platform that aims to leverage the capabilities of foundation models to tackle many AAC issues, opening new opportunities (but also challenges) for AI-enhanced AAC. We thus present a compelling vision--a roadmap towards a more inclusive society. By leveraging the capabilities of modern technologies, we aspire to not only transform AAC but also guide the way toward a world where communication knows no bounds.

Foundation Models in Augmentative and Alternative Communication: Opportunities and Challenges

TL;DR

The paper investigates how foundation models can enhance augmentative and alternative communication (AAC) by enabling scalable, personalized content generation. It introduces AMBRA, an open platform that blends cloud and edge computing with federated learning and generative AI to produce tailored messages and symbols for AAC users, including a content tokenization and symbol-generation pipeline. The authors discuss opportunities such as highly personalized content and open collaboration, alongside challenges like privacy, standardization, and symbol simplicity, proposing a roadmap that emphasizes openness and cross-institutional sharing. The work aims to democratize AAC, reduce educator workload, and foster inclusive communication through AI-enabled, context-aware content creation.

Abstract

Augmentative and Alternative Communication (AAC) are essential techniques that help people with communication disabilities. AAC demonstrates its transformative power by replacing spoken language with symbol sequences. However, to unlock its full potential, AAC materials must adhere to specific characteristics, placing the onus on educators to create custom-tailored materials and symbols. This paper introduces AMBRA (Pervasive and Personalized Augmentative and Alternative Communication based on Federated Learning and Generative AI), an open platform that aims to leverage the capabilities of foundation models to tackle many AAC issues, opening new opportunities (but also challenges) for AI-enhanced AAC. We thus present a compelling vision--a roadmap towards a more inclusive society. By leveraging the capabilities of modern technologies, we aspire to not only transform AAC but also guide the way toward a world where communication knows no bounds.
Paper Structure (9 sections, 3 figures, 1 table)

This paper contains 9 sections, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. A Primer on Augmentative and Alternative Communication (AAC)
  3. AAC Challenges
  4. Foundation Models for Content Generation
  5. Text-to-Text Generation
  6. Text-to-Image Generation
  7. AMBRA: Using Foundation Models for Pervasive and Personalized AAC
  8. AMBRA Opportunities and Challenges
  9. Conclusions and Future Directions

Figures (3)

  • Figure 1: Example of AAC-based communication where a sequence of symbols replaces a message. Symbols are created with Symwriter symwriter
  • Figure 2: Examples of AAC symbols created with DALL·E dalle, a text-to-image, transformed-based model developed by OpenAI: a) glass of water; b) cup of coffee; c) simple cup of coffee; d) "open the sink" action; e) snack; f) simple snack.
  • Figure 3: Overview of our AMBRA open platform