Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Predictive Authoring for Brazilian Portuguese Augmentative and Alternative Communication

Jayr Pereira, Rodrigo Nogueira, Cleber Zanchettin, Robson Fidalgo

TL;DR

This work tackles pictogram prediction for Brazilian Portuguese AAC by building a dedicated AAC corpus and finetuning the Brazilian Portuguese BERT (BERTimbau) with a pictogram-based vocabulary. It systematically compares representations of pictograms—captions, synonyms, definitions, and images—finding that caption- and synonym-based embeddings perform similarly, with captions delivering the best accuracy, while image embeddings offer no clear advantage. The study provides practical guidelines for corpus construction, model fine-tuning, and pictogram representation in AAC systems, and demonstrates that a synthetic, GPT-3-augmented corpus can closely match human-derived data in distribution and coverage. Limitations include lack of real-user evaluation and potential biases in synthetic data, suggesting future work to validate with AAC users and extend to other languages such as Portuguese varieties using dedicated models like Sabiá.

Abstract

Individuals with complex communication needs (CCN) often rely on augmentative and alternative communication (AAC) systems to have conversations and communique their wants. Such systems allow message authoring by arranging pictograms in sequence. However, the difficulty of finding the desired item to complete a sentence can increase as the user's vocabulary increases. This paper proposes using BERTimbau, a Brazilian Portuguese version of BERT, for pictogram prediction in AAC systems. To finetune BERTimbau, we constructed an AAC corpus for Brazilian Portuguese to use as a training corpus. We tested different approaches to representing a pictogram for prediction: as a word (using pictogram captions), as a concept (using a dictionary definition), and as a set of synonyms (using related terms). We also evaluated the usage of images for pictogram prediction. The results demonstrate that using embeddings computed from the pictograms' caption, synonyms, or definitions have a similar performance. Using synonyms leads to lower perplexity, but using captions leads to the highest accuracies. This paper provides insight into how to represent a pictogram for prediction using a BERT-like model and the potential of using images for pictogram prediction.

Predictive Authoring for Brazilian Portuguese Augmentative and Alternative Communication

TL;DR

This work tackles pictogram prediction for Brazilian Portuguese AAC by building a dedicated AAC corpus and finetuning the Brazilian Portuguese BERT (BERTimbau) with a pictogram-based vocabulary. It systematically compares representations of pictograms—captions, synonyms, definitions, and images—finding that caption- and synonym-based embeddings perform similarly, with captions delivering the best accuracy, while image embeddings offer no clear advantage. The study provides practical guidelines for corpus construction, model fine-tuning, and pictogram representation in AAC systems, and demonstrates that a synthetic, GPT-3-augmented corpus can closely match human-derived data in distribution and coverage. Limitations include lack of real-user evaluation and potential biases in synthetic data, suggesting future work to validate with AAC users and extend to other languages such as Portuguese varieties using dedicated models like Sabiá.

Abstract

Individuals with complex communication needs (CCN) often rely on augmentative and alternative communication (AAC) systems to have conversations and communique their wants. Such systems allow message authoring by arranging pictograms in sequence. However, the difficulty of finding the desired item to complete a sentence can increase as the user's vocabulary increases. This paper proposes using BERTimbau, a Brazilian Portuguese version of BERT, for pictogram prediction in AAC systems. To finetune BERTimbau, we constructed an AAC corpus for Brazilian Portuguese to use as a training corpus. We tested different approaches to representing a pictogram for prediction: as a word (using pictogram captions), as a concept (using a dictionary definition), and as a set of synonyms (using related terms). We also evaluated the usage of images for pictogram prediction. The results demonstrate that using embeddings computed from the pictograms' caption, synonyms, or definitions have a similar performance. Using synonyms leads to lower perplexity, but using captions leads to the highest accuracies. This paper provides insight into how to represent a pictogram for prediction using a BERT-like model and the potential of using images for pictogram prediction.
Paper Structure (20 sections, 8 equations, 11 figures, 2 tables)

This paper contains 20 sections, 8 equations, 11 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Language Modeling
  4. Word Embeddings
  5. Transformers
  6. Evaluating Language Models
  7. Pictogram Prediction in AAC
  8. Method
  9. Inputs
  10. Corpus Construction
  11. Collection of Sentences
  12. Data Augmentation
  13. Data Cleaning
  14. Text to Pictogram
  15. The constructed corpus
  16. ...and 5 more sections

Figures (11)

  • Figure 1: Example of a high-tech AAC system using communication cards with ARASAAC pictograms. The screenshot depicts the interface of the reaact.com.br tool, where the user can easily select communication cards from the content grid (large bottom rectangle) and arrange them sequentially to construct sentences (e.g., cat wants). Additional functionalities are accessible through the buttons located in the right sidebar, enabling utilities such as text-to-speech functionality provided by the voice synthesizer.
  • Figure 2: Flow chart for model construction.
  • Figure 3: Sentence collection participants summary.
  • Figure 4: GPT-3 text prompt for sentence generation based on examples of human-composed sentences.
  • Figure 5: GPT-3 text prompt based on a controlled vocabulary for sentence generation.
  • ...and 6 more figures