ColorCode: A Bayesian Approach to Augmentative and Alternative Communication with Two Buttons

TL;DR

ColorCode tackles the challenge of enabling communication with only two input buttons by framing AAC as Bayesian inference over the intended key. It combines a language-model-informed prior, a Beta-Bernoulli likelihood that learns click accuracy, an undo mechanism, and a color-assignment heuristic to maximize information gain and robustly correct errors. Through simulations, it achieves an average of $2.07$ clicks per character and a cross-entropy of $1.73$ bits, approaching the theoretical lower bound, and it demonstrates near-optimal error-correction performance when modeled as a binary symmetric channel. The work extends prior probabilistic AAC methods by keeping letters in static positions while reassigning colors, learning error rates online, and using entropy-based color assignments. Its open-source implementation and strong information-theoretic analysis suggest practical utility for individuals with severely limited muscle control.

Abstract

Many people with severely limited muscle control can only communicate through augmentative and alternative communication (AAC) systems with a small number of buttons. In this paper, we present the design for ColorCode, which is an AAC system with two buttons that uses Bayesian inference to determine what the user wishes to communicate. Our information-theoretic analysis of ColorCode simulations shows that it is efficient in extracting information from the user, even in the presence of errors, achieving nearly optimal error correction. ColorCode is provided as open source software (https://github.com/mrdaly/ColorCode).

Figures (3)

• Figure 1: ColorCode system after user has begun typing.
• Figure 2: Diagram demonstrating process of selecting the letter E in ColorCode.
• Figure 3: Plot comparing the information rate of ColorCode to the channel capacity of a binary symmetric channel.