Simulating Errors in Touchscreen Typing

TL;DR

Typoist tackles the problem of understanding touchscreen typing errors by modeling cognitive mechanisms that generate slips, lapses, and mistakes, and by incorporating a hierarchical supervisory-control framework within a POMDP. It extends prior models by integrating perception and memory noise, enabling joint generation and correction of a broad set of errors, and trains the policy via RL (PPO) with two-loop parameter optimization. The TypingError benchmark demonstrates that Typoist reproduces realistic error distributions across diverse user groups and conditions, often outperforming the previous state-of-the-art CRTypist in key metrics and providing human-like error-correction behavior. The approach offers practical utility for keyboard evaluation, user research, and synthetic data generation, while acknowledging limitations in autocorrection dynamics and more advanced ITE features, pointing to future work on gesture input and word prediction.

Abstract

Empirical evidence shows that typing on touchscreen devices is prone to errors and that correcting them poses a major detriment to users' performance. Design of text entry systems that better serve users, across their broad capability range, necessitates understanding the cognitive mechanisms that underpin these errors. However, prior models of typing cover only motor slips. The paper reports on extending the scope of computational modeling of typing to cover the cognitive mechanisms behind the three main types of error: slips (inaccurate execution), lapses (forgetting), and mistakes (incorrect knowledge). Given a phrase, a keyboard, and user parameters, Typoist simulates eye and finger movements while making human-like insertion, omission, substitution, and transposition errors. Its main technical contribution is the formulation of a supervisory control problem wherein the controller allocates cognitive resources to detect and fix errors generated by the various mechanisms. The model generates predictions of typing performance that can inform design, for better text entry systems.

Figures (5)

• Figure 1: We introduce the first model covering a wide spectrum of errors known to be commonplace in typing, Typoist, which simulates the way users move their eyes and fingers when they type. The figure illustrates the three main types of error covered by the model. a) Slip: accidentally double-tapping while typing rapidly, which is detected through proofreading and corrected by backspacing; b) Lapse: forgetting where the finger was. c) Mistake: missing a typo and believing it is correct.
• Figure 2: a) An information-processing view of human error wickens2021engineering assumes that a typing error can be produced by any step in a sequence of three: interpretation, intention, and execution. Slips are incorrectly executed movements, lapses are incorrect commands, and mistakes emerge when misinterpretation of the typed text leads to inappropriate decisions about what to do. b) Typoist extends the architecture that underpins CRTypist. With Typoist, the system models the cognitive processes that generate errors. Moreover, the supervisory controller can observe the consequences of errors shi2024crtypist.
• Figure 3: A simulation example involving multiple mechanisms that generate various text errors and corrections. In typing of "welcome to chi" with the Gboard interface, the following errors occur: 1) the model initially forgets to type the letter "l" (an omission error) though then quickly correcting it; 2) it accidentally types "e" instead of "w" (making a substitution error) although it corrects this mistake as well; 3) and, at the end of the sentence, it makes an insertion error by double tapping "i" -- with the model failing to detect this and submitting the text as-is.
• Figure 4: Visualization tool for exploring simulations. a) Via the settings panel, users can choose a target phrase for typing with the specified keyboard layout and adjust error parameters. b) The behavior analysis panel displays simulated gaze and finger movement to demonstrate the human error-linked behavior. To simulate different scenarios, the user can adjust parameters that affect the error-generating mechanisms in the model.
• Figure 5: Typing speed vs. error corrections. The figure shows the speed--accuracy tradeoff in both human data and the predictions.