Integrating Machine-Generated Short Descriptions into the Wikipedia Android App: A Pilot Deployment of Descartes
Marija Šakota, Dmitry Brant, Cooltey Feng, Shay Nowick, Amal Ramadan, Robin Schoenbaechler, Joseph Seddon, Jazmin Tanner, Isaac Johnson, Robert West
TL;DR
The paper addresses uneven coverage of Wikipedia short descriptions across languages and tests a Descartes-based generator integrated into the Wikimedia Android editing workflow. It reports a two-step pilot with editors and community graders across 12 languages, showing that 90% of accepted machine-generated descriptions are rated at least 3/5 and comparable to human-written text. The study analyzes acceptance rates, edits, reverts, and retention, finding that more experienced editors perform better and that the first beam yields higher-quality results, while latency and guardrails shape deployment. The results suggest that Descartes can meaningfully reduce content gaps when deployed with careful UI design, safeguards, and ongoing evaluation.
Abstract
Short descriptions are a key part of the Wikipedia user experience, but their coverage remains uneven across languages and topics. In previous work, we introduced Descartes, a multilingual model for generating short descriptions. In this report, we present the results of a pilot deployment of Descartes in the Wikipedia Android app, where editors were offered suggestions based on outputs from Descartes while editing short descriptions. The experiment spanned 12 languages, with over 3,900 articles and 375 editors participating. Overall, 90% of accepted Descartes descriptions were rated at least 3 out of 5 in quality, and their average ratings were comparable to human-written ones. Editors adopted machine suggestions both directly and with modifications, while the rate of reverts and reports remained low. The pilot also revealed practical considerations for deployment, including latency, language-specific gaps, and the need for safeguards around sensitive topics. These results indicate that Descartes's short descriptions can support editors in reducing content gaps, provided that technical, design, and community guardrails are in place.