Together or Apart? Investigating a mediator bot to aggregate bot's comments on pull requests
Eric Ribeiro, Ronan Nascimento, Igor Steinmacher, Laerte Xavier, Marco Gerosa, Hugo de Paula, Mairieli Wessel
TL;DR
The paper addresses information overload from GitHub bots during pull request reviews, especially affecting newcomers. It introduces FunnelBot, a mediator bot that aggregates and categorizes multiple bots' outputs into a single PR comment, and evaluates it against the traditional multi-bot approach through a within-subject study with 25 newcomers. Results indicate both approaches are generally perceived as useful and clear, but preferences vary about information quantity, with FunnelBot aiding quicker information location and reducing overload. The study provides empirical insights and design guidance for building bot-assisted onboarding tools in social coding platforms, highlighting the trade-offs between aggregation and contextual richness.
Abstract
Software bots connect users and tools, streamlining the pull request review process in social coding platforms. However, bots can introduce information overload into developers' communication. Information overload is especially problematic for newcomers, who are still exploring the project and may feel overwhelmed by the number of messages. Inspired by the literature of other domains, we designed and evaluated FunnelBot, a bot that acts as a mediator between developers and other bots in the repository. We conducted a within-subject study with 25 newcomers to capture their perceptions and preferences. Our results provide insights for bot developers who want to mitigate noise and create bots for supporting newcomers, laying a foundation for designing better bots.