Towards Automated Scoping of AI for Social Good Projects
Jacob Emmerson, Rayid Ghani, Zheyuan Ryan Shi
TL;DR
The paper addresses the bottleneck of problem scoping in AI for Social Good by introducing the Problem Scoping Agent (PSA), a retrieval-augmented LLM pipeline that grounds proposals in literature and real-world knowledge. PSA decomposes scoping into background, challenges, and literature-enabled solution generation, with explicit verbalized confidence to improve transparency. In experiments on 21 DSSG project summaries, PSA-generated proposals were broadly comparable to expert-authored ones and outperformed vanilla LLM baselines, while also revealing challenges in objective maintenance and generation diversity. The work highlights practical avenues for future work, including human-in-the-loop collaboration and addressing the subjectivity of scoping in social contexts.
Abstract
Artificial Intelligence for Social Good (AI4SG) is an emerging effort that aims to address complex societal challenges with the powerful capabilities of AI systems. These challenges range from local issues with transit networks to global wildlife preservation. However, regardless of scale, a critical bottleneck for many AI4SG initiatives is the laborious process of problem scoping -- a complex and resource-intensive task -- due to a scarcity of professionals with both technical and domain expertise. Given the remarkable applications of large language models (LLM), we propose a Problem Scoping Agent (PSA) that uses an LLM to generate comprehensive project proposals grounded in scientific literature and real-world knowledge. We demonstrate that our PSA framework generates proposals comparable to those written by experts through a blind review and AI evaluations. Finally, we document the challenges of real-world problem scoping and note several areas for future work.