AgentAda: Skill-Adaptive Data Analytics for Tailored Insight Discovery

TL;DR

AgentAda presents a skill-informed data analytics framework that dynamically retrieves and applies a library of 74 analytic skills to generate executable pipelines and goal-aligned insights. By coupling a Hybrid RAG skill matcher with end-to-end code generation and multimodal answer extraction, it delivers deeper, more actionable analytics than prior LLM-based tools. The KaggleBench benchmark (700 notebooks across 49 domains and 28 tasks) and the SCORER evaluation framework enable realistic, scalable assessment of insight quality and alignment with human judgments. Empirical results show superior depth, coherence, factual grounding, and accuracy across baselines, with strong generalization across models and datasets, highlighting the importance of structured skill grounding in end-to-end data-to-insight pipelines.

Abstract

We introduce AgentAda, the first LLM-powered analytics agent that can learn and use new analytics skills to extract more specialized insights. Unlike existing methods that require users to manually decide which data analytics method to apply, AgentAda automatically identifies the skill needed from a library of analytical skills to perform the analysis. This also allows AgentAda to use skills that existing LLMs cannot perform out of the box. The library covers a range of methods, including clustering, predictive modeling, and NLP techniques like BERT, which allow AgentAda to handle complex analytics tasks based on what the user needs. AgentAda's dataset-to-insight extraction strategy consists of three key steps: (I) a question generator to generate queries relevant to the user's goal and persona, (II) a hybrid Retrieval-Augmented Generation (RAG)-based skill matcher to choose the best data analytics skill from the skill library, and (III) a code generator that produces executable code based on the retrieved skill's documentation to extract key patterns. We also introduce KaggleBench, a benchmark of curated notebooks across diverse domains, to evaluate AgentAda's performance. We conducted a human evaluation demonstrating that AgentAda provides more insightful analytics than existing tools, with 48.78% of evaluators preferring its analyses, compared to 27.67% for the unskilled agent. We also propose a novel LLM-as-a-judge approach that we show is aligned with human evaluation as a way to automate insight quality evaluation at larger scale.

Figures (16)

• Figure 1: Unlike other data analytics agents, AgentAda breaks down tasks into detailed, skill-specific questions aligned with the user's goal and persona, delivering deep, insightful, and factual analysis.
• Figure 2: AgentAda's pipeline for automated insights. It first generates diverse questions from the data, then uses a RAG-based skill matcher to select relevant tools. The code generator executes the analysis, answers are derived from plots and outputs, and final insights are extracted from answers which includes statistics and visualizations.
• Figure 3: The validation loss steadily decreases during prompt optimization, indicating improved alignment between SCORER's evaluation scores and human judgments.
• Figure 4: The distribution of domains covered by KaggleBench
• Figure 5: The distribution of tasks covered by KaggleBench.