AssistedDS: Benchmarking How External Domain Knowledge Assists LLMs in Automated Data Science

TL;DR

This work introduces AssistedDS, a benchmark to quantify how LLMs leverage external domain knowledge in automated data science for tabular tasks. It combines synthetic datasets with ground-truth generative mechanisms and real Kaggle datasets, each paired with helpful and adversarial knowledge bundles to test critical evaluation. Empirical results show current LLMs often blindly adopt external hints, with adversarial or misleading content causing substantial degradation in predictive performance, and real-world Kaggle tasks revealing persistent failures in time-series handling, feature alignment, and non-numeric data processing. The study advocates developing knowledge-aware, reasoning-based automation for data science and provides public data and code for the community.

Abstract

Large language models (LLMs) have advanced the automation of data science workflows. Yet it remains unclear whether they can critically leverage external domain knowledge as human data scientists do in practice. To answer this question, we introduce AssistedDS (Assisted Data Science), a benchmark designed to systematically evaluate how LLMs handle domain knowledge in tabular prediction tasks. AssistedDS features both synthetic datasets with explicitly known generative mechanisms and real-world Kaggle competitions, each accompanied by curated bundles of helpful and adversarial documents. These documents provide domain-specific insights into data cleaning, feature engineering, and model selection. We assess state-of-the-art LLMs on their ability to discern and apply beneficial versus harmful domain knowledge, evaluating submission validity, information recall, and predictive performance. Our results demonstrate three key findings: (1) LLMs frequently exhibit an uncritical adoption of provided information, significantly impairing their predictive performance when adversarial content is introduced, (2) helpful guidance is often insufficient to counteract the negative influence of adversarial information, and (3) in Kaggle datasets, LLMs often make errors in handling time-series data, applying consistent feature engineering across different folds, and interpreting categorical variables correctly. These findings highlight a substantial gap in current models' ability to critically evaluate and leverage expert knowledge, underscoring an essential research direction for developing more robust, knowledge-aware automated data science systems. Our data and code are publicly available here: https://github.com/jeremyxianx/Assisted-DS

Figures (8)

• Figure 1: How AssistedDS establishes its unique position. Top: A human data scientist analyses data, and often critically filters external domain knowledge. This means accepting helpful hints (green) and rejecting adversarial ones (red), which leads to demonstrably better results. Bottom: An LLM can automate the same analytical steps, but its ability to judge the quality of domain knowledge is unknown. AssistedDS evaluates whether an LLM, like its human counterpart, can identify and leverage helpful information while resisting harmful advice in automated data science workflow.
• Figure 2: Overview of the AssistedDS Benchmark. Left: Dataset. We use datasets (synthetic or real-world) with defined features (e.g., Location, Decoration, House Age, School Quality) and clear train/test splits for tabular prediction tasks. Center-left: Domain Knowledge. For each dataset, we curate bundles of domain knowledge: helpful hints (e.g., “Add School Quality”) and adversarial hints (e.g., “Remove Location”), reflecting realistic expert guidance and potential pitfalls. Center: Task Bundles. Hints are grouped into tasks: Helpful-Only, Adversarial-Only, Mixed, and so on. Each represents different real-world information environments. Center-right: LLM Execution. An LLM receives both the data and bundled hints, together with the prompt, and generates a end-to-end code script. Right: Evaluation. We assess each model’s generated code by (i) valid submission rate, (ii) predictive performance (Macro-F1, RMSE, etc), and (iii) the rates at which helpful or adversarial hints are actually adopted in the generated code.
• Figure 3: Performance Change of LLMs and Human (expert) with Different Tasks in Synthetic Datasets and Kaggle Datasets . This chart displays the percentage change in model performance relative to the baseline condition (no hint provided). Such change is calculated so that positive value indicates gain in performance. The four tasks represent different bundles of helpful and adversarial hints: Helpful-only (only helpful hints provided), Adversarial-only (only adversarial hints provided), Mixed (both helpful and adversarial hints are provided), and Mixed (Misleading) (both helpful and adversarial hints are provided, with intentionally misleading prompts). Each marker is the performance change averaged over settings per task and model. Error bars show $1.96 \times$ standard errors. Notably, for both synthetic and Kaggle datasets, only human baseline persists to have no decline of performance.
• Figure 4: Example python code for generating Song Popularity dataset
• Figure 5: Helpful hints for Song Popularity