FACTS: Table Summarization via Offline Template Generation with Agentic Workflows

TL;DR

FACTS introduces offline template generation for query-focused table summarization by coupling executable SQL queries with Jinja2 templates, all validated through an ensemble LLM Council. The framework yields reusable templates that generalize across tables with the same schema while keeping raw data private, enabling fast and scalable summaries without compromising accuracy. Empirical results on FeTaQA, QTSumm, and QFMTS show FACTS outperforming strong baselines across multiple metrics, with ablations confirming the value of iterative council-based validation. The approach advances practical, privacy-conscious, multi-table reasoning for real-world data analysis tasks.

Abstract

Query-focused table summarization requires generating natural language summaries of tabular data conditioned on a user query, enabling users to access insights beyond fact retrieval. Existing approaches face key limitations: table-to-text models require costly fine-tuning and struggle with complex reasoning, prompt-based LLM methods suffer from token-limit and efficiency issues while exposing sensitive data, and prior agentic pipelines often rely on decomposition, planning, or manual templates that lack robustness and scalability. To mitigate these issues, we introduce an agentic workflow, FACTS, a Fast, Accurate, and Privacy-Compliant Table Summarization approach via Offline Template Generation. FACTS produces offline templates, consisting of SQL queries and Jinja2 templates, which can be rendered into natural language summaries and are reusable across multiple tables sharing the same schema. It enables fast summarization through reusable offline templates, accurate outputs with executable SQL queries, and privacy compliance by sending only table schemas to LLMs. Evaluations on widely-used benchmarks show that FACTS consistently outperforms baseline methods, establishing it as a practical solution for real-world query-focused table summarization.

Figures (4)

• Figure 1: Comparison between DirectSumm zhang2024qfmtsgeneratingqueryfocusedsummaries (left) and our proposed FACTS framework (right). DirectSumm prompts a large language model (LLM) with the full table and query, which may produce hallucinated values, exposes all table records to external services, and requires regeneration for each new table even under the same schema and query. In contrast, FACTS generates a reusable offline template consisting of schema-aware SQL queries and a Jinja$2$ template. The SQL queries retrieve precise values through execution, while the Jinja$2$ template renders natural language summaries, ensuring accuracy, reusability, scalability, and privacy compliance.
• Figure 2: The FACTS framework for query-focused table summarization via Offline Template Generation. (1) Schema-Guided Specification and Filtering: the agent enriches the user query with guided questions and filtering rules over the table schema, with validation from the LLM Council. (2) SQL Queries Generation: using these specifications, the agent synthesizes and iteratively improves SQL queries through execution feedback and Council validation. (3) Jinja$2$ Template Generation and Alignment: a Jinja$2$ template verbalizes SQL outputs into natural language, with LLM Council checks ensuring alignment. The final output is a reusable offline template that combines validated SQL queries with a Jinja$2$ template.
• Figure 3: Reusability analysis. Runtime for generating summaries with $1$ versus $100$ tables under the same schema and query.
• Figure 4: Scalability analysis. Runtime for generating summaries as the number of rows in each table increases.