CSQL: Mapping Documents into Causal Databases

TL;DR

Csql presents a causal knowledge discovery pipeline that converts unstructured document collections into a SQL-queryable causal database (CDB). By building on the Democritus discourse compiler, it induces a language-grounded relational schema where edges carry aggregated credibility and provenance across many local causal models, enabling deterministic, SQL-based causal analysis including backbone extraction, hub detection, and multi-step causal paths. The system supports provenance-aware auditing, cycles via strongly connected components, and even counterfactual reasoning through view rewrites, while remaining applicable to RAG-compiled corpora (e.g., TCC). This corpus-scale causal querying framework enables cross-domain analysis, discourse diagnostics, and integration with larger numerical causal inference pipelines, representing a novel causal compiler that operates over textual discourse rather than numerical data alone.

Abstract

We describe a novel system, CSQL, which automatically converts a collection of unstructured text documents into an SQL-queryable causal database (CDB). A CDB differs from a traditional DB: it is designed to answer "why'' questions via causal interventions and structured causal queries. CSQL builds on our earlier system, DEMOCRITUS, which converts documents into thousands of local causal models derived from causal discourse. Unlike RAG-based systems or knowledge-graph based approaches, CSQL supports causal analysis over document collections rather than purely associative retrieval. For example, given an article on the origins of human bipedal walking, CSQL enables queries such as: "What are the strongest causal influences on bipedalism?'' or "Which variables act as causal hubs with the largest downstream influence?'' Beyond single-document case studies, we show that CSQL can also ingest RAG/IE-compiled causal corpora at scale by compiling the Testing Causal Claims (TCC) dataset of economics papers into a causal database containing 265,656 claim instances spanning 45,319 papers, 44 years, and 1,575 reported method strings, thereby enabling corpus-level causal queries and longitudinal analyses in CSQL. Viewed abstractly, CSQL functions as a compiler from unstructured documents into a causal database equipped with a principled algebra of queries, and can be applied broadly across many domains ranging from business, humanities, and science.

Figures (4)

• Figure 1: An article in The Washington Post on the origins of human bipedal walking.
• Figure 2: Two causal models constructed from The Washington Post article on the origins of bipedal walking by human ancestors 7 million years ago. Democritus constructs thousands of such models automatically from the PDF document, which are then converted by Csql into a causal database.
• Figure 3: Csql in action (DuckDB). Example SQL queries over a Parquet-backed causal atlas (atlas_nodes.parquet, atlas_edges.parquet). (a) Extracts backbone causal relations by aggregated credibility mass. (b) Identifies causal hubs by total outgoing score mass. (c) Drills into a specific hub to recover its highest-scoring downstream influences. All results are deterministic given the compiled atlas tables.
• Figure 4: Csql counterfactual reasoning as SQL query rewriting. Left: baseline query retrieves the strongest outgoing causal generators from the hub bipedalism. Right: a hard intervention (do-cut) is implemented as a view that removes all edges whose source is bipedalism, revealing the next-strongest hubs and mechanisms in the atlas. All operations are deterministic transformations over the atlas tables.