Semantic Operators: A Declarative Model for Rich, AI-based Data Processing

TL;DR

Semantic operators extend relational-style querying to general AI-based transformations parameterized by natural language, enabling robust bulk analytics with accuracy guarantees. The LOTUS system implements this declarative model, providing gold algorithms and optimized plans that achieve large speedups (up to 1,000x) while preserving quality relative to the gold standard. The paper defines formal operator definitions, optimization techniques for semantic filters, joins, group-by, and top-k, and demonstrates effectiveness across fact-checking, biomedical multi-label classification, search, and topic analysis. These results present a practical path toward scalable, reliable AI-driven data processing over vast knowledge corpora.

Abstract

The semantic capabilities of large language models (LLMs) have the potential to enable rich analytics and reasoning over vast knowledge corpora. Unfortunately, existing systems either empirically optimize expensive LLM-powered operations with no performance guarantees, or serve a limited set of row-wise LLM operations, providing limited robustness, expressiveness and usability. We introduce semantic operators, the first formalism for declarative and general-purpose AI-based transformations based on natural language specifications (e.g., filtering, sorting, joining or aggregating records using natural language criteria). Each operator opens a rich space for execution plans, similar to relational operators. Our model specifies the expected behavior of each operator with a high-quality gold algorithm, and we develop an optimization framework that reduces cost, while providing accuracy guarantees with respect to a gold algorithm. Using this approach, we propose several novel optimizations to accelerate semantic filtering, joining, group-by and top-k operations by up to $1,000\times$. We implement semantic operators in the LOTUS system and demonstrate LOTUS' effectiveness on real, bulk-semantic processing applications, including fact-checking, biomedical multi-label classification, search, and topic analysis. We show that the semantic operator model is expressive, capturing state-of-the-art AI pipelines in a few operator calls, and making it easy to express new pipelines that match or exceed quality of recent LLM-based analytic systems by up to $170\%$, while offering accuracy guarantees. Overall, LOTUS programs match or exceed the accuracy of state-of-the-art AI pipelines for each task while running up to $3.6\times$ faster than the highest-quality baselines. LOTUS is publicly available at https://github.com/lotus-data/lotus.

Figures (9)

• Figure 1: Execution time (y-axis) and accuracy, shown in parentheses, for 3 fact-checking implementations: (i) FacTool chern_factool_2023, a recent state-of-the-art research work, (ii) a short, un-optimized LOTUS program, and (iii) the same LOTUS program optimized with accuracy guarantees. Section \ref{['sec:eval']} provides our full methodology.
• Figure 2: An example semantic operator query is shown on the left. The user function takes two two datasets and uses a series of semantic operators to to create a summary of relevant research papers from papers_df based on the user's research areas from projects_df. Given a user's query, LOTUS defines a logical plan, which it then converts to an un-optimized gold plan, that specifies how to orchestrate the LM over the data. LOTUS then generates an optimized execution plan with statistical accuracy guarantees relative to the gold plan. These optimized plans empirically demonstrate up to $1,000\times$ speedups. Finally, LOTUS executes the optimized plan and returns the results to the user.
• Figure 3: Example sem_join for matching papers and datasets.
• Figure 4: Example usage of sem_join and sem_sim_join.
• Figure 5: Example usage of sem_topk and sem_search.