Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Meta Engine: A Unified Semantic Query Engine on Heterogeneous LLM-Based Query Systems

Ruyu Li, Tinghui Zhang, Haodi Ma, Daisy Zhe Wang, Yifan Wang

TL;DR

Meta Engine tackles fragmentation in LLM-based semantic querying by introducing a unified, modular pipeline that coordinates heterogeneous backends through adapters and a central NL interface. It decomposes complex multimodal queries into modality-aligned sub-queries, routes each to the most suitable specialized system, and aggregates intermediate results into a coherent final answer. The architecture centers on five components—Natural Language Query Parser, Operator Generator, Query Router, Adapters, and Result Aggregator—with optimization like complexity checking and iterative refinement. Extensive experiments on five multimodal benchmarks demonstrate significant improvements in end-to-end answer quality and semantic correctness over strong baselines, validating the efficacy of query decomposition, adaptive routing, and multi-adapter orchestration. Overall, Meta Engine provides a practical path toward high-performance multimodal semantic analytics by effectively leveraging diverse, specialized LLM-based query systems.

Abstract

With the increasingly use of multi-modal data, semantic query has become more and more demanded in data management systems, which is an important way to access and analyze multi-modal data. As unstructured data, most information of multi-modal data (text, image, video, etc) hides in the semantics, which cannot be accessed by the traditional database queries like SQL. Given the power of Large Language Model (LLM) in understanding semantics and processing natural language, in recent years several LLM-based semantic query systems have been proposed, to support semantic querying over unstructured data. However, this rapid growth has produced a fragmented ecosystem. Applications face significant integration challenges due to (1) disparate APIs of different semantic query systems and (2) a fundamental trade-off between specialization and generality. Many semantic query systems are highly specialized, offering state-of-the-art performance within a single modality but struggling with multi-modal data. Conversely, some "all-in-one" systems handle multiple modalities but often exhibit suboptimal performance compared to their specialized counterparts in specific modalities. This paper introduces Meta Engine, a novel "query system on query systems", designed to resolve those aforementioned challenges. Meta Engine is a unified semantic query engine that integrates heterogeneous, specialized LLM-based query systems. Its architecture comprises five key components: (1) a Natural Language (NL) Query Parser, (2) an Operator Generator, (3) a Query Router, (4) a set of Adapters, and (5) a Result Aggregator. In the evaluation, Meta Engine consistently outperforms all baselines, yielding 3-6x higher F1 in most cases and up to 24x on specific datasets.

Meta Engine: A Unified Semantic Query Engine on Heterogeneous LLM-Based Query Systems

TL;DR

Meta Engine tackles fragmentation in LLM-based semantic querying by introducing a unified, modular pipeline that coordinates heterogeneous backends through adapters and a central NL interface. It decomposes complex multimodal queries into modality-aligned sub-queries, routes each to the most suitable specialized system, and aggregates intermediate results into a coherent final answer. The architecture centers on five components—Natural Language Query Parser, Operator Generator, Query Router, Adapters, and Result Aggregator—with optimization like complexity checking and iterative refinement. Extensive experiments on five multimodal benchmarks demonstrate significant improvements in end-to-end answer quality and semantic correctness over strong baselines, validating the efficacy of query decomposition, adaptive routing, and multi-adapter orchestration. Overall, Meta Engine provides a practical path toward high-performance multimodal semantic analytics by effectively leveraging diverse, specialized LLM-based query systems.

Abstract

With the increasingly use of multi-modal data, semantic query has become more and more demanded in data management systems, which is an important way to access and analyze multi-modal data. As unstructured data, most information of multi-modal data (text, image, video, etc) hides in the semantics, which cannot be accessed by the traditional database queries like SQL. Given the power of Large Language Model (LLM) in understanding semantics and processing natural language, in recent years several LLM-based semantic query systems have been proposed, to support semantic querying over unstructured data. However, this rapid growth has produced a fragmented ecosystem. Applications face significant integration challenges due to (1) disparate APIs of different semantic query systems and (2) a fundamental trade-off between specialization and generality. Many semantic query systems are highly specialized, offering state-of-the-art performance within a single modality but struggling with multi-modal data. Conversely, some "all-in-one" systems handle multiple modalities but often exhibit suboptimal performance compared to their specialized counterparts in specific modalities. This paper introduces Meta Engine, a novel "query system on query systems", designed to resolve those aforementioned challenges. Meta Engine is a unified semantic query engine that integrates heterogeneous, specialized LLM-based query systems. Its architecture comprises five key components: (1) a Natural Language (NL) Query Parser, (2) an Operator Generator, (3) a Query Router, (4) a set of Adapters, and (5) a Result Aggregator. In the evaluation, Meta Engine consistently outperforms all baselines, yielding 3-6x higher F1 in most cases and up to 24x on specific datasets.
Paper Structure (32 sections, 7 equations, 7 figures, 6 tables)

This paper contains 32 sections, 7 equations, 7 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Architecture
  4. Architecture
  5. Workflow
  6. Operators
  7. TextAnalytics
  8. TableAnalytics
  9. ImageAnalytics
  10. Querying Stages and Optimization
  11. Stage 1: Query Decomposer
  12. Checker and Decomposer
  13. Decomposer Optimization
  14. Stage 2: Sub-query Routing and Execution
  15. Stage 3: Aggregator and Final Result
  16. ...and 17 more sections

Figures (7)

  • Figure 1: An End-to-End Multimodal Analytical Query Example in Meta Engine
  • Figure 2: Architecture and End-to-End Execution Pipeline of Meta Engine for Multimodal Queries
  • Figure 3: Confidence-based Operator Ranking, Routing, and Execution in Meta Engine
  • Figure 4: Prompt for Multimodal Query Decomposition
  • Figure 5: Example of Query Decomposition and operator generation in Meta Engine
  • ...and 2 more figures