Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Bridging OLAP and RAG: A Multidimensional Approach to the Design of Corpus Partitioning

Dario Maio, Stefano Rizzi

TL;DR

This paper tackles the lack of principled corpus partitioning in large-scale Retrieval-Augmented Generation (RAG) systems. It proposes reframing multidimensional modeling from OLAP as a conceptual framework, the Dimensional Fact Model (DFM), to govern where retrieval occurs while embedding-based similarity operates within partitions. Key contributions include separating dimensional routing from semantic retrieval, enabling hierarchical routing and controlled fallbacks under incomplete metadata, and illustrating the approach with a large-scale legal-domain example. The work emphasizes governance, explainability, and adaptability over raw performance and advocates for open-source tools to support principled design of retrieval over massive unstructured corpora.

Abstract

Retrieval-Augmented Generation (RAG) systems are increasingly deployed on large-scale document collections, often comprising millions of documents and tens of millions of text chunks. In industrial-scale retrieval platforms, scalability is typically addressed through horizontal sharding and a combination of Approximate Nearest-Neighbor search, hybrid indexing, and optimized metadata filtering. Although effective from an efficiency perspective, these mechanisms rely on bottom-up, similarity-driven organization and lack a conceptual rationale for corpus partitioning. In this paper, we claim that the design of large-scale RAG systems may benefit from the combination of two orthogonal strategies: semantic clustering, which optimizes locality in embedding space, and multidimensional partitioning, which governs where retrieval should occur based on conceptual dimensions such as time and organizational context. Although such dimensions are already implicitly present in current systems, they are used in an ad hoc and poorly structured manner. We propose the Dimensional Fact Model (DFM) as a conceptual framework to guide the design of multidimensional partitions for RAG corpora. The DFM provides a principled way to reason about facts, dimensions, hierarchies, and granularity in retrieval-oriented settings. This framework naturally supports hierarchical routing and controlled fallback strategies, ensuring that retrieval remains robust even in the presence of incomplete metadata, while transforming the search process from a 'black-box' similarity matching into a governable and deterministic workflow. This work is intended as a position paper; its goal is to bridge the gap between OLAP-style multidimensional modeling and modern RAG architectures, and to stimulate further research on principled, explainable, and governable retrieval strategies at scale.

Bridging OLAP and RAG: A Multidimensional Approach to the Design of Corpus Partitioning

TL;DR

This paper tackles the lack of principled corpus partitioning in large-scale Retrieval-Augmented Generation (RAG) systems. It proposes reframing multidimensional modeling from OLAP as a conceptual framework, the Dimensional Fact Model (DFM), to govern where retrieval occurs while embedding-based similarity operates within partitions. Key contributions include separating dimensional routing from semantic retrieval, enabling hierarchical routing and controlled fallbacks under incomplete metadata, and illustrating the approach with a large-scale legal-domain example. The work emphasizes governance, explainability, and adaptability over raw performance and advocates for open-source tools to support principled design of retrieval over massive unstructured corpora.

Abstract

Retrieval-Augmented Generation (RAG) systems are increasingly deployed on large-scale document collections, often comprising millions of documents and tens of millions of text chunks. In industrial-scale retrieval platforms, scalability is typically addressed through horizontal sharding and a combination of Approximate Nearest-Neighbor search, hybrid indexing, and optimized metadata filtering. Although effective from an efficiency perspective, these mechanisms rely on bottom-up, similarity-driven organization and lack a conceptual rationale for corpus partitioning. In this paper, we claim that the design of large-scale RAG systems may benefit from the combination of two orthogonal strategies: semantic clustering, which optimizes locality in embedding space, and multidimensional partitioning, which governs where retrieval should occur based on conceptual dimensions such as time and organizational context. Although such dimensions are already implicitly present in current systems, they are used in an ad hoc and poorly structured manner. We propose the Dimensional Fact Model (DFM) as a conceptual framework to guide the design of multidimensional partitions for RAG corpora. The DFM provides a principled way to reason about facts, dimensions, hierarchies, and granularity in retrieval-oriented settings. This framework naturally supports hierarchical routing and controlled fallback strategies, ensuring that retrieval remains robust even in the presence of incomplete metadata, while transforming the search process from a 'black-box' similarity matching into a governable and deterministic workflow. This work is intended as a position paper; its goal is to bridge the gap between OLAP-style multidimensional modeling and modern RAG architectures, and to stimulate further research on principled, explainable, and governable retrieval strategies at scale.
Paper Structure (9 sections, 1 figure)

This paper contains 9 sections, 1 figure.

Table of Contents

  1. Introduction
  2. Related Work
  3. Conceptual Modeling and LLMs in OLAP
  4. Partitioning and Metadata in RAG
  5. RAG with Structured Data Sources
  6. Positioning of This Work
  7. Applying the DFM to Retrieval-Oriented Partitioning
  8. Illustrative Example
  9. Discussion and Implications

Figures (1)

  • Figure 1: DFM schema for judicial opinion chunks in the legal domain. The box represents the basic retrieval unit with its quantitative measures; dashed areas represent dimensions (e.g., Structure), while circles represent their hierarchical levels from the finest ones (those attached to the box, e.g., paragraph) to the coarsest ones (the leaves, e.g., section). Dashes on arcs model optional dimensions.