FreeChunker: A Cross-Granularity Chunking Framework

TL;DR

FreeChunker presents a cross-granularity encoding framework that treats sentences as atomic units and enables retrieval of arbitrary sentence combinations, moving beyond fixed-granularity chunking. By introducing a multi-pattern Chunk Pattern Mask and a cross-granularity encoder, it generates multiple chunk embeddings in a single forward pass while reusing sentence embeddings, reducing chunking overhead. Theoretical analysis provides worst-case and expected substitution bounds for the cross-granularity representation, and training with multi-granularity signals demonstrates stable convergence and good generalization. Empirical evaluation on LongBench V2 shows competitive retrieval performance with efficiency gains over traditional and some semantic chunking approaches, highlighting the practical benefits for diverse query intents in RAG pipelines.

Abstract

Chunking strategies significantly impact the effectiveness of Retrieval-Augmented Generation (RAG) systems. Existing methods operate within fixed-granularity paradigms that rely on static boundary identification, limiting their adaptability to diverse query requirements. This paper presents FreeChunker, a Cross-Granularity Encoding Framework that fundamentally transforms the traditional chunking paradigm: the framework treats sentences as atomic units and shifts from static chunk segmentation to flexible retrieval supporting arbitrary sentence combinations. This paradigm shift not only significantly reduces the computational overhead required for semantic boundary detection but also enhances adaptability to complex queries. Experimental evaluation on LongBench V2 demonstrates that FreeChunker achieves superior retrieval performance compared to traditional chunking methods, while significantly outperforming existing approaches in computational efficiency.

Figures (3)

• Figure 1: Cross-Granularity ChunkingFreeChunker breaks the limitation of fixed-granularity text chunking, enabling flexible extraction of sentence combinations at arbitrary granularities.
• Figure 2: The architecture of FreeChunker. Long text input is processed by the Sentenizer and Embedding Model to produce sentence embeddings. The Attention Matrix and Chunk Pattern Mask work together to generate multiple granularity chunk embeddings in a single forward pass, enabling flexible on-demand access to different chunk sizes.
• Figure 3: Training Dynamics Across Model Scales. Training and validation curves (top) and test set loss distributions (bottom) for three different model configurations, demonstrating consistent convergence and effective generalization.