LLMs Meet Isolation Kernel: Lightweight, Learning-free Binary Embeddings for Fast Retrieval

TL;DR

IKE is an ensemble of diverse (random) partitions, enabling robust estimation of ideal kernel in the LLM embedding space, thus reducing retrieval accuracy loss as the ensemble grows, and offers low memory footprint and fast bitwise computation, lowering retrieval latency.

Abstract

Large language models (LLMs) have recently enabled remarkable progress in text representation. However, their embeddings are typically high-dimensional, leading to substantial storage and retrieval overhead. Although recent approaches such as Matryoshka Representation Learning (MRL) and Contrastive Sparse Representation (CSR) alleviate these issues to some extent, they still suffer from retrieval accuracy degradation. This paper proposes \emph{Isolation Kernel Embedding} or IKE, a learning-free method that transforms an LLM embedding into a binary embedding using Isolation Kernel (IK). IKE is an ensemble of diverse (random) partitions, enabling robust estimation of ideal kernel in the LLM embedding space, thus reducing retrieval accuracy loss as the ensemble grows. Lightweight and based on binary encoding, it offers low memory footprint and fast bitwise computation, lowering retrieval latency. Experiments on multiple text retrieval datasets demonstrate that IKE offers up to 16.7x faster retrieval and 16x lower memory usage than LLM embeddings, while maintaining comparable or better accuracy. Compared to CSR and other compression methods, IKE consistently achieves the best balance between retrieval efficiency and effectiveness.

Figures (10)

• Figure 1: MRL learns adaptive embedding length by optimizing different downstream applications. CSR combines recontruction loss and constrastive loss for learning sparse representation. In contrast, IKE maps points into binary embeddings for efficient retrieval through random partitions without sophisticated learning.
• Figure 2: Effect of parameter $m$ in IKE$_{VD}$ ($t$=4096) on nDCG@10 across two LLM embedding datasets. The pentagram marker indicates the performance of IKE$_{VD}$ when $m=d$, which is equivalent to the VDeH method.
• Figure 3: A illustration of partitioning in 2D space ($\psi=4$) by different IK implementations: iForest and VD. Blue points indicate the samples used to construct the partition, and the green point represents input data to be transformed.
• Figure 4: QPS vs. MRR@10 for Four ANN Methods on the HotpotQA dataset. The curves are obtained by adjusting the ANN search parameters. The detailed configuration is described in Appendix \ref{['app:exp:llm_emb:config']}.
• Figure 5: Comparison of Retrieval Accuracy and Search Time between CSR and IKE in the exhaustive search setting on the HotpotQA (LLM2Vec) dataset.

Theorems & Definitions (14)

• Proposition 1: Full Space Coverage
• proof
• Proposition 2: Entropy Maximization
• proof
• Proposition 3: Bit Independence
• proof
• Theorem 1
• proof
• Definition 1: Base Partitioner
• Definition 2: Ideal Kernel