Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Trading Vector Data in Vector Databases

Jin Cheng, Xiangxiang Dai, Ningning Ding, John C. S. Lui, Jianwei Huang

TL;DR

This work addresses vector data trading in vector databases by framing it as an online learning problem where retrieval configurations and pricing must be learned jointly under uncertain costs and stochastic buyer feedback. It introduces a two-stage hierarchical bandit framework (VTHB) with Stage I (CCB) for adaptive retrieval configuration learning and Stage II (CPB with LAB) for pricing strategy learning, and provides regret guarantees: logarithmic for retrieval and sublinear for pricing, all with polynomial-time complexity. The approach is validated on four real-world datasets, showing significant improvements in cumulative reward and reductions in regret compared to baselines. The framework is generalizable to various indexing methods and supports cross-domain data trading scenarios, offering practical impact for scalable, cost-aware vector data marketplaces.

Abstract

Vector data trading is essential for cross-domain learning with vector databases, yet it remains largely unexplored. We study this problem under online learning, where sellers face uncertain retrieval costs and buyers provide stochastic feedback to posted prices. Three main challenges arise: (1) heterogeneous and partial feedback in configuration learning, (2) variable and complex feedback in pricing learning, and (3) inherent coupling between configuration and pricing decisions. We propose a hierarchical bandit framework that jointly optimizes retrieval configurations and pricing. Stage I employs contextual clustering with confidence-based exploration to learn effective configurations with logarithmic regret. Stage II adopts interval-based price selection with local Taylor approximation to estimate buyer responses and achieve sublinear regret. We establish theoretical guarantees with polynomial time complexity and validate the framework on four real-world datasets, demonstrating consistent improvements in cumulative reward and regret reduction compared with existing methods.

Trading Vector Data in Vector Databases

TL;DR

This work addresses vector data trading in vector databases by framing it as an online learning problem where retrieval configurations and pricing must be learned jointly under uncertain costs and stochastic buyer feedback. It introduces a two-stage hierarchical bandit framework (VTHB) with Stage I (CCB) for adaptive retrieval configuration learning and Stage II (CPB with LAB) for pricing strategy learning, and provides regret guarantees: logarithmic for retrieval and sublinear for pricing, all with polynomial-time complexity. The approach is validated on four real-world datasets, showing significant improvements in cumulative reward and reductions in regret compared to baselines. The framework is generalizable to various indexing methods and supports cross-domain data trading scenarios, offering practical impact for scalable, cost-aware vector data marketplaces.

Abstract

Vector data trading is essential for cross-domain learning with vector databases, yet it remains largely unexplored. We study this problem under online learning, where sellers face uncertain retrieval costs and buyers provide stochastic feedback to posted prices. Three main challenges arise: (1) heterogeneous and partial feedback in configuration learning, (2) variable and complex feedback in pricing learning, and (3) inherent coupling between configuration and pricing decisions. We propose a hierarchical bandit framework that jointly optimizes retrieval configurations and pricing. Stage I employs contextual clustering with confidence-based exploration to learn effective configurations with logarithmic regret. Stage II adopts interval-based price selection with local Taylor approximation to estimate buyer responses and achieve sublinear regret. We establish theoretical guarantees with polynomial time complexity and validate the framework on four real-world datasets, demonstrating consistent improvements in cumulative reward and regret reduction compared with existing methods.

Paper Structure

This paper contains 48 sections, 6 theorems, 27 equations, 9 figures, 2 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Background and Motivation
  3. Key Contributions
  4. Related Works
  5. Data Trading Methods
  6. Vector Database Optimization
  7. Multi-armed Bandits Frameworks
  8. System Model
  9. Vector Queries
  10. Vector Indexing
  11. HNSW
  12. IVF
  13. Extensibility to Other Indexing Methods
  14. Online Vector Data Trading
  15. Data Buyer Modeling
  16. ...and 33 more sections

Key Result

Theorem 1

Suppose the configuration setting satisfies the gap-dominant reward structure, the expected cumulative configuration regret $R^{c}_{\varsigma}(T_{\varsigma})$ of Algorithm 2 over $T_{\varsigma}$ rounds is bounded by:

Figures (9)

  • Figure 1: Similarity-based retrieval from vector databases.
  • Figure 2: Vector-based Data trading system.
  • Figure 3: Framework Overview.
  • Figure 4: Illustration of the clustering function $h(\cdot)$.
  • Figure 5: Comparison of methods on cumulative regret.
  • ...and 4 more figures

Theorems & Definitions (9)

  • Definition 1: Approximate Nearest Neighbor (ANN) Query
  • Remark 1
  • Definition 2: Gap-Dominant Reward Structure
  • Theorem 1: Regret Bound of CCB under Gap-Dominant Reward Structure
  • Theorem 2: Regret Bound of CCB under General Structure
  • Lemma 1: Approximation Error Bound
  • Lemma 2: Regret Bound of LAB
  • Theorem 3: Regret Bound of CPB
  • Theorem 4: Overall Time Complexity of VTHB