Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Uniformly Rotated Mondrian Kernel

Calvin Osborne, Eliza O'Reilly

TL;DR

The paper tackles the limitation of axis-aligned Mondrian random features by introducing a uniformly rotated Mondrian kernel that induces rotation-invariance through a random $SO_d$ rotation prior to Mondrian tessellation. It derives a closed-form isotropic limiting kernel $k_\infty$ given by $k_\infty(x,x') = \frac{1}{\omega_d} \int_{S^{d-1}} e^{-\lambda \|x-x'\|_2 \|v\|_1} \mathrm{d}v$ and proves an exponential-type uniform convergence rate for the finite-feature kernel $k_M$ to this limit, $\mathbb{P}[\sup_{x,x'\in\mathcal{X}} |k_M - k_\infty| > \delta] = \mathcal{O}(M^{d + d/(2d+1)} e^{-M\delta^2/(4d+2)})$. The work also develops a geometric analysis of the typical cell in rotated Mondrian tessellations and introduces a lifting technique to bound circumradius, enabling precise constants in the rate bound. Empirically, the uniformly rotated kernel matches or surpasses the Mondrian kernel on datasets debiased from coordinate axes while maintaining favorable computational properties, and it converges toward isotropy at a rate comparable to other random tessellation-based features. Overall, the approach offers a practical, efficient path to isotropic kernel approximations via rotation before Mondrian partitioning, with potential extensions to isotropic Mondrian forests and data-driven directional distributions.

Abstract

Random feature maps are used to decrease the computational cost of kernel machines in large-scale problems. The Mondrian kernel is one such example of a fast random feature approximation of the Laplace kernel, generated by a computationally efficient hierarchical random partition of the input space known as the Mondrian process. In this work, we study a variation of this random feature map by applying a uniform random rotation to the input space before running the Mondrian process to approximate a kernel that is invariant under rotations. We obtain a closed-form expression for the isotropic kernel that is approximated, as well as a uniform convergence rate of the uniformly rotated Mondrian kernel to this limit. To this end, we utilize techniques from the theory of stationary random tessellations in stochastic geometry and prove a new result on the geometry of the typical cell of the superposition of uniformly rotated Mondrian tessellations. Finally, we test the empirical performance of this random feature map on both synthetic and real-world datasets, demonstrating its improved performance over the Mondrian kernel on a dataset that is debiased from the standard coordinate axes.

The Uniformly Rotated Mondrian Kernel

TL;DR

The paper tackles the limitation of axis-aligned Mondrian random features by introducing a uniformly rotated Mondrian kernel that induces rotation-invariance through a random rotation prior to Mondrian tessellation. It derives a closed-form isotropic limiting kernel given by and proves an exponential-type uniform convergence rate for the finite-feature kernel to this limit, . The work also develops a geometric analysis of the typical cell in rotated Mondrian tessellations and introduces a lifting technique to bound circumradius, enabling precise constants in the rate bound. Empirically, the uniformly rotated kernel matches or surpasses the Mondrian kernel on datasets debiased from coordinate axes while maintaining favorable computational properties, and it converges toward isotropy at a rate comparable to other random tessellation-based features. Overall, the approach offers a practical, efficient path to isotropic kernel approximations via rotation before Mondrian partitioning, with potential extensions to isotropic Mondrian forests and data-driven directional distributions.

Abstract

Random feature maps are used to decrease the computational cost of kernel machines in large-scale problems. The Mondrian kernel is one such example of a fast random feature approximation of the Laplace kernel, generated by a computationally efficient hierarchical random partition of the input space known as the Mondrian process. In this work, we study a variation of this random feature map by applying a uniform random rotation to the input space before running the Mondrian process to approximate a kernel that is invariant under rotations. We obtain a closed-form expression for the isotropic kernel that is approximated, as well as a uniform convergence rate of the uniformly rotated Mondrian kernel to this limit. To this end, we utilize techniques from the theory of stationary random tessellations in stochastic geometry and prove a new result on the geometry of the typical cell of the superposition of uniformly rotated Mondrian tessellations. Finally, we test the empirical performance of this random feature map on both synthetic and real-world datasets, demonstrating its improved performance over the Mondrian kernel on a dataset that is debiased from the standard coordinate axes.

Paper Structure

This paper contains 18 sections, 6 theorems, 71 equations, 4 figures.

Table of Contents

  1. INTRODUCTION
  2. Related Work
  3. BACKGROUND
  4. UNIFORMLY ROTATED MONDRIAN KERNEL
  5. MAIN RESULTS
  6. Limiting Uniformly Rotated Mondrian Kernel
  7. Uniform Convergence of Uniformly Rotated Mondrian Kernel
  8. CELLS OF UNIFORMLY ROTATED MONDRIAN TESSELLATIONS
  9. PROOFS OF MAIN RESULTS
  10. EXPERIMENTS
  11. Experimental Procedures
  12. Discussion of Results
  13. CONCLUSION
  14. APPENDIX
  15. Proof of Equation \ref{['e:vZ_bnds']}
  16. ...and 3 more sections

Key Result

Theorem 1

The limiting kernel of the uniformly rotated Mondrian process with lifetime $\lambda > 0$ in $\mathbb{R}^d$ is of the form where $\omega_d$ is the surface area of the unit ball in $\mathbb{R}^d$.

Figures (4)

  • Figure 1: Sample Mondrian Tessellations on $\mathcal{X} = [0, 1]^2$ with $\lambda = 1$.
  • Figure 2: Sample Isotropic Poisson Hyperplane Tessellation and Poisson Manhattan Tessellation on $\mathcal{X} = [0, 1]^2$ with $\lambda = 1$.
  • Figure 3: Limiting Kernel for the Uniformly Rotated Mondrian Process on $\mathbb{R}^2$ with lifetime $\lambda = 1$.
  • Figure 4: Five Experiments Evaluating the Uniformly Rotated Mondrian Kernel in Regression Problems.

Theorems & Definitions (6)

  • Theorem 1
  • Theorem 2
  • Theorem 3
  • Lemma 1
  • Proposition 1
  • Theorem 3