ColorGradedGaussians: Palette-Based Color Grading for 3D Gaussian Splatting via View-Space Sparse Decomposition

Abstract

Professional color editing requires precise control over both color (hue and saturation) and lightness, ideally through separate, independent controls. We present a real-time interactive color editing framework for 3D Gaussian Splatting (3DGS) that enables palette-based recoloring, per-palette tone curves for color-aware lightness adjustment, and accurate pixel-level constraints -- capabilities unavailable in prior palette-based 3DGS methods. Existing approaches decompose colors at the primitive level, optimizing per-Gaussian palette weights before splatting. However, sparse primitive-level weights do not guarantee sparse pixel-level decompositions after alpha-blending, causing palette edits to affect unintended regions and degrading editing quality. We address this through view-space palette decomposition, splatting weights instead of colors to optimize the observable appearance of the scene. We introduce a geometric loss using inverse barycentric coordinates to enforce consistent sparsity patterns, ensuring similar colors share similar decompositions. Our approach achieves superior editing quality compared to primitive-space methods, enabling professional color grading workflows for 3DGS scenes with real-time interaction.

Figures (7)

• Figure 1: Our method (top) enables independent lightness control per palette color on 3DGS via tone curves. We can independently add contrast to green (S-curve) and adjust purple's tonal range (black control points). PaletteGaussian PaletteGaussian's linear RGB mixing (bottom) lacks this capability. Palette color changes affect hue, saturation, and lightness simultaneously, preventing independent tonal control.
• Figure 2: Training Pipeline. Our method trains 3D Gaussians (left, red label) that splat palette weights $\widetilde{\mathbf{W}}$ and lightness $L$ in CIELAB color space, with the palette $\mathbf{P}$ (right, red label) optimized in ab-space only. The splatted weights are supervised by computing differentiable inverse barycentric coordinates from the input images with respect to the current palette state, yielding geometric target weights $\mathbf{W}_{\text{bary}}$. The geometric loss $\mathcal{L}_{\text{geo}}$ (Eq. \ref{['eq:geometric_loss']}) enforces consistency between $\widetilde{\mathbf{W}}$ and $\mathbf{W}_{\text{bary}}$, while the lightness loss $\mathcal{L}_{\text{lightness}}$ (Eq. \ref{['eq:lightness_loss']}) directly supervises the splatted lightness $L$ against ground truth $L_{\text{gt}}$. Red dashed arrows indicate gradient flow, with losses backpropagating to both the 3D Gaussian parameters and the palette $\mathbf{P}$. The final rendered image is obtained by multiplying the splatted weights with the palette and concatenating with the lightness channel to form complete Lab images. Additional regularization terms (not shown) include palette regularization (Sec. \ref{['sec:palette_reg']}) and sparsity regularization on the weights (Eq. \ref{['eq:sparsity_loss']}). (Fern from the LLFF dataset mildenhall2019local.)
• Figure 3: ColorGradedGaussians can edit a variety of scene in real time by placing pixel-level constraints on reference view. Datasets (top to bottom): Ignatius from Tanks&Temple knapitsch2017tanks; Materials from Blender mildenhall2021nerf; Fortress from LLFF mildenhall2019local.
• Figure 4: Effect of the geometric sparsity loss $\mathcal{L}_{\text{geo}}$. With $\mathcal{L}_{\text{geo}}$, palette weights are consistent and edits remain localized; without, inconsistent decompositions cause visible color bleeding (yellow arrows and FLIP andersson2020flip). (DrJohnson from Deep Blending hedman2018deep.)
• Figure 5: Grey penalization ablation. Without $\mathcal{L}_{\text{grey}}$, grey dominates the palette weights, resulting in an unrepresentative palette and reduced editability, as chromatic colors (pink and green) receive negligible weights. Enforcing $\mathcal{L}_{\text{grey}}$ yields a compact palette in which chromatic colors meaningfully control the scene, improving editability without sacrificing reconstruction quality (PSNR, SSIM, and LPIPS are shown inset). The palette plot is shown in ab-space with a consistent relative scale. (Caterpillar from Tanks&Temple knapitsch2017tanks.)