Neural Poisson Solver: A Universal and Continuous Framework for Natural Signal Blending

TL;DR

Implicit neural representations (INRs) provide a continuous mapping from coordinates to signal values but pose challenges for gradient‑domain blending due to discretization and boundary handling. The authors propose Neural Poisson Solver, a boundary‑condition‑free, plug‑and‑play framework that blends INRs by optimizing a Poisson equation via a gradient‑guided neural solver, using a gradient fidelity loss $L_{grad}$ and a color‑consistency loss $L_{color}$ balanced by $\lambda$, operating directly on continuous INRs. The method supports signals across dimensions, including 2D images and 5D radiance fields, leveraging Disorder‑Invariant Implicit Neural Representation (DINER) for high‑frequency detail. Experimental results report Poisson PDE error reductions by $1/10$ to $1/1000$ relative to classical solvers and natural, artifact‑free blending in both 2D and NeRF‑based data, with memory considerations for high‑dimensional tasks. This work provides a versatile, priors‑free tool for INR‑based editing with broad implications for computational photography and neural rendering.

Abstract

Implicit Neural Representation (INR) has become a popular method for representing visual signals (e.g., 2D images and 3D scenes), demonstrating promising results in various downstream applications. Given its potential as a medium for visual signals, exploring the development of a neural blending method that utilizes INRs is a natural progression. Neural blending involves merging two INRs to create a new INR that encapsulates information from both original representations. A direct approach involves applying traditional image editing methods to the INR rendering process. However, this method often results in blending distortions, artifacts, and color shifts, primarily due to the discretization of the underlying pixel grid and the introduction of boundary conditions for solving variational problems. To tackle this issue, we introduce the Neural Poisson Solver, a plug-and-play and universally applicable framework across different signal dimensions for blending visual signals represented by INRs. Our Neural Poisson Solver offers a variational problem-solving approach based on the continuous Poisson equation, demonstrating exceptional performance across various domains. Specifically, we propose a gradient-guided neural solver to represent the solution process of the variational problem, refining the target signal to achieve natural blending results. We also develop a Poisson equation-based loss and optimization scheme to train our solver, ensuring it effectively blends the input INR scenes while preserving their inherent structure and semantic content. The lack of dependence on additional prior knowledge makes our method easily adaptable to various task categories, highlighting its versatility. Comprehensive experimental results validate the robustness of our approach across multiple dimensions and blending tasks.

Figures (8)

• Figure 1: Overview. Our method processes various dimensions of INRs effectively, without relying on prior knowledge. It utilizes a blending INR to carry the combined signal and introduces a Neural Poisson Solver for solving the Poisson equation during training and optimization of the blending outcome. In the Neural Poisson Solver, $\mathcal{L}_{\text{grad}}$ leverages the continuous nature of INR to provide gradients with infinite resolution, enhancing smoothness by integrating gradients from multiple directions, thereby minimizing the jaggedness of the blending result. Moreover, our approach obviates the need for traditional boundary conditions required in solving variational problems, employing $\mathcal{L}_{\text{color}}$ to broaden the receptive field of the blending area for a more seamless blending outcome. Finally, we introduce a hyperparameter $\lambda$ to fine-tune the balance between $\mathcal{L}_{\text{grad}}$ and $\mathcal{L}_{\text{color}}$, facilitating the achievement of varied and natural blending styles.
• Figure 2: In PIE perez2023poisson, the selection of mask shapes significantly influences the final blending outcomes, particularly regarding texture and color details. Masks that are too closely positioned can result in color bleeding artifacts within the synthesized image areas. In contrast, our method, which does not rely on boundary conditions, is not impacted by the proximity of mask shapes.
• Figure 3: Neural Blending Operator. When $\lambda<1$, the blending operation accentuates the gradient within the blending region $\Omega$, thereby more effectively preserving the intricate details of the scene. However, this approach may lead to a noticeable color discrepancy between the blending edge $\partial \Omega$ and the background INR $\mathcal{S}$. On the other hand, as $\lambda$ increases, the focus shifts towards ensuring a smooth transition at $\partial \Omega$. While this method facilitates a seamless blend, it might slightly diminish the precision of the scene's detailed features.
• Figure 4: Displaying the blending results of the PIE method perez2023poisson and our approach across different 2D scenes. The first and third columns show the source and target scenes for two tasks, respectively. The second and fourth columns respectively showcase the blending outcomes and related details.
• Figure 5: Our method in Radiance Fields blending. The first column shows the original scenes. The second column employs a common replacement blending approach, directly substituting the region of interest $\Omega$ in $\mathcal{S}_{\Theta}$ with $c_{\mathcal{G}}(\mathbf{x})$, rendered based on $\alpha_{\mathcal{G}}(\mathbf{x})$. Columns three to five demonstrate the blending results achieved with our method, showcasing the naturalness and consistency of the blend from different perspectives.