Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning Instance-Specific Parameters of Black-Box Models Using Differentiable Surrogates

Arnisha Khondaker, Nilanjan Ray

TL;DR

This work is able to learn input-specific parameters for a black box in this work for the first time, and demonstrates a significant increase in PSNR and a notable improvement in SSIM nearing 0.93.

Abstract

Tuning parameters of a non-differentiable or black-box compute is challenging. Existing methods rely mostly on random sampling or grid sampling from the parameter space. Further, with all the current methods, it is not possible to supply any input specific parameters to the black-box. To the best of our knowledge, for the first time, we are able to learn input-specific parameters for a black box in this work. As a test application, we choose a popular image denoising method BM3D as our black-box compute. Then, we use a differentiable surrogate model (a neural network) to approximate the black-box behaviour. Next, another neural network is used in an end-to-end fashion to learn input instance-specific parameters for the black-box. Motivated by prior advances in surrogate-based optimization, we applied our method to the Smartphone Image Denoising Dataset (SIDD) and the Color Berkeley Segmentation Dataset (CBSD68) for image denoising. The results are compelling, demonstrating a significant increase in PSNR and a notable improvement in SSIM nearing 0.93. Experimental results underscore the effectiveness of our approach in achieving substantial improvements in both model performance and optimization efficiency. For code and implementation details, please refer to our GitHub repository: https://github.com/arnisha-k/instance-specific-param

Learning Instance-Specific Parameters of Black-Box Models Using Differentiable Surrogates

TL;DR

This work is able to learn input-specific parameters for a black box in this work for the first time, and demonstrates a significant increase in PSNR and a notable improvement in SSIM nearing 0.93.

Abstract

Tuning parameters of a non-differentiable or black-box compute is challenging. Existing methods rely mostly on random sampling or grid sampling from the parameter space. Further, with all the current methods, it is not possible to supply any input specific parameters to the black-box. To the best of our knowledge, for the first time, we are able to learn input-specific parameters for a black box in this work. As a test application, we choose a popular image denoising method BM3D as our black-box compute. Then, we use a differentiable surrogate model (a neural network) to approximate the black-box behaviour. Next, another neural network is used in an end-to-end fashion to learn input instance-specific parameters for the black-box. Motivated by prior advances in surrogate-based optimization, we applied our method to the Smartphone Image Denoising Dataset (SIDD) and the Color Berkeley Segmentation Dataset (CBSD68) for image denoising. The results are compelling, demonstrating a significant increase in PSNR and a notable improvement in SSIM nearing 0.93. Experimental results underscore the effectiveness of our approach in achieving substantial improvements in both model performance and optimization efficiency. For code and implementation details, please refer to our GitHub repository: https://github.com/arnisha-k/instance-specific-param
Paper Structure (19 sections, 5 equations, 9 figures, 4 tables, 3 algorithms)

This paper contains 19 sections, 5 equations, 9 figures, 4 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Background
  3. Black-Box Approximation
  4. Black-Box Parameter Optimization
  5. Modeling the Black-Box System
  6. Training an Approximator Model to Learn the Black-Box Function
  7. Proposed Method
  8. Dynamic Optimization with Differentiable Surrogate
  9. Learning Parameters of Black-Box
  10. Experimental Setup and Results
  11. Dataset
  12. Evaluation Metrics
  13. Black Box: BM3D
  14. Algoritm 1: Black-Box Optimization
  15. Algoritm 2: Dynamic Black-Box Optimization
  16. ...and 4 more sections

Figures (9)

  • Figure 1: Illustration of instance-specific parameter learning in BM3D denoising application (Algorithm \ref{['algo3']}). The surrogate model $f_{\text{am}}$ takes the noisy image and parameters as input channels (\ref{['fig:proxy_unetr']}). The loss $\omega$ is updated using the model output and BM3D output. The parameter learner $f_{\text{pl}}$ outputs a parameter layer, which in the case of BM3D is $H \times W \times 5$. This layer is averaged across each channel and quantized back into the parameter space to obtain the BM3D output with updated parameters. The input to the parameter learner is noisy images, and $\psi$ is updated using the approximator output and ground truth images.
  • Figure 2: Each sample of data comprises three key elements: (a) the initial unprocessed noisy image, (b) the image processed using the BM3D algorithm with specified parameters, and (c) the corresponding ground truth image with low noise.
  • Figure 3: Architecture of the Surrogate Model($f_{\text{am}}$), UNETR. Output sizes are given for patch resolution = 16, filters = [32 , 64, 128, 256], attention heads = 12 and embedding size = 78. Parameter layer is concatenated with input as well as hidden layers 3, 6 and 9.
  • Figure 4: Illustration of the losses during Algorithm \ref{['algo1']} (a) shows the surrogate approximator loss, whereas (b) shows the parameter optimization loss over epochs.
  • Figure 5: Illustration of the losses during Algorithm \ref{['algo2']} (a) shows the surrogate approximator loss, whereas (b) shows the parameter optimization loss over epochs.
  • ...and 4 more figures