Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hardware-Friendly Input Expansion for Accelerating Function Approximation

Hu Lou, Yin-Jun Gao, Dong-Xiao Zhang, Tai-Jiao Du, Jun-Jie Zhang, Jia-Rui Zhang

TL;DR

Experimental results demonstrate that input-space expansion significantly accelerates training convergence and enhances approximation accuracy and enhances approximation accuracy and reducing final MSE by 66.3\% for the optimal 5D expansion.

Abstract

One-dimensional function approximation is a fundamental problem in scientific computing and engineering applications. While neural networks possess powerful universal approximation capabilities, their optimization process is often hindered by flat loss landscapes induced by parameter-space symmetries, leading to slow convergence and poor generalization, particularly for high-frequency components. Inspired by the principle of \emph{symmetry breaking} in physics, this paper proposes a hardware-friendly approach for function approximation through \emph{input-space expansion}. The core idea involves augmenting the original one-dimensional input (e.g., $x$) with constant values (e.g., $π$) to form a higher-dimensional vector (e.g., $[π, π, x, π, π]$), effectively breaking parameter symmetries without increasing the network's parameter count. We evaluate the method on ten representative one-dimensional functions, including smooth, discontinuous, high-frequency, and non-differentiable functions. Experimental results demonstrate that input-space expansion significantly accelerates training convergence (reducing LBFGS iterations by 12\% on average) and enhances approximation accuracy (reducing final MSE by 66.3\% for the optimal 5D expansion). Ablation studies further reveal the effects of different expansion dimensions and constant selections, with $π$ consistently outperforming other constants. Our work proposes a low-cost, efficient, and hardware-friendly technique for algorithm design.

Hardware-Friendly Input Expansion for Accelerating Function Approximation

TL;DR

Experimental results demonstrate that input-space expansion significantly accelerates training convergence and enhances approximation accuracy and enhances approximation accuracy and reducing final MSE by 66.3\% for the optimal 5D expansion.

Abstract

One-dimensional function approximation is a fundamental problem in scientific computing and engineering applications. While neural networks possess powerful universal approximation capabilities, their optimization process is often hindered by flat loss landscapes induced by parameter-space symmetries, leading to slow convergence and poor generalization, particularly for high-frequency components. Inspired by the principle of \emph{symmetry breaking} in physics, this paper proposes a hardware-friendly approach for function approximation through \emph{input-space expansion}. The core idea involves augmenting the original one-dimensional input (e.g., ) with constant values (e.g., ) to form a higher-dimensional vector (e.g., ), effectively breaking parameter symmetries without increasing the network's parameter count. We evaluate the method on ten representative one-dimensional functions, including smooth, discontinuous, high-frequency, and non-differentiable functions. Experimental results demonstrate that input-space expansion significantly accelerates training convergence (reducing LBFGS iterations by 12\% on average) and enhances approximation accuracy (reducing final MSE by 66.3\% for the optimal 5D expansion). Ablation studies further reveal the effects of different expansion dimensions and constant selections, with consistently outperforming other constants. Our work proposes a low-cost, efficient, and hardware-friendly technique for algorithm design.
Paper Structure (27 sections, 6 equations, 4 figures, 4 tables)

This paper contains 27 sections, 6 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Neural Networks for Function Approximation
  4. Symmetry Breaking in Optimization
  5. Hardware-Friendly Optimization Strategies
  6. Methodology
  7. Theoretical Foundation of Symmetry Breaking
  8. Input Expansion Implementation
  9. Network Architecture and Training Details
  10. Experiments and Results
  11. Experimental Design
  12. Function Benchmark with Mathematical Characterization
  13. Controlled Experimental Setup
  14. Evaluation Framework
  15. Results and Analysis
  16. ...and 12 more sections

Figures (4)

  • Figure 1: Comprehensive ablation study of symmetry breaking with expanded input dimensionality across 10 benchmark functions. Functions 1-5: Multi-frequency sine, Square wave, Sawtooth wave, Triangle wave, and Modulated sine.
  • Figure 2: Comprehensive ablation study of symmetry breaking with expanded input dimensionality across 10 benchmark functions. Functions 6-10: Frequency chirp, Duty-cycle square wave, Van der Pol oscillator, Weierstrass function, and Comb function.
  • Figure 3: Experiments on ablation using different constant values in 5-dimensional extension. Functions 1-5: Multi-frequency sine, Square wave, Sawtooth wave, Triangle wave, and Modulated sine.
  • Figure 4: Experiments on ablation using different constant values in 5-dimensional extension. Functions 6-10: Frequency chirp, Duty-cycle square wave, Van der Pol oscillator, Weierstrass function, and Comb function.