Inverting Non-Injective Functions with Twin Neural Network Regression

TL;DR

A deterministic framework that finds input parameters to a given target variable of non-injective functions that are a) defined by data or b) known as mathematical formula is proposed.

Abstract

Non-injective functions are not invertible. However, non-injective functions can be restricted to sub-domains on which they are locally injective and surjective and thus invertible if the dimensionality between input and output spaces are the same. Further, even if the dimensionalities do not match it is often possible to choose a preferred solution from many possible solutions. Twin neural network regression is naturally capable of incorporating these properties to invert non-injective functions. Twin neural network regression is trained to predict adjustments to well known input variables $\mathbf{x}^{\text{anchor}}$ to obtain an estimate for an unknown $\mathbf{x}^{\text{new}}$ under a change of the target variable from $\mathbf{y}^{\text{anchor}}$ to $\mathbf{y}^{\text{new}}$. In combination with k-nearest neighbor search, I propose a deterministic framework that finds input parameters to a given target variable of non-injective functions. The method is demonstrated by inverting non-injective functions describing toy problems and robot arm control that are a) defined by data or b) known as mathematical formula.

Figures (10)

• Figure 1: Reformulation of a regression problem: A traditional solution to a regression problem consists of finding an approximation to the function that maps a data point $x$ to its target value $f(\mathbf{x})=y$. Twinned regression methods solve the problem of mapping a pair of inputs $x_1$ and $x_2$ to the difference between the target values $F(x_2,x_1)=y_2-y_1$. The resulting function can then be employed as an estimator for the original regression problem $y_2=F(x_2,x_1)+y_1$ given a labelled anchor point $(x_1,y_1)$. Twinned regression methods must satisfy loop consistency: predictions along each loop sum to zero: $F(x_1,x_2)+F(x_2,x_3)+F(x_3,x_1)=0$.
• Figure 2: Comparison between forward and inverse approximations of the function $f(x)=x^3-x$ using traditional neural networks and twin neural network regression. Traditional neural networks are capable of approximating the forward function, fail however at retrieving the inverse function because of non-injectivity. TNNR can be employed to fully approximate all branches of the inverse function.
• Figure 3: Twin Neural Network Regression for the inverse of non-injective functions: Twin Neural Networks are trained to predict the difference $\mathbf{x}^\text{new}-\mathbf{x}^\text{anchor}$ from inputs $(\mathbf{y}^\text{new},\mathbf{y}^\text{anchor},\mathbf{x}^\text{anchor})$, which is used to estimate the input $\mathbf{x}^\text{new}$ corresponding to the target $\mathbf{y}^\text{new}$. In contrast to TNNR for approximating the forward functions, it is necessary to include a third input variable $\mathbf{x}^\text{anchor}$ because $F$ behaves differently in regions around different anchors $\mathbf{x}_i^\text{anchor}$ that each exhibit the same target $\mathbf{y}^\text{anchor}$.
• Figure 4: Employing the k-nearest-neighbors(kNN) algorithm to find the inverse of the function $f(x)=x^3-x$. The distance that kNN uses to choose the closest neighbor is the absolute value of the difference in $y$. Since non-injectivity prevents averaging of kNN predictions, it is necessary to choose to make predictions based solely on one anchor each time. The 5 anchors define 5 different preimages. The image contains a very unfavorable choice of anchors, highlighting the problems of choosing kNN to find the inverse of functions. The lack of sufficient anchors leads to the assignment of wrong branches to make predictions. Increasing the number of anchors substantially will lead to an image similar to the right in figure \ref{['fig:combined_prediction_example']}.
• Figure 5: A first attempt at employing TNNR to solve find the inverse of the function $f(x)=x^3-x$. The anchors are chosen as nearest neighbors based on distance in $y$. While TNNR with nearest neighbor anchors has similar flaws as kNN, one can clearly see that some branches are very well approximated, even in the region where the function is non-injective. The lack of sufficient anchors still leads to the assignment of wrong branches to make predictions. Increasing the number of anchors from 5 to 600 corresponds to the TNNR prediction in image \ref{['fig:combined_prediction_example']}.