How Auto-Encoders Could Provide Credit Assignment in Deep Networks via Target Propagation

TL;DR

This work introduces target propagation as an alternative to backpropagation by using layer-wise denoising auto-encoders to generate reconstruction-based targets that steer intermediate representations toward higher-probability configurations. By formulating a KL(Q||P) objective over a stack of auto-encoders, the approach provides a principled, layer-local training signal that can handle discrete latent variables, multi-modal data, and recurrent structures, while potentially improving biological plausibility. The paper develops a detailed training scheme, discusses sampling from the model, and explores top-level priors (Gaussian, factorial, Parzen) and nearest-neighbor strategies to maintain tractable, structured generation. It also offers extensions to supervised, semi-supervised, and structured-output settings and outlines key questions and conjectures to guide future mathematical and empirical validation, including potential brain-inspired mechanisms for credit assignment.

Abstract

We propose to exploit {\em reconstruction} as a layer-local training signal for deep learning. Reconstructions can be propagated in a form of target propagation playing a role similar to back-propagation but helping to reduce the reliance on derivatives in order to perform credit assignment across many levels of possibly strong non-linearities (which is difficult for back-propagation). A regularized auto-encoder tends produce a reconstruction that is a more likely version of its input, i.e., a small move in the direction of higher likelihood. By generalizing gradients, target propagation may also allow to train deep networks with discrete hidden units. If the auto-encoder takes both a representation of input and target (or of any side information) in input, then its reconstruction of input representation provides a target towards a representation that is more likely, conditioned on all the side information. A deep auto-encoder decoding path generalizes gradient propagation in a learned way that can could thus handle not just infinitesimal changes but larger, discrete changes, hopefully allowing credit assignment through a long chain of non-linear operations. In addition to each layer being a good auto-encoder, the encoder also learns to please the upper layers by transforming the data into a space where it is easier to model by them, flattening manifolds and disentangling factors. The motivations and theoretical justifications for this approach are laid down in this paper, along with conjectures that will have to be verified either mathematically or experimentally, including a hypothesis stating that such auto-encoder mediated target propagation could play in brains the role of credit assignment through many non-linear, noisy and discrete transformations.

Figures (5)

• Figure 1: For any intermediate layer $h$, we consider as generative model the combination of the top-level prior $P(h)$ (e.g. represented by a deep denoising auto-encoder), a decoder $P(x|h)$, and an encoder $Q(h|x)$. The training criterion requires the encoder and decoder to reconstruct $x$ well (lower auto-encoding loop), while $P(h)$ models well the transformation of data $x$ into $h$ through $Q(h|x)$, while the entropy of $Q(h|x)$ is increased where possible. It means that the top-level auto-encoder (upper auto-encoding loop) does a good job of modeling $h\sim Q(H|x)$ with data $x$, and that the encoder produces samples that are easy to model by $P(h)$.
• Figure 2: The training procedure involves running both a long-loop encode/decode path (in thick black, bold going up, producing representations $h_l$, and dashed going down, producing reconstructions $\hat{h}_l$) and all the short-loop encode/decode paths associated with each layer (in red, curved). The representations $h_l$ and reconstructions $\hat{h}_l$ basically become targets for each other, while each layer is trained to be a good auto-encoder.
• Figure 3: Illustration of auto-encoders allowing good targets at level $l$ to be propagated into good targets at level $l-1$ simply because decoder $g_l$ is approximately inverting the encoder $f_l$. If $\hat{h}_l$ is a good target for $H_l$, around $h_l$, i.e., if cost($H_l=\hat{h}_l) <\,$cost$(H_l=h_l)$ and $f_l(g_l(\hat{h}_l))\approx \hat{h}_l$, i.e., the encoders are good inverses of each other in the neighborhood of the reconstructions $\hat{h}_l$, then similarly we obtain that cost($H_{l-1}=\hat{h}_{l-1})<\,$cost$(H_{l-1}=h_{l-1})$, i.e., the reconstructed $\hat{h}_{l-1}$ is also a good target for $H_{l-1}$.
• Figure 4: Generalization to supervised training, with $y$ appearing as an extra input of the top-level auto-encoder. The reconstruction $\hat{h}$ of $h$ when $y$ is given provides a supervised training signal for changing the encoder $Q(h|x)$. It can be made discriminant by considering as well the reconstruction obtained when $y$ is missing.
• Figure 5: Architecture associated with the structured output or multi-modal cases (two modalities shown, here, $x$ and $y$). Each modality has its own representation, but a top-level deep auto-encoder learns to represent their joint or conditional distribution. The top-level auto-encoder can be used to sample (MCMC) or predict (MAP) some modalities given others.