Improved Forward-Forward Contrastive Learning
Gananath R
TL;DR
The paper tackles the challenge of biologically plausible learning without backpropagation by building on Forward-Forward ideas. It eliminates FFCL's final backprop-based stage and uses two parallel networks with local losses to update weights without global error signals. On MNIST, the method yields convergent learning, with losses decaying and test accuracy reaching about 63%. The work advances biologically inspired learning by removing backprop and reducing training complexity, with implications for neuroscience and energy-efficient AI.
Abstract
The backpropagation algorithm, or backprop, is a widely utilized optimization technique in deep learning. While there's growing evidence suggesting that models trained with backprop can accurately explain neuronal data, no backprop-like method has yet been discovered in the biological brain for learning. Moreover, employing a naive implementation of backprop in the brain has several drawbacks. In 2022, Geoffrey Hinton proposed a biologically plausible learning method known as the Forward-Forward (FF) algorithm. Shortly after this paper, a modified version called FFCL was introduced. However, FFCL had limitations, notably being a three-stage learning system where the final stage still relied on regular backpropagation. In our approach, we address these drawbacks by eliminating the last two stages of FFCL and completely removing regular backpropagation. Instead, we rely solely on local updates, offering a more biologically plausible alternative.