Forward Forward Algorithm in Python Unofficial LeetArxiv Implementation

Forward-Forward (Hinton, 2022) is a biologically-plausible backpropagation alternative that achieves ~96% (Löwe, 2023) accuracy on MNIST without flowing gradients.

It vastly outperforms Target Propagation (Bengio, 2014), another biologically plausible backprop alternative, that reaches ~39% accuracy on MNIST.

This is part of our Alternatives To Backpropagation series:

1. Target Propagation: Autoencoders are great at reconstruction and can be used to learn backprop
2. Belief Propagation: Training using Optimal Transport theory
3. Forward-Forward: We can train individual layers to discriminate data like GANs, and sum their correctness without backprop

Getting Started

We provide a Jupyter Notebook written to be followed alongside this LeetArxiv guide.