Hebbian Learning

Neurons that fire together wire together.

The principle

Donald Hebb proposed in 1949 that when two neurons are active at the same time, the connection between them should strengthen. This simple rule — "neurons that fire together wire together" — turns out to be sufficient for building associative memory.

No backpropagation

Unlike modern deep learning, Hebbian learning is local. Each connection updates based only on the activity of the two neurons it connects. There is no global error signal, no loss function, no gradient propagation through layers. This makes it biologically plausible and computationally cheap.

The tradeoff: Hebbian learning doesn't optimize for a specific task. Instead, it captures the statistical structure of whatever patterns are presented — which is exactly what you want for associative memory.

In Membot

When you build a cartridge with the --train flag, Membot trains Hebbian weights on the embedded vectors. The result is a weight matrix (~128 MB) saved as _brain.npy.

These weights enable three capabilities:

• Associative recall — retrieve memories from partial or noisy input
• Noise tolerance — the attractor dynamics correct for corrupted bits
• Cross-modal binding — link text, image, and audio representations

Cross-modal binding

By presenting CLIP (Contrastive Language-Image Pre-training) and CLAP (Contrastive Language-Audio Pre-training) embeddings alongside text embeddings during training, the Hebbian weights learn associations across modalities. Three training passes are typically used:

After training, a text query can retrieve images, and an audio clip can retrieve text — the Hebbian weights bridge the modalities.

Further reading

Neuromorphic Substrate — the physics layer
CLIP — image embeddings
CLAP — audio embeddings