Machines that learn the way brains do.
Cadence is a new kind of neural network. There is no forward pass and no backward pass. Each unit repairs its own state from its neighbours until the network comes to rest. The resting state is the answer, and learning is the same process nudged toward a goal.
Live field · units settling with no global clock
We are building artificial intelligence that works like animal and human brains. Local, continuous, plastic for its whole life, and able to repair itself. We think that is the road to general intelligence.
Brains and transformers live in different worlds.
A transformer is trained in a regime no brain has ever been in: a static corpus, a global gradient, a freeze, then a window of recent tokens as the only memory. A brain lives in the other one. Cadence is built for that other regime.
| Transformer | Brain | Cadence | |
|---|---|---|---|
| Learning signal | A global gradient, sent backwards through every layer | Local, at each synapse | Local at each connection, and certified local by a reference engine |
| Memory | A context window of recent tokens | Synaptic strengths | Connection strengths, with no window |
| Lifetime | Train once, then freeze | Plastic every moment | Plastic for life, with consolidation, pruning and sprouting |
| Inference | One pass through a fixed stack | Activity settles into a state | Activity settles into a state |
| After damage | Retrain | Recovers | Recovers under its own local rule |
One rule for answering and learning.
Settle
Clamp the input and let every unit repair its own state from what its neighbours send. When nothing moves, read the answer.
Nudge
Settle twice more with the output pulled gently toward the target, once each way. Labels, a teacher's moves and rewards all enter through this one door.
Update locally
Each connection changes on the difference it saw between the two nudged states. A connection reads two activations, and a unit reads one.
Cadence builds on equilibrium propagation, on the link between modern Hopfield networks and attention, and on the observer consensus mathematics of our physics program. Attention turns out to be one settlement step of a Hebbian memory. The rest of the settlement, and a life for the connections, is what Cadence adds.
Measured, with receipts.
Every number below comes from a receipt that binds it to the exact code and data. Held-out data is read once, and the backprop baseline is trained in the same script.
Learning from reward
On cart-pole, Cadence reaches the solved threshold in under half the steps of a PPO network, with 1,716 parameters against 9,155.
Cheap decisions
A trained policy decides in two warm settlement steps, a fraction of the compute of a PPO network and 20 times less than a PPO transformer.
Memory without a window
Each key and value is stored as one Hebbian imprint. Recall stays perfect from 4 to 128 pairs, and one read takes two settlement steps.
Self-repair
After half its input pixels go dark, Cadence wins back 85% of the lost accuracy in 100 updates under its own local rule.
Bach chorales
Continuing chorales chord by chord, Cadence beats the same-size backprop network on accuracy and on bits per chord.
Connectome scale
The same dynamics settle a complete fruit-fly nervous system read from published connectomes.
Cart-pole, recall and repair results come from our forthcoming paper. Library benchmarks and the examples ladder are public, with receipts in the repositories.
Where we push next.
The receipts also show where Cadence trails established methods. Those gaps set our research agenda.
Built for new hardware
A settlement takes tens of steps, so training on a CPU runs 10 to 100 times slower than backprop. Settlement is local and parallel, which suits neuromorphic and analog hardware that computes in place.
Text modelling
On next-character prediction Cadence reaches 3.33 bits per character, behind a small transformer at 3.08. Closing that gap is a priority.
Long-horizon reward
The local update tracks the true policy gradient with a correlation of 0.95 to 0.98. Assigning credit across long episodes, as in Pong and Hopper, is work in progress.
Continual tasks
Across 30 permuted-MNIST tasks seen once each, Cadence starts level with Adam and drifts three points by the end.
Ten rungs, from digits to a composer.
Each rung is a tutorial, a script and a receipt. Several ship with pages that run the same settlement in the browser that the receipt scored in Python.
- 01DigitsHandwritten 8×8 digits at 96% held-out accuracy in 20 epochs.
- 02ImagesMNIST at 97.4% in 10 epochs, with two independent backends agreeing on every prediction.
- 03Connect FourImitates a depth-four search and plays in the browser with no lookahead.
- 04PongA paddle that learns from raw pixels and reward.
- 05TextNext-character prediction on Shakespeare from a sixteen-character window.
- 06Sign writerSees a sign and writes it with a two-joint arm, matching its teacher on unseen signs.
- 07Bach choralesContinues chorales chord by chord, ahead of the same-size backprop network.
- 08Cart-poleClassic control learned from reward.
- 09C. elegansThe worm's published connectome under a declared protocol, tested against a shuffled control.
- 10ComposerA plastic network that listens to a piano library for ten days, then composes from silence by imagining each phrase and scoring how it sounds.
Install it in one line.
NumPy only by default. Add the torch extra for CUDA and Apple silicon, or numba for fused CPU kernels. Python 3.11 or newer, MIT licensed, fully typed.
# install $ pip install cadence-net $ pip install "cadence-net[accel]" # CUDA + Apple silicon import cadence as cd wiring = cd.layered(64, 32, 10, density=1.0, seed=0) learner = cd.Learner( cd.Settlement(wiring, cd.learning_rule()), wiring.sets["output"], cd.LearnerConfig(eta=3.0, beta=0.1, temperature=0.1)) for idx in batches: learner.step(drive[idx], labels[idx]) # settle, nudge, update locally learner.accuracy(drive_test, labels_test) # 0.96 on 8x8 digits
Born in physics.
Cadence grew out of our physics research. Observer Patch Holography describes the universe as observer patches reaching consensus, and Cadence builds learning machines from the same patch-net mathematics.