Rust · reverse-mode autograd · no ML frameworks
Synapse is a reverse-mode autograd engine you can read node by node, and a tiny MLP built on top of it, trained on XOR. No ML frameworks, no tensor crates. Here is the real thing, training in your browser.
Not a canned animation. The buttons below drive a scalar autograd engine and a 2 4 4 1 tanh MLP ported line for line from src/value.rs and src/nn.rs, running right here in JavaScript. Every click runs a real forward pass, a real backward pass through the computation graph, and a real gradient descent step.
Loss curve, drawn live from the real MSE at each epoch
Decision boundary, the trained net evaluated over a grid
| x1 | x2 | target | prediction | result |
|---|
Every value in Synapse remembers how it was made. Add, multiply, subtract, divide, raise to a power, negate, or pass through tanh, and the operation is recorded as a node in a graph with pointers back to its inputs. Call backward() once on the output and gradients for every value in the graph fall out through the chain rule, in reverse topological order, with shared subexpressions correctly accumulating gradient from every place they are used.
A scalar Value wraps its data, its gradient, and the operation and parent nodes that produced it.
One backward() call walks the graph in reverse dependency order and pushes gradients through every op.
A value used twice, like a * a, sums gradient contributions from each use instead of overwriting.
Clone it, run it, test it. That is the whole workflow.
cargo run --release # trains XOR, prints the loss curve
cargo test # gradient check, shared-node test, XOR training test