← All projects
Machine learning & AI · Python
N
nnscratch
A feed-forward network from first principles — hand-derived backprop, SGD/Adam, live in your browser. No PyTorch, no autograd.
01 Train live
Same layer maths as the Python package: Dense + He init, ReLU, softmax cross-entropy whose gradient folds the Jacobian into p − y, batch-averaged gradients, and an optimiser of your choice.
class 0
class 1
background = predicted class
cross-entropy loss per epoch (log scale)
02 What is implemented
- Dense layers with He/Xavier init; gradients averaged over the batch so the learning rate means the same thing at batch size 8 and 128.
- ReLU / sigmoid / tanh / softmax — softmax max-shifted for stability.
- Cross-entropy after softmax:
dL/dz = p − yin one step avoids building the full Jacobian. - SGD with momentum and Adam with bias correction.
- Gradient check in the package compares every analytic gradient to a central finite difference (~1e-8).
XOR is impossible without a hidden layer — if the boundary collapses to a line, something is wrong with backprop.