Backpropagation without the scare factor

Backpropagation means: send the error backward through the network so each weight learns how much it contributed to the mistake.

In our tiny model there is only one layer, so “backward” looks like:

error = prediction - actual
weight1 -= learning_rate × error × study_hours
weight2 -= learning_rate × error × sleep_hours
bias    -= learning_rate × error

That error × input pattern is the seed of full backprop. Deep nets chain the same idea with calculus (chain rule) so every layer gets a fair share of blame.

Backprop is not magic. It is bookkeeping: who caused how much error, nudge them opposite that direction.

What this tutorial does not cover

Being upfront builds trust. Real stacks usually add:

• More layers and neurons — depth creates feature hierarchies
• Better losses — cross-entropy for classification, MSE for regression
• Optimizers — Adam, momentum, weight decay
• Regularization — dropout, early stopping, data augmentation
• Scale — batching, GPUs, distributed training
• Frameworks — PyTorch, TensorFlow, JAX

None of that changes the core loop you practiced:

Predict → measure error → adjust → repeat.

Final recap

You built a tiny neural network by hand.

Idea	You did it
Neuron	input × weight + bias
Probability	sigmoid
Decision	threshold 0.5
Loss	squared error
Training	epoch loop updating weights
Multi-input	w1, w2
Backprop preview	error × input updates

Runnable code: githubRepo/2026/05/20/tiny-neural-network-by-hand/.

Final quiz — test your knowledge

Where to go next

• Add more rows and features in the repo’s examples/ folder
• Read about logistic regression—you basically built one
• Try a framework tutorial next, but trace one batch: loss.backward()

When you finish the quiz, head to the completion page for a summary and project ideas.