The setup

Start random. Start wrong.

A fresh network's dials are random noise — ask it to read a digit and it guesses like a dice roll. Training never writes rules; it only ever reduces wrongness.