CNNs, RNNs & Transformers: The Neural Network Family Tree, Explained

Our digit-reading network ended on a confession: it treated each image as 784 unrelated numbers. It had no idea pixel 15 sits next to pixel 16 — shuffle the pixels of every image the same way and it would have trained just as happily. It succeeded anyway because MNIST is tiny and tidy. On real photos, real audio, real language, ignoring the shape of the data is fatal.

That's what architecture means in deep learning, and it's the final basic you need. The neurons never change — every family below is still weighted votes with biases and bends, trained by the same guess-score-nudge loop. What changes is the wiring: each architecture bakes an assumption about the data directly into how the neurons connect.

Feedforward networks: the plain stack

The architecture this whole guide has used so far — layers in a row, every neuron connected to everything in the previous layer — is the feedforward network (or multilayer perceptron, MLP, in older papers). It's the vanilla of the family: no assumptions about space or order, just "here are some numbers, find the relationships."

That neutrality is its niche. For tabular data — a loan application, a churn record, sensor readings — the columns really are just independent measurements with no meaningful "neighbors," and the plain stack fits naturally. It's also hiding inside every fancier architecture below, doing the workaday number-crunching between the clever parts.

Its weakness is exactly what our digit-reader confessed: connect everything to everything and the network must rediscover from scratch that pixels have neighborhoods — while the parameter count explodes. One layer on a modest 1000×1000 photo already needs billions of weights. For perception, brute force is the wrong shape.

CNNs: pattern-detectors on patrol

The convolutional neural network fixed vision with one idea: learn a small pattern-detector once, then slide it across the whole image.

Instead of every neuron seeing every pixel, a CNN learns tiny filters — each looking at a small patch, say 3×3 pixels. Each filter is a specialist: one fires on vertical edges, another on a patch of orange, another on a curve. The filter slides ("convolves") across the image, checking its patch everywhere, producing a map of where its pattern showed up.

Two properties made this the architecture that cracked vision in 2012, when a CNN called AlexNet crushed the field's benchmark image-recognition contest and set off the modern deep-learning boom:

• Reuse. A cat's ear looks the same in the top-left or bottom-right of a photo. A CNN learns "ear-ness" once, in one small filter, instead of relearning it for every location — collapsing billions of would-be weights into thousands.
• Locality, then hierarchy. Small patches feed into layers that scan maps of patches, so the features-built-from-features story from chapter one — edges → textures → parts → objects — is wired into the architecture rather than hoped for.

Every time your phone finds faces in your photo library or a car reads a stop sign, this is the machinery running.

RNNs: reading with a running memory

Now for sequences — text, audio, anything where order is the whole point. "Dog bites man" and "man bites dog" contain identical words; only sequence distinguishes news from noise.

The classic answer was the recurrent neural network: read the sequence one step at a time, and pass along a running memory. At each word, the network combines the new input with a summary of everything so far, and hands the updated summary to the next step — like reading with a single sticky note you keep rewriting as you go.

Elegant — with two flaws that defined an era:

• The memory fades. Everything ever read gets squeezed through that one rewritten note. By word 200, word 3 has been diluted toward nothing — and if word 3 was the name of the murderer, too bad. (Upgrades like LSTMs stretched the memory considerably, but never truly solved long range.)
• One step at a time. Step 500 can't be computed before step 499. That's a hard sequential bottleneck — precisely the thing modern GPU hardware, built to do millions of things at once, cannot accelerate. Training big RNNs on big data was grindingly slow.

RNNs powered a real decade — the speech recognition and machine translation systems of the mid-2010s, including the 2016 neural version of Google Translate — and today they've largely passed the torch. Both flaws fell at once, to one paper.

Transformers: everything attends to everything

In 2017, Google researchers published "Attention Is All You Need" and reset the field. The Transformer discards the running memory entirely. Its move: process every position in the sequence simultaneously, and let each one look directly at every other via a mechanism called attention.

As the network processes each word, attention lets that word ask, in effect, "which other words matter for making sense of me?" — and pull information straight from them. In "The animal didn't cross the street because it was too tired," the word it attends strongly to animal, hardly at all to street. Every word queries the whole sentence at once; no sticky note, no fading relay.

Both RNN flaws die instantly:

• Long range is free. Word 3 and word 3000 are one attention-hop apart — nothing fades with distance. This is what makes today's giant context windows possible.
• Massively parallel. With no step-by-step dependency, the whole sequence is processed at once — a perfect match for GPUs. Training could suddenly scale to internet-sized data, and the scaling era of AI began.

The T in GPT stands for Transformer. ChatGPT, Claude, Gemini, Llama — every modern LLM is this architecture, stacked deep and trained on trillions of tokens. And it escaped language: Vision Transformers treat image patches as words, and the diffusion models behind AI image generation lean on attention to bind your prompt to the picture. The family tree's newest branch is busily absorbing the others' habitats.

Choosing from the tree

The decision, compressed:

The guide, in one breath

You now hold the complete basic picture of the machinery behind modern AI:

• A neural network is a giant adjustable function — numbers in, numbers out, knowledge stored entirely in its weights.
• Its only component is a tiny weigh-add-shift-squash unit, repeated by the million — with the activation function's bend making depth possible at all.
• It learns by guess → score → blame → nudge, over epochs of data, assembling unwritable rules out of pure error-pressure.
• And it comes in architectures — feedforward, CNN, RNN, Transformer — each wiring a different assumption about data into the same atoms.

One honest closing note: we can name every part, yet why a particular trained network makes a particular decision remains largely opaque — billions of tuned dials don't come with explanations. Reading those dials is a young research field (interpretability), and one of the most consequential open problems in AI.

From here, pick your next safari: how this machinery scales into large language models, the wider craft of machine learning around the model, or how the same networks learned to paint and film.