Newsletter
Book guide

A Brief Introduction to Neural Networks — What to Read Instead (and First)

An original guide to learning neural networks from scratch in 2027: the classic texts, the modern interactive courses, and a reading order that works.

The original Engatica ebook page pointed readers at a classic introductory PDF. This replacement is an original reading guide to the same subject, kept at the same address.

A reading order that works

  1. Intuition first: the 3Blue1Brown neural-networks video series — four videos that make backpropagation feel inevitable.
  2. Build one by hand: Andrej Karpathy's 'Neural Networks: Zero to Hero' — you will implement backprop yourself, which changes how you read everything after.
  3. Then the theory: the opening chapters of Dive into Deep Learning or Goodfellow, Bengio and Courville's Deep Learning for the mathematical spine.
  4. Then go deep on LLMs: Andrej Karpathy's 'Deep Dive into LLMs like ChatGPT' (February 2025) — the full pipeline from pretraining through supervised fine-tuning and RLHF to reasoning models, in the same from-first-principles style, installing the habit of treating a model as lossy compression of its training data rather than a knowledge source. Follow it with his 'How I Use LLMs' (February 2025) for the applied counterpart.
  5. Then specialise: transformers and LLM internals via our LLM hub reading path.

Why bother in the agent era?

Because every abstraction leaks. When your agent misbehaves, the people who can reason one level below the framework fix it in an afternoon; everyone else files a ticket into the void.

2027 update: one new step, one reframe

Two of this guide's anchors have grown since we wrote it, and the reading order improves if you use the new material. First, after Karpathy's Zero to Hero, insert his Deep Dive into LLMs like ChatGPT (February 2025, 3 hours 31 minutes) before you specialise [1][2]. It covers the full modern pipeline — pretraining, supervised fine-tuning, RLHF and reasoning models — in the same from-first-principles style, and it installs the single most useful sceptical habit a practitioner can have: treat the model as lossy compression of its training data, not as a knowledge source. Follow it with How I Use LLMs (February 2025) for the applied counterpart [2]. Second, extend the 3Blue1Brown step: Sanderson's deep-learning series no longer stops at backpropagation — it now runs through transformers and a visual account of attention, plus a short explainer of LLMs made for the Computer History Museum [3][4]. Watch those between steps 1 and 2 and the jump to transformers in step 4 mostly disappears. One reframe: Goodfellow, Bengio and Courville's Deep Learning is still the mathematical spine of step 3, but it is a 2016 text that predates the transformer era — read it for theory, and take current practice from D2L and the LLM material above.

Frequently asked questions

Is everything in this guide free?

Almost all of it. The 3Blue1Brown videos and Andrej Karpathy's courses are free on YouTube, and Dive into Deep Learning is free online. The one paid classic is Goodfellow, Bengio and Courville's Deep Learning — though its chapters circulate freely from the authors' site. You can follow the entire reading order, in sequence, without spending anything until you decide you want the theory text on paper.

What order should I read and watch in?

The guide's order: intuition first with 3Blue1Brown — now extended through transformers and attention — then build a network by hand with Karpathy's Zero to Hero, then the theory spine in Dive into Deep Learning or Goodfellow. New in this edition: insert Karpathy's Deep Dive into LLMs like ChatGPT (February 2025) after Zero to Hero, before you specialise. Each step makes the next one feel half as hard.

Who is this guide for?

Working engineers and technically curious readers who intend to build with LLMs and agents, not just use them. The premise is that every abstraction leaks: when an agent misbehaves, the people who can reason one level below the framework fix it in an afternoon, and everyone else files a ticket. If you only need AI vocabulary for the boardroom, the AI For Dummies review is the gentler on-ramp.

Why was Karpathy's Deep Dive into LLMs added as a step?

Because it closes the guide's biggest gap. Zero to Hero teaches how networks learn; the February 2025 Deep Dive (3 hours 31 minutes) walks the full modern pipeline — pretraining, supervised fine-tuning, RLHF and reasoning models — in the same first-principles style. It also installs the sceptical habit the guide ends on: treat a model as lossy compression of its training data, not as a knowledge source.

Sources

  1. Andrej Karpathy (Deep Dive into LLMs like ChatGPT, 5 February 2025) — AI Wiki (2026)
  2. Andrej Karpathy — Podcast Summaries (Deep Dive into LLMs, 05 Feb 2025; How I Use LLMs, 27 Feb 2025) — Podwise (2026)
  3. Visualizing Attention — Grant Sanderson, 3Blue1Brown Substack (2024)
  4. Large Language Models Explained Briefly — Grant Sanderson, 3Blue1Brown Substack (2024)
  5. 3Blue1Brown: Neural Networks (liveVideo edition) — Manning Publications (2026)