Why this chapter matters for PMs
Before you can reason about LLMs, RAG, or agents, you need the machine-learning fundamentals underneath them — and they're simpler than the jargon suggests. The whole chapter teaches one habit: when a model is shown to you as a single accuracy number, you know exactly which questions expose what that number is hiding. That habit is the difference between a PM who rubber-stamps a model launch and one who can actually review it.
Examples are grocery-delivery flavoured — swap in your own domain when you present.
Chapter summary
Every concept here is one angle on the same throughline: never trust a single number until you know what it was measured against, at every layer of the stack. For Chapter 1 that layer is the data layer. A headline "98% accuracy" means nothing until you ask which set it came from — because training accuracy measures memorisation, and validation accuracy is the only one that estimates the real world. The two-line loss curve, not a headline number, is what tells you whether the model generalised; the moment training and validation loss diverge is the moment overfitting begins, and it's exactly what early stopping watches for. Splits keep the test set an honest blind exam, distribution shift is what breaks a frozen model when the world moves, and hyperparameter tuning is the cheapest lever to fix an underperformer. The pre-ship review ties it together: make each hidden failure mode answer for itself before launch. Later chapters carry the same discipline up the stack — to tokens and latency, then to evals and benchmarks.
Chapter 1 quiz
5–8+ questions · 80% to pass · a failed quiz links back to the cards to review.