One observation from the neural network series worth noting upfront: watching is not enough. To genuinely internalize how these systems work, you have to build. The "Zero to Hero" name is honest — but the hero part requires getting your hands dirty with actual code. What follows are the ideas that stuck, and some reflections on what they mean beyond the technical.

How LLMs Are Actually Built

Let me try to distill what I learned. Modern LLM training happens in three broad stages.

The first stage is pre-training: the model is fed an enormous corpus of data — essentially the entire internet — and through a neural network, it adjusts the positions of tensors in high-dimensional space to build up its parameter weights. The second stage is supervised fine-tuning: the model is shown examples of how to respond, like a teacher working through sample problems with a student. The third stage is reinforcement learning: the model starts solving problems on its own and adjusts based on how well it did against a reference standard.

A useful analogy: pre-training is like a student absorbing all the foundational material for a course. Fine-tuning is the teacher demonstrating how to solve problems. Reinforcement learning is the student doing practice problems independently and self-correcting from the answer key.

In terms of scalability, the fine-tuning stage is labor-intensive and hard to scale. The first and third stages, however, are where machines can run largely on their own. One of the most striking findings in pre-training is the scaling law: model capability is positively correlated with both the volume of training data and the number of parameters — and there doesn't appear to be an obvious ceiling. Even if training data eventually runs out, you can still improve the model by increasing its parameters (essentially, giving it more dimensions to think in).

The AlphaGo example makes this vivid. If you only use supervised fine-tuning — feeding the AI records of human games — the AI's ceiling is the best human player. But with reinforcement learning, where the AI plays against itself in a domain with clear win conditions, it blows past human performance entirely. The same logic probably applies to programming and mathematics: domains with verifiable answers are where machines can genuinely run away from us.

For domains without clear right answers — which joke is funnier, which piece of writing is better — Anthropic and others have developed RLHF (Reinforcement Learning from Human Feedback), where humans rank outputs and those rankings train a reward model. Karpathy himself notes this isn't "real" RL in the strict sense. And there's something philosophically troubling about it: you're essentially crystallizing human subjectivity and feeding it to a machine. Researchers have found that if you train long enough, the system will discover how to "game" the reward signal and produce results that are bizarre by any commonsense standard. Practitioners have to step in and stop training before it goes off the rails.

This pattern felt oddly familiar to me — it's remarkably similar to what happens with political correctness in human institutions. Once subjective judgments get codified and run through an optimization loop long enough, the outputs stop resembling anything like the original intention.

What AI Is — And What It Might Not Be

Karpathy describes an LLM as a kind of cheese: mostly solid and highly capable, but with small holes scattered throughout. You need to stay aware of where the holes are and validate the output accordingly. For coding and chess, the cheese is quite solid. For open-ended judgment calls, watch your step.

He also offered what I think is the most evocative description of AI I've come across: we're not building an animal, we're summoning a ghost. It's not a God-made creature. It might have strange, seemingly uncanny powers, while simultaneously being immature and unstable. The parallel to Yann LeCun's point is interesting — we still can't build a whole cat or a whole dog, let alone a whole person. What we've built is something stranger and narrower.

Another frame that stuck with me: AI as a compression of human knowledge. Learning itself might ultimately be a form of compression — and machine learning has found the mathematical formula for it. Physics is the field where humans found mathematical expressions for the laws of the physical world; AI might be where we found mathematical expressions for the laws of intelligence. But Karpathy is careful about how far this extends: what we've formalized is essentially a replication of cortex function — the part of the brain responsible for reasoning, language, and pattern recognition. We've probably only captured a sliver of intelligence even by that measure. (Learning to drive is still much harder for a machine than for a teenager — which tells you that driving draws on more than cortex function alone.)

Still, pre-training plus reinforcement learning plus continuous updating from new sources is no longer just a compressed knowledge store. It's starting to look like something that can genuinely think, at least in limited domains.

The Human Question

All of this circles back to the hardest question: how does AI compare to humans?

I keep running into the same wall. We have surprisingly little settled understanding of what a human being actually is. Ask what's most valuable about a person, and you'll get as many answers as there are people asking, with no one able to persuade anyone else. Without a clear anchor for what we are, it's difficult to evaluate what AI is relative to us.

From a traditional Chinese medicine perspective, for instance, the brain isn't even considered an organ in the classical taxonomy — it's not among the five viscera and six bowels. What constitutes life, in that framework, is distributed differently: the heart houses shen (spirit), the liver houses hun (soul), the lungs house po (corporeal soul). Current machine learning doesn't seem to touch any of that.

The Tao Te Ching draws a useful distinction: "In pursuit of learning, every day something is acquired. In pursuit of the Tao, every day something is dropped." Machine learning is, at its core, a massive and extraordinarily scalable version of the first kind of pursuit — the accumulation kind. The question AI forces on us might be: do we actually understand the second kind? And if intelligent people have been equating their cognitive output with their entire worth, AI will feel like an existential threat. But if the mind is only part of what a person is, then the disruption is real but bounded.

What I can't quite articulate — but feel fairly confident about — is that there remains something humans have that machines don't reach. Not intelligence in the narrow sense. Something else.

On Hallucination and Creativity

One of the most interesting side notes: AI hallucination and AI creativity might be two names for the same thing.

When an AI invents a fact about a person who doesn't exist, that's hallucination — the system doesn't know the boundaries of its own knowledge. But in Go, when reinforcement learning causes the AI to play moves no human has ever tried, that's creativity — or at least something that functions like it, because the feedback loop is tight and verifiable.

The difference seems to be the quality of the feedback loop. In domains where results are clear and consequences are direct, hallucination becomes invention. If you trained an AI on traditional Chinese medicine and gave it actual patients, with real outcomes feeding back into its training, could it eventually surpass a master physician? I'm not sure — but I don't think it's obviously impossible.

What I'm more skeptical about is whether machines can independently develop something like aesthetic judgment — subjective, yet shared; personal, yet intersubjective. That seems harder.

What This Means for Software Engineering (and Probably Other Things)

Seven months ago, Karpathy was publicly saying he only used AI for autocomplete because the models had too many quirks for anything more serious. Four months later, he reversed course — said he felt behind — and joined Anthropic.

That trajectory feels right to me. AI's ability in domains like coding is improving fast. But I still think there's a durable gap between what the machine can do and what the work actually requires: deciding what to build in the first place, developing genuine insight, knowing when you're in a dead loop and how to break out, and taking responsibility for complex outcomes. A world of ghosts without a human threading the needle would probably collapse quickly. Sustainability seems to require something biological — or at least something with skin in the game.

The Most Practically Useful Insight

The single thing that's changed how I think about my own workflow: Karpathy mentioned he rarely reads an entire book anymore. Instead, he uses LLMs to generate summaries and knowledge maps, then chooses to read deeply only when he wants to.

That reframing is genuinely useful. Think of a capable LLM as a reading companion that can ingest a codebase, a research paper, or an industry's worth of material and pre-digest it for you. Getting oriented in a new domain used to require months of immersion; now you can reach a working mental model much faster. Deep expertise still requires personal practice — you still have to do the reinforcement learning yourself — but the ramp-up to having a real idea is dramatically shorter.

The bar for raw reading and writing effort has dropped. What you do with the time that frees up is, as ever, the interesting question.

That said, there's a flip side worth keeping in mind. When your information intake bandwidth increases this dramatically, your brain is actually processing a much larger volume of material — even if the friction of getting it in has dropped. The cognitive load doesn't disappear; it shifts. So alongside all the productivity gains, don't forget to rest. Recovery isn't optional when the throughput goes up.

Notes from Andrej Karpathy's neural network and LLM series — recommended for anyone who wants a grounded, hands-on understanding of how these systems actually work.