tutorial · intermediate · evergreen
Let's build GPT: from scratch, in code, spelled out.
Andrej Karpathy · Jan 17, 2023 · 1h 56m
What this video covers
Andrej Karpathy builds a GPT from scratch in code, following Attention Is All You Need and GPT-2/3, in about two hours using PyTorch. He connects it to ChatGPT and assumes familiarity with his earlier makemore videos.
Why it surfaced in Hacker News
It is the go-to recommendation for learning how transformers and LLMs actually work, praised for its clarity and Karpathy's teaching.
10recommendation score
- 14
- distinct recommenders
- 8
- recommendation months
- 4
- strong recommendations
Recommendation context
Short excerpts from positive mentions. Open the source to read the full Hacker News discussion.
“Andrej Karpathy's GPT video is a must have companion for this”
“I was blown away”
“the best video I have seen on the topic”