This project builds and trains GPT-2, a transformer language model that learns to predict the next token, or piece of text, and generates text one token at a time. It follows the from-scratch approach in Karpathy’s course and GPT-2 walkthrough, connecting neural-network fundamentals with the model architecture, training loop, and text generation in Python and PyTorch.
I completed Andrej Karpathy’s Neural Networks: Zero to Hero series and trained GPT-2 from scratch. Training took four days, but it was worth it. Thanks to badmephisto!
