NanoChat
lastmod 2025-10-15created 2025-10-151 edits
The best ChatGPT that $100 can buy
About
- Andrej karpathy's nanochat is an open-source project for building your own ChatGPT-like conversational LLM on a $100 budget
- The full pipeline (tokenization, pretraining, midtraining, fine-tuning, evaluation, inference, web serving) is implemented in a concise, hackable way
- Training finishes within 4 hours on a single 8XH100 GPU node with one script; the d26 model, aiming for GPT-2-level performance, takes about 12 hours for about $300, and the $1000 tier takes 41.6 hours
- An extremely concise codebase of 8,300 lines in 44 files at 330KB, providing a strong baseline that is easy to read and fork, with no complex config objects or model factories
- Being developed as the capstone project of Eureka Labs' LLM101n course, with the goal of advancing the state of the art for micro models accessible on budgets under $1000