Nano-llama31

lastmod 2024-08-16created 2024-08-161 edits

A nanoGPT-style version of Llama 3.1

Features

• A nanoGPT-style version of Llama 3.1 implemented by Andrej karpathy
• A minimal, dependency-free implementation of the Llama 3.1 architecture that makes training, fine-tuning, and inference very simple
  • Fewer dependencies and smaller code than the Meta and HuggingFace implementations
• Currently focuses on the 8B base model of Llama 3.1
• WIP: still under development and not finished

See also

• LLaMA
• Llama 3.1

Favorite site

• Github - karpathy/nano-llama31