ChatLLaMA

lastmod 2023-03-02created 2023-03-021 edits

Open source implementation for LLaMA-based ChatGPT training process. Faster and cheaper training than ChatGPT (wip)

Features

• LLaMA, recently released by Meta, is not fine-tuned with the RLHF training process
• ChatLLaMA is an RLHF implementation using LLaMA, fully open source, for building ChatGPT-style services
• Trains about 15 times faster than ChatGPT
• Built-in DeepSpeed ZERO support to speed up the fine-tuning process
• Supports all of LLaMA's 7B, 13B, 33B, and 65B, so you can choose the size that suits you

See also

• LLaMA - 65b-parameter LLM released by Meta
• ChatGPT

Favorite site

• nebullvm/apps/accelerate/chatllama at main · nebuly-ai/nebullvm