Alpaca

lastmod 2023-04-19created 2023-03-147 edits

Alpaca: A Strong Open-Source Instruction-Following Model

A strong open-source Instruction-Following model at the GPT-3.5 level

Features

• Stanford CRFM fine-tuned Meta's LLaMA 7B on 52K Instruction-Following data
• Behaves similarly to OpenAI's GPT-3.5 (text-davinci-003), but is very small and cheap
• The training recipe and data are released, and the model weights are planned for release later
• Released for research purposes only; commercial use is prohibited
• Fine-tuning took 3 hours on eight 80GB A100s, which is possible for under about 100 dollars in the cloud

See also

• GPT
• KoAlpaca - Korean Alpaca model
• Alpaca.cpp - running a ChatGPT-like model on your own laptop
• Alpaca-LoRA - Low-Rank LLaMA Instruct-Tuning
• Code Alpaca - LLaMA trained on code generation instructions
• LLaMA
• RedPajama - an open-source project that recreates the LLaMA dataset

Favorite site

• Stanford CRFM
• Github - tatsu-lab/stanford_alpaca
• Alpaca Demo site