OpenOrca

lastmod 2023-07-01created 2023-07-012 edits

Open-source dataset & instruction-tuned language model

Features

• An open-source replication of the work in Microsoft's Orca paper
  • About 1 million FLANv2 entries augmented with GPT-4 completions
  • About 3.5 million FLANv2 entries augmented with GPT-3.4 completions
• Includes all 75k CoT entries from the FLAN-1m dataset
• Full-weight fine-tuning on the LLaMA-13B foundation is currently in progress, and performance should come close to Microsoft's
  • OpenOrca-LLaMA-13b is expected to be released around mid-July
• Looking for GPU compute sponsors to apply OpenOrca to other platforms (Falcon, LLaMA, MPT, etc.)

See also

• LLM

Favorite site

• OpenOrca - Open-source dataset & instruction-tuned language model | GeekNews