StableLM
lastmod 2023-04-21created 2023-04-211 edits
StableLM: Stability AI Language Models
Features
- 3B/7B models released; 15B/30B/65B models to be released, with plans up to 175B
- Models are licensed under CC BY-SA-4.0, allowing commercial use with attribution
- Trained on a new dataset based on the open dataset The Pile, but three times larger at 1.5T tokens
- Context length is 4096 tokens
- As a PoC, also released StableLM-Tuned-Alpha-7B, fine-tuned following the Alpaca procedure
- Uses five conversational datasets: Stanford's Alpaca, Nomic-AI's gpt4all, RyokoAI's ShareGPT52K datasets, Databricks labs' Dolly, Anthropic's HH
- Chatbot demo available on Hugging Face