StarCoder

lastmod 2023-05-10created 2023-05-101 edits

A code LLM released by the BigCode project

Features

• A 15.5B model trained on 1 trillion tokens across 80+ programming languages
• Uses MQA for efficient generation and has an 8192-token context window
  • StarCoderBase : trained on the The-Stack v1.2 (6TB) dataset, which contains data in more than 80 languages
  • StarCoder : a model further trained on Python
• Provides a Chat Demo and a VSCode extension

See also

• BigCode
• LLM
• SantaCoder - A code generation model trained with 1.1 billion (1.1B) parameters

Favorite site

• Huggingface - bigcode (BigCode)