Skip to content
Search

Yi

lastmod 2024-05-20created 2024-05-201 edits

A series of large language models trained from scratch by developers @01-ai

Next-generation open-source bilingual LLM

Features

  • Next-generation open-source large language models trained from scratch by developers
  • Aimed at being bilingual models and trained on a 3T multilingual corpus, showing strong performance in language understanding, commonsense reasoning, reading comprehension, etc.
  • Based on 6B and 34B pretrained language models, extended to chatbot models, 200K long-context models, depth-upscaled models, and vision-language models
  • The Yi-34B-Chat model
    • Ranked 2nd on the AlpacaEval leaderboard after GPT-4 Turbo, outperforming other LLMs
    • Outperforms existing open-source models in both English and Chinese, taking 1st place on various benchmarks
  • Adopts the same model architecture as Llama but is not a derivative of Llama. Does not use Llama's weights
  • Available in various sizes, and the models can be fine-tuned for specific requirements
    • Chat models
      • Yi-34B-Chat, Yi-34B-Chat-4bits, Yi-34B-Chat-8bits
      • Yi-6B-Chat, Yi-6B-Chat-4bits, Yi-6B-Chat-8bits
    • Base models
      • Yi-34B, Yi-34B-200K
      • Yi-9B
      • Yi-6B, Yi-6B-200K

Hacker News comments

  • Yi: Open Foundation Models by 01.AI | Hacker News
  • The Yi-34B-Chat model ranked 2nd on the AlpacaEval leaderboard after GPT-4 Turbo, outperforming other LLMs such as GPT-4, Mixtral, and Claude.
  • The Yi-34B model took 1st place among open-source models on English and Chinese benchmarks. This is based on the Hugging Face Open LLM Leaderboard (pretrained models) and C-Eval.
  • The source code in the repository is under the Apache 2.0 license, but the weights are not.
  • The Yi models fail a particular test prompt. Despite several attempts, Yi picked one of the other answers as the winner every time.
  • The name "01.ai" is the same as the name of the first AI nation in the movie 'The Matrix', which waged war on humanity and enslaved humans, hinting that it is not a good omen.
  • The performance of the Yi models is attributed to data quality resulting from data engineering efforts.
  • The Yi 34B Chat model did not do well on the NYT Connections benchmark and ranked 22nd on the LMSYS Elo-based leaderboard. It performs better in Chinese.
  • Seeing these models improve gives hope that within 2-3 years, mobile-first LLMs will improve text-to-speech and typing prediction while also greatly reducing battery drain.
  • There is also a new model called Yi-9B.

Yi 1.5

Yi-1.5 is an upgraded version of Yi, delivering stronger performance in coding, math, reasoning, and instruction-following capability.

  • An upgrade of Yi, the next-generation open-source large language model trained from scratch by developers
  • Pretrained on a 500B-token corpus and fine-tuned on 3M diverse fine-tuning samples
  • Compared to Yi, Yi-1.5 delivers stronger performance in coding, math, reasoning, and instruction following, while still maintaining excellent capabilities in language understanding, commonsense reasoning, and reading comprehension
  • Available in 3 model sizes: 34B, 9B, 6B

See also

Favorite site

author Zer0 sitemap