Skip to content
Search

Llama 3.3

lastmod 2024-12-11created 2024-12-112 edits

With 70B, a GPT-4 class model can now run on a laptop

About

  • Meta's Llama 3.3 70B model, with GPT-4 class performance, can now run on an ordinary laptop (64GB MacBook Pro M2)
  • A major advance achieved about 20 months after the release of LLaMA in March 2023
  • Ran it by downloading 42GB of model data locally via Ollama (it can also run with Apple's MLX)
  • Requires 64GB of RAM; memory usage is high while running, so closing other resource-intensive apps is recommended

Performance and benchmarks

  • Ranked 19th on the LiveBench benchmark, outperforming Claude 3 Opus and on a level similar to GPT-4 Turbo
  • Recorded top-tier performance especially in the Instruction Following evaluation
  • Can perform various tasks such as text generation, code writing, and SVG image generation

Increasingly powerful models can now run on personal devices

Other notable locally runnable LLMs

Qwen2.5-Coder-32B

  • Developed by Alibaba's Qwen research team and released under the Apache 2.0 license
  • Shows excellent performance on code generation tasks and can be used freely
  • Its open-source license makes it an even more meaningful tool for developers

QwQ

  • Implements a chain-of-thought pattern similar to OpenAI's o1 series
  • Can solve complex problems step by step
  • Impressive in that it runs smoothly even in a local environment

Meta's Llama 3.2

  • The 1B and 3B models can run even on small computers such as the Raspberry Pi
  • Offers excellent performance for its size, and also includes 11B and 90B multimodal vision models with image processing capabilities
  • There is a wide range of choices from small to large models
  • These models show that LLM technology has moved beyond server-grade hardware and can run on ordinary personal computers
  • In particular, advances in open-source models are expanding the environment in which developers can freely experiment and build applications

Outlook

  • Major advances are expected in multimodality and model efficiency
  • Progress is expected to continue focusing on practical, efficient task performance rather than AGI
  • The current models alone are expected to enable productive work for the next few years

See also

Favorite site

author Zer0 sitemap