Skip to content
Search

Llama2.c

lastmod 2023-08-01created 2023-07-272 edits

Llama 2 inference code written in pure C

Llama2.c is Inference Llama 2 written in pure C, developed by Andrej Karpathy. The project is a lightweight library for machine learning model inference; because it is written in C, it can be used on a variety of platforms. The library is designed to run fast on the CPU, especially for small models.

Summary

This library is a lightweight library for machine learning model inference; because it is written in C, it can be used on a variety of platforms. The library is designed to run fast on the CPU, especially for small models. The library is designed to run fast on the CPU, especially for small models.

The project offers various optimization options that let users run the code faster depending on their system. For example, compile options such as -O3, -Ofast, -ffast-math, -funsafe-math-optimizations, and -march=native can be used. These options include optimizations such as vectorization, loop unrolling, and branch prediction, and some of them may violate the C/IEEE specifications.

The project is still under development and has several issues to resolve. For example: why there is a leading space when running ./run in the C sampling code, multi-query support for small models, and support for inference beyond the maximum sequence length.

Features

  • GCC optimization options: learn more about GCC's various optimization options.
  • GCC floating-point math: get detailed information about the -ffast-math and -funsafe-math-optimizations options.
  • Andrej Karpathy's GitHub: check out Andrej Karpathy's other projects.
  • Llama2.c license: check the project's MIT license.
  • Llama2.c contributors: see the people who contributed to this project.

See also

Favorite site

author Zer0 sitemap