Web LLM
lastmod 2023-04-18created 2023-04-181 edits
Running LLMs in the browser with WebGPU acceleration
Features
- Runs LLMs such as LLaMA and Vicuna in the browser using WASM, without a server
- Accelerated with WebGPU, but also supports the GPU backends supported by Apache TVM (CUDA, OpenCL, Vulkan)
- Uses MLC (Machine Learning Compilation) technology