MLC LLM
lastmod 2023-05-25created 2023-05-251 edits
Enable everyone to develop, optimize and deploy AI models natively on everyone's devices.
Open-source solution that lets you deploy LLM models to any device
Features
- Aims to let anyone develop and optimize AI models and deploy them to any device
- Uses MLC (Machine Learning Compilation) and Apache TVm Unity
- Supported platforms
- iPhone/iPad/Android
- Metal GPU and Intel&ARM MacBooks
- AMD/Intel and NVidia GPUs (Vulkan on Windows/Linux)
- NVidia GPUs (CUDA on Windows/Linux)
- WebGPU in the browser