Uzu
lastmod 2025-07-17created 2025-07-171 edits
High-performance AI inference engine for Apple Silicon
About
- Rust-based inference engine for running AI models fast on Apple Silicon
- Hybrid architecture that selectively uses GPU kernels or MPSGraph underneath CoreML
- Uses its own model format; various models such as Llama3 can be converted with the lalamo tool
- Mostly faster than llama.cpp, and notably 13x faster processing on Qwen3-0.6B
- Flexible development and integration through Swift bindings, a CLI interface, a Rust API, etc.
- Maximizes performance with a modular design and the unified memory of Apple devices, and ensures reliability and scalability by providing verifiable inference paths