LocalAI
lastmod 2025-03-31created 2023-06-153 edits
Self-hosted, community-driven, local OpenAI-compatible API. Drop-in replacement for OpenAI running LLMs on consumer-grade hardware. Free Open Source OpenAI alternative. No GPU required. LocalAI is an API to run ggml compatible models: llama, gpt4all, rwkv, whisper, vicuna, koala, gpt4all-j, cerebras, falcon, dolly, starcoder, and many other
A self-hostable OpenAI-compatible API
Features
- Use various LLMs through an OpenAI-compatible REST API
- Works without a GPU. GPU acceleration supported for llama.cpp-compatible LLMs
- Supports multiple models, plus audio transcription, text generation, and image generation (Stable Diffusion)
- Once loaded, models are kept in memory for fast inference
- Supports all ggml-compatible models: llama, gpt4all, rwkv, whisper, vicuna, koala, gpt4all-j, cerebras, falcon, dolly, starcoder, ...
See also
- ChatGPT
- OpenAI
- LLM
- llama
- llama.cpp
- LM Studio
- LangChain - A library that connects LLMs to the outside world
- Open WebUI
- ollama - Ease of use, support for many models, efficient resource management
- vLLM - High-performance inference, memory efficiency, large-scale batch processing
- LocalAI - Supports many model types (text, speech, image)
- FastChat - Supports many open-source models, provides a web UI