Lanarky
lastmod 2023-06-19created 2023-06-191 edits
Open-source framework to deploy LLM applications in production. Built on top of FastAPI
Open-source framework for deploying LLMs to production
Features
- Based on FastAPI
- Multi-mode token streaming
- Simple gradio chatbot for rapid prototyping
- Supports LangChain applications
- Multiple LLM caching strategies
- Planned: support for LlamaIndex, Guidance, Rebuff, etc., and SQL DB integration