GPTCache
lastmod 2023-04-25created 2023-04-251 edits
A library that caches answers to LLM queries
Features
- Caches LLM answers to reduce an organization's LLM costs and speed up responses
- Uses a semantic caching strategy to find similar or related queries and raise the cache hit rate
- Converts queries into embeddings with an embedding algorithm and performs related-item search on these embeddings through a vector store
LLM Adapter : supports OpenAI ChatGPT and LangChain (Bard/Anthropic/LLaMA and others planned) MultiModal Adapter : OpenAI Image Create, OpenAI Audio Transribe, HuggingFace Stable Diffusion Embedding Generator : OpenAI, ONNX, HuggingFace, Cohere, fastText, SentenceTransformers Cache Storage : SQLite, PostgreSQL, MySQL, SQLServer,.. Vector Store : Mulvus, Zilliz Cloud, FAISS, Hnswlib Cache Manager : LRU, FIFO See also
Favorite site