LMQL
lastmod 2023-05-17created 2023-05-171 edits
A language for querying LLMs
Features
- LLMs (large language models) can be queried in natural language, but sometimes an LLM does not accurately understand human intent
- To address this, researchers at SRIlab (Secure, Reliable, and Intelligent Systems Lab) of ETH Zürich in Switzerland proposed a concept called LMP (Language Model Programming)
- LMQL (Language Model Query Language) is the language and platform that implements the LMP concept
- Declarative syntax similar to SQL statements
- Imposes clear constraints on LLM output to control its behavior so that unexpected results do not occur
- Implemented in Python, so Python functions and control flow can be used in AI queries
- Can use GPT-4 via the OpenAI API as well as locally hosted transformer models
- Saves consumed tokens, cutting the cost of paid API usage by 26–85%
- Paper: https://doi.org/10.48550/arXiv.2212.06094
- Documentation: https://docs.lmql.ai/
- Playground: https://lmql.ai/playground
- Github: https://github.com/eth-sri/lmql