Guidance

lastmod 2023-05-17created 2023-05-171 edits

A guidance language for controlling LLMs

Features

• A language that controls language models more effectively than conventional prompting.
• Based on Handlebars templates; lets you specify output structure and method and control logical flow.
• Uses a key/value cache inside the prompt and runs inference only on the parts you want, improving inference performance.
• Goes beyond simple control of output structure to guarantee valid syntax such as JSON.
• Supports easy integration with HuggingFace models.

See also

• LLM

Favorite site

• microsoft/guidance: A guidance language for controlling large language models.
• A guidance language for controlling LLMs | Hacker News
  • The repository has many examples, and the examples alone are quite interesting. I especially like the part where, when structured output is needed, the LLM generates only the content parts rather than the whole thing, cutting inference time in half.