Skip to content
Search

DSPy

lastmod 2024-12-12created 2024-12-121 edits

A framework for programming (rather than prompting) language models

About

  • DSPy (Declarative Self-improving Python) is a framework designed to control language models (LMs) through programming
  • Lets you quickly build modular AI systems, from simple classifiers to complex RAG pipelines and agent loops, and also provides algorithms for optimizing prompts and weights
  • By writing composable Python code instead of brittle prompts, you can quickly build modular AI systems and train LMs to produce high-quality output

Describe AI behavior as code, not strings, using Modules

  • Building reliable AI systems requires iterating quickly
  • But maintaining prompts is hard, because strings or data have to be modified every time you change the LM, metrics, or pipeline
  • DSPy was developed to separate the definition of an LM system from messy choices about specific LMs or prompting strategies
  • DSPy's programming approach
    • DSPy shifts the focus from tweaking prompt strings to programming with structured, declarative natural-language modules
    • For every AI component in the system, you specify input/output behavior as a signature and choose a module to assign an LM calling strategy
    • DSPy expands signatures into prompts and parses typed outputs, so you can write AI systems that are easy to use, portable, and optimizable

Optimizers tune the prompts and weights of AI modules

  • DSPy provides tools that compile high-level code with natural-language annotations into low-level computations, prompts, or weight updates
  • This lets you align the LM with the structure and metrics of your program
  • When you change the code or metrics, you can simply recompile to match
  • DSPy optimization process
    • If you have a few dozen or a few hundred representative inputs for a task and a metric that can measure the quality of system output, you can use a DSPy Optimizer
      • dspy.BootstrapRS synthesizes good few-shot examples for every module
      • dspy.MIPROv2 proposes better natural-language instructions for every prompt and explores them intelligently
      • dspy.BootstrapFinetune builds datasets for modules and uses them to fine-tune the LM weights of the system

The DSPy ecosystem advances open-source AI research

  • Compared with large language models, DSPy's modular paradigm gives more researchers the opportunity to improve the compositional architectures, inference-time strategies, and optimizers of LM programs in a distributed way
  • This has the benefit of giving DSPy users more control, helping them iterate faster, and letting their programs improve over time by applying the latest optimizers or modules
  • DSPy research began at Stanford NLP in February 2022, building on lessons learned from developing early compound LM systems such as ColBERT-QA, Baleen, and Hindsight
  • First released as DSP in December 2022 and evolved into DSPy in October 2023; thanks to 250 contributors, tens of thousands of people have been able to learn how to build and optimize modular LM programs
  • The DSPy community has produced many research results on optimizers such as MIPROv2, BetterTogether, and LeReT, and on program architectures such as STORM, IReRa, and DSPy Assertions
  • It also has many successful applications to new problems, such as PAPILLON, PATH, WangLab@MEDIQA, UMD's prompting case study, and Haize's Red-Teaming program
  • Beyond these, many open-source projects, production applications, and other use cases have proven DSPy's practicality

Install 

pip install -U dspy

Usage

You can authenticate by setting the OPENAI_API_KEY env variable or passing api_key below. 

import dspy
lm = dspy.LM('openai/gpt-4o-mini', api_key='YOUR_OPENAI_API_KEY')
dspy.configure(lm=lm)

Try the examples below after configuring your lm above. Adjust the fields to explore what tasks your LM can do well out of the box. Each tab below sets up a DSPy module, like dspy.Predict, dspy.ChainOfThought, or dspy.ReAct, with a task-specific signature. For example, question -> answer: float tells the module to take a question and to produce a float answer. 

math = dspy.ChainOfThought("question -> answer: float")
math(question="Two dice are tossed. What is the probability that the sum equals two?")

Output: 

Prediction(
    reasoning='When two dice are tossed, each die has 6 faces, resulting in a total of 6 x 6 = 36 possible outcomes. The sum of the numbers on the two dice equals two only when both dice show a 1. This is just one specific outcome: (1, 1). Therefore, there is only 1 favorable outcome. The probability of the sum being two is the number of favorable outcomes divided by the total number of possible outcomes, which is 1/36.',
    answer=0.0277776
)

See also

Favorite site

author Zer0 sitemap