dspy

DSPy, developed by Stanford NLP, is a framework for programming rather than prompting language models. Instead of hand-writing and iterating on prompt strings, developers declare Signatures (typed input/output contracts) and compose them into Modules such as Predict, ChainOfThought, ReAct, or ProgramOfThought, building multi-step LM pipelines the same way they’d assemble layers in a neural network. The framework grew out of the earlier Demonstrate-Search-Predict (DSP) research project and has become the reference implementation for treating prompt engineering as a compilation problem rather than a manual craft.

What differentiates DSPy is its teleprompters: optimization algorithms (BootstrapFewShot, MIPROv2, GEPA, COPRO, GRPO) that search over instructions, few-shot demonstrations, and even model weights to maximize a program’s score against a metric and dataset, using the built-in Evaluate harness. Adapters (Chat, JSON, XML, TwoStep, BAML) translate the same signature into the right request/response format across more than 100 LM providers via LiteLLM, so a pipeline optimized against one model can be re-compiled for another without rewriting prompts by hand.

What You Get

• Signature and Module abstractions for declaring LM programs as typed Python code instead of prompt strings
• A library of built-in modules (Predict, ChainOfThought, ReAct, ProgramOfThought, BestOfN, Refine, CodeAct) covering common LM reasoning patterns
• Teleprompters/optimizers (BootstrapFewShot, MIPROv2, GEPA, GRPO, COPRO) that automatically tune instructions, demonstrations, and model weights
• An Evaluate harness for scoring pipelines against a dataset and metric function in parallel
• Adapters (Chat, JSON, XML, TwoStep, BAML) that translate signatures into provider-specific formats via LiteLLM, so one program runs across 100+ LM providers
• Async, streaming, and disk-backed caching support for production deployment

Common Use Cases

• Building multi-stage RAG pipelines that retrieve, rerank, and synthesize answers
• Constructing tool-using agent loops (ReAct) that self-correct across multiple LM calls
• Automatically optimizing few-shot prompts and instructions for a task instead of hand-tuning them
• Distilling or fine-tuning a smaller model from a bootstrapped teacher pipeline (BootstrapFinetune, GRPO)
• Running systematic evaluations of an LM pipeline against a labeled dataset and metric

Under The Hood

Architecture DSPy is organized as layered, composable primitives rather than a monolith: dspy/signatures defines the typed input/output contract for an LM call (signature.py, 865 lines), dspy/primitives/module.py provides the base Module/Parameter class that every building block (dspy/predict/predict.py, chain_of_thought.py, react.py, program_of_thought.py) extends, and dspy/adapters (base.py, chat_adapter.py, json_adapter.py, xml_adapter.py, two_step_adapter.py, baml_adapter.py) translates a Signature plus its inputs into the concrete request format a given LM backend expects, then parses the response back into typed outputs. dspy/clients (lm.py, _litellm.py, provider.py, cache.py) wraps the actual model call behind a BaseLM interface backed by LiteLLM, with disk-based caching. On top of this, dspy/teleprompt (bootstrap.py, mipro_optimizer_v2.py, gepa/, grpo.py) implements the optimization layer: it runs a program repeatedly over training examples, mutates prompts/demonstrations/weights, and re-scores via dspy/evaluate. A global dspy.dsp.utils.settings singleton threads configuration (active LM, adapter, caching) through the call stack without every module needing it passed explicitly. Swapping the core Signature or Module base would ripple through every predictor, adapter, and optimizer, since they all depend on that shared contract.

Tech Stack DSPy is a Python 3.10-3.14 package built with setuptools, depending on litellm (>=1.65.8) for unified access to 100+ LM providers, pydantic (>=2.11) for typed signature fields and structured outputs, orjson/json-repair for robust JSON parsing of LM responses, diskcache for on-disk response caching, tenacity and cachetools for retry/caching utilities, and cloudpickle for serializing compiled programs. Optional extras add anthropic, weaviate-client, mcp, langchain_core, optuna, and numpy for specific integrations. The gepa[dspy] optimizer ships as a first-class dependency rather than an extra, signaling how central prompt-evolution optimization is to the project. CI runs via GitHub Actions (run_tests.yml, precommits_check.yml, dependency-range.yml) with a dedicated docs-publish workflow for the MkDocs-based dspy.ai site.

Code Quality The repository ships an extensive test suite (97 test_*.py files under tests/, mirroring the source tree’s adapters/, clients/, predict/, teleprompt/, evaluate/, streaming/, and reliability/ subpackages) using pytest, pytest-mock, pytest-asyncio, and pytest-xdist for parallel runs. Linting is enforced through ruff with an explicit [tool.ruff.lint] rule set (including RUF ruff-specific rules) and per-file ignores for test fixtures, plus pre-commit hooks checked in CI. Error handling is deliberately typed rather than swallowed: dspy/utils/exceptions.py defines a hierarchy of specific errors (LMRateLimitError, LMTimeoutError, LMAuthError, ContextWindowExceededError, etc.) with an is_retryable_lm_error helper, and predict.py sanitizes unsafe LM state keys on load rather than silently accepting them. Naming and module boundaries are consistent (clients, adapters, predict, teleprompt, primitives), and the codebase is close to 100% Python (99.5%) with a small JS helper for a sandboxed code interpreter.

What Makes It Unique While most LLM frameworks treat the prompt as the unit a developer hand-edits, DSPy treats it as a compiled artifact: a program is expressed purely as Signatures and Modules, and a teleprompter (MIPROv2, GEPA, BootstrapFewShot, GRPO) searches over instructions, few-shot demonstrations, and even weights to optimize a chosen metric, with the resulting “compiled” program serializable and reusable. This separates what the program should do from how the prompt achieves it, letting the same program be re-optimized for a different model or re-scored on a different dataset without manual re-prompting. Combined with provider-agnostic adapters and a first-class evaluation harness, DSPy positions prompt engineering as a research problem with reproducible optimization algorithms rather than an ad hoc practice.