DSPy

DSPy (Declarative Self-improving Python) is a framework from Stanford NLP for building modular AI systems on top of language models. Instead of hand-tuning brittle prompt strings, you write compositional Python code using signatures and modules, and DSPy compiles that into effective prompts — and optionally optimizes the model’s weights.

It provides algorithms (“optimizers”/teleprompters) that automatically improve the prompts and few-shot examples driving your program, whether you’re building simple classifiers, sophisticated RAG pipelines, or multi-step agent loops. DSPy is model-agnostic via LiteLLM and has become one of the most widely used frameworks for structured LLM programming.

What You Get

• Declarative signatures and composable modules (Predict, ChainOfThought, ReAct) for building LLM programs
• Optimizers/teleprompters that automatically improve prompts and few-shot examples from data and a metric
• Model-agnostic LM access through LiteLLM plus adapters, retrievers, and evaluation tooling
• Support for classifiers, RAG pipelines, and agent loops as ordinary, testable Python code

Common Use Cases

• Building RAG pipelines whose prompts are optimized rather than hand-tuned
• Creating agent loops and tool-using LLM programs as modular Python code
• Systematically optimizing prompts/few-shot examples against an evaluation metric

Under The Hood

Architecture — DSPy is organized around a few layers: signatures define typed I/O contracts; predict and primitives implement modules (Predict, ChainOfThought, ReAct) that render signatures into prompts via adapters; clients wrap LMs (through LiteLLM); teleprompt and propose hold the optimizers that search over prompts and few-shot demonstrations; and evaluate/datasets supply the metric-driven feedback loop that optimization runs against. retrievers and streaming round out RAG and incremental-output support.

Tech Stack — Python, built on pydantic v2 for typed signatures, litellm for model-agnostic LM access, plus openai, orjson, diskcache, tenacity, cloudpickle, and gepa for optimization. Packaging is via pyproject/hatch.

Code Quality — A large, very actively maintained project (health 89) with a broad test suite mirroring the package structure (adapters, clients, evaluate, predict, teleprompt) and a conftest-based harness. Modular boundaries keep the many subsystems independently testable.

API Design — The programming model is distinctive and powerful: declaring a signature and composing modules reads like ordinary Python, and swapping an optimizer in requires little code change. The learning curve is real — the declarative/optimizer mental model differs from prompt engineering — but the docs at dspy.ai and extensive examples soften it.