LangSmith Python SDK

langsmith is the official Python SDK for LangSmith, LangChain’s platform for tracing, evaluating, and monitoring LLM applications in production. It ships a @traceable decorator and a RunTree API for capturing nested execution traces from any Python code — with or without LangChain — plus wrappers that auto-instrument OpenAI, Anthropic, and Gemini clients.

Beyond tracing, the SDK covers the full evaluation lifecycle: building datasets from captured runs, running LLM-graded and custom evaluators, and a pytest plugin for expressing evals as ordinary test cases. Recent releases add a sandboxed code-execution client with AWS/GCP auth proxies and filesystem mounts, plus OpenTelemetry-based instrumentation for LiveKit and Pipecat voice agents and native support for the OpenAI Agents SDK, Google ADK, and Strands Agents.

What You Get

• @traceable decorator and RunTree API for capturing nested execution traces from arbitrary Python functions
• Pre-built wrappers (wrap_openai, wrap_anthropic, wrap_gemini) that auto-instrument popular LLM SDK clients
• An evaluation framework for turning production runs into datasets and running LLM-graded or custom evaluators
• A pytest plugin (langsmith[pytest]) for writing evals as ordinary test functions with automatic result logging
• A SandboxClient for running untrusted or agent-generated code with AWS/GCP auth proxies, workspace secrets, and filesystem mounts
• An async client (AsyncClient) and OpenTelemetry-based integrations for LiveKit, Pipecat, and other voice-agent stacks

Common Use Cases

• Debugging a LangChain or raw-Python agent by inspecting nested traces of every LLM call, tool call, and intermediate output
• Building regression test suites for prompts and agents using datasets exported from real production traces
• Running LLM-as-judge evaluations to score groundedness, correctness, or custom criteria across model or prompt changes
• Auto-instrumenting OpenAI, Anthropic, or Gemini calls in an existing codebase with a one-line client wrapper
• Giving an agent a sandboxed execution environment with scoped cloud credentials via the AWS/GCP auth proxy

Under The Hood

Architecture The SDK layers around a central Client (langsmith/client.py, ~11.8k lines) that wraps LangSmith’s REST/OpenAPI-generated _openapi_client and owns background-threaded batching and compression for run submission (_internal/_background_thread.py, _compressed_traces.py, _multipart.py), so tracing calls never block the caller’s hot path. Above the client sits run_trees.py’s RunTree, a tree-structured run model with post/patch/end lifecycle methods that the @traceable decorator (run_helpers.py, ~2.3k lines) builds on top of using contextvars to propagate parent-run context across sync, async, and generator call boundaries transparently. Framework-specific behavior (wrappers/_openai.py, _anthropic.py, _gemini.py; integrations/livekit, pipecat, google_adk, openai_agents_sdk) is isolated into swappable adapter modules that wrap or subclass a target SDK’s client rather than touching the core tracing path, and evaluation/ (_runner.py, _arunner.py, evaluator.py) is a separate subsystem built on the same Client for reading and writing datasets. Changing the core RunTree/run-lifecycle contract would ripple through every wrapper and integration module since they all assume the same post/patch/end shape.

Tech Stack Built for Python 3.10+ with hatchling as the build backend and pydantic 2.x for all schema models (schemas.py, ~1.7k lines). Networking runs through both requests (sync) and httpx (async AsyncClient), with requests-toolbelt for multipart run uploads, zstandard for trace compression, and orjson for fast JSON serialization outside PyPy. Run and example IDs use uuid-utils and hashing uses xxhash; websockets backs a live-tail feature. Optional extras add opentelemetry-sdk/api for OTel-based tracing (used by the LiveKit, Pipecat, and otel integrations), a langsmith-pyo3 Rust extension, and a pytest plugin (registered via the pytest11 entry point) for eval-as-test workflows. The repository is a polyglot monorepo — this pypi package lives under python/ alongside a parallel js/ TypeScript SDK sharing the same OpenAPI-generated client contract.

Code Quality Testing is extensive: over a hundred test files across tests/unit_tests, tests/integration_tests, tests/evaluation, and tests/external, using pytest with pytest-asyncio, pytest-xdist for parallelism, pytest-cov, pytest-retry for flaky-network tests, and pytest-socket to enforce that unit tests make no real network calls; HTTP-dependent tests replay through vcrpy cassettes. The package ships a py.typed marker and is type-checked with mypy (pydantic mypy plugin, with the generated _openapi_client/ excluded) plus ruff for lint and format, both gated by GitHub Actions workflows. A dedicated workflow guards against hand-edits to the generated OpenAPI client, and dependabot keeps dependencies current — signaling a codebase treated as a stable public SDK rather than an internal tool.

What Makes It Unique What’s distinctive isn’t a novel algorithm but the breadth of a single, framework-agnostic tracing primitive: @traceable works identically whether wrapping a raw function, a LangChain Runnable, sync/async/generator code, or a third-party SDK client via wrap_openai/wrap_anthropic/wrap_gemini, all propagated through contextvars rather than requiring explicit context threading. The evaluation subsystem closes the loop from trace to test by letting any historical run become a dataset row and any dataset become a pytest test case via the bundled pytest plugin, a tighter tracing-to-testing integration than most competing observability SDKs offer. More recent additions — a SandboxClient for running agent-generated code behind AWS/GCP auth proxies with workspace-secret injection and scoped filesystem mounts, plus OTel-based instrumentation for voice-agent stacks (LiveKit, Pipecat) — extend the same Client and tracing foundation into agent-execution and voice-agent observability rather than staying a pure logging SDK.