langchain-ibm

langchain-ibm is the officially maintained bridge between LangChain and IBM’s watsonx.ai foundation-model platform, wrapping the ibm-watsonx-ai SDK so watsonx models slot directly into LangChain’s chat, embedding, and reranking abstractions. It ships four primary integration points: ChatWatsonx for chat-style inference with full tool-calling and streaming support, WatsonxEmbeddings for vector generation, WatsonxRerank for reordering retrieved documents by relevance, and WatsonxLLM for classic text-completion models — plus a WatsonxToolkit/WatsonxTool agent-toolkit layer and a SQL agent toolkit built on IBM’s SQL query service.

Under the hood it normalizes credentials across IBM Cloud, watsonx.ai, and Cloud Pak for Data authentication paths, translates LangChain’s message and tool-call schemas into the shapes watsonx’s Gateway and native APIs expect, and repairs malformed JSON tool arguments returned by models via json-repair. Because it targets one named external platform rather than exposing a general-purpose capability, it functions as an SDK layer within the broader LangChain ecosystem — teams already building on LangChain use it as a drop-in provider swap to run the same chains, agents, and retrieval pipelines against watsonx-hosted models instead of OpenAI or Anthropic.

What You Get

• ChatWatsonx — a LangChain-native chat model with streaming, async, and OpenAI-style tool/function calling against watsonx.ai and the watsonx Gateway
• WatsonxEmbeddings — vector embedding generation for retrieval and semantic search pipelines
• WatsonxRerank — a cross-encoder reranker for reordering retrieved documents by relevance to a query
• WatsonxLLM — classic completion-style text generation for non-chat foundation models
• WatsonxToolkit/WatsonxTool agent toolkit for exposing IBM-hosted tools to LangChain agents, plus a dedicated SQL agent toolkit for querying databases through watsonx
• Unified credential handling across IBM Cloud API keys, watsonx.ai project/space IDs, and Cloud Pak for Data (CP4D) username/password auth

Common Use Cases

• Swapping a LangChain app’s model provider to watsonx-hosted foundation models without rewriting chains or agents
• Building RAG pipelines that use WatsonxEmbeddings for indexing and WatsonxRerank to improve retrieval precision before generation
• Running tool-calling agents against IBM’s Granite or partner models through ChatWatsonx’s OpenAI-compatible tool interface
• Natural-language-to-SQL agents that query relational databases via the langchain-ibm SQL agent toolkit
• Enterprises standardized on IBM Cloud/CP4D that need LangChain compatibility without leaving IBM’s governance and deployment boundary

Under The Hood

Architecture The package is organized as one of several libraries in the langchain-ai/langchain-ibm monorepo, structured under libs/ibm/langchain_ibm/ with one module per LangChain abstraction: chat_models.py, embeddings.py, llms.py, and rerank.py, plus an agent_toolkits/ subpackage split into utility (the general WatsonxToolkit/WatsonxTool) and sql (a dedicated SQL agent toolkit built on a utilities/sql_database.py wrapper). Each model class subclasses the matching LangChain Core base class (BaseChatModel, embeddings/reranker interfaces) and delegates the actual inference call to ibm_watsonx_ai’s ModelInference/Gateway clients, with a shared utils.py handling credential resolution, parameter extraction, and translation between LangChain’s message/tool schemas and watsonx’s request format — a thin, focused adapter layer rather than an independent execution engine.

Tech Stack Built for Python 3.10–3.14 using PDM as the build backend, with langchain-core (>=1.3.3) and ibm-watsonx-ai (>=1.3.37, or >=1.5.13 on 3.14) as the core runtime dependencies, json-repair for salvaging malformed model-emitted tool arguments, and an optional pyarrow-backed sql_toolkit extra. Dependency and dev tooling runs through uv, with ruff for lint/format (full ALL ruleset with targeted ignores) and mypy --strict for type-checking, including a follow-untyped-imports override for the upstream ibm_watsonx_ai package.

Code Quality Tests are split into tests/unit_tests (mocked APIClient/ModelInference/Gateway objects, run on every PR) and tests/integration_tests (real watsonx credentials, run daily), covering chat models, embeddings, rerank, the toolkit, and SQL utilities, including LangChain’s own standard compliance test suites (test_chat_models_standard.py, test_embeddings_standard.py, test_tools_standard.py). CI runs lint, type-check, unit tests, and codespell via reusable GitHub Actions workflows, and pyproject.toml carries explicit CVE-driven constraint pins (urllib3, requests, langsmith) with dated comments explaining each — a level of dependency hygiene beyond most integration packages of this size.

API Design The public surface mirrors LangChain’s own model conventions closely — ChatWatsonx, WatsonxEmbeddings, WatsonxLLM, and WatsonxRerank all accept the familiar model_id/url/project_id/params constructor shape and drop into existing chains with no special-casing, so developers already fluent in LangChain’s provider pattern face almost no new API surface to learn. The main friction is IBM-specific: users must supply a project_id or space_id and a region-specific url, and deprecated top-level imports (WatsonxTool, WatsonxToolkit) emit clear deprecation warnings pointing at the new agent_toolkits.utility path rather than breaking silently.