DDGS

DDGS (published on PyPI historically as duckduckgo-search, with active development now shipping under the ddgs package name) is a Python metasearch library that queries multiple web search backends — DuckDuckGo, Bing, Brave, Google, Yandex, Yahoo, Mojeek, Startpage, Wikipedia, Grokipedia, and Anna’s Archive — through a single DDGS class, returning normalized results for text, images, videos, news, and books. Each backend is implemented as a pluggable subclass of BaseSearchEngine, and the library dispatches queries across engines concurrently with a ThreadPoolExecutor, then deduplicates and ranks results with a built-in similarity filter.

Beyond the core Python API, the project ships a ddgs CLI (built on Click), an optional FastAPI-based REST API server, and an MCP (Model Context Protocol) server that exposes the same six search functions as tools for LLM agents — making it a common building block for giving AI agents live web search.

What You Get

• A single DDGS class with .text(), .images(), .videos(), .news(), and .books() methods covering five result categories across ten-plus backends
• A pluggable backend architecture (BaseSearchEngine subclasses under ddgs/engines/) for DuckDuckGo, Bing, Brave, Google, Yandex, Yahoo, Mojeek, Startpage, Wikipedia, Grokipedia, and Anna’s Archive
• A ddgs command-line tool for running any search category straight from the terminal
• An optional FastAPI REST server (ddgs api) exposing the same search categories as HTTP endpoints, deployable via the included Dockerfile or docker-compose.yml
• An optional MCP server (ddgs mcp) that exposes text/image/video/news/book search and content extraction as tools for MCP clients like Claude Desktop and Cursor

Common Use Cases

• Giving an LLM agent live web search via the bundled MCP server, without building a search integration from scratch
• Scraping search-engine results for research, OSINT, or content-aggregation tooling
• Standing up a self-hosted search API behind FastAPI for internal tools or dashboards
• Querying multiple search engines in parallel and letting backend="auto" fall back automatically when one engine is rate-limited

Under The Hood

Architecture DDGS uses a clean plugin architecture: BaseSearchEngine (ddgs/base.py) is an abstract generic class defining the contract every backend implements (build_payload, result extraction via XPath, a declared category and provider), and ddgs/engines/__init__.py registers concrete subclasses — one file per engine (bing.py, brave.py, duckduckgo.py, google.py, yandex.py, wikipedia.py, annasarchive.py, and category-specific variants like duckduckgo_images.py) — in an ENGINES lookup. The DDGS façade class (ddgs/ddgs.py) resolves and caches engine instances per category/backend in _get_engines(), fans requests out concurrently with a ThreadPoolExecutor, and merges/dedupes results through a ResultsAggregator (ddgs/results.py) ranked by a SimpleFilterRanker (ddgs/similarity.py). The CLI (cli.py), FastAPI layer (api_server/), and MCP server all sit as thin wrappers over this same DDGS core rather than duplicating logic, so extending to a new engine or delivery surface stays localized.

Tech Stack The project targets Python 3.10+ with a small, deliberate dependency set: click for the CLI, primp — a Rust-backed HTTP client that impersonates real browser TLS/HTTP fingerprints — for making requests, and lxml for HTML parsing and XPath-based result extraction (engines declare items_xpath/elements_xpath as class attributes). Optional extras add fastapi+uvicorn for the REST server and mcp for the Model Context Protocol server. Packaging uses setuptools with a dynamic version sourced from ddgs.__version__, and dev tooling runs ruff (lint + format), mypy (with lxml-stubs and other type stub packages), and pytest/pytest-trio.

Code Quality The codebase is fully type-hinted (generics, ClassVar, a py.typed marker for downstream type-checking) and CI (.github/workflows/python-package.yml) runs ruff check, ruff format --check, and strict mypy on every push across Python 3.10/3.14 and Ubuntu/macOS/Windows, with a retry-on-failure pytest pass. Test coverage itself is thin, though: tests/ddgs_test.py and tests/cli_test.py total under 150 lines and are live-network smoke tests (assert len(results) > 0 against real search engines, with a time.sleep(2) between tests to dodge rate limits) rather than mocked unit tests, so they verify the pipes work end to end but not the extraction logic in isolation.

API Design The public surface is deliberately small: one DDGS class, five verb-shaped methods (text, images, videos, news, books) with a consistent query/region/safesearch/timelimit/max_results/page/backend signature across categories, so DDGS().text("query") is the entire onboarding cost. What differentiates it from a typical single-engine scraper is shipping an MCP server as a first-class extra out of the box — search_text, search_images, search_news, search_videos, search_books, and extract_content are already wired as agent tools, which is an unusually agent-ready design choice for a search-scraping library.