unstructured-python-client

unstructured-client is the Speakeasy-generated Python SDK for Unstructured’s hosted Platform API, the commercial counterpart to the open-source unstructured library. Instead of running partitioning models locally, it gives Python code a typed, synchronous-or-async HTTP client for submitting documents to Unstructured’s API and getting back structured elements, plus endpoints for building full ETL workflows — connecting sources, destinations, and scheduled jobs that pull unstructured data into vector stores and data pipelines.

Beyond the generated request/response surface, the SDK carries meaningful custom logic on top of the raw API contract: a split_pdf_hook that transparently splits large PDFs into page-range batches, sends them concurrently, and reassembles the response elements in order, plus NDJSON streaming support for the partition endpoint so large documents can be processed without holding the full response in memory. Retry policies, custom HTTP clients, and per-request debugging hooks are all first-class configuration options.

What You Get

• A typed UnstructuredClient with sync and async methods for every Platform API operation (partition, workflows, jobs, sources, destinations, templates, general)
• Automatic large-PDF splitting via split_pdf_hook, which pages a document into concurrent batches server-side calls and reassembles ordered elements transparently
• NDJSON streaming responses for the partition endpoint, writing elements to a temp file instead of buffering the full response in memory
• Configurable retry policies (RetryConfig with backoff strategy) settable globally or per operation
• Pluggable custom HTTP clients (sync HttpClient / async AsyncHttpClient protocols) for swapping in your own httpx configuration, proxies, or instrumentation
• Structured error types (UnstructuredClientError and subclasses) exposing status code, headers, and raw response body

Common Use Cases

• Submitting PDFs, Office documents, or images to the partition endpoint to extract structured elements for a RAG pipeline
• Building an ETL workflow that connects a source (e.g. an S3 bucket or SharePoint site) to a destination (e.g. a vector database) via the Workflow Endpoint
• Processing large, multi-hundred-page PDFs without hitting request size or memory limits, using the SDK’s automatic PDF-splitting hook
• Streaming very large documents as NDJSON instead of parsing a single large JSON payload into memory
• Automating document-connection checks and job monitoring for a production ingestion pipeline via the jobs and destinations sub-clients

Under The Hood

Architecture The SDK follows Speakeasy’s generated-client pattern: sdk.py defines UnstructuredClient, a thin composition root that lazily constructs sub-SDKs (Destinations, Jobs, Sources, Templates, Workflows, General) from a shared SDKConfiguration, each sub-SDK extending BaseSDK for request building, auth injection, and response unmarshalling. What sets this repo apart from a purely generated client is _hooks/custom/, a hook system (BeforeRequestHook, AfterSuccessHook, AfterErrorHook, SDKInitHook) that lets hand-written Python intercept the generated request/response cycle without touching generated files — split_pdf_hook.py is the largest of these, orchestrating PDF page splitting, concurrent sub-requests via a thread pool, and response element reassembly entirely from hook callbacks. Swapping the core abstraction (the hook protocol) would require re-plumbing every custom behavior layered on the generated base.

Tech Stack Python 3.11+, built on httpx for both sync and async HTTP, pydantic v2 for request/response models, pypdf and pypdfium2 for the PDF-splitting hook’s page manipulation, aiofiles for async file I/O in NDJSON streaming, and requests-toolbelt for multipart form encoding. Packaging uses setuptools with a dynamic version pulled from _version.py, and dependency management runs on uv with a committed lockfile.

Code Quality Testing is comprehensive and split by concern: _test_unstructured_client/unit covers request utilities, custom hooks, retries, server URL handling, NDJSON streaming, PDF utilities, and the split-PDF hook’s sync/async behavior in isolation; _test_unstructured_client/integration and _test_contract run against a live or mocked Platform API across three Python versions in CI. The GitHub Actions pipeline runs unit, integration, platform-integration, and contract test jobs plus a dedicated lint job on every push and PR, and mypy type-checking is configured project-wide with pylint as the linter. Generated files carry a Code generated by Speakeasy — DO NOT EDIT header, keeping the human-maintained surface area limited to the _hooks/custom/ directory and tests.

What Makes It Unique Most generated API clients stop at typed request/response wrappers, but this SDK layers meaningful client-side logic onto the generated core: the split-PDF hook transparently works around the Platform API’s per-request document size limits by chunking a PDF into page ranges, dispatching them as concurrent requests, and stitching the returned elements back into original page order — behavior a caller would otherwise have to hand-roll themselves. The NDJSON elements-file mode is a similar accommodation, letting very large partition responses stream to disk rather than forcing a full in-memory JSON parse.