AssemblyAI Python SDK

The AssemblyAI Python SDK wraps AssemblyAI’s speech-to-text and audio understanding API in a typed Python client. It ships three transcription paths behind a consistent interface: a polling Transcriber for long-form prerecorded audio and audio-intelligence features (speaker labels, chapters, sentiment, entity detection, content safety, IAB topic categorization), a SyncTranscriber for short clips that returns a transcript in a single low-latency request, and a RealTimeTranscriber that streams live audio over a websocket for microphones, telephony, and voice agents.

Every transcriber has a matching asyncio counterpart built on a dedicated AsyncClient, so synchronous scripts and asyncio services use the same request/response models and configuration surface. Configuration is centralized through a global aai.settings object and a TranscriptionConfig/SyncTranscriptionConfig object per call, both backed by Pydantic with dual support for Pydantic v1 and v2. The project is tested across a matrix of dependency floors and ceilings (httpx, websockets, pydantic) via tox, in addition to mypy type-checking and ruff linting in CI.

What You Get

• A typed Python client (ships py.typed) for AssemblyAI’s prerecorded, sync, and real-time streaming transcription APIs.
• Matched sync/async classes for every transcription mode (Transcriber/AsyncTranscriber, SyncTranscriber/AsyncSyncTranscriber, RealTimeTranscriber/AsyncRealTimeTranscriber), sharing a common Client/AsyncClient HTTP layer.
• Pydantic-backed configuration and response models (TranscriptionConfig, Transcript, TranscriptGroup, SyncTranscriptResponse, streaming models.py) with dual Pydantic v1/v2 compatibility.
• Typed request/response models for audio-intelligence add-ons: speaker labels, auto chapters, sentiment analysis, auto-highlights, entity detection, content safety, IAB categories, and PII redaction.
• A dedicated migration guide (MIGRATION.md) documenting the breaking changes from the pre-1.0 API surface.

Common Use Cases

• Transcribing recorded meetings and support calls into searchable text with speaker labels.
• Building voice agents and telephony pipelines that need live audio converted to text mid-call with low latency.
• Flagging risky user-uploaded audio/video on media platforms using content-safety and topic-detection results.
• Generating chapters, highlights, and PII-redacted transcripts for podcast and video captioning tools.

Under The Hood

Architecture The SDK is organized as three versioned API surfaces (prerecorded/v2, sync/v1, streaming/v3), each pairing a synchronous and asyncio implementation against shared _base.py request-building logic, so adding a transport mode never duplicates the audio-intelligence data model. A single client.py/async_client.py pair owns the underlying httpx.Client/httpx.AsyncClient pool, header construction, and connection limits (_build_headers, _build_limits are shared free functions, not duplicated per transcriber), while transcriber.py and sync_api.py re-export the versioned classes as top-level shortcuts (aai.Transcriber) for backward compatibility. types.py centralizes the Pydantic config and response models that every version reads from and writes to, so switching between prerecorded, sync, and streaming does not mean switching data shapes.

Tech Stack Built on httpx for both sync and async HTTP transport, websockets for the real-time streaming client, and Pydantic (v1.10+ or v2, selected by Python version, with pydantic-settings on newer Pythons) for config and response validation, plus typing-extensions for back-compat typing. Packaging is plain setuptools (setup.py), with a tox.ini matrix that exercises the dependency floors (httpx 0.22/0.24, pydantic 1.10, websockets 11.0) on one Python version and the latest dependency set across Python 3.9-3.12, rather than a full cartesian-product matrix.

Code Quality The test suite is extensive and organized per feature (test_auto_chapters.py, test_content_safety.py, test_streaming.py, test_streaming_async.py, test_streaming_dual_channel.py, test_transcriber_backwards_compat.py, and more), using pytest, pytest-asyncio, pytest-httpx for mocked HTTP, and factory-boy fixtures (factories.py) for building test data, with coverage collected via pytest-cov. CI runs ruff format --check, ruff lint, and mypy on changed files separately from the test matrix, and the package ships a py.typed marker backing its type annotations.

API Design The public API favors a small number of entry points over deep configuration: aai.settings.api_key sets global auth once, TranscriptionConfig captures per-call options, and each transcription mode exposes an identically-shaped sync/async pair (Transcriber/AsyncTranscriber, etc.) so switching from a script to an asyncio service touches only the class name. The extensive README (1,400+ lines) documents every feature with a runnable example, and a dedicated MIGRATION.md walks through the 1.0 breaking changes (removal of LeMUR and audio-capture extras) rather than leaving users to diff changelogs.