Groq Python Library

groq is Groq’s official Python library for its LPU-backed LLM inference API, providing typed synchronous and asynchronous clients for chat completions, audio transcription/translation, batching, and embeddings. It’s generated with Stainless, giving it the same typed, resource-oriented shape as other modern provider SDKs (OpenAI, Anthropic) — full type definitions for request params and response fields, built on httpx and pydantic.

Because it’s maintained directly by Groq and mirrors their REST API 1:1 (documented alongside the library in api.md), it’s the reference way to call Groq’s fast-inference endpoints from Python without hand-writing HTTP requests or response parsing.

What You Get

• Synchronous and asynchronous clients (Groq / AsyncGroq) sharing the same resource-based API surface
• Full type definitions for request parameters and response fields via pydantic, giving IDE autocomplete and validation
• Chat completions, audio transcription/translation, batching, embeddings, and file/model management resources
• Automatic retries with backoff and configurable timeouts built into the underlying httpx-based transport
• A generated api.md reference kept in sync with the client’s method surface

Common Use Cases

• Calling Groq’s low-latency chat completion endpoints from a Python backend or agent framework
• Transcribing or translating audio via Groq’s Whisper-backed audio endpoints
• Batching large volumes of completion requests through the batches resource for cost/throughput efficiency
• Generating embeddings for retrieval or search pipelines backed by Groq-hosted models

Under The Hood

Architecture - The client is generated by Stainless from an OpenAPI-style spec, giving it the now-familiar shape used by OpenAI’s and Anthropic’s official SDKs: src/groq/resources/ groups endpoints by domain (chat/, audio/, batches.py, embeddings.py, files.py, models.py), src/groq/types/ holds the corresponding pydantic request/response models, and src/groq/lib/ and _utils/ provide shared transport, streaming, and serialization helpers consumed by every resource. Both the sync Groq and async AsyncGroq clients are generated from the same resource definitions, so the two stay behaviorally identical.

Tech Stack - Python 3.10+ built on httpx for transport (supporting both sync and async), pydantic (1.9–3.x) for typed models, anyio for async-runtime portability, and distro/typing-extensions for smaller platform/typing needs. Dependencies are pinned via lockfiles (requirements.lock, requirements-dev.lock) and releases are automated through release-please-config.json.

Code Quality - tests/ contains 28 test files exercising the resource clients, request building, and error handling, consistent with the rigor typical of Stainless-generated SDKs. Because the code is generated rather than hand-written, consistency across resources is very high (identical patterns for pagination, streaming, and error types throughout), though it means idiosyncratic per-endpoint logic is harder to hand-tune outside the generator pipeline.

API Design - The API is deliberately uniform and low-boilerplate: instantiate Groq(api_key=...) once, then call .chat.completions.create(...), .audio.transcriptions.create(...), etc. with typed keyword arguments, and swap to AsyncGroq with an await for async code with no other changes. This resource-oriented, generated-SDK style trades some hand-crafted ergonomics for very fast onboarding for anyone already familiar with OpenAI-style client SDKs.