minio-py

minio-py is MinIO’s officially maintained Python SDK for Amazon S3 compatible object storage. It wraps the full S3 API surface — buckets, objects, multipart uploads, presigned URLs, versioning, replication, lifecycle rules, object lock, and server-side encryption — behind a single synchronous Minio client class with nearly 100 public methods.

Because it speaks the S3 protocol rather than a MinIO-proprietary one, the same client works unmodified against MinIO servers, AWS S3, and any other S3-compatible endpoint, which makes it a common building block for self-hosted storage backends in Python applications.

The library is dependency-light (urllib3, certifi, argon2-cffi, pycryptodome) and ships full type hints via a py.typed marker, with an opt-in RDMA transport (via a bundled minio-cpp/libs3rdma bridge) for high-throughput GPU and HPC workloads that need to bypass the normal HTTP path.

What You Get

• A synchronous Minio client covering the full S3 surface: buckets, objects, multipart uploads, copy/compose, and stat/list operations
• Presigned URL generation for GET/PUT/POST so browsers or other services can upload and download directly without proxying through your app
• Bucket and object policy management: versioning, lifecycle rules, object lock/retention, replication, and notification configuration
• Server-side encryption helpers (SSE-C, SSE-KMS, SSE-S3) and checksum/hash utilities built on pycryptodome
• Pluggable credential providers (static keys, environment variables, IAM/STS, chained providers) under minio.credentials
• An opt-in RDMA path (via libminiocpp.so) for put_object/get_object that accepts GPU buffer pointers directly, for HPC/AI workloads

Common Use Cases

• Self-hosting an S3-compatible object store behind a Python backend for user uploads, backups, or generated artifacts
• Generating presigned upload/download URLs so a frontend can transfer files directly to storage without routing bytes through the API server
• Migrating an existing boto3/S3 integration to a self-hosted MinIO cluster with minimal API-shape changes
• Building data pipelines that read/write large objects (datasets, model checkpoints) with optional RDMA acceleration for GPU-resident buffers
• Enforcing object lock, retention, and lifecycle policies for compliance-sensitive storage from application code

Under The Hood

Architecture The library is organized as a flat, focused package: minio.py defines the ~99-method Minio client that is the sole entry point for API calls, while cross-cutting concerns are split into dedicated modules — signer.py for AWS SigV4 request signing, credentials/ for a pluggable provider chain (static keys, env vars, IAM/STS, chaining), xml.py for request/response (de)serialization against S3’s XML wire format, models.py for typed response objects, and rdma.py as an isolated, strictly opt-in bridge into a native libminiocpp.so/libs3rdma transport for GPU/HPC buffer transfer. This keeps the common HTTP+XML path fully pure-Python while letting the RDMA path degrade gracefully to a no-op if the native library isn’t present, and means changing the core client would ripple through every module since they all key off the same request-building conventions in helpers.py and args.py. Tech Stack The SDK depends on urllib3 directly for HTTP transport (including its Retry and Timeout utilities) rather than a higher-level client like requests, plus certifi for CA bundles, argon2-cffi and pycryptodome for cryptographic operations (checksums, SSE-C key handling), and typing-extensions for compatibility typing on older Python versions. It targets Python 3.10+ (including free-threaded 3.14t) and ships no async variant — everything is synchronous, blocking I/O through urllib3’s connection pooling. Code Quality The repository carries parallel unit (tests/unit/) and functional (tests/functional/) test suites — the former mocking HTTP responses per-operation, the latter running against a real MinIO server spun up in CI — and a make check target that chains pylint, isort --diff, autopep8 --diff, and mypy before any test runs, with the CI matrix covering Python 3.10 through 3.14t across Linux, macOS, and Windows. Exceptions are modeled as a typed hierarchy rooted at MinioException (e.g. InvalidResponseError, ServerError) rather than bare exceptions, and the package ships a py.typed marker with mypy enforced in CI, so the public API is meant to be consumed with full type checking on the caller’s side. What Makes It Unique Unlike most S3 SDKs, minio-py ships an explicit, opt-in RDMA transport for put_object/get_object that accepts raw buffer-protocol objects — including GPU device pointers from CuPy or PyTorch — as data=/into= arguments, dispatching to a native libminiocpp.so bridge over RoCE or InfiniBand when present and falling back to the normal HTTP path otherwise. This lets AI/HPC workloads move large tensors or checkpoints in and out of object storage without an intermediate host-memory copy, a capability generalist S3 clients like boto3 don’t offer.