pybase64

pybase64 wraps libbase64, a widely used C library, to give Python a Base64 codec that runs many times faster than the standard library’s base64 module. It automatically selects the best available SIMD instruction set at runtime — SSSE3, SSE4.1/4.2, AVX2, or AVX512-VBMI on x86, NEON on ARM — and falls back to a pure-Python implementation transparently when no compiled extension is available, so the same code works everywhere from CPython to PyPy, GraalPy, and WebAssembly builds.

The public API mirrors Python’s built-in base64 module (b64encode, b64decode, standard_b64encode, urlsafe_b64decode, and friends), making it a near drop-in replacement. Recent releases layer on options the stdlib version lacks — configurable padding, character validation via ignorechars, and a wrapcol line-wrap parameter — while a bundled pybase64 command-line tool provides encode, decode, and benchmark subcommands for quick terminal use.

What You Get

• Drop-in base64-module-compatible API (b64encode, b64decode, standard_b64encode, urlsafe_b64decode, encodebytes)
• Runtime SIMD dispatch (SSSE3/SSE4.1/SSE4.2/AVX2/AVX512-VBMI/NEON) with automatic pure-Python fallback when no extension is available
• A pybase64 CLI with encode, decode, and benchmark subcommands
• Prebuilt wheels across an unusually wide platform matrix — CPython 3.9-3.15, free-threaded builds, PyPy, GraalPy, Android, iOS — each with an embedded SBOM

Common Use Cases

• Encoding binary payloads (images, files, tokens) for JSON/HTTP transport in latency-sensitive services
• Speeding up base64-heavy data pipelines (ETL, logging, message queues) without changing calling code
• Decoding large volumes of URL-safe base64 tokens (JWTs, signed URLs) with strict validation
• Benchmarking base64 throughput across SIMD levels via the bundled CLI before adopting the dependency

Under The Hood

Architecture pybase64’s __init__.py first tries to import compiled symbols from pybase64._pybase64 (a C extension built from src/pybase64/_pybase64.c, which wraps the vendored libbase64 git submodule and dispatches SIMD codepaths at runtime via _pybase64_get_simd_flags.c/h); on ImportError — e.g. on a platform where the extension wasn’t built — it falls back to _fallback.py, a pure-Python reimplementation of the same functions layered on the stdlib’s binascii/base64. Public convenience wrappers such as standard_b64encode and urlsafe_b64decode in __init__.py simply forward to whichever backend was selected, so callers never see which path is active; get_version() exposes that detail on request. It’s a clean adapter/strategy pattern: one API, two interchangeable implementations chosen automatically.

Tech Stack The project is pure Python 3.9+ at the interface layer with an optional C extension built via setuptools (pyproject.toml + setup.py) against the vendored aklomp/base64 submodule. Wheels are produced with cibuildwheel across an extensive matrix — manylinux/musllinux (x86, ARM, RISC-V, ppc64le, s390x), macOS via delocate, Windows including ARM64, MSYS2, iOS, Android, PyPy, and GraalPy. Dependency and dev-tooling management runs through uv (a committed uv.lock and dependency-groups in pyproject.toml), tests run under pytest with pytest-codspeed for benchmarking, C coverage is gathered with coverage.py + gcovr, and docs are built with Sphinx (furo theme) on Read the Docs. The published package itself has zero runtime dependencies.

Code Quality tests/test_pybase64.py (over 1,000 lines) plus test_main.py, test_packaging.py, and test_benchmark.py form an extensive parametrized pytest suite exercising both the C-extension and fallback backends, run under pytest-run-parallel for thread-safety checks on 3.13+. ruff is configured with lint.select = ["ALL"] and mypy runs in strict mode (disallow_any_explicit, disallow_untyped_defs, warn_unreachable, and more) across three Python versions via pre-commit; separate CI workflows (test.yml, coverage.yml, sanitizers.yml, msys2.yml, benchmark.yml) run sanitizer builds, cross-platform tests, and performance regression checks, and zizmor lints the GitHub Actions workflows themselves for security issues. This is unusually rigorous engineering for a small utility library.

API Design The library is close to a zero-friction adoption: swap the import base64 statement for import pybase64 and existing b64encode/b64decode calls keep working, since the function names and default behavior mirror the stdlib module. Newer keyword-only parameters (padded, ignorechars, canonical, wrapcol) extend behavior without breaking existing call sites, type stubs (_pybase64.pyi, a py.typed marker) give full IDE/mypy support, and get_version()/get_license_text() helpers make it easy to confirm which SIMD path is active in production. The bundled CLI mirrors the library’s own options for ad-hoc encode/decode/benchmark use from the terminal.