universal-pathlib

universal-pathlib extends Python’s standard pathlib API to any filesystem supported by fsspec. Its central class, UPath, behaves like pathlib.Path but transparently targets local disk, cloud object stores (S3, GCS, Azure), HTTP, FTP, in-memory, GitHub, Hugging Face, and many other backends selected by URL scheme.

This lets you write path manipulation and file I/O code once using familiar methods like .exists(), .read_text(), and the / join operator, then run it unchanged against whichever storage backend a URL points to. It is maintained under the fsspec organization and pairs with fsspec extras for backend-specific dependencies.

What You Get

• UPath, a pathlib.Path-compatible class that works across many storage backends
• Scheme-based backend selection (s3://, gs://, az://, http://, memory://, github://, and more) via fsspec
• Familiar pathlib operations: joining with /, .name/.stem/.suffix, .exists(), .read_text(), globbing, and iteration
• One code path that runs unchanged on local disk or remote cloud storage
• Integration with fsspec extras so you add only the backend dependencies you need

Common Use Cases

• Writing storage-agnostic file I/O that runs on local disk in dev and S3/GCS in production
• Manipulating cloud object-store paths with the ergonomics of pathlib
• Building data pipelines that read and write across multiple filesystem backends
• Porting existing pathlib-based code to remote storage with minimal changes

Under The Hood

Architecture The public entry point is the UPath class in the upath package, which subclasses and mirrors the semantics of pathlib/pathlib_abc. When you construct a UPath from a URL, it parses the scheme and binds to the corresponding fsspec AbstractFileSystem, then implements pathlib methods (stat, exists, open, iterdir, glob, joins) by delegating to that filesystem’s operations. Per-scheme behavior lives in dedicated implementation classes so each backend can adapt path semantics as needed.

Tech Stack Pure Python (3.9+) built on filesystem_spec (fsspec) for backend access and on the pathlib-abc design for path semantics. Optional backends (s3fs, gcsfs, adlfs, etc.) are pulled in through fsspec extras. Code is formatted with black and documented via Read the Docs, with example Jupyter notebooks.

Code Quality The project has a broad test suite exercised in CI across supported filesystems and Python versions, a maintained changelog, black formatting, and 30-plus contributors under the fsspec umbrella. Because path semantics differ subtly between backends, much of the code is careful edge-case handling validated by the per-filesystem tests.

API Design The API’s core strength is familiarity: if you know pathlib, you already know UPath. Construction from a URL string is the only new concept, and the standard operators and methods behave as expected. The main friction is remembering to install the right fsspec extra for a given backend, which the docs call out clearly.