huggingface_hub

huggingface_hub is the official Python library and command-line client for interacting with the Hugging Face Hub, the platform hosting hundreds of thousands of open models, datasets, and Spaces. It wraps the Hub’s REST API behind a single package that ships both a Python SDK (from huggingface_hub import hf_hub_download) and the hf CLI, giving humans and coding agents the same surface for downloading files, uploading repositories, running inference against deployed models, and managing repos programmatically.

Beyond simple file transfer, the library handles the operational complexity of working with the Hub at scale: chunked, resumable downloads with local caching, LFS/Xet-backed large file uploads, repository and commit management, model card metadata, webhooks, OAuth login flows, and typed clients for Inference Providers and Jobs. It is the dependency that libraries like transformers, diffusers, and datasets build on to talk to the Hub, and is equally usable standalone for any project that needs to read or write Hub-hosted artifacts.

What You Get

• The hf CLI (hf auth login, hf download, hf upload, hf models ls, hf jobs run) for scripting Hub operations from the terminal or CI
• A Python SDK (hf_hub_download, snapshot_download, upload_file, upload_folder, create_repo) for downloading and publishing files and repos in code
• A resumable, chunk-deduplicated local cache (backed by Xet) so repeated downloads of large models and datasets don’t re-fetch unchanged data
• HfApi — a full typed client for repo management, discussions, webhooks, commits, and repository search across models, datasets, and Spaces
• Inference clients (sync and async) for calling models deployed via Inference Providers, plus a Jobs API for running arbitrary workloads on Hugging Face infrastructure
• Model Card and OAuth/OIDC helpers for documenting models and building Hub-authenticated applications

Common Use Cases

• Downloading pretrained model weights or dataset files into a local cache from application or training code
• Publishing a fine-tuned model, dataset, or Space to the Hub from a training script or CI pipeline
• Building a coding agent or CLI tool that needs to search, download, or run inference against Hub-hosted models
• Managing Hub repositories programmatically — creating repos, committing files, handling large uploads via LFS/Xet
• Running Hugging Face-hosted Jobs for remote compute without provisioning separate infrastructure

Under The Hood

Architecture - The package is organized around a thin, functional download/upload layer (file_download.py, _snapshot_download.py, _commit_api.py, _upload_large_folder.py) and a broad HfApi client (hf_api.py, 36k+ lines across the package) that wraps every Hub REST endpoint — repos, discussions, webhooks, collections, papers, Spaces, and Jobs. The cli/ subpackage layers a Click-based hf command over the same SDK calls, so CLI and Python usage share one code path. Inference (inference/) and serialization (serialization/) are kept as separate subpackages so heavier optional dependencies (torch, safetensors) don’t load unless used. Tech Stack - Core runtime dependencies are deliberately minimal: httpx for HTTP, filelock/fsspec for local cache safety, hf-xet for chunk-deduplicated large-file transfer, click for the CLI, and pyyaml/packaging/tqdm as small utilities; heavier integrations (torch, fastai, gradio, mcp, oauth/fastapi) are all pushed into extras, keeping a bare pip install huggingface_hub lightweight. Code Quality - The tests/ directory is extensive (36k+ lines, dozens of test_*.py files covering caching, CLI, commit API, inference, and buckets) with pytest markers to separate Xet-dependent, production-Hub, and deprecated-API tests; ruff and mypy/ty are configured for linting and type-checking, and the codebase makes consistent use of type hints and dataclasses throughout hf_api.py and dataclasses.py. API Design - The public API favors small, single-purpose functions (hf_hub_download, upload_file, create_repo) for common tasks while exposing the full surface through one HfApi class for advanced use, and the same operations are mirrored 1:1 between the Python SDK and the hf CLI, minimizing the learning gap between scripting and terminal use; documentation is hosted separately at huggingface.co/docs/huggingface_hub with dedicated guides per feature area (download, upload, inference, jobs, search).