clean-fid

clean-fid is a PyTorch library for evaluating generative models by computing the Frechet Inception Distance (FID) and Kernel Inception Distance (KID) between image sets. It was created to fix the subtle but consequential inconsistencies - incorrect anti-aliased resizing and image quantization - that cause FID scores to differ across implementations and make published numbers hard to compare.

It provides one-call functions to score a folder of images against precomputed reference statistics for popular datasets, compare two folders directly, register custom dataset statistics, and compute FID using either the classic InceptionV3 features or newer CLIP features.

What You Get

• compute_fid and compute_kid functions that score a folder of images against precomputed dataset statistics or another folder
• Correct anti-aliased resizing and image quantization that eliminate cross-implementation FID discrepancies
• Support for both InceptionV3 features and CLIP features (clip-fid)
• Tools to register and reuse custom dataset statistics via make_custom_stats
• A leaderboard of reference numbers for common generative-modeling tasks

Common Use Cases

• Benchmarking a GAN or diffusion model against a standard dataset like FFHQ or CIFAR with reproducible FID
• Comparing two folders of generated versus real images directly
• Registering custom reference statistics for a private dataset and scoring against it repeatedly
• Reporting FID/KID in a paper using a consistent, widely accepted implementation

Under The Hood

Architecture - The core lives in cleanfid/fid.py, which exposes user-facing entry points (compute_fid, compute_kid, make_custom_stats) that pipe image folders through a feature extractor and then a distance calculation. Feature extraction is delegated to features.py plus InceptionV3 backbones (inception_pytorch.py, inception_torchscript.py) or CLIP features (clip_features.py), while resize.py implements the mathematically correct anti-aliased resizing that is the project’s central contribution. downloads_helper.py fetches precomputed reference statistics, and frechet_distance/kernel_distance compute the final metrics.

Tech Stack - Python on top of PyTorch and torchvision for the neural feature extractors, NumPy and SciPy for the Frechet/kernel distance math, Pillow for image loading, tqdm for progress, and OpenAI CLIP for CLIP-FID. Packaged with setuptools.

Code Quality - The codebase is organized by concern with clearly named modules, though test coverage is light (a single device_handling_test.py). The value is in correctness of the resizing and quantization steps rather than breadth of unit tests, and the implementation is backed by a peer-reviewed CVPR paper.

API Design - The public API is deliberately minimal and task-oriented: point compute_fid at a folder and a dataset name and get a score. Sensible defaults (dataset, resolution, feature model) keep the common path to a single line, while keyword arguments expose CLIP features and custom statistics for advanced use.