kornia

Kornia is an open-source differentiable computer vision library built directly on top of PyTorch. Instead of wrapping OpenCV or PIL, every operation — filtering, geometric transforms, color conversion, augmentation, feature detection, and camera/epipolar geometry — is implemented as a native, autograd-compatible tensor operation, so gradients flow through the entire vision pipeline end to end.

The library covers a wide surface area: over 500 operators spanning classic image processing (Gaussian/Sobel/median filters, histogram equalization, morphology), a full augmentation pipeline (AugmentationSequential, VideoSequential, AutoAugment/RandAugment), 3D and projective geometry (homographies, epipolar geometry, camera calibration, pose estimation), and research-grade feature matching models (LoFTR, LightGlue, DISK, DeDoDe) alongside pretrained models like SAM and YuNet.

Kornia is maintained with unusual rigor for a vision library — it tracks per-module float16/bfloat16 numerical conformance against a float32 baseline directly in its README, runs a dedicated API-surface snapshot test, and states in its own roadmap that it is pursuing “reference implementation and executable specification” status for differentiable vision in the PyTorch ecosystem.

What You Get

• 500+ differentiable vision operators covering filters, color conversion, geometric transforms, and morphology — all GPU-accelerated PyTorch tensor ops
• A full augmentation pipeline (AugmentationSequential, VideoSequential, PatchSequential) with AutoAugment, RandAugment, and TrivialAugment policies built in
• 3D and projective geometry: homography and essential/fundamental matrix estimation, camera calibration, epipolar geometry, PnP solvers, and Lie-group utilities
• Pretrained research models exposed through a consistent tensor API — LoFTR and LightGlue for feature matching, DISK/DeDoDe descriptors, SAM for segmentation, YuNet for face detection
• Cross-framework export via the transpiler module (to_jax, to_numpy, to_tensorflow using ivy) and ONNX export support

Common Use Cases

• Building end-to-end trainable data augmentation pipelines that run on-GPU inside the training loop instead of on CPU via a DataLoader
• Adding differentiable geometric losses (e.g. reprojection or homography error) that need gradients to flow back through the transform itself
• Classic image-processing preprocessing (blur, edge detection, color-space conversion) inside a PyTorch model without a NumPy/OpenCV round-trip
• Feature matching and pose/camera estimation for structure-from-motion, SLAM, or registration research built on PyTorch tensors

Under The Hood

Architecture Kornia is organized as a set of domain-specific subpackages under kornia/ (filters, geometry, augmentation, color, feature, losses, metrics, models, morphology, onnx, contrib, sensors, tracking, transpiler), sitting on top of a kornia/core layer that provides shared primitives (module.py, ops.py, check.py, tensor_wrapper.py). kornia/__init__.py explicitly imports filters and geometry before any other subpackage, with an inline comment warning that changing the order risks circular imports — evidence of a layered dependency graph where the core and geometry/filters modules sit beneath everything else. Most public functions are pure, differentiable tensor-in/tensor-out functions (e.g. gaussian_blur2d in kornia/filters/gaussian.py) rather than stateful classes, with nn.Module wrappers layered on top for use inside training pipelines; the augmentation subpackage builds a container/pipeline system (kornia/augmentation/container, random_generator) directly on top of the geometry transform primitives, so a change to the core transform layer would ripple through augmentation, feature, and losses.

Tech Stack Python 3.11+, built on PyTorch >=2.0.0 as the tensor/autograd engine, with kornia_rs (a Rust extension, >=0.1.9) handling performance-critical IO and image operations, plus NumPy for interop. Packaged with hatchling via PEP 621 pyproject.toml with dynamic versioning; development environments are reproduced with both pixi (pixi.toml/pixi.lock, including a CUDA environment) and uv (uv.lock). Optional extras pull in diffusers, transformers, onnx/onnxruntime/onnxscript for ONNX export, ivy for the cross-framework transpiler, and opencv-python/matplotlib for docs and examples. Documentation is built with Sphinx (furo theme, autodoc, typehints) and hosted on ReadTheDocs. CI on GitHub Actions is split into dedicated lint, typecheck, coverage, doc-links, CPU test, scheduled test, and PyPI-release workflows.

Code Quality The tests/ tree mirrors the kornia/ module layout with 259 test files, plus a dedicated smoke_test.py, an API-surface snapshot test (test_api_surface.py against api_surface.json) that catches accidental public-API changes, and test_import.py. Numerical precision is treated as a first-class quality concern: the README publishes per-module float16/bfloat16 pass-rate tables measured against a float32 CPU/CUDA baseline, refreshed by hand outside CI. Ruff is configured for both linting and formatting (with dedicated isort, pydocstyle, mccabe, and pylint sections in pyproject.toml), a separate typecheck.yml workflow runs static type checking, and pre-commit hooks enforce a license-header check on every file. Functions consistently use from __future__ import annotations, full type hints, and explicit runtime guards through kornia.core.check (KORNIA_CHECK, KORNIA_CHECK_SHAPE) rather than letting shape mismatches fail silently deep in a computation.

What Makes It Unique Kornia’s defining choice is implementing every vision operation — filters, geometric transforms, augmentations, feature detectors, camera and epipolar geometry, and losses — as a native differentiable PyTorch operation rather than wrapping OpenCV or PIL, so gradients flow through an entire vision pipeline end to end (enabling things like a learned augmentation policy or camera calibration trained jointly with the model it feeds). It packages research-grade feature matchers (LoFTR, LightGlue, DISK, DeDoDe) and pretrained models (SAM, YuNet, MobileViT) behind one consistent tensor API, and its transpiler module lets the same differentiable ops run outside PyTorch entirely (JAX, NumPy, TensorFlow, via ivy). The project’s own roadmap frames it as aiming for “reference implementation and executable specification” status for differentiable vision — explicit conformance tests and published numerical-precision guarantees ahead of raw feature growth, a level of rigor that is unusual for a vision library.