clarity-upscaler

Clarity AI Upscaler is an open-source AI image enhancement tool that brings commercial-grade upscaling within reach of developers, digital artists, and photographers. It wraps the AUTOMATIC1111 Stable Diffusion WebUI with an opinionated, GPU-accelerated pipeline combining ControlNet tile resampling, Tiled Diffusion, and custom LoRAs to deliver photorealistic detail at resolutions up to 13,000x13,000 pixels — matching or exceeding tools like Magnific at no cost.

The tool supports multiple deployment modes: as a Cog-packaged predictor for cloud platforms like Replicate, as a ComfyUI workflow node, as an AUTOMATIC1111 WebUI extension, or via the hosted API at ClarityAI.co. Flux-based upscaling (available through the hosted app only) extends the system to faces, text, and artistic content using Flux LoRAs. The open-source core remains focused on Stable Diffusion–based workflows.

Under the hood, Clarity AI decomposes large scale factors into multi-step 2x passes, progressively reducing denoising strength to avoid artifacts. Its seamless pattern upscaling algorithm uses a two-pass pixel-shift-and-inpaint method to eliminate tiling seams — a technique not found in standard upscaling tools. The handfix module uses MediaPipe hand detection to surgically crop, upscale, and reinsert hand regions, addressing one of diffusion upscaling’s common failure modes.

What You Get

• 13K Resolution Upscaling - Upscale images to resolutions up to 13,000x13,000 pixels using progressive multi-step 2x passes with Tiled Diffusion to manage VRAM.
• Seamless Pattern Upscaling - Two-pass pixel-shift-and-inpaint algorithm that eliminates tiling seams in repeating textures like fabrics, wallpapers, and backgrounds.
• ControlNet Tile Guidance - Uses the control_v11f1e_sd15_tile ControlNet model to preserve structural resemblance during upscaling, with configurable weight (resemblance parameter).
• LoRA and Custom Checkpoint Support - Download LoRA weights on-the-fly from URLs (including CivitAI) and load custom safetensors checkpoints for personalized upscaling styles.
• Hand Fix Module - Detects hands in images with MediaPipe, crops and upscales them independently, then reinserts the result into the full image to fix common hand distortions.
• Flexible Output Formats - Save results as PNG (lossless), JPEG, or WebP with 95% quality optimization for web or print delivery.
• Mask-Based Selective Upscaling - Apply a mask image to preserve specific areas during upscaling and composite the result back onto the original at original resolution.
• Pre-Downscaling - Optionally resize input images down before upscaling to improve model performance and reduce noise amplification on already-large inputs.
• Cog/Replicate Deployment - Fully packaged Cog predictor for one-command GPU container deployment on Replicate or any Cog-compatible infrastructure.
• ComfyUI Node Integration - Official ComfyUI node available through ComfyUI Manager with API key support for cloud-backed inference.

Common Use Cases

• Restoring archival photography - A museum digitization project uses multi-step upscaling with high resemblance settings to restore and enlarge scanned 35mm film photographs to print-ready 8K resolution.
• Generating gallery-quality art prints - A digital artist upscales AI-generated 1024px paintings to 13K for large-format gallery prints using custom fine-tuned LoRAs to maintain stylistic consistency.
• E-commerce product image enhancement - An online retailer integrates the Cog API into their image processing pipeline to automatically upscale supplier-provided low-resolution product photos to meet marketplace requirements.
• Seamless texture creation for 3D artists - A game developer uses the pattern upscaling mode to upscale tileable PBR textures without introducing seams that would be visible on 3D surfaces.
• Portrait and headshot enhancement - A portrait photographer combines the handfix module with full-image upscaling to produce clean, high-resolution headshots from camera RAW exports.
• Anime and illustration upscaling - A content creator uses the flat2DAnimerge checkpoint with appropriate LoRAs to upscale manga panels and fan art to 4K resolution for digital display and printing.

Under The Hood

Architecture Clarity AI Upscaler functions as an orchestration layer over the AUTOMATIC1111 Stable Diffusion WebUI, exposing its inference pipeline through a Cog Predictor class interface. The architecture cleanly separates initialization (model loading, extension wiring via callback hooks), prediction orchestration (multi-step scale factor decomposition into sequential 2x passes), and payload construction (tiled diffusion, ControlNet, and VAE parameters assembled as pure functions returning dictionaries). Extension integration relies on the script_callbacks system inherited from AUTOMATIC1111, coupling the tool to upstream global state rather than explicit dependency injection. Data flows from file path input through base64 encoding to a JSON API payload, then to the Stable Diffusion WebUI img2img endpoint, and finally through PIL post-processing to file output. The pattern upscaling path adds a secondary loop with pixel-shifting and inpainting mask generation to heal tiling seams, which is compositionally sensible but adds branching complexity to the main prediction method.

Tech Stack Powered by Python 3.10 with PyTorch 2.0 and CUDA-accelerated GPU inference, with xformers providing memory-efficient attention optimization. The inference backend is the AUTOMATIC1111 Stable Diffusion WebUI, accessed via its internal FastAPI REST interface. Upscaling combines ControlNet tile resampling, the Tiled Diffusion MultiDiffusion algorithm, and Tiled VAE for VRAM-efficient large-image processing. Model assets include RealESRGAN (4x-UltraSharp) for initial enhancement and GFPGAN for face restoration. Cog handles GPU environment packaging and provides the Predictor interface for Replicate deployment. Weight downloads use pget for fast parallel checkpoint fetching. Computer vision post-processing uses OpenCV and Pillow, while MediaPipe handles hand detection in the handfix module. Ruff handles Python linting and pytest covers integration tests.

Code Quality The project includes an integration test suite using pytest that exercises the Stable Diffusion WebUI API endpoints with HTTP requests to a running server, verifying status codes. Test fixtures are cleanly structured with session-scoped reuse and clear separation between image encoding utilities and endpoint tests. However, no unit tests exist for the core upscaling logic in the predictor module itself, and error handling is limited throughout — the main prediction flow has no try/except blocks, leaving failures as unhandled exceptions. Type annotations are absent from the primary codebase, reducing static analysis effectiveness. The Ruff linter is configured but with complexity and import-sort rules disabled. The pattern upscaling section contains abundant commented-out debug code, and inline documentation is sparse across the codebase.

What Makes It Unique The most distinctive technical contribution is the seamless pattern upscaling algorithm: a two-pass pixel-shift-and-inpaint approach that eliminates tiling artifacts in repeating textures. The first pass shifts pixels 50% diagonally and inpaints the center cross seam region using a dynamically generated cross-shaped mask; the second pass offsets 33% and inpaints the remaining edge seams with reduced creativity to maintain coherence. This multi-pass seam healing is not found in standard upscaling tools. The hand fixup pipeline using MediaPipe detection, surgical crop-upscale-reinsert, and optional Gaussian-blurred mask compositing addresses a common failure mode in diffusion-based portrait upscaling. The multi-step scale factor decomposition with progressive denoising strength reduction produces demonstrably better output quality versus single-pass approaches at equivalent scale factors.

Self-Hosting

Clarity AI Upscaler is released under the GNU Affero General Public License v3.0 (AGPL-3.0). This is a strong copyleft license that allows you to run, study, modify, and redistribute the software freely — including for commercial purposes — provided that any modifications to the source code are also released under the same license. Critically, AGPL-3.0 extends copyleft obligations to network use: if you run a modified version of this software as a service over a network, you must make the modified source code available to users of that service. Self-hosters using the software privately or within their own organization without exposing it as a public service face no additional obligations.

Running Clarity AI yourself requires substantial infrastructure: a modern NVIDIA GPU with at least 8GB of VRAM (more for 13K outputs), CUDA drivers, a full Python 3.10 environment with PyTorch, and multiple gigabytes of model weights to download (checkpoints, LoRAs, ControlNet models, embeddings, VAE). Setup is handled via the provided Cog container or manual installation following AUTOMATIC1111 WebUI procedures, both of which involve downloading several third-party models from Hugging Face. There is no automated update mechanism — you are responsible for tracking upstream changes, managing model versions, and ensuring environment stability. GPU memory management for large-image upscaling requires careful parameter tuning (tile sizes, VAE tile sizes) to avoid out-of-memory errors.

The hosted application at ClarityAI.co and the ComfyUI API node offer a managed alternative, removing infrastructure concerns entirely and providing access to Flux-based upscaling (which is not open-source and not available for self-hosting). The hosted service handles model updates, GPU provisioning, and scale automatically. There is no formal SLA, enterprise support tier, or high-availability guarantee mentioned for either the self-hosted or hosted variants. Developers integrating Clarity AI into production pipelines should plan for operational overhead around GPU provisioning, model management, and AGPL compliance review if they intend to modify and distribute the code or run it as a public service.