Unsloth

Unsloth is a local AI workbench that started as a fine-tuning library and now ships in three forms: a native desktop app for Windows, macOS and Linux; Unsloth Studio, a browser UI served from your own machine; and Unsloth Core, the importable Python package. All three sit on the same training and inference engine, so a model you fine-tune in the UI is the same one you can drive from a notebook.

The training side covers LoRA, QLoRA, full fine-tuning, pretraining and reinforcement learning including GRPO and DPO, with FP8 and vision RL on consumer cards. The speed and memory claims — roughly twice as fast with a large VRAM reduction — come from hand-written Triton kernels and custom autograd paths rather than from a different training algorithm, so results are meant to match what the standard stack produces.

The running side turns the same machine into a local inference server. It loads GGUF and MLX builds through llama.cpp, serves them over an OpenAI-compatible API, and connects coding agents such as Claude Code and Codex to a local model with a single command. Private web search, deep research, RAG over your own documents, image and video diffusion, and speech models are all handled in the same interface.

Hardware coverage is unusually wide for this category: NVIDIA, AMD, Intel and CPU-only machines, a Vulkan backend, multi-GPU setups, and Windows, Linux, WSL and macOS, with a separate ROCm image for AMD.

What You Get

• A native desktop app for Windows, macOS and Linux, plus a self-hosted web UI and a pip-installable library, all on one engine
• Fine-tuning across LoRA, QLoRA, full fine-tuning, pretraining and reinforcement learning, with FP8 and vision support
• A local inference server for GGUF, MLX, diffusion, speech and embedding models, behind an OpenAI-compatible API
• Export to GGUF, NVFP4 and FP8, with importance-matrix quantisation and direct upload to Hugging Face
• Dataset construction from PDFs, CSVs and DOCX files through Data Recipes
• Remote access over your LAN or a Cloudflare HTTPS tunnel, with server-side tools switchable off

Common Use Cases

• Adapting an open model to a private domain on a single consumer GPU
• Serving a local model behind an OpenAI-compatible endpoint for coding agents
• Quantising a fine-tuned model to GGUF and publishing it to Hugging Face
• Running training and inference on AMD, Intel or CPU-only hardware
• Answering questions over internal documents without sending them to a cloud provider

Under The Hood

Architecture Three delivery surfaces rest on one Python engine: a Rust-based desktop shell, a web UI served by an async Python API, and an importable training library. The API layer is route-oriented with schema-validated request models, and the automation surface deliberately calls those same route functions rather than reimplementing training or export, so there is one code path per capability. Heavy work is isolated by process — export runs in its own subprocess and coexists with training and inference rather than displacing them — and concerns are separated into authentication, storage, session state, model registry, integrations and plugins. The defining decision is that the training core patches the upstream machine-learning stack at import time instead of forking it, which keeps it current with fast-moving libraries but makes upstream releases a standing source of breakage. Working against the layering, several modules have grown to a size where they function as subsystems held in a single file, which is the clearest structural weakness.

Tech Stack Python with FastAPI and Pydantic on the backend, PyTorch underneath, with Triton kernels, bitsandbytes, PEFT, TRL, Transformers, TorchAO and vLLM in the training path, and llama.cpp, MLX, stable-diffusion.cpp and whisper.cpp for local inference. The frontend is React 19 and TypeScript built with Vite, styled with Tailwind and Radix-based components, using TanStack Router and Table, Zustand for state and Dexie for browser-side persistence. The desktop app is Tauri, so the shell is Rust. Tooling is Ruff for Python, Biome and ESLint for the frontend, pre-commit for enforcement, GitHub Actions for continuous integration, and Docker images for both CUDA and ROCm. Cloudflare tunnels provide remote access and Hugging Face Hub handles model distribution.

Code Quality Testing is the strongest part of this codebase. Coverage is extensive and regression-shaped: individual tests are named for the exact failure they pin, with dedicated suites for security, cross-version compatibility, installer behaviour and platform parity. The shared harness is notably careful, pre-loading device detection under a mocked accelerator probe so GPU code paths execute on processor-only runners, isolating compilation caches before the framework loads, and ordering imports to avoid a known dependency-binding failure — each constraint documented with the upstream bug that forced it. Comments throughout explain causes rather than restating code. Type annotations are used consistently on both sides of the stack, linting and formatting are enforced automatically, and continuous integration is comprehensive, covering security auditing, dependency-lock verification, installer malware scanning and clean-machine installation across operating systems. Error handling is explicit, with independent subsystem status gathered so one failing backend degrades rather than blanking the whole response, and credential comparison done in constant time on raw bytes for a documented reason. The counterweight is module size, which makes some areas hard to review.

What Makes It Unique Most projects in this space either wrap the standard training stack or replace it. Unsloth does neither: it patches the upstream libraries as they load, applying an extensive layer of compatibility and performance fixes on import, and pairs that with its own kernels and autograd paths for the memory-critical operations. That choice is what lets it track fast-moving upstream releases while still changing their behaviour. The second distinguishing decision is scope — the same core is delivered as a library, a self-hosted web UI and a signed desktop application carrying prebuilt inference, runtime and language stacks, with installer behaviour verified against operating-system security tooling. Supporting hardware that the surrounding ecosystem largely ignores, and testing accelerator code paths on machines without accelerators, are both unusual engineering commitments.

Self-Hosting

Unsloth is dual-licensed. The core package is Apache-2.0, which allows commercial use, modification and redistribution with no obligation to publish your changes. The Studio UI and its backend are AGPL-3.0, which is the part most self-hosters actually run, and that distinction matters: AGPL’s network clause means that if you modify Studio and offer it to users over a network, you owe those users the modified source. Running it unmodified for yourself or your team triggers nothing. There are no licence keys, no paid tiers and no gated features — a search of the codebase for licence checks and pro-tier flags returns nothing at all.

Operationally this is a workstation-class application before it is a server one. It needs a GPU to be useful for training, substantial disk for model weights and caches, and the install pulls a large prebuilt stack. You carry responsibility for GPU drivers, updates and the storage those model files consume. The remote access paths deserve care: server-side tools are enabled by default, exposing the UI over a network prompts for an administrative password, and the documentation is direct about binding to loopback or using a tunnel rather than publishing ports. Anyone exposing it beyond a trusted network should disable tools and treat the password as a real credential.

Compared with a hosted fine-tuning service, what you give up is elasticity and support. There is no SLA, no managed scaling to larger cards when a job needs them, no managed backups of trained artifacts, and no one to escalate to when a training run fails overnight. What you gain is that models, datasets and documents never leave your hardware, and cost is your electricity and the card you already own rather than per-token or per-GPU-hour billing.