Handy

Handy is a cross-platform desktop application that transcribes your speech entirely on your own machine and pastes the result into whatever text field is active. There are no cloud uploads, no subscriptions, and no data leaving your computer. Press a hotkey, speak, and your words appear — that is the complete workflow.

The application is built on Tauri 2, pairing a React and TypeScript settings UI with a Rust backend that handles audio capture, voice activity detection, and ML inference. It supports multiple transcription engines in parallel: the Whisper family (Small, Medium, Turbo, Large) with GPU acceleration via CUDA, Metal, and ROCm, and the CPU-optimised Parakeet V3 which adds automatic language detection without manual selection. Additional ONNX-based engines include Moonshine, SenseVoice, GigaAM for Russian, Canary, and Cohere.

After transcription, an optional post-processing step lets you refine the raw text through any OpenAI-compatible API — including local models — using configurable LLM prompts. Custom word dictionaries let you correct proper nouns and technical terms that models frequently misrecognise. The result is a tool that fits into accessibility workflows, note-taking pipelines, and developer automation with equal ease.

Handy is explicitly designed to be the most forkable speech-to-text application. The codebase prioritises clarity over cleverness: a clean separation between the Rust backend and the TypeScript frontend, type-safe bindings generated with Specta, and a contributing philosophy that favours bug fixes and stability over feature growth.

What You Get

• Offline Transcription with Multiple Engines - Runs Whisper (Small/Medium/Turbo/Large) with GPU acceleration or Parakeet V3 for CPU-only use, plus Moonshine, SenseVoice, GigaAM, Canary, and Cohere via ONNX — all locally with zero cloud dependency.
• Direct Paste into Any App - Transcribed text is typed or pasted into the currently focused window automatically, supporting xdotool, wtype, dotool, enigo, and clipboard methods depending on your platform and display server.
• Flexible Hotkey System - Configurable global keyboard shortcuts for start/stop and push-to-talk modes, with support for independent left/right modifiers, the macOS Globe key, and Wayland via CLI flags or Unix signals.
• LLM Post-Processing Pipeline - After transcription, optionally pass the raw text to any OpenAI-compatible API endpoint (including local Ollama, Z.AI, and others) with custom prompt templates and structured output support to clean, reformat, or translate the transcription.
• Custom Word Dictionary - Define correction rules for proper nouns, technical terms, or brand names that base models frequently misrecognise, applied as a post-transcription filter in the Rust backend.
• CLI and Signal Remote Control - Send —toggle-transcription, —cancel, or —toggle-post-process flags to a running instance from any terminal or hotkey daemon; on Linux, SIGUSR1/SIGUSR2 signals provide daemon-friendly control without spawning a new process.
• Raycast Extension - Official Raycast integration for macOS that lets you start and stop recording, browse transcript history, switch models, and manage your custom dictionary directly from the Raycast launcher.
• Audio Feedback and Overlay - Configurable sound themes (start/stop chimes), a recording overlay window that shows recording state, and a volume visualiser give clear feedback without switching windows.
• Internationalization - Full i18n support via i18next with multiple language locales maintained by community contributors, including RTL layout support.

Common Use Cases

• Accessibility for mobility-impaired users - Someone with a hand injury, RSI, or motor disability uses Handy as a full keyboard replacement, dictating emails, code comments, and documents with a push-to-talk hotkey configured to a foot pedal or single key.
• Privacy-first transcription for journalists and researchers - A reporter records sensitive interview notes offline so source audio never touches a remote server, then uses LLM post-processing via a local Ollama instance to clean punctuation without any cloud exposure.
• Developer automation on Linux Wayland - A developer running Sway or Hyprland configures SIGUSR2 in their Sway config to trigger transcription, filling in command arguments and commit messages by voice without leaving the terminal.
• Multilingual dictation with automatic language detection - A bilingual writer switches between English and a second language mid-session using the Parakeet V3 model’s automatic language detection, with SenseVoice swapped in for higher accuracy on accented speech.
• Structured note capture with LLM cleanup - A product manager dictates rough meeting notes in push-to-talk mode, then the post-processing step reformats them into a bullet-point summary via a configured prompt before pasting into Notion.

Under The Hood

Architecture Handy separates concerns into two well-defined layers: a Rust backend managing all system integration, audio processing, and ML inference, and a React/TypeScript frontend that handles only settings display and user interaction. The bridge between them is generated at build time with Specta and tauri-specta, producing fully typed TypeScript bindings from Rust command signatures, which eliminates an entire class of type mismatch bugs at the boundary. The Rust side coordinates transcription lifecycle events through a single-threaded TranscriptionCoordinator that serialises all hotkey, signal, and async pipeline events via an mpsc channel, preventing the race conditions that plague multi-threaded recording state machines. The action system abstracts keyboard shortcut behaviour behind a ShortcutAction trait, making it straightforward to add new binding types without modifying the coordinator. Frontend state is managed by Zustand stores with subscribeWithSelector middleware, keeping UI reactivity localised to components that care about specific slices of settings.

Tech Stack The application is a Tauri 2 desktop app combining Rust for the backend and React 18 with TypeScript for the settings UI, bundled by Vite and styled with Tailwind CSS v4. Audio capture uses cpal for cross-platform device enumeration and recording, rubato for sample rate conversion, and a custom FrameResampler that feeds the VAD pipeline. Voice activity detection runs via a Silero model wrapped in the vad-rs crate. Transcription dispatches to whisper-rs for Whisper models and to transcribe-rs for the ONNX engine suite (Parakeet, Moonshine, SenseVoice, GigaAM, Canary, Cohere). Post-processing calls any OpenAI-compatible REST endpoint via reqwest with provider-specific auth header handling for OpenAI, Anthropic, and open-router style services. Persistence uses tauri-plugin-store for settings and tauri-plugin-sql (SQLite) for transcript history. Zod validates settings shapes on the frontend, and i18next handles localisation.

Code Quality The codebase demonstrates disciplined Rust practices: RAII guards (LoadingGuard, FinishGuard) ensure atomic state cleanup even on panics, log macros are used consistently throughout rather than silent error swallowing, and the settings module employs custom Serde deserializers with explicit backward-compatibility handling for migrated field formats. TypeScript coverage is strong with generated bindings ensuring correctness at the Tauri boundary and Zod schemas enforcing settings shapes. The test suite is limited — only two shallow Playwright smoke tests verify the dev server responds — which is a meaningful gap for a project with extensive platform-specific input and audio paths. ESLint and Prettier are configured and enforced, and the CI pipeline runs formatting checks. Contributing guidelines are explicit about the current feature freeze prioritising bug-fix stability.

What Makes It Unique Handy’s primary differentiator is the breadth of local model support combined with the LLM post-processing layer: rather than committing to a single transcription engine, the TranscriptionManager dispatches to eight distinct engine types through a common interface, letting users trade accuracy, speed, and language coverage without any code changes. The dual keyboard backend — choosing between Tauri’s global shortcut plugin and the custom handy-keys library at runtime with automatic fallback — solves a class of system-level shortcut conflicts that competing tools work around with documentation warnings. The OpenCC integration for Traditional/Simplified Chinese conversion and the Unix signal interface for Wayland hotkey daemons reflect genuine community-driven engineering decisions rather than theoretical feature additions.

Self-Hosting

Handy is released under the MIT License, which grants unrestricted rights to use, copy, modify, merge, publish, distribute, sublicense, and sell copies of the software. There are no copyleft obligations — you can embed Handy in a proprietary product, distribute it as part of a commercial application, or fork it internally without publishing your changes. The only requirement is retaining the copyright notice in distributed copies. For organisations evaluating the legal posture, MIT is the permissive baseline with no viral implications.

Running Handy yourself is straightforward from an operational perspective: it is a single-user desktop application with no server component, no database to maintain, no network services to monitor, and no infrastructure dependencies beyond the machine it runs on. The application stores settings in a local Tauri plugin store and transcript history in a local SQLite database. Updates ship as versioned GitHub releases, and the built-in updater checks for new versions automatically — though applying them requires user action. There is no fleet management, no centralised configuration, and no telemetry by default. The operational burden is exactly equivalent to managing any other desktop application.

There is no paid cloud tier, hosted version, or managed offering for Handy. What you give up compared to a SaaS transcription service is the convenience of managed model hosting, synchronised history across devices, and a support SLA. Whisper model files range from several hundred megabytes to several gigabytes and must be downloaded separately on first use, which can be a friction point on constrained connections or locked-down enterprise networks. GPU acceleration for Whisper requires appropriate drivers (CUDA on Windows/Linux, Metal on macOS) and has known crash issues on some system configurations that the project openly documents as needing contributor help.