Frog

Frog is a small, animated desktop pet that lives on your Mac and doubles as a dictation engine, meeting-notes recorder, and talking voice assistant. Hold the fn key and speak, and Frog transcribes it directly into whatever app has focus — a terminal, an editor, an email, a coding-agent prompt. Right-click it to start recording a call, and it records both sides of the conversation, transcribes them, labels the speakers, and writes a summary once the call ends.

Hold right Option and talk to it directly and Frog answers out loud, remembers facts you tell it (“my sister’s name is Anna”), and can set voice reminders. Everything — speech recognition, speaker diarization, and the conversational brain — runs as local models downloaded once on first use, so nothing you say leaves the machine unless you opt into an external Claude API key for meeting summaries.

Under the hood it pairs Apple’s on-device SpeechAnalyzer (or an optional NVIDIA Parakeet v3 Core ML model) for transcription, FluidAudio for speaker diarization, and a locally run Gemma or Qwen2.5 GGUF model via llama.cpp for the pet’s personality and memory. The pet’s body itself is a three.js scene rendered in a transparent WKWebView, with a small, documented JS contract (setState, setLevel, lookAt) that makes it straightforward to reskin as a different creature.

Frog is MIT licensed and ships as a signed macOS app you can download directly, or build from source with SwiftPM and Node — no Xcode project required. It targets Apple Silicon Macs running macOS 26, since it depends on the newest on-device SpeechAnalyzer and Apple Foundation Models APIs.

What You Get

• Push-to-talk dictation that types transcribed speech directly into the focused app via the fn key, with a vocabulary/replacement system for names and jargon it mishears
• One-click meeting recording that captures both microphone and system audio, transcribes each, labels speakers via diarization, and writes a title and summary
• A talking companion mode (hold right Option) backed by a local LLM that answers out loud, remembers durable facts about you, and can set and speak voice reminders
• A choice of on-device speech engines (Apple SpeechAnalyzer or NVIDIA Parakeet v3) and four local LLM tiers auto-picked by your Mac’s RAM, from a 1GB Qwen2.5 model up to a 17GB Gemma model
• A fully customizable three.js desktop pet with a documented state/animation contract, so the whole creature can be reskinned by editing a single JS file

Common Use Cases

• Dictating long prompts into Claude Code, Codex, or a terminal instead of typing them by hand
• Recording a client or team call and getting a speaker-labeled transcript and summary without a cloud meeting-notes subscription
• Talking through a task out loud and having the pet remember personal facts and spoken reminders across sessions
• Running dictation and meeting notes fully offline for privacy-sensitive work where audio can’t leave the machine
• Teaching the app project-specific vocabulary (names, acronyms) so dictation accuracy improves over time

Under The Hood

Architecture AppDelegate.swift is the central coordinator wiring PetPanel, AudioRecorder, HotkeyMonitor, MeetingRecorder, Brain, StatusBar, and Wander as a flat set of collaborator objects composed via dependency injection in one @MainActor class; execution forks into three flows — push-to-talk dictation (startListening/stopListening), the conversational assistant (startTalk/stopTalk), and call recording (MeetingRecorder.start/stop) — each running async Task blocks that call into a Transcriber protocol implementation and back onto the main actor to update UI state via JS calls into the WKWebView-hosted pet. Data flows through event/callback closures (onStateChange, onSamples, onTalking) rather than a formal pub/sub pipeline, with state persisted through a couple of JSON-backed singletons (Store.shared, Mind.shared). Because AppDelegate owns nearly every collaborator directly, changing its shape would ripple through most of the app — a reasonable, if monolithic, coordination style for a single-window menu-bar utility of this size.

Tech Stack The app is Swift 6.2 (language mode 5), built with SwiftPM only and no Xcode project, targeting macOS 26 on Apple Silicon. It depends on FluidAudio for on-device ASR and speaker diarization Core ML models, and LLM.swift wrapping llama.cpp for local GGUF model inference, alongside Apple’s own Speech, AVFoundation, and Foundation Models frameworks. The desktop pet itself is a separate Vite and three.js frontend bundled to a single file via vite-plugin-singlefile and loaded into a WKWebView, with a build script gluing the Swift and web builds into one distributable .app.

Code Quality No test files or test target exist anywhere in the repository, so there is no automated coverage. Error handling generally favors typed throws and do/catch around async engine calls, logging failures and surfacing a visible “confused” pet state on failure, though some force-unwrapped optionals around file writers point to areas of optional-safety debt. Naming is clear and consistent across transcription engines and recorder types, and there is no CI configuration present in the repo.

What Makes It Unique The defining choice is running the full pipeline — speech-to-text, speaker diarization, a conversational LLM with memory, and text-to-speech — entirely on-device on Apple Silicon, including auto-selecting an LLM tier sized to the Mac’s available RAM and extracting durable memory and reminders as structured JSON straight from the local model’s own output, with no external service required. Treating a customizable three.js “pet” persona as a first-class, swappable interface via a small documented JS contract is also a distinctive product decision compared to typical dictation-utility UIs.

Self-Hosting

Licensing Model MIT licensed — all features available with no restrictions or license keys required.

Self-Hosting Restrictions None. Frog is a local macOS application, not a hosted service; there is no cloud tier or paywall. The only optional external dependency is a user-supplied Claude API key to swap in for the default on-device Apple Foundation Models summarizer.

Enterprise Features Not applicable — Frog has no paid tier, team plan, or enterprise edition.