Speakr

Speakr is a self-hosted web application that transforms audio and video recordings into organized, searchable, AI-powered notes. Built for privacy-conscious individuals and teams, it runs entirely on your own infrastructure so sensitive conversations never leave your control. From one-click Docker deployment to GPU-accelerated WhisperX diarization, Speakr handles the full pipeline from capture to insight.

The platform supports a connector-based transcription architecture that auto-detects your preferred engine — whether that is self-hosted WhisperX for best-in-class diarization and voice profiles, OpenAI’s gpt-4o-transcribe-diarize for cloud simplicity, Mistral Voxtral, VibeVoice, or Azure OpenAI. Smart tags carry their own AI prompts that stack and layer, transforming raw transcripts into recipes, action item lists, study notes, or any other structured format you define. Groups with granular sharing permissions, OIDC SSO, retention policies, signed webhooks, and a full REST API make Speakr a serious platform rather than a weekend project.

Version 0.9.0 elevated Speakr’s mobile experience to first-class status with a proper bottom-nav detail view, a drag-to-dismiss upload sheet, and a redesigned recording surface that works edge-to-edge on phones. A new Stats tab shows per-speaker breakdowns of speaking time, turn count, and words per minute. Server-side recording sessions enable long multi-hour captures that survive page reloads, and a Phase 1-3 webhook system with HMAC signing and exponential-backoff retry connects Speakr to automation tools like n8n, Zapier, and Make.

Speakr ships as an installable Progressive Web App with a web share target so your OS can push audio files directly from the share sheet. Seven languages are fully localized, including English, French, German, Spanish, Russian, Simplified Chinese, and Brazilian Portuguese.

What You Get

• Connector-Based Transcription Engine - Auto-detects and routes to WhisperX, OpenAI gpt-4o-transcribe-diarize, Mistral Voxtral, VibeVoice, Azure OpenAI, or any custom ASR endpoint based on your environment configuration, with capability flags driving chunking, diarization, and format handling per provider.
• Speaker Diarization and Voice Profiles - Automatic who-said-what labeling across every supported backend; WhisperX additionally builds persistent voice embeddings that recognize the same speaker across different recordings over time.
• Smart Tags with Prompt Stacking - Each tag carries its own AI system prompt and ASR settings, and multiple tags layer their instructions so a recording tagged with both ‘Lecture’ and ‘Biology 301’ gets study-note formatting focused on biological terminology.
• Signed Webhooks with Retry - HMAC-signed outbound notifications for recording lifecycle events with exponential-backoff retry, SSRF protection, and delivery log visibility in account settings — connects directly to n8n, Zapier, Make, or any HTTP endpoint.
• Per-Speaker Stats Tab - Speaking time, percentage of total audio, turn count, word count, and words-per-minute for each identified speaker, displayed as a proportion bar table on desktop and a card stack on mobile.
• Multi-User Groups with Granular Permissions - Create groups with group-scoped tags that automatically share recordings to every member; individual shares support view, edit, and reshare controls; admin-controlled public links are available for external access.
• Retention Policies with Auto-Deletion - Set custom retention periods per group or tag with per-recording override protection; compliance-driven teams can enforce 7-year retention for legal consultations while auto-purging 14-day-old standups.
• Server-Side Recording Sessions - With ENABLE_SERVER_RECORDING_CHUNKS enabled, long browser captures stream to the server in the background, survive page reloads, and support multi-hour sessions up to a configurable ceiling without holding audio in memory.
• REST API v1 with Swagger UI - Full programmatic access with bearer-token authentication covering uploads, transcription, speaker management, sharing, and user metadata; batch tag operations and speaker identification endpoints included.
• Progressive Web App with Share Target - Installable on mobile and desktop; the OS share sheet can push audio files directly into Speakr from any app, and in-app recordings capture microphone, system audio, or both simultaneously via virtual-device mixing.
• Inquire Mode with Semantic Search - Natural-language chat across your entire recording library using sentence-transformers embeddings with graceful fallback to keyword search when the embedding model is not available.
• Automated Export to Obsidian, Logseq, or Any Folder - Completed transcripts are written to templated files in a watched output directory automatically, enabling zero-touch integration with note-taking systems and documentation wikis.
• Custom Vocabulary and Initial Prompts - Per-user, per-tag, and per-folder hotword lists bias the transcription model toward domain-specific jargon; an initial prompt adds free-text context to steer vocabulary and style.

Common Use Cases

• Legal consultations - Law firms assign a group tag with a 7-year retention policy so every client conversation is automatically shared with the legal team and preserved for compliance without manual filing.
• Research interviews - Academics apply a protected tag and Obsidian auto-export to ensure raw audio is retained indefinitely and transcripts land in their vault the moment processing completes.
• Engineering architecture discussions - Development teams use a protected group tag so technical decision recordings are shared automatically with the entire engineering org and never expire.
• Multi-language product teams - International teams set language detection per recording or per folder, and the seven-language UI ensures every team member reads the interface in their own language.
• Podcast or lecture production - Content creators use SRT export templates and the per-speaker stats tab to produce subtitle files and understand which speaker dominated the recording before editing.
• Sales call review - Sales teams auto-tag inbound calls with a 1-year retention policy, use the per-recording chat to ask ‘what objections came up?’ and share view-only links with managers for coaching.
• Daily standups - Teams apply a Standup tag with 14-day retention and group auto-share so routine meetings are available to all members and automatically cleaned up after the retention window expires.
• Medical or therapeutic consultations - Practitioners self-host on air-gapped infrastructure to ensure patient audio never leaves the clinic, relying on WhisperX locally for both transcription and diarization without any cloud dependency.

Under The Hood

Architecture Speakr is a Flask monolith organized into clearly delineated layers — API blueprints, service modules, SQLAlchemy models, and background task processing — though some cross-cutting concerns such as authentication and configuration are woven directly into the application factory. The most architecturally sophisticated subsystem is the transcription pipeline: an abstract base class hierarchy with a capability enum system and a singleton ConnectorRegistry handle provider auto-detection, per-connector specification declarations, and runtime switching without application restarts. Audio chunking is handled by a separate service that reads ConnectorSpecifications to decide whether chunking is required, what the chunk size and overlap should be, and how to reassemble diarized segments across chunk boundaries with speaker continuity. Event-driven features including webhook dispatch, retention cleanup, and export templating are implemented as background tasks driven by SQLAlchemy-backed job queues rather than a dedicated message broker, keeping the deployment footprint minimal while supporting retry and error recovery.

Tech Stack The backend is Python 3.11 with Flask 2.3, SQLAlchemy, Flask-Login, Flask-WTF for CSRF protection, Flask-Limiter for rate limiting, and flask-openapi3 with Pydantic v2 for the typed REST API surface exposed through Swagger UI. Storage defaults to SQLite with a separate directory for uploaded media, but PostgreSQL is fully supported and tested in CI via psycopg2. Speaker embedding matching uses sentence-transformers with numpy and scikit-learn for cosine similarity; this dependency is optional and the application degrades gracefully to keyword search when not installed. The frontend is a server-rendered Jinja2 application with Vue 3 composables and vanilla JavaScript modules — no build pipeline is required in production because assets are pre-bundled offline and served statically. Gunicorn with optimized worker and timeout settings is the production WSGI server. Deployment is via Docker with a multi-stage Dockerfile that vendors custom FFmpeg and ffprobe binaries to keep the image portable and predictable across host distributions.

Code Quality The test suite is extensive across more than 50 test files covering unit behavior, integration paths through real SQLite databases, authorization boundary tests for every sharing and group endpoint, connector architecture conformance, and edge cases including race conditions in the job queue and video retention behavior under large file probes. Pytest with a conftest.py that provisions a throw-away isolated database ensures tests are safe to run anywhere without affecting developer data. Error handling is layered: custom exception classes for each service boundary, fallback parsing for malformed LLM JSON responses, and explicit validation at API contract points. Type annotations and dataclasses are used throughout the service and connector layers; the API layer adds Pydantic models for request and response validation. The connector base class enforces an abstract interface that new providers must implement, preventing silent capability mismatches.

What Makes It Unique Speakr’s most distinctive technical contribution is the connector capability flag system combined with adaptive audio chunking: each connector declares its constraints as a dataclass and the chunking service reads those constraints at runtime to decide whether to chunk, how large each chunk should be, and how to handle overlap — meaning adding support for a new ASR provider requires only a new connector class with accurate specifications rather than changes to any core processing logic. The tag prompt stacking model is unusual in self-hosted tools: multiple tags contribute their system prompts in order, enabling compositional AI behavior without requiring users to author monolithic prompts. The signed webhook system with SSRF protection and DNS-level IPv6 site-local rejection reflects serious attention to the security implications of outbound webhooks in self-hosted contexts. Persistent voice embeddings that survive across recordings via the WhisperX backend give Speakr a durable speaker identity layer that pure cloud APIs do not offer.

Self-Hosting

Speakr is dual-licensed under the GNU Affero General Public License v3.0 for open source use and a separate commercial license for organizations that cannot comply with the AGPL’s copyleft terms. The AGPL v3 requires that any modified version of Speakr used to provide a network service must make its source code available to users of that service — this is the key self-hosting consideration. If you deploy Speakr internally for your own organization and do not distribute it to external parties, the AGPL typically imposes no obligation beyond keeping the source available within your organization. For SaaS providers, software vendors, or anyone integrating Speakr into a proprietary product, the commercial license is available directly from the Speakr maintainers.

Running Speakr yourself is a meaningful operational commitment. The Docker deployment is well-documented and the image is self-contained with vendored FFmpeg, but production readiness requires a persistent volume strategy for uploaded media, a database backup schedule (SQLite or PostgreSQL), TLS termination via a reverse proxy, and attention to file size limits and Gunicorn worker timeouts for large audio files. The optional WhisperX ASR service is a separate GPU-backed container that adds GPU scheduling, model caching, and independent update cycles to the operational surface. Recording sessions, webhook delivery retries, retention cleanup, and embedding re-indexing all run as background tasks within the main process — there is no separate worker fleet to manage, but this also means a process restart interrupts any in-flight background work.

There is no hosted or managed version of Speakr, so everything a managed SaaS would provide — automatic security patches, monitored uptime, HA failover, cloud backup, and a support SLA — falls entirely to the self-hoster. The active release cadence (40+ releases, multiple per week at peak) is a double-edged sword: the software improves rapidly but staying current requires frequent image pulls and migration runs. The community is active on GitHub with eight contributors and open issues, but there is no paid support tier or enterprise support contract currently available beyond the commercial license itself.