Quickwit is a high-performance, cloud-native search engine built for observability, serving as a cost-effective open-source alternative to Datadog, Elasticsearch, Loki, and Tempo. Designed for log and trace analytics, it enables sub-second search directly on object storage without requiring expensive, tightly coupled storage systems. It targets DevOps teams and SREs managing large-scale telemetry data who need scalable, low-cost search without vendor lock-in.
Built in Rust using the Tantivy search library, Quickwit features a decoupled compute-and-storage architecture with stateless indexers and searchers. It supports cloud storage backends like Amazon S3, Azure Blob, and Google Cloud Storage, and integrates natively with OpenTelemetry, Jaeger, Grafana, Kafka, and Kinesis. Deployment options include Kubernetes via Helm, single-node setups, and serverless environments like AWS Lambda.
What You Get
- Sub-second search on cloud storage - Queries execute directly on S3, Azure Blob, or GCS with optimized I/O paths and smart scheduling, eliminating the need for expensive local storage or indexing clusters.
- Elasticsearch-compatible API - Supports ES query DSL, aggregations, and ingest APIs, enabling seamless migration from Elasticsearch or OpenSearch without rewriting log shippers or dashboards.
- OpenTelemetry and Jaeger native support - Ingests logs and traces directly via OTLP and Jaeger gRPC protocols, with built-in data parsing and indexing pipelines.
- Schemaless and strict schema indexing - Dynamically parses unstructured logs while supporting optional schema enforcement for structured data and predictable query performance.
- Decoupled compute and storage - Indexers and searchers are stateless, allowing independent scaling of ingestion and query workloads with no shared storage dependencies.
- Grafana data source integration - Official Grafana plugin enables visualizing logs and traces directly in Grafana dashboards with full query and aggregation support.
- Multi-tenancy and partitioning - Supports multiple indexes with fine-grained access control and data partitioning for isolated tenant workloads in shared environments.
- Retention policies and GDPR deletion - Automates data lifecycle management with configurable retention rules and targeted delete tasks for compliance with data privacy regulations.
Common Use Cases
- Running a high-volume log pipeline on S3 - A DevOps team uses Quickwit to index 5TB/day of application logs from Vector and Fluent Bit, reducing storage costs by 80% compared to Elasticsearch while maintaining sub-second search latency.
- Centralizing distributed traces without sampling - A cloud-native startup uses Quickwit with Jaeger to store 100% of traces for 90 days, cutting telemetry costs by 90% while enabling deep-dive root-cause analysis.
- Building a serverless log analytics platform - Stedi leverages Quickwit on AWS Lambda to offer search-as-a-service to customers, scaling from zero cost at rest to petabyte-scale with no operational overhead.
- Replacing Elasticsearch in a regulated environment - A financial services firm migrates from Elasticsearch to Quickwit to reduce licensing costs and implement GDPR-compliant data deletion without changing their existing Kibana dashboards.
Under The Hood
Architecture
- Modular monolithic structure with Rust crates enforcing clear separation between ingestion, indexing, and search layers
- Dependency injection via Rust’s type system and service registries, avoiding global state through factory patterns
- Event-driven pipelines using Pulsar and Kafka for decoupled, async workflows
- Dockerized multi-service orchestration with profile-based service grouping and environment-aware configuration
- UI and backend as separate build targets with clean API boundaries, integrated via static asset embedding
- Makefile-driven build system with multi-stage Docker builds and cross-compilation for consistent deployment
Tech Stack
- Rust core with tokio and cargo, compiled via multi-stage Docker builds targeting lightweight Linux images
- Node.js 24 and Next.js frontend bundled and embedded into the main binary
- PostgreSQL and Apache Pulsar as primary persistent and event streaming layers, configured for high availability
- LocalStack and Confluent Local for local testing of cloud services without external dependencies
- Comprehensive Docker Compose setups with healthchecks and volume management for end-to-end development
- Build pipeline supports cross-compilation to multiple architectures with versioned image tags derived from Git metadata
Code Quality
- Extensive test coverage including unit, integration, and end-to-end scenarios with robust async test harnesses
- Clear service boundaries and protocol buffers for inter-component communication
- Comprehensive error handling using anyhow and domain-specific error types with precise variants
- Strong type safety through serde, explicit configuration schemas, and compile-time validation
- Consistent naming and layer separation across CLI, config, and service components
- Automated testing with Playwright for UI and TLS/GRPC for security and connectivity validation
What Makes It Unique
- Native query language with real-time syntax highlighting and field completion powered by index metadata
- Dynamic OpenAPI documentation auto-generated and embedded directly in the UI
- Lightweight, embeddable search UI components designed for observability pipeline integration
- Clipboard-optimized API URL display with intelligent truncation and secure fallbacks
- Unified frontend system synchronizing query parsing, autocomplete, and visualization through a single extensible configuration