Quickwit is a high-performance, cloud-native search engine designed specifically for observability workloads—handling logs, distributed traces, and soon metrics. Built in Rust, it eliminates the operational complexity of traditional search engines like Elasticsearch by decoupling compute and storage, enabling sub-second queries directly on object storage (S3, Azure Blob, GCS). This architecture reduces costs by up to 10x compared to Elasticsearch while maintaining full compatibility with Elasticsearch/OpenSearch APIs. Quickwit is ideal for DevOps teams, SREs, and platform engineers managing large-scale telemetry data in cloud environments who need scalable, low-cost search without vendor lock-in.
Unlike monolithic systems, Quickwit’s stateless indexers and searchers allow independent scaling of ingestion and query layers. It supports schemaless indexing for flexible log formats while also enforcing strict schemas for performance-critical use cases. With native integrations for Jaeger, OpenTelemetry, Grafana, and Kafka, Quickwit serves as a drop-in replacement for existing observability pipelines without requiring changes to data producers.
What You Get
- Elasticsearch-compatible API - Full support for Elasticsearch query DSL, aggregations, and ingest APIs; allows seamless migration from Elasticsearch/OpenSearch using existing log shippers like Vector or Fluent Bit without code changes.
- Sub-second search on cloud storage - Direct querying of data stored in S3, Azure Blob Storage, or Google Cloud Storage with optimized IO paths and columnar index structures, eliminating the need for expensive local SSDs.
- Jaeger-native distributed tracing - Native integration with Jaeger backend to store and query trace data without requiring additional adapters or custom plugins.
- OpenTelemetry-native ingestion - Direct support for OTLP logs and traces via HTTP/gRPC endpoints, enabling modern observability pipelines without intermediaries.
- Schemaless and strict schema indexing - Flexible field inference for unstructured logs or enforced schemas via JSON mapping files for predictable performance and query reliability.
- Grafana data source plugin - Official Grafana plugin to visualize logs and traces directly in dashboards using native Quickwit queries.
- Kubernetes-ready with Helm chart - Official Helm charts for deploying Quickwit clusters in Kubernetes with configurable resource limits, storage classes, and ingress settings.
- Multi-tenancy and retention policies - Support for multiple indexes with per-index retention rules, enabling cost-controlled data lifecycle management and GDPR-compliant deletion tasks.
- High availability for search - Stateless searcher nodes can be horizontally scaled; indexing HA is available when using Kafka as a data source.
Common Use Cases
- Building a multi-tenant SaaS dashboard with log isolation - A platform provider uses separate Quickwit indexes for each customer, applying retention policies to auto-delete logs after 30 days and using multi-tenancy to enforce access controls without shared infrastructure.
- Creating a cost-efficient log pipeline for 10k+ microservices - A cloud-native company replaces Elasticsearch with Quickwit on S3, reducing storage costs by 80% while maintaining sub-second search latency across all service logs ingested via Fluent Bit and Vector.
- Problem: High Elasticsearch bills due to redundant indexing → Solution: Quickwit on S3 - Teams struggling with expensive Elasticsearch clusters migrate to Quickwit, storing raw logs in S3 and using stateless indexers that scale with ingestion load—cutting infrastructure costs by up to 10x.
- DevOps teams managing observability across hybrid clouds - Engineers deploy Quickwit on AWS EKS with S3 storage and integrate it with Jaeger for traces, Grafana for dashboards, and Kafka for buffering logs from on-prem systems—all using a unified, open-source stack.
Under The Hood
Quickwit is a high-performance, distributed search and analytics engine built with Rust, designed for scalable log management, real-time event processing, and observability. It emphasizes modular architecture and efficient data handling across distributed systems.
Architecture
Quickwit adopts a service-oriented, layered architecture that enables modular development and clear separation of concerns across its core components.
- The system is structured around distinct modules for ingestion, indexing, and query execution, promoting loose coupling and scalability.
- It implements a distributed design pattern that allows for horizontal scaling across nodes and cloud environments.
- Core services are organized with well-defined interfaces, supporting extensibility and integration with observability ecosystems.
Tech Stack
The project is built entirely in Rust, leveraging asynchronous runtime and systems-level performance for efficient data processing.
- It integrates with cloud storage systems like S3 and Kubernetes, enabling seamless deployment in modern infrastructures.
- Docker and Makefiles are used for containerization and build orchestration, supporting streamlined development and deployment workflows.
- Rust’s native testing capabilities and integration tests form the backbone of its quality assurance practices.
Code Quality
The codebase reflects a mixed quality profile with strengths in backend Rust components and room for improvement in frontend testing.
- Testing is comprehensive in core modules but lacks full coverage in UI and integration layers, indicating potential gaps in validation.
- Error handling follows standard Rust practices with appropriate use of Result types and structured logging for debugging.
- Consistent naming conventions and style guidelines are enforced through documentation and linting configurations.
What Makes It Unique
Quickwit distinguishes itself through its schemaless ingestion model and flexible indexing engine tailored for modern observability needs.
- It uniquely supports real-time analytics, log search, and trace visualization within a single platform without requiring separate tools.
- The system’s extensible architecture allows for custom data pipelines and integrations with popular observability frameworks like Jaeger and OpenTelemetry.
- Its distributed design enables efficient handling of large-scale event streams with minimal latency and high throughput.