Open Source Data Engineering Apps

Open-source ELT platform with 600+ connectors for moving data from any source to warehouses, lakes, and AI agents.

Define, schedule, and monitor complex data workflows as Python code — with a powerful UI, 80+ provider integrations, and battle-tested scalability across thousands of production deployments.

Collaborate on high-quality AI training data with a self-hosted annotation platform built for LLMs, NLP, and multimodal models.

The most popular Kubernetes-native workflow engine for orchestrating containerized DAGs, ML pipelines, CI/CD, and parallel batch jobs at scale.

A distributed stream processing engine written in Rust that lets you write SQL to run stateful, real-time computations over data streams with subsecond results.

Run AI workloads at scale with a Pythonic serverless runtime that handles GPU inference, background jobs, and sandboxes with zero infrastructure overhead.

Open-source column-oriented database that delivers real-time analytical queries on petabyte-scale data with millisecond latency.

An incremental data indexing engine that keeps AI agent context perpetually fresh by reprocessing only what changed.

Open-source enterprise data warehouse unifying analytics, vector search, full-text search, and AI agent orchestration in a single Rust-built engine on S3.

Open-source BI tool with drag-and-drop dashboards, 20+ data source connectors, and AI-powered natural language queries — a self-hosted alternative to Tableau.

A next-generation documentation site generator for dbt Core projects — lineage explorer, health scoring, and full-text search for teams without access to dbt Cloud's built-in docs features.

The SQL database you can branch, merge, diff, and clone — Git for your data, MySQL-compatible and ready for multi-agent AI workflows.

The dbt-native data observability CLI that turns your existing dbt tests and metadata into anomaly detection, lineage graphs, and Slack/Teams alerts — no separate platform required.

A visual and textual programming platform for data prep and analysis where the node graph and the underlying Enso code are always perfectly in sync, built by an Alteryx co-founder on a GraalVM engine.

Turn SQL queries and markdown files into polished, interactive data apps and business intelligence reports — no drag-and-drop, no GUI, just code.

Visual ETL that compiles to Polars — build pipelines on a canvas, export as standalone Python, and run anywhere without platform lock-in.

Open-source, fully-scriptable data ingestion engine that streams events from web, apps, and APIs to any data warehouse in real time.

Event-driven orchestration platform for data, AI, and infrastructure workflows — define everything in YAML, run anywhere at scale.

ktx builds a self-improving context layer over your data warehouse so AI agents like Claude Code and Codex query it with approved metric definitions instead of reinventing SQL logic from scratch.

Label Studio is an open-source, multi-type data labeling platform that lets teams annotate images, text, audio, video, and time series data with a configurable XML-based UI and export annotations in formats ready for any ML framework.

The open-source Looker alternative that turns your dbt project's metrics and dimensions into governed, self-serve charts and dashboards — no license key required.

A reactive Python notebook that eliminates hidden state, runs reproducibly, and deploys as a web app or script — stored as pure Python, built for the AI era.

OpenDuck brings MotherDuck-style cloud capabilities to self-hosted DuckDB — attach remote databases, run hybrid queries across local and remote nodes, and own your data with an open gRPC and Arrow IPC protocol.

Postgres-native ETL that streams change data capture in real time to Snowflake, BigQuery, ClickHouse, S3, and Kafka — up to 10x faster than general-purpose pipelines, managed through a familiar Postgres SQL interface.

Production-grade open source web scraping engine that turns any URL into clean markdown for AI agents — with built-in anti-bot bypass, proxy rotation, and browser session management.

Build analytics dashboards, reports, and customer-facing analytics by writing pure SQL — powered by DuckDB and designed for self-hosted deployment.

Drop your files and search them instantly — no vector DB, no indexing pipeline, just raw data queried by a self-evolving intelligence layer.

superglue is an AI-agent-driven integration engine that turns plain-English descriptions of enterprise systems into production-grade API tools, ERP/CRM connectors, and data pipelines — self-hosted or cloud, Y Combinator-backed (W25).

Federated AI search and RAG across 100+ enterprise sources—no data extraction, no vector database required.

Single C++ binary SQL engine for real-time stream processing, ETL, and analytics on Kafka, Redpanda, and ClickHouse with sub-millisecond latency.

Open-source event tracking infrastructure built on Kafka and ClickHouse that handles thousands of events per second on a single node, with full Segment API compatibility and no cookies.

A Rust-based real-time data processing engine for AI/ML feature computation, built on Apache DataFusion and Arrow — positioned as an alternative to Flink, Spark, Chronon, and OpenMLDB with unified streaming, batch, and request-time execution.