Open Source Data Engineering Apps
Discover open source data engineering tools for building reliable data pipelines, ETL processes & scalable data storage. Unlock the power of your data!
Apps in Data Engineering
ClickHouse
Analytics · Data Engineering · Databases
Open-source column-oriented database that delivers real-time analytical queries on petabyte-scale data with millisecond latency.
DataEase
AI Assistants · Analytics · Data Engineering
Open-source BI tool with drag-and-drop dashboards, 20+ data source connectors, and AI-powered natural language queries — a self-hosted alternative to Tableau.
Databend
Data Engineering · Databases
Open-source enterprise data warehouse unifying analytics, vector search, full-text search, and AI agent orchestration in a single Rust-built engine on S3.
Kestra
Automation · Data Engineering · Devops
Event-driven orchestration platform for data, AI, and infrastructure workflows — define everything in YAML, run anywhere at scale.
Timeplus Proton
Analytics · Data Engineering
Single C++ binary SQL engine for real-time stream processing, ETL, and analytics on Kafka, Redpanda, and ClickHouse with sub-millisecond latency.
cocoindex
AI Development · Data Engineering
An incremental data indexing engine that keeps AI agent context perpetually fresh by reprocessing only what changed.
Arroyo
Analytics · Data Engineering
A distributed stream processing engine written in Rust that lets you write SQL to run stateful, real-time computations over data streams with subsecond results.
Volga
Data Engineering
A Rust-based real-time data processing engine for AI/ML feature computation, built on Apache DataFusion and Arrow — positioned as an alternative to Flink, Spark, Chronon, and OpenMLDB with unified streaming, batch, and request-time execution.
Enso
Analytics · Data Engineering · Low Code Platforms
A visual and textual programming platform for data prep and analysis where the node graph and the underlying Enso code are always perfectly in sync, built by an Alteryx co-founder on a GraalVM engine.
openduck
Data Engineering · Databases
OpenDuck brings MotherDuck-style cloud capabilities to self-hosted DuckDB — attach remote databases, run hybrid queries across local and remote nodes, and own your data with an open gRPC and Arrow IPC protocol.
About Data Engineering
Data engineering focuses on building and maintaining robust data pipelines that enable organizations to make data-driven decisions. These tools are essential for turning raw data into actionable insights, automating data workflows, and ensuring data quality.
Typical features within this category include:
- Data Integration: Connecting to various data sources (databases, APIs, cloud storage) and ingesting data.
- Data Transformation: Cleaning, validating, enriching, and transforming data into usable formats using techniques like ETL (Extract, Transform, Load).
- Data Storage: Managing and organizing data in efficient and scalable storage systems (data warehouses, data lakes).
- Data Pipeline Automation: Scheduling and monitoring data workflows to ensure reliability and consistency.
- Data Quality & Governance: Implementing checks and balances to maintain data accuracy, completeness, and security.
Data engineering solves critical problems such as siloed data, inefficient workflows, and a lack of reliable data for analytics. By streamlining the data process, organizations can unlock business value faster, improve decision-making accuracy, and gain a competitive edge. Furthermore, robust data pipelines are foundational for machine learning initiatives, enabling teams to build and deploy predictive models with confidence.
