Open Source Data Engineering Apps
Discover open source data engineering tools for building reliable data pipelines, ETL processes & scalable data storage. Unlock the power of your data!
Apps in Data Engineering
Argo Workflows
Data Engineering · Devops
The most popular Kubernetes-native workflow engine for orchestrating containerized DAGs, ML pipelines, CI/CD, and parallel batch jobs at scale.
DataEase
AI Assistants · Analytics · Data Engineering
Open-source BI tool with drag-and-drop dashboards, 20+ data source connectors, and AI-powered natural language queries — a self-hosted alternative to Tableau.
Kestra
Automation · Data Engineering · Devops
Event-driven orchestration platform for data, AI, and infrastructure workflows — define everything in YAML, run anywhere at scale.
Dolt
Data Engineering · Databases · Developer Tools
The SQL database you can branch, merge, diff, and clone — Git for your data, MySQL-compatible and ready for multi-agent AI workflows.
PeerDB
Data Engineering · Databases
Postgres-native ETL that streams change data capture in real time to Snowflake, BigQuery, ClickHouse, S3, and Kafka — up to 10x faster than general-purpose pipelines, managed through a familiar Postgres SQL interface.
Rill
Analytics · Data Engineering
The fastest BI tool for humans and agents — define metrics, models, and dashboards as code and query them instantly on ClickHouse or DuckDB.
Beta9
AI Development · Automation · Data Engineering
Run AI workloads at scale with a Pythonic serverless runtime that handles GPU inference, background jobs, and sandboxes with zero infrastructure overhead.
shaper
Analytics · Data Engineering
Build analytics dashboards, reports, and customer-facing analytics by writing pure SQL — powered by DuckDB and designed for self-hosted deployment.
Enso
Analytics · Data Engineering · Low Code Platforms
A visual and textual programming platform for data prep and analysis where the node graph and the underlying Enso code are always perfectly in sync, built by an Alteryx co-founder on a GraalVM engine.
About Data Engineering
Data engineering focuses on building and maintaining robust data pipelines that enable organizations to make data-driven decisions. These tools are essential for turning raw data into actionable insights, automating data workflows, and ensuring data quality.
Typical features within this category include:
- Data Integration: Connecting to various data sources (databases, APIs, cloud storage) and ingesting data.
- Data Transformation: Cleaning, validating, enriching, and transforming data into usable formats using techniques like ETL (Extract, Transform, Load).
- Data Storage: Managing and organizing data in efficient and scalable storage systems (data warehouses, data lakes).
- Data Pipeline Automation: Scheduling and monitoring data workflows to ensure reliability and consistency.
- Data Quality & Governance: Implementing checks and balances to maintain data accuracy, completeness, and security.
Data engineering solves critical problems such as siloed data, inefficient workflows, and a lack of reliable data for analytics. By streamlining the data process, organizations can unlock business value faster, improve decision-making accuracy, and gain a competitive edge. Furthermore, robust data pipelines are foundational for machine learning initiatives, enabling teams to build and deploy predictive models with confidence.
