All 389 Dependencies
Every package Apache Airflow depends on, ranked by repo health score.
Apache Airflow is an open-source platform for programmatically authoring, scheduling, and monitoring workflows. Built at Airbnb and donated to the Apache Software Foundation, it treats pipelines as first-class Python code, giving data engineers the full power of the language to express complex task dependencies, parameterize runs, and version-control their pipelines alongside application code.
At the core of Airflow is the Directed Acyclic Graph (DAG) — a declarative representation of tasks and their dependencies. The scheduler continuously evaluates DAGs, queues tasks that are ready to run, and dispatches them to workers via pluggable executors: a local multiprocess executor for single-node deployments, a Celery executor for distributed queues, or a Kubernetes executor that launches each task as its own Pod. As of Airflow 3.0, the architecture has been refactored into a modular multi-package layout with a dedicated Task SDK, a clean FastAPI-based execution API, and first-class support for asset-driven scheduling.
Airflow ships with an extensive provider ecosystem covering over 80 external systems — AWS, GCP, Azure, Snowflake, dbt, Databricks, Spark, and many more — each providing pre-built operators, hooks, and sensors. The web UI offers real-time DAG graph visualization, task-level log streaming, backfill controls, and variable and connection management. Its plugin architecture allows teams to extend virtually every layer of the system without forking core code.
The project is maintained by a large and active Apache community with over 3,000 contributors and is in production at hundreds of organizations including Airbnb, LinkedIn, Twitter, ING, and NASA. It is the de facto standard for Python-native workflow orchestration in data engineering and MLOps.