Apache AirflowWorkflow38/100 via “distributed task execution with pluggable executors”
Industry-standard workflow orchestration.
Unique: Pluggable executor architecture decouples task scheduling from execution infrastructure, allowing same DAG code to run on laptop (LocalExecutor), Celery cluster, or Kubernetes without modification. Supervisor process on workers manages task lifecycle with subprocess isolation, enabling graceful shutdown and resource cleanup. XCom system provides lightweight inter-task communication via database, avoiding need for external message passing for small payloads.
vs others: More flexible executor abstraction than Prefect (which is cloud-first) or Dagster (which couples execution to deployment), but requires more operational overhead than managed services like AWS Step Functions or Google Cloud Workflows.