Prefect

lastmod 2026-11-10created 2024-06-042 edits

Prefect is a workflow orchestration tool for defining, running, and observing data pipelines in Python. You define flows and tasks by adding decorators to ordinary Python functions, so the code is simpler than in Airflow, where DAGs must be written in a separate syntax. It provides retries, scheduling, logs, and a run-state UI, and comes as open source and as a cloud service.

See also

• Workflow management system (WfMS; WFMS; Workflow Engine)
• Kubeflow
• Airflow
• JupyterFlow
• Argo Workflows