Kestra Workflow Orchestration Platform
Open-source event-driven orchestration and scheduling platform for data, AI, and infrastructure workflows.
- Source repo
- kestra-io/kestra
- Stars
- ★ 30k
- Last updated
- today
- License
- Apache-2.0
- Primary language
- Java
- FA score
- 53/100 · Major gaps
At a glance
- How it runs
- Works with
- Universal · cross-platform
- Cost
- Free tier plus a paid hosted plan
- Setup effort
- Medium · a few setup steps
- You'll need
- Typical use
- Data engineers can use Kestra to orchestrate ETL pipelines, extracting data from databases, cloud storage, or APIs, transforming it, and loading it into data warehouses.
- Not a fit if
- Teams that must run fully offline without fetching plugins from Maven Central
- Users who want a lightweight script and won't learn YAML and Kestra concepts
- Small environments that can't run Docker or Kubernetes
- Source review
- 53/100 · Major gaps 1 safety controls not found
What does this agent do, and when should you use it?
Kestra is an open-source, event-driven orchestration platform for data, AI, and infrastructure workflows. It unifies scheduled and event-driven automation behind a declarative, language-agnostic interface, bringing Infrastructure as Code best practices to data, process, and microservice pipelines. The platform offers an intuitive UI and code editor for building workflows visually or via YAML, with features like Git version control integration, a rich plugin ecosystem, and high availability and fault tolerance for scaling to millions of workflows. Kestra supports local and remote execution, including Docker and Kubernetes, and provides monitoring and notification integrations.
Kestra defines workflows as YAML files that are parsed and executed by its core engine. Users can create and manage workflows via the UI or API, using the built-in code editor with syntax highlighting, auto-completion, and real-time validation. Workflows can be triggered on a schedule or by events such as file arrivals or message queue events. Tasks include running scripts in various languages (Python, Node.js, R, Go, Shell, etc.), executing SQL queries, making HTTP requests, running Docker containers, or launching Kubernetes jobs. Workflows support subflows, retries, timeouts, error handling, input/output variables, conditional branching, dynamic tasks, and parallel task execution. Kestra integrates with Git for version control and offers a Terraform provider, and can be deployed on AWS via CloudFormation or GCP via Terraform.
- Data engineers can use Kestra to orchestrate ETL pipelines, extracting data from databases, cloud storage, or APIs, transforming it, and loading it into data warehouses.
- Platform teams can use Kestra for infrastructure automation, such as running scheduled scripts, executing Ansible playbooks, or triggering Kubernetes jobs.
- AI engineers can leverage Kestra to orchestrate machine learning pipelines, including data preprocessing, model training, and deployment, with integration for MLOps tools.
- Operations teams can use Kestra to monitor and manage critical business processes, receiving notifications via Slack, email, or PagerDuty.
- Developers can use Kestra's local development mode to quickly start in Docker, test workflows, and collaborate via Git version control.
How do you install or deploy this agent?
Kestra can be deployed via Docker. Ensure Docker is installed, then run:
docker run --pull=always -it -p 8080:8080 --user=root \
--name kestra --restart=always \
-v kestra_data:/app/storage \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /tmp:/tmp \
kestra/kestra:latest server localFor Windows PowerShell, use backticks; for CMD, use ^. Access the UI at http://localhost:8080. Other deployment options include Docker Compose, Podman, Kubernetes, and cloud deployments (AWS CloudFormation, GCP Terraform).
How do you use this agent?
After launching Kestra, create a new flow in the UI. For example, create a flow named hello_world with:
id: hello_world
namespace: dev
tasks:
- id: say_hello
type: io.kestra.plugin.core.log.Log
message: "Hello, World!"Run the flow and view the output. Use the built-in editor to write YAML and the topology editor to visualize workflows. Triggers can be scheduled or event-based.
What are this agent's strengths and limitations?
- Declarative YAML interface with UI and Git version control for Infrastructure as Code.
- Rich plugin ecosystem supporting multiple programming languages, cloud services, and databases.
- Supports event-driven and scheduled workflows with high availability and fault tolerance.
- Visual construction with real-time validation lowers the learning curve.
- Requires learning YAML syntax and Kestra concepts, which may be steep for new users.
- Docker deployment may be heavy for small use cases.
- Plugin ecosystem, while extensive, may have varying quality and maintenance.
How does this agent compare with similar options?
Kestra can be compared to other orchestration platforms such as Apache Airflow or Prefect. Compared to Airflow, Kestra emphasizes event-driven and declarative configuration, while Airflow is primarily Python-based with DAGs.
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| Kestra Workflow Orchestration Platform This agent | 53 · Major gaps | Self-hosted serviceFreemium | ★ 30k | today | Java | — |
| Agentspan | 62 · Some gaps | CLIFree + model costs | ★ 497 | 23d ago | TypeScript | OpenAI API · Claude API |
| Agency Orchestrator: One Sentence to Orchestrate an AI Expert Team | 61 · Some gaps | CLIFree + model costs | ★ 2.4k | today | TypeScript | Codex · Claude Code · OpenAI API · Claude API |
| Dagu Workflow Orchestrator | 74 · Some gaps | CLIFreemium | ★ 4.4k | today | Go | Codex · Claude Code · Claude API |
How does FollowAgents rate this agent?
Why each dimension lost points
Evidence shows: README emphasizes workflows as code and Git version control integration, but does not explicitly mention least privilege; user confirmation mechanism not mentioned; data flow transparency limited, only mentions UI and API modifications auto-adjust YAML; sensitive data handling not explicit; dependency security has CodeQL scanning and SECURITY.md reporting process; external effects include executing scripts, calling APIs, but authorization mechanism not explicit; rollback not mentioned; source attribution has Apache 2.0 license and copyright notice. Deductions: lack of specific evidence for least privilege, user confirmation, data flow transparency, sensitive data handling, external effect authorization, and rollback.
Evidence shows: README describes features consistently without contradictions; dependency availability is indicated through Docker images and plugin ecosystem, but no dependency list provided; failure messages not explicit. Deductions: insufficient evidence for failure messages and dependency availability.
Evidence shows: README clearly identifies target users for data, AI, and infrastructure workflows, with rich scenarios; capability boundaries are explained through plugin ecosystem and task types; trigger precision is defined through event-driven and scheduled triggers; environment fit supports Docker, Kubernetes, and cloud platforms. Deductions: limited details on capability boundaries and trigger precision.
Evidence shows: README has clear structure with table of contents; installation instructions detailed (Docker commands); naming stable (Kestra brand); examples and FAQ include Hello World example; known limitations not explicit; license is Apache 2.0; versioning and changelog via GitHub releases and badge; maintenance responsibility via contribution guide and community channels. Deductions: known limitations not explicit.
Evidence shows: output usability via UI and YAML definitions; marginal value via unified scheduling and event-driven; cost-benefit via open source and community support. Deductions: lack of specific performance or cost data.
Evidence shows: claims in README are supported by documentation links, but no specific evidence provided; cross-source verification limited; facts and inferences not clearly separated. Deductions: insufficient claim traceability, cross-source verification, and fact-inference separation.
- Not found in source: confirmation before actingTurn on (or add) a confirmation step before it acts, and try it in a sandbox or test environment before real data.
- Publisher identity is unverified; assess supply chain risks carefully.
- Feature claims in README lack specific evidence; consult official docs and tests.
- Least privilege and user confirmation mechanisms are not explicit; configure security policies yourself.
Recent releases
Latest entries from the project's GitHub Releases. Notes are quoted from upstream as written, not rewritten or tested by FollowAgents.
-
Release notes excerpt
Changelog 📘 Subtasks version - b531f5c update to version '1.3.43' 🐛 Bug Fixes core - db82eb4 keep the value timezone in the date filter, closes #11129 - b5caed0 stop retry race from duplicating task runs and ending executions early (#18947), closes #18947 flow - ea2daee preserve metadata for failed flows (#19814), closes #17806 #17808 #19814 flows - b7c74bc stop the file watcher rewriting unchanged flows in a loop, closes #6847 storage-local - 97a934a create destination parent on move and move/delete companion metadata (#18836), closes #18836 system …View on GitHub ↗ -
Release notes excerpt
Changelog 📘 Subtasks version - b59444f update to version '2.0.6' 🐛 Bug Fixes core - 019fb8e neutralize formula characters in CSV exports - 043f765 keep the value timezone in the date filter, closes #11129 - 264562d stop retry race from duplicating task runs and ending executions early (#18947), closes #18947 execution - cbcae8e fail on unknown onChildFailure executions - 314cdba retry a task with CREATE_NEW_EXECUTION inside a loop, closes #20515 - 728931e complete loops with empty URI values (#20561), closes #20011 #20561 …View on GitHub ↗ -
Release notes excerpt
Changelog 🚀 Features core - 777ceef centralized HTTP retry mechanism in core (#19904), closes #19904 worker - 0b35619 reach namespace files through the controller, configurably - 0dff4d1 make the fetched job buffer size configurable 📘 Subtasks core - fe0ba63 remove duckdb ld_preload workaround from selfrun.sh (#20249), closes #20249 version - 172dd0f update to version '2.0.5' 🐛 Bug Fixes core - 12312b7 fall back to the default tab when no tab is requested (#19923), closes #19918 #19923 execution …View on GitHub ↗