Astronomer Agents: AI Agent Toolkit for Airflow Data Engineering
AI agent tooling for data engineering workflows—unifying Airflow orchestration, warehouse analytics, and DAG development in one package.
- Source repo
- astronomer/agents
- Stars
- ★ 454
- Last updated
- 15d ago
- License
- Apache-2.0
- Primary language
- Python
- FA score
- 46/100 · Major gaps
At a glance
- How it runs
- Works with
- Universal · cross-platformClaude CodeChatGPT (Partial support)
- Cost
- Free software; you pay for model usage
- Setup effort
- Medium · a few setup steps
- You'll need
- Typical use
- A data engineer using Claude Code inside an Airflow project wants to write a new DAG; they call the
authoring-dagsskill for best-practice suggestions and runtesting-dagsto validate locally. - Not a fit if
- Teams that need fully offline operation: warehouse and Airflow access require network calls
- Source review
- 46/100 · Major gaps 1 safety controls not found
What does this agent do, and when should you use it?
Astronomer Agents is an Apache-2.0 licensed open-source project by Astronomer that equips AI coding agents with specialized capabilities for data engineering. The repository delivers three interlocking pieces: an MCP server (astro-airflow-mcp) exposing Airflow's REST API as Model Context Protocol tools for DAG management, triggering, and log retrieval; a CLI utility named af for terminal-based Airflow interaction; and over a dozen Skills—reusable prompt/instruction modules—that extend agents like Claude Code and Cursor with domain knowledge for warehouse discovery, lineage tracing, DAG authoring, testing, deployment, dbt integration, and Airflow upgrades. Installation is client-driven: a Claude Code plugin, a Cursor one-click MCP setup, or a generic npx skills add command for 25+ agents. The MCP server is compatible with any Airflow 2.x/3.x REST API endpoint, making it viable for both Astro and self-hosted deployments.
Concretely, the agent performs these operations: The Airflow MCP server (launched via uvx astro-airflow-mcp --transport stdio) converts Airflow REST API interactions into MCP tools, letting AI assistants list DAGs, trigger runs, fetch task logs, and check system health. The af CLI (invoked with uvx --from astro-airflow-mcp af) enables commands such as af health, af dags list, and af runs trigger <dag_id>. Skills extend agents with workflows: warehouse-init profiles configured databases (Snowflake, PostgreSQL, BigQuery, SQLAlchemy-compatible) to generate .astro/warehouse.md schema docs; analyzing-data runs SQL on a background Jupyter kernel to answer business questions; profiling-tables computes table statistics; lineage skills (tracing-downstream-lineage, tracing-upstream-lineage) analyze data flow; DAG skills (authoring-dags, testing-dags, debugging-dags) support code creation and validation; cosmos-dbt-core guides dbt Core integration; and migrating-airflow-2-to-3 assists version upgrades. Configuration lives in ~/.astro/agents/warehouse.yml and .env for warehouse credentials, with Airflow connection via environment variables or auto-discovery from a local dags/ folder.
- A data engineer using Claude Code inside an Airflow project wants to write a new DAG; they call the
authoring-dagsskill for best-practice suggestions and runtesting-dagsto validate locally. - An analyst who needs to understand warehouse schemas runs
warehouse-initto generate.astro/warehouse.md, then usesanalyzing-datato query customer tables via natural language. - A platform team debugging a failed DAG uses the
debugging-dagsskill for deep root-cause analysis and theafCLI to inspect task logs. - A data governance engineer traces the impact of a schema change using
tracing-downstream-lineageandtracing-upstream-lineageskills. - A team porting dbt Core projects into Airflow employs the
cosmos-dbt-coreskill to integrate with Astronomer Cosmos. - An organization upgrading from Airflow 2.x to 3.x follows the
migrating-airflow-2-to-3skill to refactor DAGs safely.
How do you install or deploy this agent?
Prerequisites: Python with uv (or pip), Node.js for npx, and appropriate client. Installation options: 1. Claude Code: run claude plugin marketplace add astronomer/agents then claude plugin install astronomer-data@astronomer. 2. Cursor: use the one-click install button or manually add to ~/.cursor/mcp.json: {"mcpServers":{"airflow":{"command":"uvx","args":["astro-airflow-mcp","--transport","stdio"]}}}. 3. Generic MCP clients: execute uvx astro-airflow-mcp --transport stdio (optionally set AIRFLOW_API_URL, AIRFLOW_USERNAME, AIRFLOW_PASSWORD). 4. Skills: run npx skills add astronomer/agents --skill '*' (omit --skill to select individually). Configure warehouse connections in ~/.astro/agents/warehouse.yml and credentials in ~/.astro/agents/.env.
How do you use this agent?
- Trigger warehouse initialization: invoke
/astronomer-data:warehouse-initin the agent to generate.astro/warehouse.md. 2. Ask natural-language queries like "Show me revenue trends by product" or "Create a DAG that loads data from S3 to Snowflake daily"; the agent auto-invokes relevant skills. 3. Use theafCLI for direct terminal operations: aliasaf='uvx --from astro-airflow-mcp af', thenaf health,af dags list,af runs trigger <dag_id>. 4. For remote Airflow, setAIRFLOW_API_URL,AIRFLOW_USERNAME,AIRFLOW_PASSWORD, orAIRFLOW_AUTH_TOKENenvironment variables. Run from a directory containingairflow.cfgordags/to enable auto-discovery.
What are this agent's strengths and limitations?
- Deep Airflow integration: MCP server and
afCLI cover DAG listing, triggering, and log retrieval, reducing context switching. - Extensive Skills library spans the entire data engineering lifecycle—warehouse discovery, lineage, authoring, testing, debugging, deployment, dbt, and migration—and auto-triggers from natural language.
- Client-agnostic distribution: works with Claude Code via plugin, Cursor via MCP, and any MCP-compatible client; Skills installable to 25+ agents via npx.
- Open-source friendly: compatible with self-hosted Apache Airflow 2.x/3.x, not locked to Astronomer's managed platform.
- Warehouse connectors are built-in for only Snowflake, PostgreSQL, and BigQuery; other databases require SQLAlchemy configuration and manual driver installation.
- Runtime dependencies: requires
uvx(Python) andnpx(Node.js) availability, plus a network connection for package fetching and remote API calls. - The
afCLI collects anonymous telemetry unless disabled, which may raise data-governance concerns. - Advanced features like human-in-the-loop (HITL) to
airflow-hitlskill require Airflow 3.1+ and may not apply to older deployments.
How does this agent compare with similar options?
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| Astronomer Agents: AI Agent Toolkit for Airflow Data Engineering This agent | 46 · Major gaps | Agent plugin / skillFree + model costs | ★ 454 | 15d ago | Python | Claude Code |
| DecisionBox | 44 · Major gaps | Self-hosted serviceFree + model costs | ★ 120 | 1d ago | Go | OpenAI API · Claude API |
| QuantDinger AI Trading OS | 89 · Good | Self-hosted serviceFree + model costs | ★ 13k | today | Python | Codex · Claude Code · OpenAI API |
| ktx — Context layer for data agents | 64 · Some gaps | CLIFree + model costs | ★ 1.6k | 3mo ago | TypeScript | Codex · Claude Code |
How does FollowAgents rate this agent?
Why each dimension lost points
Evidence: SECURITY.md provides vulnerability disclosure process; README mentions anonymous telemetry with opt-out; configuration uses environment variables and file permissions, but least privilege principle not explicit. Deductions: no explicit least privilege, user confirmation, data flow transparency, sensitive data handling details, dependency security audit, external effects control, rollback mechanism, source attribution.
Evidence: README and CI config consistent; tests disable telemetry; CI matrix covers Python and Airflow versions. Deductions: dependency availability not explicit, failure messages not detailed.
Evidence: README clearly identifies target users (data engineers) and scenarios (data discovery, DAG development); skill descriptions clear; supports multiple clients and environments. Deductions: capability boundaries not fully explicit, trigger precision relies on skill descriptions, environment fit not comprehensive.
Evidence: README well-structured; installation instructions detailed; examples and FAQ provided; Apache-2.0 license complete. Deductions: naming stability not explicit, known limitations not fully documented, versioning/changelog missing, maintenance responsibility not explicit.
Evidence: Output usability high (generates warehouse.md etc.); marginal value clear (automates data engineering workflows). Deductions: cost-benefit not quantified, relies on external services.
Evidence: README and CI config traceable, but no independent verification. Deductions: claim traceability limited, cross-source corroboration insufficient, fact-inference separation not explicit.
- Not found in source: rollback or recovery pathBack up first, or work on a git branch or snapshot, so its changes can be undone.
- Repository includes telemetry; verify data collection scope.
- Relies on external services (PyPI, GitHub); ensure supply chain security.
- No changelog provided; assess upgrade risks.
Recent releases
Latest entries from the project's GitHub Releases. Notes are quoted from upstream as written, not rewritten or tested by FollowAgents.
-
astro-airflow-mcp-0.9.1 astro-airflow-mcp 0.9.1
Release notes excerpt
Patch release: fixes 421 Misdirected Request on the plugin's /mcp/v1/ endpoint with FastMCP 3.4.3+, and pins fastmcp to a bounded range. What's fixed since 0.9.0 Fix 421 Misdirected Request from FastMCP's host guard (#241) FastMCP 3.4.3 shipped HostOriginGuardMiddleware enabled by default: it only accepts loopback Host headers (127.0.0.1/localhost/::1) plus the ASGI scope["server"] host, and returns 421 Misdirected Request for anything else. In plugin mode the MCP app is embedded in the Airflow webserver and reached over the Deployment's own hostname, so with an …View on GitHub ↗ -
astro-airflow-mcp-0.9.0 astro-airflow-mcp 0.9.0
Release notes excerpt
Minor release: every MCP tool now returns structured, agent-actionable errors and carries MCP safety annotations, plus af runs list gains end-date filters. What's new since 0.8.2 Structured tool errors + MCP safety annotations (#221) Tools previously collapsed every failure to a bare str(e) — an agent couldn't tell a wrong dag_id (fix the arg and retry) from Airflow being unreachable (stop and surface), since both came back as the same opaque string. …View on GitHub ↗ -
astro-airflow-mcp-0.8.2 astro-airflow-mcp 0.8.2
Release notes excerpt
Patch release: fix af instance discover silently reporting "No instances discovered" when ASTRO_API_TOKEN is set in the environment. What's fixed since 0.8.1 - Skip preamble lines when parsing astro CLI table output (#213): When ASTRO_API_TOKEN is set, astro CLI prepends Using an Astro API Token to stdout. The af table parser was treating that line as the header row, collapsing every astro deployment list row to a single column called using_an_astro_api_token and dropping every deployment - discover would silently return zero instances. The parser now walks lines from the …View on GitHub ↗
FAQ
Is this only for Astronomer's Astro platform?
AIRFLOW_API_URL to your self-hosted instance. Skills are tool-agnostic and work with any Airflow deployment.What credentials and permissions do I need?
warehouse.yml and secrets in .env. Schema discovery via warehouse-init only scans databases listed in the config; querying can access any database your user has permissions for.