Maze: Distributed Framework for LLM Agents
Turn LLM agent programs into distributed, observable workflows scheduled across heterogeneous GPU/CPU/IO resources, with unified recovery, model serving, and artifact management.
Workflow docs mention task sandboxes and relative paths but no least-privilege model for agent tasks, so least_privilege is 1; cancel/retry are user-initiated yet no confirmation mechanism for risky operations, user_confirmation is 1; artifact addressing, workspace-to-sandbox staging rules, and model-route states are documented, data_flow_transparency is 2; OpenAI-compatible endpoints are used with zero guidance on keys or sensitive data, sensitive_data_handling is 1; versions are pinned but include suspicious entries (zmq==0.0.0 is a placeholder package; transformers>=5.12.1 does not match any published series), dependency_security is 1; the system auto-deploys/scales model instances and reserves GPUs with no described safeguards, external_effects is 1; retry, cancel, and restart-safe run discovery exist but no rollback semantics, rollback is 1; citation and acknowledgements are present but the author email is a [email protected] placeholder and repository URLs conflict, source_attribution is 2.
README claims GA-grade capabilities while pyproject declares Development Status 4 - Beta, and News entries are dated 2026 (future), self_consistency is 1; PyPI install, vllm extra, and pinned ray give adequate dependency availability, dependency_availability is 2; structured error fields (error_type, retryable, origin, node_id, attempt, traceback) are thoroughly documented, failure_messages is 2.
Targets distributed LLM-agent development and research with static/dynamic workflow, app-spec, and Workbench examples, audience_and_scenarios is 2; it explicitly states GAIA templates are integration exercises not benchmark claims and distinguishes missing_model from undeployed checkpoints, capability_boundaries is 2; trigger/dispatch semantics for agent tasks are thinly described, trigger_precision is 1; ports, env vars, Python versions, and model dir are documented, but the OS-Independent classifier sits uneasily with GPU cluster reality, environment_fit is 2.
README is well organized (architecture, quick start, workflows, ops, Workbench), information_architecture is 2; pip, source install, start and worker commands are given, install_notes is 2; package name maze-agent vs repo Maze, and pyproject Homepage points to QinbinLi/Maze while README points to maze-agent/Maze, naming_stability is 1; code and YAML examples are rich but there is no FAQ, examples_and_faq is 2; no known-limitations section beyond scattered boundary notes, known_limitations is 1; full MIT text in LICENSE consistent with pyproject, license is 3; a version and News timeline exist but no formal CHANGELOG, versioning_changelog is 2; maintainer is the placeholder '[email protected]' with no real contact or governance, maintenance_responsibility is 1.
Unified Run API, CLI, and Workbench outputs (result summaries, logs, artifact references) are usable, output_usability is 2; workflow contracts, heterogeneous scheduling, and durable runs above Ray carry plausible marginal value though without comparison to alternatives, marginal_value is 2; GPU deployment and reservation carry significant cost with no cost/benefit guidance, cost_benefit is 1.
Performance and stability claims (deterministic GPU cleanup, restart-safe discovery, DCT EMA scheduling gains) have no attached tests or data to trace statically, claim_traceability is 1; README, pyproject, and LICENSE contradict each other (2026 dates, two different repository URLs, placeholder email), cross_source_corroboration is 1; however the GAIA template vs benchmark-accuracy distinction is explicit, so fact_inference_separation is 2.
- pyproject.toml author/maintainer emails are the placeholder '[email protected]'; maintainer identity cannot be verified.
- Suspicious dependency entries: zmq==0.0.0 is a placeholder package, not real pyzmq; transformers>=5.12.1 does not match any published version series — verify dependency resolution before installing.
- README and pyproject point to different repository URLs (maze-agent/Maze vs QinbinLi/Maze), and News/citation dates are 2026 (future); scrutinize supply-chain provenance.
- The system auto-deploys and scales in/out vLLM/Transformers instances and reserves GPUs; review resource and external-effect controls before production use.
- No guidance on API keys or sensitive data handling is provided; manage credentials yourself when wiring OpenAI-compatible endpoints.
- The package declares Beta status while the README claims GA capabilities; this is a static review and no runtime claims were executed or verified.
What does this agent do, and when should you use it?
Maze (GitHub: maze-agent/Maze, MIT-licensed) is an open-source framework that turns agent programs into distributed, observable workflows. It comprises Maze Core (persisted Runs, events, logs, artifacts), a Ray-based scheduler with independent gpu/cpu/io queues and FCFS/HACS algorithms, a Python SDK, a LangGraph adapter, and a visual Workbench. All clients submit DAGs through the same maze.workflow/v1 contract, so static @workflow definitions, dynamic DynamicRuns, and maze.yaml application specs share one execution surface. The framework also discovers local model checkpoints, deploys reusable vLLM or Transformers instances on demand, and manages GPU leases with LRU scale-in. Task failures carry structured error fields and support retries, timeouts, cancellation, and restart-safe state recovery. Its research paper was accepted to SC26, with contributions from Huazhong University of Science and Technology, Huawei, and other institutions.
Users define task DAGs with resource declarations via @task/@workflow decorators or a maze.yaml app spec, then submit them through MaClient (client.create_workflow_from, client.create_dynamic_run) or POST /workflows/submit. Core assigns a run_id; the scheduler orders ready tasks via FCFS or HACS across gpu/cpu/io queues, a node placement strategy (e.g., least-loaded) picks a registered node, and Ray executes on head/worker nodes. Model tasks declaring a model_anchor trigger checkpoint discovery under model_cache and on-demand deployment of vLLM/Transformers instances with an injected OpenAI-compatible endpoint. During execution, Maze persists events, logs, placement, and content-addressed artifacts (maze://artifacts/sha256/<hash>), queryable and operable via CLI (maze runs/cluster commands), SDK, or Workbench, including cancel and retry.
- ML platform engineers running agent workflows with inference steps on multi-GPU clusters who need GPU, CPU, and IO tasks to queue independently
- Research teams that want to build workflows visually in Maze Workbench with a DAG editor, reusable task catalogs, and server-side workspaces
- Pipeline developers who need to append tasks at runtime (DynamicRun) rather than pre-defining a fixed DAG
- Teams using LangGraph that need durable execution, restart recovery, and cross-node scheduling via the LangGraph adapter
- Ops engineers who prefer declaring application jobs (command, conda env, artifacts, timeouts, retry policy) in maze.yaml within a unified Run history
- Cluster administrators who need warm standby workers, crash re-registration, and scheduler-managed GPU leases with LRU scale-in for model instances
What are this agent's strengths and limitations?
- Independent gpu/cpu/io queues prevent one resource class from blocking another, plus the paper-aligned HACS scheduling algorithm with tunable MAZE_HACS_* parameters
- Run state persists across process restarts, retaining task state, structured errors, events, logs, retries, and content-addressed artifacts
- Scheduler-managed local model execution: automatic vLLM/Transformers deployment, explicit Model Wait state, GPU lease reuse, and LRU scale-in
- SDK, LangGraph, Workbench, and app specs converge on the maze.workflow/v1 contract and the same Core Run APIs, avoiding divergent execution paths
- Distributed execution is built on Ray, and every Ray node must additionally register as a Maze worker, adding a two-layer runtime to operate
- Distributed runs without shared storage require the Head content-addressed artifact store, and task code must use relative workspace paths
- Model execution depends on node-local checkpoints: missing_model means no eligible node reported the checkpoint, so you must provision model_cache yourself
- The GAIA templates (reason/file/speech/vision) are workflow integration examples; the README explicitly disclaims any reproduced GAIA benchmark accuracy
- Recent timeline (SC26 paper, 2026 updates) means limited evidence of production community adoption
How do you install or deploy this agent?
Install from PyPI: pip install maze-agentOr from source:
git clone https://github.com/maze-agent/Maze.git
cd Mazepip install -e .
Requires a Python environment and Ray; distributed deployment needs network access to the head node.
How do you use this agent?
- Start head and Workbench: maze start --head --port 8000 --playground --detach (Workbench at http://localhost:5173, Core API at http://localhost:8000).
- Add a worker: maze start --worker --addr HEAD_IP:8000 --agent --heartbeat-interval 20; every Ray node must also register as a Maze worker before it receives tasks.
- Inspect the cluster: maze cluster resources --server-url http://HEAD_IP:8000.
- Define a @workflow and submit via MaClient, or run an app spec with maze run maze.yaml --wait.
- Operate runs: maze runs list / show / logs / retry <run_id>, or SDK calls like client.get_run and client.cancel_run.
- Manage the service: maze status, maze doctor, maze stop.