Coze Loop
A self-hosted platform for AI agent prompt development, evaluation, and execution tracing.
- Source repo
- coze-dev/coze-loop
- Stars
- ★ 5.8k
- Last updated
- 1d ago
- License
- Apache-2.0
- Primary language
- Go
- FA score
- 26/100 · Major gaps
At a glance
- How it runs
- Works with
- Universal · cross-platformOpenAI API
- Cost
- Free software; you pay for model usage
- Setup effort
- Medium · a few setup steps
- You'll need
- Typical use
- A prompt engineer wants to interactively test a prompt and compare outputs from different LLMs before release.
- Not a fit if
- Teams wanting zero-config setup without supplying a model API key and Endpoint ID
- Teams without a Kubernetes cluster and Nginx Ingress that still want the Helm path
- Source review
- 26/100 · Major gaps 8 safety controls not found
What does this agent do, and when should you use it?
Coze Loop is a developer-oriented platform for the AI agent lifecycle, spanning prompt development, debugging, evaluation, and monitoring. Its visual Playground supports interactive prompt testing, comparison of outputs from different LLMs, and prompt version management. The evaluation module manages evaluation sets, evaluators, and experiments for automated checks of prompt and Coze agent outputs, including accuracy, conciseness, and compliance. Its observability workflow uses SDK trace reporting and query to capture the path from user input through model calls and tool execution, including intermediate results and exceptions. The open-source edition can be deployed with Docker Compose or a Kubernetes Helm Chart, with model settings supplied in deployment configuration files.
After deployment, Coze Loop accepts prompt-debugging inputs in its Playground, calls configured models, and displays outputs for comparison; it also stores prompt versions. Its evaluation workflow uses evaluation sets, evaluators, and experiments to run automated tests and produce experiment-result statistics. Execution traces can be reported through an SDK, then queried in the platform to inspect user input, prompt parsing, model invocations, tool execution, intermediate results, and exceptions. For Docker Compose, users edit api_key and model in release/deployment/docker-compose/conf/model_config.yaml and run make compose-up; the Helm path uses release/deployment/helm-chart/umbrella/conf/model_config.yaml and make helm-up.
- A prompt engineer wants to interactively test a prompt and compare outputs from different LLMs before release.
- A team responsible for Coze agent quality needs evaluation sets, evaluators, and experiments to check accuracy, conciseness, or compliance.
- A platform or operations engineer needs to investigate model calls, tool execution, exceptions, and intermediate results within an agent request.
- A development team wants to self-host an agent-development and tracing platform in a Docker environment while configuring its own model credentials.
- A team with Kubernetes, Nginx Ingress, Kubectl, and Helm wants to deploy the platform through its Helm Chart.
How do you install or deploy this agent?
Install and start Docker Engine. Run:
git clone https://github.com/coze-dev/coze-loop.git
cd coze-loopEdit release/deployment/docker-compose/conf/model_config.yaml and set api_key and model; the README uses Volcengine Ark as its example. Then run:
make compose-upThe service starts in development mode by default. Open http://localhost:8082 in a browser.
How do you use this agent?
Before first use, configure a model API key and model Endpoint ID in model_config.yaml. After startup, open http://localhost:8082 and use the Playground to debug and compare prompt outputs; organize automated evaluation with evaluation sets, evaluators, and experiments; and query execution paths after integrating SDK trace reporting. For Kubernetes, configure release/deployment/helm-chart/umbrella/conf/model_config.yaml and templates/ingress.yaml, then run make helm-up.
What are this agent's strengths and limitations?
- Combines a prompt Playground, version management, evaluation sets/evaluators/experiments, and trace observability in one platform.
- Traces cover user input, prompt parsing, model calls, tool execution, intermediate results, and exceptions rather than only final answers.
- Offers two documented self-hosted deployment modes: Docker Compose and Kubernetes Helm.
- Explicitly supports OpenAI, Volcengine Ark, and other models, with multi-LLM support described through the Eino framework.
- Deployment requires configuring a model API key and model Endpoint ID, making a model service an external dependency.
- The Docker path requires Docker Engine; the Kubernetes path additionally requires a cluster, Nginx Ingress, Kubectl, and Helm.
- The documented public-network risks include account registration, listening-address configuration, SSRF, and some API horizontal-privilege-escalation concerns, so operators must assess and mitigate security exposure.
- The README does not provide a complete supported-model list for the open-source edition or describe feature differences across model configurations.
How does this agent compare with similar options?
The README positions this project as the open-source edition of the commercial Coze Loop: it provides core foundational modules at no charge and is intended to be customized and extended for business needs.
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| Coze Loop This agent | 26 · Major gaps | Self-hosted serviceFree + model costs | ★ 5.8k | 1d ago | Go | OpenAI API |
| Future AGI | 51 · Major gaps | Self-hosted serviceFreemium | ★ 2.3k | 3d ago | Python | Claude Code · OpenAI API · Claude API |
| Laminar | 41 · Major gaps | Self-hosted serviceFreemium | ★ 3.5k | today | TypeScript | OpenAI API · Claude API |
| MLflow | 64 · Some gaps | CLIFree + model costs | ★ 29k | today | Python | Claude Code · OpenAI API · Claude API |
How does FollowAgents rate this agent?
Why each dimension lost points
Evidence shows no explicit permission model, user confirmation, data flow transparency, sensitive data handling, dependency security, external effects, rollback, or source attribution. README mentions public deployment risks (e.g., SSRF, horizontal privilege escalation) but provides no mitigations. Hence all trust criteria score 0.
Self-consistency: README and CI workflows are consistent on Go version and build steps, but no runtime behavior evidence. Dependency availability: CI pins versions (e.g., golangci-lint v2.2.1) but no lock files or integrity checks. Failure messages: No documentation of error handling or user-facing failure messages.
Audience and scenarios: README clearly targets developers and describes full lifecycle management. Capability boundaries: Feature list outlines main modules but lacks detailed limitations. Trigger precision: No trigger mechanisms or precise conditions. Environment fit: Provides Docker and Helm deployment, supports multiple models, but does not specify all environment requirements.
Information architecture: README is well-structured with TOC and feature list. Install notes: Provides Docker and Helm steps. Naming stability: No stability promises for APIs or configs. Examples and FAQ: Quick start examples but no FAQ. Known limitations: Only mentions public network security risks, not comprehensive. License: Apache-2.0 license file complete. Versioning and changelog: No CHANGELOG or version history. Maintenance responsibility: Contribution guide and security reporting channel exist, but maintainers not clearly identified.
Output usability: Features described concretely but no actual output examples. Marginal value: Full lifecycle management offers differentiated value. Cost-benefit: Open-source free, but deployment and configuration costs not assessed.
Claim traceability: Feature claims in README not linked to specific code or tests. Cross-source corroboration: Only README, no other sources. Fact-inference separation: No distinction between facts and inferences.
- Not found in source: least-privilege scopingGrant only what the task needs: a dedicated account or read-only token, scoped to specific directories and repos.
- Not found in source: confirmation before actingTurn on (or add) a confirmation step before it acts, and try it in a sandbox or test environment before real data.
- Not found in source: data-flow disclosureWatch which external services it contacts (proxy or firewall logs) and keep sensitive data out until you know where it goes.
- Not found in source: sensitive-data handlingUse dedicated, low-privilege, revocable API keys — never production credentials — and keep secrets out of logs.
- Not found in source: dependency securityPin versions and run a dependency audit (npm audit, pip-audit) before installing; prefer running it in a container.
- Not found in source: disclosed external effectsEstablish which external systems it writes to, sends to or changes, and verify with test accounts or repos before production.
- Not found in source: rollback or recovery pathBack up first, or work on a git branch or snapshot, so its changes can be undone.
- Not found in source: verifiable attributionInstall from the official repo or registry and check the publisher and URL to avoid look-alike packages.
- Public deployment has security risks (e.g., SSRF, horizontal privilege escalation); assess and mitigate before deployment.
- No permission model or user confirmation mechanism; sensitive operations may lack control.
- Dependencies are not pinned, posing supply chain risks.
Recent releases
Latest entries from the project's GitHub Releases. Notes are quoted from upstream as written, not rewritten or tested by FollowAgents.
-
Release notes excerpt
Image Version > cozedev/coze-loop:1.5.1 > cozedev/coze-loop-python-faas:1.0.0 What's Changed * [feat][all] Release 1.4.0 by @lsy357 in https://github.com/coze-dev/coze-loop/pull/230 * [fix]auto task by @zero3233-bd in https://github.com/coze-dev/coze-loop/pull/232 * [feat][frontend] code evaluator route by @wec4dre96-w in https://github.com/coze-dev/coze-loop/pull/231 * [fix] auto task by @zero3233-bd in https://github.com/coze-dev/coze-loop/pull/235 * [feat][all] release v1.4.1 by @lsy357 in https://github.com/coze-dev/coze-loop/pull/236 …View on GitHub ↗ -
Release notes excerpt
Image Version > cozedev/coze-loop:1.5.0 > cozedev/coze-loop-python-faas:1.0.0 What's Changed * [feat][all] Release 1.4.0 by @lsy357 in https://github.com/coze-dev/coze-loop/pull/230 * [fix]auto task by @zero3233-bd in https://github.com/coze-dev/coze-loop/pull/232 * [feat][frontend] code evaluator route by @wec4dre96-w in https://github.com/coze-dev/coze-loop/pull/231 * [fix] auto task by @zero3233-bd in https://github.com/coze-dev/coze-loop/pull/235 * [feat][all] release v1.4.1 by @lsy357 in https://github.com/coze-dev/coze-loop/pull/236 …View on GitHub ↗ -
Release notes excerpt
Image Version > cozedev/coze-loop:1.4.1 > cozedev/coze-loop-python-faas:1.0.0 What's Changed * [feat][frontend] code evaluator route (#231) by @wec4dre96-w in https://github.com/coze-dev/coze-loop/pull/234 Full Changelog: https://github.com/coze-dev/coze-loop/compare/v1.4.0...v1.4.1View on GitHub ↗