Vet Code Change Verifier
Reviews code changes and coding-agent behavior to surface issues that deserve developer attention.
Per-dimension scores and reasoning
The README explains that repository snapshots, diffs, and optional conversation history are sent to the selected inference provider and states that the project performs no data collection. The GitHub Action example limits permissions to contents: read and pull-requests: write, and the shell-execution risk of --history-loader is explicitly disclosed. Deductions apply because redaction, provider retention, secret protection, and complete transmission boundaries are not documented; most dependencies are not locked, while Actions and installation material use mutable tags or main; proactive skill runs, PR comments, registry downloads, and cache writes lack a unified per-action confirmation or reversal mechanism. Authorship, repository, and licensing are attributed, but individual maintenance ownership is not identified.
The README, package metadata, and release workflows are broadly consistent about the product, Python requirement, CLI entry point, license, and publishing process. Distinct exit codes cover success, runtime errors, configuration errors, and findings. Deductions apply because concrete error messages, diagnostic guidance, and fallback behavior are not shown; many runtime dependencies have no upper or exact pin, and operation also depends on model providers, subscription-backed agent harnesses, and a remote registry.
The developer audience and terminal, agent-skill, CI, and pull-request scenarios are thoroughly identified. Multiple providers, OpenAI-compatible endpoints, custom model definitions, team profiles, and issue-guide customization provide substantial flexibility. Deductions apply because unsupported languages, repository types, and review classes are not enumerated; proactive skill execution lacks precise trigger and suppression rules; Python versions and installation methods are covered, but operating-system, Git-state, network, and resource constraints are incomplete.
The README has strong organization and covers CLI, skill, CI, configuration, output, and support paths. Both automated and manual installation instructions are concrete. AGPL-3.0-only is consistent across the README, pyproject, and complete license text. Deductions apply because verify-everything, vet, and the vet repository coexist as names; examples are useful but no real FAQ or systematic troubleshooting section is present; known limitations are sparse beyond the history-loader warning and Beta classifier; semantic package versions and tag-driven releases exist, but no separate changelog is supplied; Imbue, Issues, and Discord establish an update path without named maintainers, response expectations, or a support policy.
Text, JSON, and GitHub output modes plus distinct exit codes make results usable from terminals and CI, while the described review pipeline plausibly adds coverage beyond tests and linters. Deductions apply because no JSON schema, representative finding structure, severity semantics, or false-positive workflow is provided. Claims about finding issues missed by tests and linters have no benchmarks, case results, or evaluation evidence in the supplied files. The software is free, but inference cost, token use, runtime, and recommended model capacity are not quantified.
Many operational claims connect to concrete commands, configuration fields, exit codes, package metadata, dependencies, and publishing workflows. License, version, and release assertions receive useful corroboration across the README, LICENSE, pyproject, and workflows. Deductions apply because core quality claims, the no-data-collection statement, and the effectiveness of filtering and deduplication remain README assertions without supplied implementation, tests, or independent evidence. Marketing conclusions are generally distinguishable from configuration facts, but assumptions, inferences, and evidence levels are not explicitly labeled.
- The recommended skill installer downloads and immediately executes a shell script from the main branch; pin it to a reviewed commit and inspect it before use.
- --history-loader executes arbitrary shell commands with the current user's privileges, so shared configuration presets must be reviewed as executable code.
- Source code, diffs, and conversation history may be sent to external model providers; verify provider retention terms and remove secrets before using private repositories.
- The GitHub Action example references imbue-ai/vet@main and release workflows use mutable Action tags; supply-chain-sensitive deployments should pin full commit digests.
- Most Python dependencies are not exactly locked, and the remote model registry can be updated independently; production use should add lockfiles, provenance checks, and an update-review process.
- AGPL-3.0-only can impose source-availability obligations for modified network services; obtain a license-compliance review before integrating or hosting it.
What does this agent do, and when should you use it?
Vet is a standalone verification tool for code changes and coding-agent behavior. It snapshots a repository and its diff, and can include the stated goal and agent conversation to check both implementation correctness and goal adherence. It runs from the terminal, as an installed coding-agent skill, or in pull-request workflows through a reusable GitHub Action. Vet invokes configured models, then filters and deduplicates their findings into a final issue list available as text, JSON, or GitHub-oriented output. It supports built-in Anthropic, OpenAI, and Gemini models plus custom models exposed through OpenAI-compatible endpoints, with requests sent directly to the selected inference provider. It fits developers and teams that want model-driven review in local or CI workflows while retaining control of provider credentials and review configuration.
Vet captures the current repository state and diff, optionally adds a user goal and coding-agent conversation, and can compare changes against a ref supplied with --base-commit. It runs model checks for issues such as incorrect code and divergence from the requested goal, then filters and deduplicates the findings. The CLI starts with vet "<goal>" and can use direct model APIs or an existing coding-agent subscription through --agentic with --agent-harness claude, codex, or opencode. Model definitions can come from .vet/models.json, the user-level models.json, built-in definitions, or the remote registry cached by vet --update-models. Its reusable GitHub Action sets up Python, installs Vet, computes the merge base, and posts the review to the pull request. Output formats are text, json, and github; exit codes distinguish success with no findings (0), runtime failure (1), invalid usage or configuration (2), and detected issues (10).
- A developer who used a coding agent runs Vet before committing to check whether the implementation still matches the original request and whether the new code contains defects.
- A team performing a storage-layer refactor compares the work with main through --base-commit main to review only the relevant change set.
- A GitHub project adds the reusable Action to review every non-draft pull request and post findings directly to the PR.
- A developer with Claude Code, Codex, or OpenCode access uses --agentic to run verification through an existing supported subscription instead of calling model APIs directly.
- A team standardizes CI reviews with TOML profiles for model selection, enabled issue codes, output format, and thresholds, while tailoring checks through guides.toml.
- An organization using a private or third-party model gateway defines an OpenAI-compatible provider, model ID, context window, and output limit in models.json.
What are this agent's strengths and limitations?
- It evaluates both the code diff and, when supplied, the user goal and agent conversation, so it can flag behavioral mismatch as well as implementation problems.
- The same verifier works through a local CLI, an agent skill, and a reusable GitHub Action for pull-request review.
- Built-in providers, custom OpenAI-compatible endpoints, and an updateable model registry give teams multiple model-selection paths.
- Text, JSON, and GitHub output plus distinct exit codes make the result practical to consume in CI and other automation.
- Named TOML profiles and issue-specific guides let teams standardize settings and tailor review criteria.
- Vet requires no product account and states that it collects no data; inference requests go directly to the user's provider.
- Verification depends on model inference; API operation requires provider credentials and may incur charges from that provider.
- The AGPL-3.0-only license requires legal review for organizations planning to modify, distribute, or offer the software over a network.
- The --history-loader option executes its configured shell command as the current user, so shared commands and presets must be reviewed before use.
- The documented GitHub Action defaults to Anthropic models, requiring an ANTHROPIC_API_KEY secret and permission to write pull-request feedback.
- Registry models do not become available until vet --update-models downloads and caches their definitions.
- The supplied material does not state supported Python versions, measured review accuracy, or performance limits for large repositories, so adopters must validate those locally.
How do you install or deploy this agent?
Install the CLI with one of the documented package commands:
pip install verify-everythingor:
pipx install verify-everythingor:
uv tool install verify-everythingTo install the coding-agent skill, run this from the target repository and choose project-level or user-level installation when prompted:
curl -fsSL https://raw.githubusercontent.com/imbue-ai/vet/main/install-skill.sh | bashProject-level installation writes the skill under .agents/skills/vet/, .opencode/skills/vet/, .claude/skills/vet/, and .codex/skills/vet/ at the repository root. User-level installation uses the corresponding user skill directories. API-based operation requires credentials for the selected inference provider; the documented GitHub Action example stores ANTHROPIC_API_KEY as a repository secret.
How do you use this agent?
From the Git repository to be checked, run a first review with:
vet "Implement X without breaking Y"To compare the work against the main branch, run:
vet "Refactor storage layer" --base-commit mainTo use a supported coding-agent harness and subscription, run:
vet "Implement X without breaking Y" --agentic --agent-harness claudeSupported --agent-harness values are claude, codex, and opencode. For a custom OpenAI-compatible model, define its provider and model in .vet/models.json or the user-level models.json, then invoke it by name:
vet "Harden error handling" --model gpt-5.2Run vet --update-models to download and cache remote registry definitions. Teams can keep named defaults in .vet/configs.toml and customize valid issue-code guidance in .vet/guides.toml with prefix, suffix, or the mutually exclusive replace field. For pull requests, create a GitHub Actions workflow using imbue-ai/vet@main with contents: read and pull-requests: write permissions.