SkillHone
Turn the real Skill failures your coding agents hit into tested, reviewable, continually improving skills — with every decision preserved as local Git history.
- Source repo
- Tencent/SkillHone
- Stars
- ★ 169
- Last updated
- 1mo ago
- License
- NOASSERTION
- Primary language
- TypeScript
- FA score
- 85/100 · Good
At a glance
- How it runs
- Works with
- Universal · cross-platformCodex · Claude Code
- Cost
- Free, no paid service needed
- Setup effort
- Low · running in minutes
- You'll need
- Typical use
- A developer using Claude Code or Codex who discovers a Skill references missing files and wants the agent to capture the evidence and return a tested local PR
- Not a fit if
- Teams needing a hosted service or central console
- Users who only want single-prompt tuning without maintaining Skill repositories
- Environments without a local coding-agent runtime
- Source review
- 85/100 · Good
What does this agent do, and when should you use it?
SkillHone is an open-source tool from Tencent for continual agent skill evolution through persistent decision history. When coding agents like Claude Code or Codex hit a broken Skill during real work — a missing script, an API change, an instruction that stopped working — they report the reproducible failure on the spot. SkillHone queues it as an Issue, repairs the complete Skill repository (SKILL.md, scripts, references, assets, tests), runs the regression test, and produces a local PR for human approval. Each Skill has its own repository and decision history — failures, changes, test outcomes, rejected candidates — which the repairing agent cannot rewrite while it works. Two modes cover quick maintenance without building benchmarks and evaluation-driven optimization against a frozen Eval repository. It originated from an EMNLP 2026 Industry Track paper and is released under the MIT License.
SkillHone consumes runtime feedback from coding agents: an agent records reproducible Skill failure evidence while working, and SkillHone converts it into an Issue in that Skill's repository. It then orchestrates the repair — updating SKILL.md, scripts/, references/, assets/, and hidden regression tests in .test/ in one atomic change — verifies it with the regression test, and packages the Issue, Git diff, run record, and local PR into an approval inbox. The local workbench shows every Skill's Issues, tests, repair runs, commits, changed files, PR approval state, and Wiki records. Automatic local merge is opt-in and still requires linked tests to pass; pushing to a remote remains a separate user action. Full mode adds baseline runs, candidate iterations, and validation gates against a separate frozen Eval repository.
- A developer using Claude Code or Codex who discovers a Skill references missing files and wants the agent to capture the evidence and return a tested local PR
- A team maintaining many custom Skills that needs an auditable decision history (failure cause, test, diff, approval state) for every change
- An engineer tired of benchmark-building overhead who wants everyday maintenance driven by real failures (Quick mode)
- A research or platform team running evaluation-driven optimization over whole Skill repositories (Full mode with a frozen Eval repo)
- A user wary of autonomous skill rewrites who wants human approval for every PR and tamper-proof repair history
How do you install or deploy this agent?
SkillHone installs directly from GitHub — no package registry account and no hosted service. Give this prompt to your coding agent (Claude Code, Codex, etc.):
Install SkillHone from the `main` branch of
https://github.com/Tencent/SkillHone. Follow
docs/install/skillhone.md, make the SkillHone Skills and CLI available to
this Agent, and verify the installation without changing unrelated files.
Use the documented `--install-links=true` Git installation command.Full installation documentation lives at docs/install/skillhone.md in the repository; once installed, the SkillHone Skills and CLI are available to that agent. A Chinese guide is at docs/README.zh.md.
How do you use this agent?
- Work normally in an agent equipped with SkillHone; when the agent hits a Skill failure it records reproducible evidence on the spot.
- SkillHone queues the problem as an Issue in the Skill's repository and automatically runs repair, regression testing, and a local PR.
- Open the local workbench (see the examples/phoenix-tracing/ example) to inspect the failure, the test, the exact diff, and approval state.
- Decide whether to merge in Review mode; automatic local merge is opt-in and still requires linked tests to pass; pushing is manual.
- For broader optimization jobs, set up a separate frozen Eval repository and run baselines, candidate iterations, and validation gates in Full mode.
What are this agent's strengths and limitations?
- Repairs the whole Skill repository (SKILL.md, scripts, references, tests) rather than just rewriting prompts, fixing problems prompt-only optimizers cannot such as missing scripts or broken parsers
- Every repair is tied to an Issue, a regression test, a Git diff, a run record, and a local PR; failed or untested changes never enter the review queue
- Each Skill has its own decision history the repairing agent cannot rewrite, so the next agent continues from evidence instead of rediscovering failures
- Review mode keeps merge decisions with the user by default, and pushing is always a separate manual action
- Backed by an EMNLP 2026 Industry Track paper; in open-web evaluation, evolved Skills improved 15.8 points on GAIA over the reported commercial-retrieval research agent
- Depends on an agentskills.io-compatible runtime (Claude Code, Codex, OpenClaw, Hermes); the core workflow is unusable without a local coding agent
- The GitHub repository license is marked NOASSERTION — verify the LICENSE file and third-party notices before adopting
- Repair quality depends on agent-reported evidence and regression test coverage; the docs do not state false-positive or failed-repair rates
- Merge policy and pushing must be configured and maintained by the user — extra overhead for teams expecting full automation
- Full mode requires building a separate frozen Eval repository, a non-trivial upfront investment
How does this agent compare with similar options?
The README contrasts SkillHone with prompt-only optimizers, which can only rewrite instructions and cannot fix a missing script or broken parser; SkillHone optimizes the entire Skill repository and proves the fix with regression tests.
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| SkillHone This agent | 85 · Good | Agent plugin / skillFree | ★ 169 | 1mo ago | TypeScript | Codex · Claude Code |
| Self-Learning Skills | 56 · Major gaps | Agent plugin / skillFree + model costs | ★ 971 | 1mo ago | — | Codex · Claude Code |
| Cline Pilot | 41 · Major gaps | Agent plugin / skill | ★ 2 | 20d ago | Python | Codex · Claude Code |
| LangWatch | 78 · Good | CLIFreemium | ★ 5k | today | TypeScript | OpenAI API · Claude API |
How does FollowAgents rate this agent?
Why each dimension lost points
Strong evidence: CI uses contents:read only, doctor probes in read-only permission mode, PR merge requires explicit --confirm with review-by-default, auto-merge is opt-in and the tool never pushes; tests prove credential redaction (issues, wiki, config output) and import rejection of literal secrets; web server binds 127.0.0.1; no third-party runtime deps, pinned build deps, hash-verified gitleaks; takeover/copy modes keep backups and refuse to overwrite files changed after import; failed repairs preserve partial commits and support retry. Deductions: external_effects scored 2 because the tool writes AGENTS.md and takes over user runtime directories via symlinks — with backups and conflict refusal, but irreversibility of takeover is only evidenced in tests; source_attribution scored 2 since THIRD_PARTY_NOTICES.md is referenced but not shown and the publisher is registry-unverified.
Tests cover self-consistency: structured failure.kind values (tool-call-protocol, tool-roundtrip-unverified), no PR on failed test gates, idempotent reinstall, pnpm9→npm fallback retaining pnpm11 build approval flags. Deduction: dependency_availability scored 2 because install paths depend on external tools (pnpm/npm, gitleaks download, separately installed DeepSeek Harness) whose availability is asserted but not statically provable.
Audience and scenarios are clear: Quick/Full mode table, multi-runtime install (codex/claude-code/cursor/zcode) with test backing, explicit Node engines, CI-verified Git install without a checkout cwd. Deductions: capability_boundaries scored 2 — the DeepSeek Harness native build scripts as a larger trust boundary is noted, but boundaries of what SkillHone can repair lack dedicated documentation; trigger_precision scored 2 — trigger policies (immediate/queued/scheduled) appear in tests but user-facing trigger semantics rely mainly on README prose.
Good information architecture: docs/, examples/, skills/, src/ layers, navigable README with a Chinese version; install docs are actually executed in CI; stable naming (skillhone, @skillhone/cli, skillhone-auto-optimization). Deductions: examples_and_faq scored 2 — an example fixture is referenced but no FAQ/troubleshooting section; known_limitations scattered; LICENSE file is MIT while registry metadata says NOASSERTION — metadata inconsistency; versioning_changelog only 2.0.0-dev.0 plus a News list, no formal CHANGELOG; maintenance_responsibility lacks maintainer commitments or update cadence.
Strong output usability: --json structured output, local workbench showing issues/tests/diffs/PRs/approval state, PR bodies with change summaries and test counts; clear marginal value — persisting failure evidence normally lost in chat as Issue + regression test + PR, repairing the whole Skill repo rather than only prompts. Deduction: cost_benefit scored 2 — model/token/time costs of optimization runs are not quantified in the sample, and the GAIA/WebWalkerQA gains are paper claims, not repo-verifiable cost/benefit.
Most key claims trace to in-repo tests (redaction, confirmation gates, install verification, signed tamper-evident audit) and CI steps. Deductions: claim_traceability scored 2 — the 15.8/3.2-point improvements rest on an arXiv paper unverifiable in static review plus external issue/PR links; cross_source_corroboration scored 2 — no independent corroboration within the sample; fact_inference_separation scored 2 — the README mixes paper results with current capabilities, though tests vs. claims are broadly distinguished.
- Publisher is registry-unverified and license metadata is NOASSERTION; verify the LICENSE yourself (the in-repo file is MIT).
- The tool takes over Skill directories in user Agent runtimes via symlinks and writes AGENTS.md; backups and conflict refusal exist, but confirm backup locations and recovery paths before enabling takeover.
- The GAIA/WebWalkerQA gains are paper claims — external evidence not verifiable in static review; do not treat them as validated results.
- DeepSeek Harness is installed separately and its native build scripts are treated as a larger trust boundary; pin the tested version when updating.
- This is a static source review; no install or repair flow was executed. Verify actual behaviour against revision-pinned evidence independently.