SkillAlchemy
An open-world agent skill creation system that turns underspecified briefs and public sources into installable, reusable agent skills.
Evidence is limited to README, LICENSE, and package.. The system fetches open-world sources and writes to output/, but no permission scoping or network behavior is documented (least_privilege=1); no user confirmation or review step is described for generation/installation (user_confirmation=1); data flows — what is fetched and what leaves the machine — are entirely undisclosed (data_flow_transparency=1); "Distill people" builds Persona Skills from public information about real individuals with no privacy-handling policy (sensitive_data_handling=1); only an npx install path is given, with no dependency manifest, lockfile, or audit note (dependency_security=1); file writes and network access are external effects with no boundary statement (external_effects=1); no uninstall, rollback, or recovery path is mentioned at all (rollback=0); the paper, authors, and arXiv link are present, and attribution is good, but the publisher is unverified, so source_attribution=2.
README and package. are broadly consistent on name and positioning, but the content is thin with no internal files to cross-check (self_consistency=1); reliance on npx skills and target agent environments with no availability guarantees or version requirements (dependency_availability=1); no error handling, failure messages, or degradation behavior is described anywhere (failure_messages=0).
The Features list gives concrete scenarios (RAG review, methodology distillation, persona skills) and a clear audience (audience_and_scenarios=2); the boundary of produced-skill reliability is one sentence about evidence scoping with no constraints (capability_boundaries=1); triggering is free-form natural language with no precise conditions or misfire protection (trigger_precision=1); Claude Code and Codex are explicitly supported with install commands (environment_fit=2).
README is well structured, but TECHNICAL.md and the skills/ directory are not present in the evidence, so architecture confirmability is low (information_architecture=1); three install paths (agent instruction, npx, individual skills) are clearly described (install_notes=2); the SkillAlchemy name is consistent across README and package. (naming_stability=2); usage examples exist but no FAQ (examples_and_faq=1); no known limitations, failure cases, or applicability boundaries at all (known_limitations=0); MIT LICENSE file is complete and consistent with metadata (license=3); package. says v1.0 but there is no CHANGELOG or version history (versioning_changelog=1); no maintainer, update policy, or support channel, and publisher identity is unverified (maintenance_responsibility=0).
Output path (output/) is explicit, but the artifact format, structure, and quality standard are undefined in the visible files (output_usability=1); claimed gains such as the 55.8% pass rate cannot be verified from the repository, so marginal value is asserted rather than shown (marginal_value=1); no discussion of costs (tokens, network fetching) versus benefits (cost_benefit=1).
Performance claims point to an arXiv paper, but the identifier 2608.23417, the 2026 date, and model names such as DeepSeek-V4-Pro, Opus 4.8, and GPT-5.5 cannot be corroborated at this static review point, making the headline claims non-traceable (claim_traceability=1); no second in-repo source corroborates the benchmark results (cross_source_corroboration=0); experimental results and product description are interleaved without evidence labeling (fact_inference_separation=1).
- The benchmark figures (55.8% pass rate, etc.), the arXiv identifier, and the model names in the README cannot be verified in this static review; do not treat them as established fact.
- The system performs open network fetching and local file writes with no documented permission boundaries, confirmation mechanism, or rollback; trial it in an isolated environment first.
- 'Distill people' generates Persona Skills from public information about real individuals — a privacy and reputational risk, with no privacy-handling policy in the repository.
- Publisher identity is unverified, and the repository lacks maintainer and update-commitment information; long-term viability is unknown.
- No known limitations, no CHANGELOG, and no failure-handling documentation; audit the actual generated skill content yourself before production adoption.
What does this agent do, and when should you use it?
SkillAlchemy is an open-source, open-world agent skill creation system accompanying the paper "SkillAlchemy: Open-World Agent Skill Creation." It targets unfamiliar capabilities where briefs omit requirements, expert procedures may not exist, and execution traces are unavailable. It frames skill creation as a source-grounded procedure-admission problem: it first discovers implicit requirements missing from the brief, then acquires grounded procedures from documentation, repositories, papers, and issue reports, and determines whether evidence supports a reusable instruction, a scoped example, or exclusion. It compiles admitted knowledge into Skill packages that agents can load and use directly, written to the output/ directory of the active project. It works with both Claude Code and Codex and bundles individually installable skills such as Lens and LEAP. On 87 SkillsBench v1.1 tasks it averages a 55.8% task pass rate across four agent–model configurations, slightly exceeding human-curated skills. The project is released under the MIT License.
SkillAlchemy's end-to-end flow: it reads an underspecified skill brief (e.g., "create a Skill for reviewing RAG systems"), first recovering omitted implicit requirements, constraints, and operational dimensions. It then acquires candidate procedures from heterogeneous public sources — documentation, repositories, papers, issue reports — and evaluates the breadth of evidence behind each, admitting reusable instructions, scoping context-specific examples, or excluding unsupported content. It can also distill people into Persona Skills from public evidence (decisions, failures, values, communication patterns), distill methods into executable skills with conditions, steps, branches, and failure handling, and fuse existing workflows into new capabilities. Finally it compiles admitted procedures, examples, references, and supporting resources into an installable Skill package written to output/. Skills install via npx skills add into Claude Code or Codex.
- An engineer who needs an agent skill for an unfamiliar domain (e.g., RAG system review) but lacks expert procedure docs, wanting the system to recover missing requirements from public docs and papers
- A knowledge worker who wants to turn a book, methodology, repository, or interview into an executable skill with conditions, branches, and failure handling
- A team lead managing agents who wants to fuse existing workflows or domain knowledge into one new capability
- A researcher in automated skill creation who needs to compare auto-generated skills against human-curated ones on SkillsBench
- A Claude Code or Codex user who wants to install bundled skills like Lens or LEAP individually to enhance their agent
What are this agent's strengths and limitations?
- Backed by quantitative evaluation: on 87 SkillsBench v1.1 tasks it averages 55.8% pass rate across four agent–model configurations, highest in 3 of 4, +19.9 points over no-skill execution
- Matches and slightly exceeds human-curated skills (54.4%) on average, and beats the strongest automated baseline by 8.6 points
- Distinctive evidence-admission mechanism separates reusable instructions from context-specific examples rather than treating all retrieved findings as universally valid
- Works with both Claude Code and Codex; skills are individually installable and the MIT license eases integration
- Depends on Claude Code or Codex as the runtime agent; no support for other agent platforms is documented
- Not a clean sweep: under the Codex + DeepSeek-V4-Pro configuration (43.9%) it falls below human-curated skills (45.7%)
- Installation relies on the npx skills CLI (Node.js required); offline or network-restricted operation is undocumented, and open-world source acquisition depends on network access
- The paper is a 2026 arXiv preprint (arXiv:2608.23417), not yet peer-reviewed; technical details should be independently verified
How do you install or deploy this agent?
The easiest path is to ask Claude Code or Codex: "Install SkillAlchemy from https://github.com/agentsope/SkillAlchemy and show me how to use it." Or install from the command line: npx skills add agentsope/SkillAlchemy. Node.js is required for npx. Individual components can also be installed: npx skills add agentsope/SkillAlchemy/skills/Lens, npx skills add agentsope/SkillAlchemy/skills/LEAP, or any skills/<skill-name>.
How do you use this agent?
After installation, describe the skill you want and specify sources, e.g.: "Use SkillAlchemy to create a Skill for reviewing RAG systems. Use public documentation and research papers as sources." It discovers implicit requirements, acquires procedures from public sources, determines evidence-backed scope, and writes the generated Skill package to output/ in the active project; the agent can then load and use the skill directly. Supported target agents are Claude Code and Codex.
How does this agent compare with similar options?
The README's evaluation table compares it directly with Anthropic Skill-Creator, OpenAI Skill-Creator, OpenSkill, MUSE-Autoskill, and human-curated skills; on average SkillAlchemy (55.8%) beats these automated baselines (best: MUSE-Autoskill at 47.2%) and slightly exceeds human-curated skills (54.4%), making it a fit when you need to create skills in unfamiliar domains rather than relying on vendor Skill-Creator templates.