Yao Meta Skill
Engineering, evaluating, governing, and packaging repeatable workflows as portable agent skills.
The README specifies local-first metadata-only telemetry, avoidance of raw-content collection, explicit sources, approval-gated adaptation proposals, permission probes, release locks, and a public-claim guard. An interface fixture also shows policies forbidding remote inline execution and allowing remote metadata only, providing substantive design evidence for least privilege, confirmation, data flow, and sensitive-data handling. Deductions apply because most guarantees remain product descriptions rather than supplied implementation code; global installation, GitHub metadata access, browser native messaging, and telemetry imports have external effects without fully shown per-action consent and recovery boundaries. Dependency contents, vulnerability scanning, locking, and supply-chain policy are absent. Rollback is only indirectly represented by expiring waivers, upgrade checks, and release governance. Attribution reaches the repository and “Yao Team,” but the publisher is unknown and no verified individual or organization is identified.
The README, CI workflow, and test helper present a broadly consistent Python 3.11, make ci-test, packaging, install-simulation, permission-probe, and report-generation workflow. Invalid-YAML and missing-field fixtures show attention to failure paths, while the helper captures stdout/stderr and checks return codes. Deductions reflect that only orchestration fragments are supplied, without core implementations, assertions, results, or complete user-facing error messages. The contents of requirements-ci.txt are missing, and availability or graceful degradation for npx, GitHub, browser integration, and target clients is mostly asserted.
The source distinguishes creation, team hardening, beta release, and post-release maintenance scenarios and explicitly contrasts when YAO, Anthropic, and OpenAI approaches fit. Skill IR, multiple compilation targets, four archetypes, compatibility records, and stated exclusions provide thorough capability boundaries. Trigger precision loses a point because only a few positive natural-language examples are shown, with no detailed ambiguity, conflict, or negative-trigger rules. Environment fit is useful but centered on Python 3.11, Codex, and named adapters; Windows, restricted-network operation, shell differences, and client-version requirements are underdocumented.
The README provides strong navigation, an architecture flow, lifecycle explanations, a command catalog, and linked companion reports. Global installation, restart requirements, development setup, and CI commands are concrete. The complete MIT text is present, while beta status, single-reviewer evidence, missing provider execution, native-permission gaps, and absent real-client telemetry are candidly disclosed, justifying full marks for license and known limitations. Deductions apply because the supplied README truncates during Quick Start, preventing confirmation of a complete FAQ, example quality, or all troubleshooting notes. The 1.0-to-2.0 story is clear, but no supplied changelog, tags, or release migration record corroborates it. Maintenance is attributed only to an unverified “Yao Team,” with no clear contact, support channel, or accountable update owner.
The system targets usable installable packages, JSON/Markdown/HTML reports, review pages, compiled adapters, and operational queues; the test helper demonstrates how many of those artifacts are assembled. A shared IR plus evaluation, governance, and portable release controls offers plausible marginal value over a prompt-writing guide. Deductions apply because no substantive generated-output samples or independent usage outcomes are supplied. The 91.5 benchmark and 5/5 preference result are project-reported, and the latter uses one reviewer with empty per-case rationales. The large command and governance surface may impose considerable maintenance and context costs, with no quantified effort, latency, or scale threshold.
Many claims are mapped to named reports, ledgers, manifests, commands, and test stages, and the helper connects several of those claims to specific scripts and artifact paths, giving the project a useful internal traceability structure. The README carefully separates local engineering scores, single-reviewer preference evidence, planned external evidence, and completed provider evidence, thoroughly distinguishing facts from inference. Deductions apply because the cited reports, benchmark methodology, decisions, core scripts, and test results are not included in the supplied source. Cross-source corroboration is consequently limited to the README, CI configuration, fixtures, and one orchestration helper, which cannot statically validate most quality and safety claims.
- This assessment covers only the supplied static excerpts; no code, tests, installation, or network behavior was executed.
- Do not treat the README's 91.5 score, 5/5 blind-review result, or “world-class” governance features as independently verified; the referenced reports and decision records were not supplied.
- Installation uses npx and can write globally across multiple agents. Pin and inspect dependencies, confirm write scope, and validate in an isolated environment before adoption.
- Browser native messaging, GitHub metadata access, and telemetry imports broaden data and permission boundaries. Verify that implementations consistently enforce metadata-only collection, raw-content exclusion, and approval gates.
- Publisher identity is unknown. Enterprise or critical-process use should separately establish maintainers, update channels, vulnerability response, and succession arrangements.
What does this agent do, and when should you use it?
Yao Meta Skill is an engineering and governance system for reusable agent skills, not merely a prompt generator. It turns workflows, transcripts, prompt sets, notes, and runbooks into an intent model and platform-neutral Skill IR, then produces a lean SKILL.md, interface metadata, references, scripts, evaluations, and reports. Its unified scripts/yao.py CLI covers initialization, validation, trigger optimization, target compilation, output evaluation, permission probing, packaging, install simulation, and release-evidence checks. Compilers target OpenAI, Claude, generic Agent Skills-compatible packages, and VS Code-oriented workflows, although some runtime features degrade to metadata: all four currently probed adapters lack native permission enforcement. It is best suited to teams converting personal operating knowledge into durable, reviewable assets; its governance and evidence machinery will likely be excessive for a one-off prompt.
The workflow begins with raw prompts, transcripts, documents, notes, runbooks, or repeated operational tasks. quickstart clarifies the job, outputs, exclusions, constraints, and quality standards before generation. export_skill_ir.py builds the Skill IR from SKILL.md, agents/interface.yaml, manifests, evaluations, resources, and reports; compile_skill.py and cross_packager.py then produce contracts and packages for OpenAI, Claude, generic, Agent Skills-compatible, and VS Code-oriented targets. trigger_eval.py, run_eval_suite.py, optimize_description.py, and judge_blind_eval.py test activation accuracy, blind holdouts, adversarial cases, and route collisions, while run_output_eval.py, run_output_execution.py, and adjudicate_output_review.py create assertion grades, execution records, blind-review packs, and adjudication reports. governance_check.py, trust_check.py, probe_runtime_permissions.py, verify_package.py, simulate_install.py, and world-class-claim-guard examine ownership metadata, trust boundaries, permission declarations, archive safety, temporary installation, and public-claim limits. Outputs include installable skill packages, target adapters, registry metadata, and HTML, Markdown, and JSON evidence such as Skill Overview, Review Studio, regression history, portability, adoption drift, and release reports.
- An agent developer has a frequently reused prompt set, notebook, or operating procedure and wants an installable skill with explicit triggers, inputs, outputs, and exclusions.
- An internal tooling team is promoting a personal skill into shared infrastructure and needs ownership, lifecycle metadata, evaluations, permission notes, release gates, and review evidence.
- A team maintaining skills across multiple clients wants to compile OpenAI, Claude, generic Agent Skills, and VS Code-oriented adapters from one Skill IR while tracking semantic degradation.
- A maintainer preparing a beta release needs package verification, install simulation, Python compatibility checks, runtime-permission probes, and cross-report evidence validation.
- A skill-library operator wants metadata-only adoption and drift signals for missed triggers, output failures, or script errors, followed by approval-gated improvement proposals.
- An evaluation engineer needs train, development, visible holdout, blind holdout, adversarial, and route-confusion tests for skill activation descriptions, with auditable promotion records.
What are this agent's strengths and limitations?
- A platform-neutral Skill IR acts as the shared semantic layer for generating multiple target contracts, adapters, and compatibility records.
- Evaluation extends beyond file structure to trigger regressions, blind and adversarial holdouts, an independent rubric judge, output assertions, execution evidence, and human-review packs.
- Release controls are concrete and executable: archive verification, temporary install simulation, permission probes, evidence-consistency checks, and a public-claim guard.
- The unified scripts/yao.py CLI, Makefile, and GitHub Actions workflow provide a repeatable local engineering path.
- Metadata-only telemetry explicitly excludes command arguments and raw content, while durable changes remain approval-gated proposals.
- The large collection of reports, ledgers, gates, and evaluation components creates substantial adoption and maintenance overhead for small or one-off skills.
- The reported OpenAI, Claude, generic, and VS Code adapters currently rely on permission metadata rather than native enforcement: zero native-enforcement adapters and four fallbacks.
- The repository describes itself as beta-ready, but provider-backed production evidence, independent model execution, broader human evidence, and real-client telemetry remain separate unfinished evidence tasks.
- The published 5/5 blind comparison involved only one reviewer, and its per-case rationale fields were still empty.
- Local development requires Python 3.11 or newer; global installation uses npx, and target packages still require review of unsupported features and degradation notes.
- The project's own onboarding and review-experience score is 6.5/10, identified as its clearest area for improvement.
How do you install or deploy this agent?
Install globally for Codex:
npx -y skills add yaojingang/yao-meta-skill -a codex -g -yTo install for every agent supported by the installer:
npx -y skills add yaojingang/yao-meta-skill -a '*' -g -yRestart the client after installation. Local development requires Python 3.11 or newer:
python3.11 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install --requirement requirements-ci.txt
make ci-testThe npx installation step requires network access. Local development and artifact generation require shell and filesystem access. No API credential is listed as a base requirement, although GitHub benchmark scanning, update checks, and PR metadata inspection require access to GitHub.
How do you use this agent?
After installation and restart, invoke it with requests such as “create a skill from this workflow,” “improve this existing skill,” or “add evals to this skill.” A first CLI workflow is:
python3 scripts/yao.py quickstart --output-dir .
python3 scripts/yao.py validate my-skill
python3 scripts/yao.py skill-ir . --output-json skill-ir/examples/yao-meta-skill.json
python3 scripts/yao.py compile-skill . --target openai --target claude --target generic --target vscode
python3 scripts/yao.py package . --platform generic --output-dir dist
python3 scripts/yao.py testTo create a named package directly:
python3 scripts/yao.py init my-skill --description "Describe what the skill does."
python3 scripts/yao.py validate my-skillReview reports/skill-interpretation.html first, followed by reports/skill-overview.html and reports/review-studio.html for interpretation, scores, blockers, permission approvals, and evidence paths. Before release, run the package-verify, install-simulate, runtime-permissions, and evidence-consistency gates.
How does this agent compare with similar options?
The repository positions Anthropic Skill Creator as a conversation-first, human-guided iteration method and OpenAI Skill Creator as a compact guide to writing lean, context-efficient skill instructions. Yao Meta Skill targets the heavier case: team assets requiring evaluation, governance, evidence, release gates, and cross-platform packaging. Its internal weighted engineering review scores Yao at 91.5, Anthropic at 67.5, and OpenAI at 50.5, but explicitly labels those figures as local project evidence rather than independent proof of world-class readiness. The repository also recommends a hybrid approach: draft conversationally, then use Yao to harden boundaries, evaluations, packaging, and release evidence.