OSS Autopilot
An AI workflow engine for open source contributors: track PRs, act on maintainer feedback, diagnose CI failures, and discover matched issues to keep contribution velocity up.
Strong evidence: SECURITY.md details non-persisted tokens, execFileSync array-only subprocess calls, 0o600/0o700 permissions, atomic writes, wrapUntrustedContent injection fencing with a regression corpus; CI forbids mcp__* wildcards in agent frontmatter; external effects are gated by human-in-the-loop (post/claim require explicit approval; overnight mode never pushes/posts/merges). Deductions: dependency_security only 2 because the pnpm audit step in ci.yml is continue-on-error (despite a documented reason), so dependency auditing is not a hard gate; data_flow_transparency 2 because README references external oss-widgets.vercel.app endpoints, in tension with the 'GitHub only' claim; source_attribution 2 because the named author's identity is unverified and claims like '3rd-largest Ink contributor' cannot be corroborated in-repo.
Structured CLI errors ({ success, data, error, timestamp }), an FAQ troubleshooting section (gh auth, build failures, missing PRs), and CI gating that fails loudly when the e2e bundle is missing support dependency availability and failure messages. Deduction: self_consistency only 2 — the README contradicts itself on numbers ('2,600+ tests' vs '3,000+ across 120+ files'; 8 agents in one place vs 7 in the table; '26 commands' in SECURITY.md vs '35+' in README), irreconcilable under static review.
Three deployment models (plugin/MCP/CLI), a clearly scoped individual-contributor audience, and a candid Limitations section (GitHub-only, 1,000-PR Search API cap, no team workflows) support audience and boundaries. Deductions: trigger_precision 2 — agents are 'automatically dispatched by Claude based on context' but exact trigger conditions are not fully evidenced in the files; environment_fit 2 — requires Node>=22, gh CLI, pnpm, and first-run build failures require manual find+rebuild; cross-platform automation is only partially shown.
Clear information architecture (monorepo diagram, command and config tables), complete install notes for all three paths, LICENSE file matching the MIT badge, release-please automated versioning with CI guards enforcing server./package./README version consistency, a specific Limitations section, and a thorough FAQ. Deduction: maintenance_responsibility 2 — a solo maintainer with a security-advisory path and a supported-versions table, but update continuity rests on an unverified individual.
Strong output usability: -- structured output, markdown/badge export, structured report directory. Deductions: marginal_value and cost_benefit both 2 — the 'every feature came from real usage' narrative is plausible but not statically verifiable; npx @latest by default pulls unpinned latest versions (supply-chain and behavior-drift exposure); the claimed 5-minute daily workflow benefit is asserted, not demonstrated in evidence.
Extensive issue/PR references (#1053, #1372, #1651), dedicated docs (repo-scores.md, anti-llm-policy.md) explaining decision heuristics, and generally clean separation of facts from rationale. Deductions: claim_traceability 2 — aggregate metrics (test counts, release counts) have no in-repo itemization to check; cross_source_corroboration 2 — CI embeds doc/code consistency guards (tool counts, reference.md sync) but static review cannot confirm they pass; fact_inference_separation 2 — promotional narrative ('built and used daily', contributor ranking) is intermixed with verifiable facts.
- The pnpm audit CI step is continue-on-error, so dependency-vulnerability auditing is not a hard gate; run your own audit before adoption.
- npx @latest pulls unpinned latest versions by default — supply-chain and behavior-drift exposure; pin versions.
- README self-contradicts on counts (tests, agents, commands); verify metrics independently before citing them.
- Publisher identity is unverified; claims such as '3rd-largest Ink contributor' are not independently corroborated.
- Overnight/unattended mode prepares local worktrees and writes reports; although it claims never to push/post, review its artifacts on first use.
What does this agent do, and when should you use it?
OSS Autopilot (GitHub: costajohnt/oss-autopilot) is an open source TypeScript tool shipped in three forms: a Claude Code plugin, an MCP server, and a standalone CLI, for managing personal open source contributions at scale. Each run live-fetches all your PRs via GitHub's Search API, flagging maintainer feedback, CI failures, and merge conflicts, and matches new issues to your contribution history. The pnpm-workspaces monorepo contains three npm packages: @oss-autopilot/core, @oss-autopilot/mcp, and @oss-autopilot/dashboard (Preact + Vite). Its architecture pairs a deterministic core with an AI orchestration layer: PR status classification, CI failure triage, and state management live in tested TypeScript (3,000+ tests), not in prompts. It is strictly human-in-the-loop: nothing is posted to GitHub without your explicit approval.
When you run /oss for the daily check, the tool calls GitHub's Search API to fetch PRs you authored and enriches each with CI status, review decisions, merge-conflict detection, maintainer comment classification, and checklist completion — no PR data is stored locally. The pr-responder agent drafts replies to maintainer feedback; pr-health-checker diagnoses CI failures under a deterministic taxonomy (actionable / fork limitation / auth gate / infrastructure); issue-scout searches and vets new issues matched to your history; repo-evaluator scores repositories on two 1-10 scales (cached history score from your own PR outcomes, fresh health score from a weighted rubric) and runs anti-LLM policy detection that scans CONTRIBUTING/CODE_OF_CONDUCT/README and skips projects rejecting AI-assisted contributions. /oss-overnight runs unattended, prepares fix branches in local worktrees, and writes a morning report — it never pushes, posts, or merges. The MCP server exposes 30 tools, 6 resources, and 4 prompts; every CLI command supports structured -- output ({ success, data, error, timestamp }).
- An individual contributor juggling 10+ active PRs across upstream repos who wants a 5-minute daily triage routine for CI failures and maintainer replies.
- A developer who does not use Claude Code but works in Cursor, Claude Desktop, Codex, or Windsurf, and wants the same capabilities via the MCP server.
- A developer integrating PR monitoring or issue discovery into scripts, importing runDaily/runSearch from @oss-autopilot/core and consuming -- output.
- A contributor who wants a live contributions badge on their GitHub profile README, using stats --badge and the oss-widgets Shields.io endpoints.
- A long-term contributor in AI-friendly projects who wants /pr-ready pre-push review loops and per-repo guidelines to avoid shipping rough code to maintainers.
What are this agent's strengths and limitations?
- Three deployment forms share one core library: Claude Code plugin (8 specialized agents + 9 slash commands), MCP server (30 tools / 6 resources / 4 prompts), and standalone CLI — switch clients without losing functionality.
- Deterministic-core architecture: PR classification, CI failure triage, and state management are validated by 3,000+ tests in TypeScript, giving reproducible behavior independent of prompt quality.
- Production-grade GitHub API integration: ETag-based caching, rate-limit backoff with retries, bounded concurrency pools, and paginated fetching, correctly handling cross-fork diff ranges, squash counting, and --head flags — built for daily use without hitting limits.
- Strong human-in-the-loop guardrails: nothing is posted without explicit approval; factual claims in drafts are verified against the diff; state files are written with 0o600 permissions and Zod-validated on read.
- GitHub only: GitLab, Bitbucket, and other forges are explicitly unsupported.
- A 1,000-PR cap: GitHub's Search API returns at most 1,000 results per query, so the oldest records may be truncated past that.
- Individual-contributor focus: no team dashboards, shared state, or multi-user workflows.
- The best experience is tied to the Claude Code plugin ecosystem; MCP/CLI paths cover the same core features but reach the specialized agents only through tools and commands.
- Overnight scheduling is documented only for launchd; no systemd or other scheduler support is documented.
How do you install or deploy this agent?
Option 1 (recommended): in Claude Code run /plugin marketplace add costajohnt/oss-autopilot, then /plugin install oss-autopilot@oss-autopilot, restart Claude Code, and run /setup-oss. Option 2 (MCP server for Cursor / Claude Desktop / Codex / Windsurf): run npx @oss-autopilot/core@latest init <your-github-username> once, then add {"mcpServers":{"oss-autopilot":{"command":"npx","args":["@oss-autopilot/mcp@latest"]}}} to your MCP client config. Option 3 (standalone CLI): npx @oss-autopilot/core daily -- or npm install -g @oss-autopilot/core. All paths require GitHub authentication (the CLI reuses the gh auth token automatically; if it fails, run brew install gh && gh auth login). Node.js is required.
How do you use this agent?
Daily workflow (~5 min): 1) run /oss to see what needs attention; 2) work through critical issues (CI failures, maintainer comments, conflicts); 3) done. Overnight mode: /oss-overnight prepares fix branches in local worktrees and writes reports to ~/.oss-autopilot/reports/ without pushing, posting, or merging; schedule it via oss-autopilot overnight schedule --install (launchd). Other commands: /oss-search to find issues, /oss-dashboard for the interactive dashboard (also npx @oss-autopilot/core dashboard serve at http://localhost:3000), /oss-guidelines for per-repo guidelines, /pr-ready for the pre-push review loop, /setup-oss for interactive configuration. Configuration lives in ~/.oss-autopilot/state.; key settings include githubUsername, maxActivePRs (default 10; 3-5 recommended when starting), dormantDays (default 30), minStars (default 50), excludeRepos/excludeOrgs, and boostIssueTypes.