PaperOrchestra Skill Pack
Turn any coding agent into an automated AI research-paper writer: submission-ready LaTeX from your experiment materials, with no API keys and no LLM SDKs.
Evidence: scripts declared purely deterministic, no network/LLM; API keys read from env only, never committed; aggregator has depth limits, prints discovery summary for review before reading, strips PII, flags [UNVERIFIED]; worklog-based rollback mentioned. Deductions: default --search-roots includes the whole home directory (~), broader than needed; the 'print summary then user reviews' confirmation is asserted but no enforced confirmation gate is visible in supplied files; dependencies use >= with no pins or hashes; attribution claims for paper-excerpted prompts cannot be independently verified.
Evidence: internally consistent skill structure; README matches requirements.txt (no LLM SDKs); schema validation, halt rules, coverage gates support consistency; documented 429 back-off and key fallback. Deductions: failure handling is mostly narrative; the silent fallback when SEMANTIC_SCHOLAR_API_KEY is absent can mask misconfiguration; failure-message quality is not verifiable statically.
Evidence: explicitly multi-host (Claude Code, Cursor, Antigravity, Cline, Aider, OpenCode) with per-host integration docs; aggregator self-skips when inputs exist; parameters (--agents, --since, --depth) fit varied environments. Deductions: only Claude Code gets concrete install commands, other hosts are deferred to a referenced doc; trigger precision via SKILL.md description field is untested.
Evidence: clear repo layout (skills/examples/docs), LICENSE, CITATION.cff, toy example, paper-fidelity map; MIT license includes excerpt attribution. Deductions: no CHANGELOG, no version numbers or release history; no evidence of maintenance responsibility or update path; known limitations not systematically documented; registry-level license metadata is NOASSERTION.
Evidence: outputs well defined (LaTeX paper, workspace structure, autorater reports); marginal value clear — turns paper prompts into host-executable skills runnable with zero API keys; cost info given (Exa ~$0.007/query, S2 rate limits). Deductions: paper-quality claims (50–68% win margins) are imported from an unverifiable external source, not evidenced in-repo; cost-benefit holds mainly for research-writing users.
Evidence: prompts carry page-number headers, paper-fidelity.md maps design decisions to pages, CITATION.cff plus BibTeX, out-of-paper improvements explicitly marked. Deductions: the cited arXiv:2604.05018 (dated 2026) is the sole external corroboration and cannot be cross-checked; benchmark claims are asserted, not independently evidenced; no test suite is declared.
- The aggregator defaults to scanning your entire home directory (~); always pass explicit --search-roots and --since to avoid the agent ingesting unrelated or sensitive chat logs.
- The unverified arXiv reference (2604.05018, dated 2026) is the sole source of all effectiveness claims; verify benchmark win-rate figures independently before relying on them.
- Dependencies use loose >= constraints with no hashes; pin versions yourself in enterprise settings.
- No versioning, no changelog, unclear maintenance responsibility; check commit history and issue activity before production use.
- Missing S2 key silently falls back to unauthenticated mode; if you depend on higher rate limits, confirm with --check-key.
What does this agent do, and when should you use it?
PaperOrchestra is a pluggable skill pack from the GitHub repository Ar9av/PaperOrchestra that converts the five-agent paper-writing pipeline described in arXiv:2604.05018 into skills executable inside coding agents such as Claude Code, Cursor, Antigravity, Cline, Aider, and OpenCode. Its core components are seven skills (an orchestrator plus outline, plotting, literature-review, section-writing, and content-refinement agents, along with benchmark and autorater skills) and an optional agent-research-aggregator. The execution model is instructions plus deterministic helpers: each skill bundles a SKILL.md instruction document, verbatim paper prompts and JSON schemas in references/, and purely local scripts in scripts/; all LLM reasoning and web search are delegated to the host agent's own tools, so the pipeline itself runs with zero API keys. Inputs are idea.md, experimental_log.md, a LaTeX template, and conference guidelines under workspace/inputs/, and outputs include a structured outline, figures, a citation-integrated literature review, and a final LaTeX paper. It suits researchers who already have experiment records and want their preferred coding agent to handle paper writing.
The orchestrator drives the pipeline: outline-agent turns the idea, experimental log, template, and guidelines into a structured outline JSON; plotting-agent (~20–30 LLM calls) executes the plotting plan to render figures and conceptual diagrams, optionally using PaperBanana for publication-quality diagrams with a matplotlib fallback; literature-review-agent (~20–30 calls) web-searches candidates, verifies them against the Semantic Scholar endpoint (Levenshtein >70, cutoff, dedup), and drafts Intro and Related Work with ≥90% citation integration — steps 2 and 3 run in parallel; section-writing-agent makes one multimodal call to draft remaining sections, build tables from the experimental log, and splice figures; content-refinement-agent (~5–7 calls) runs simulated peer review, accepting or reverting edits under strict halt rules. paper-writing-bench reverse-engineers raw materials from an existing paper to build benchmark cases, and paper-autoraters runs the paper's own raters (Citation F1, LitReview 6-axis quality, SxS paper and litreview quality). Deterministic scripts handle JSON schema validation, Levenshtein fuzzy matching, BibTeX formatting, dedup, LaTeX sanity checks, and coverage gates. The optional agent-research-aggregator converts scattered research notes from .claude/, .cursor/, .antigravity/, and .openclaw/ caches into pipeline inputs via four phases: discover_logs.py discovery, per-batch LLM extraction, a single synthesis call into synthesis., and format_po_inputs.py formatting.
- An ML researcher who ran experiments through Claude Code but never wrote a clean experiment log uses agent-research-aggregator to extract idea.md and experimental_log.md from agent caches before the full pipeline.
- A PhD student with structured experiment records and a conference LaTeX template generates a figure-, citation-, and section-complete submission draft inside Cursor or Antigravity.
- A research team evaluating paper-writing quality builds benchmark cases with paper-writing-bench and quantifies outputs with paper-autoraters' Citation F1 and SxS raters.
- A literature-survey task: drafting an Introduction and Related Work with high citation integration grounded in web search and Semantic Scholar verification.
- A budget-conscious solo researcher whose coding agent has native web search runs the entire pipeline without buying any API keys.
What are this agent's strengths and limitations?
- Host-agnostic: the same skills run in Claude Code, Cursor, Antigravity, Cline, Aider, and OpenCode — no lock-in to a single vendor ecosystem.
- Zero-key architecture: all LLM reasoning and web search use the host agent's own tools; the repo contains no embedded LLM calls or key requirements.
- High paper fidelity: every agent prompt is reproduced verbatim from Appendix F of arXiv:2604.05018, with a design-decision-to-page map in docs/paper-fidelity.md.
- Auditable deterministic layer: schema validation, BibTeX formatting, LaTeX checks, and orphan-citation gates are network-free local scripts with predictable behavior.
- Built-in evaluation: paper-autoraters reuses the paper's own Citation F1, LitReview 6-axis, and SxS raters for quantitative output assessment.
- Depends on host-agent capabilities: users need a coding agent with web search and multimodal reasoning; without one the pack cannot run.
- Optional components add external dependencies: PaperBanana requires cloning, configuration, and a Gemini or OpenRouter key; the free Semantic Scholar endpoint is capped at ~1 QPS with 429 back-off on large runs.
- The GitHub license field is NOASSERTION while the README claims MIT — verify the LICENSE file before adopting.
- The underlying paper is a 2026 preprint (arXiv:2604.05018); its benchmark claims (50–68% literature-review win margin, etc.) are self-reported and independently unverified.
- Users must supply their own conference LaTeX template and guidelines, and must manually review aggregator-produced idea.md and experimental_log.md.
How do you install or deploy this agent?
- Clone: git clone <repo-url> ~/paper-orchestra && cd ~/paper-orchestra
- Install deterministic helpers only: pip install -r requirements.txt (no LLM SDKs, no API keys required).
- Symlink skills into your host's skill directory, e.g. for Claude Code:
mkdir -p ~/.claude/skills
for s in paper-orchestra outline-agent plotting-agent literature-review-agent section-writing-agent content-refinement-agent paper-writing-bench paper-autoraters agent-research-aggregator; do
ln -sf ~/paper-orchestra/skills/$s ~/.claude/skills/$sdone
For Cursor / Antigravity / Cline / Aider, follow the per-host instructions in skills/paper-orchestra/references/host-integration.md. Optional integrations: a free Semantic Scholar API key (export SEMANTIC_SCHOLAR_API_KEY; absent the script silently falls back to the unauthenticated endpoint), the PaperBanana figure backend (clone it, add a Gemini or OpenRouter key, export PAPERBANANA_PATH), and Exa search (export EXA_API_KEY).
How do you use this agent?
With structured inputs: run python skills/paper-orchestra/scripts/init_workspace.py --out workspace/, drop idea.md, experimental_log.md, template.tex, and conference_guidelines.md into workspace/inputs/, then tell your agent "Run the paper-orchestra pipeline on ./workspace". With scattered research: just say "Write a paper from my work in ~/my-project" — the aggregator scans the directory, produces the input files automatically, and hands off after your review. You can run aggregation phases standalone, e.g. python skills/agent-research-aggregator/scripts/discover_logs.py --search-roots ~/my-project --out workspace/ara/discovered_logs.. A ready-to-run toy case lives at examples/minimal/.