paper2code: Paper-to-Code Skill
Turn any arXiv paper into a working implementation with citation anchoring
- Source repo
- PrathamLearnsToCode/paper2code
- Stars
- ★ 1.5k
- Last updated
- 6mo ago
- License
- MIT
- Primary language
- Python
- FA score
- 36/100 · Major gaps
At a glance
- How it runs
- Works with
- Platform-specificClaude Code
- Cost
- Free software; you pay for model usage
- Setup effort
- Low · running in minutes
- You'll need
- Typical use
- A researcher wants to quickly understand a paper's implementation details without reading a lengthy codebase.
- Not a fit if
- Teams using coding agents other than Claude Code
- Users expecting a fully trained, reproducible result
- Teams needing datasets downloaded and training run for them
- Source review
- 36/100 · Major gaps 8 safety controls not found
What does this agent do, and when should you use it?
paper2code is an Agent skill for Claude Code that takes an arXiv paper URL and produces a structured code project where every line of key code references the exact paper section and equation it implements. It performs an ambiguity audit before writing any code, classifying each implementation choice as SPECIFIED, PARTIALLY_SPECIFIED, or UNSPECIFIED, and flags unspecified choices with [UNSPECIFIED] comments and alternatives. The generated output includes a src directory with model, loss, train, data, evaluate, and utils modules, a configs/base.yaml for hyperparameters, and a pedagogical walkthrough notebook. It mines appendices, footnotes, and figure captions as first-class sources, and emphasizes honest uncertainty. The skill is installed via npx and used through slash commands in Claude Code.
paper2code is triggered by the /paper2code command, accepting full arXiv URLs or bare IDs like 1706.03762. It fetches the paper, parses it, runs an ambiguity audit, then generates an output folder (e.g., attention_is_all_you_need/) containing README.md, REPRODUCTION_NOTES.md (the audit), requirements.txt, and src/ files: model.py, loss.py, train.py, data.py, evaluate.py, utils.py, plus configs/base.yaml and notebooks/walkthrough.ipynb. Every non-trivial code decision is anchored with §X.Y or §X.Y, Eq. N comments; unspecified choices are marked with [UNSPECIFIED] comments. The skill supports options like --framework jax, --mode full, and --mode educational.
- A researcher wants to quickly understand a paper's implementation details without reading a lengthy codebase.
- An ML engineer needs to verify a paper's reproducibility and see which hyperparameters are explicitly specified.
- A student wants to learn how each paper component maps to code through a runnable notebook.
- A developer needs to generate a paper implementation in a specific framework like JAX.
- A paper author wants to check for ambiguities in their own paper to improve clarity.
How do you install or deploy this agent?
Install using npx: npx skills add PrathamLearnsToCode/paper2code/skills/paper2code. You'll be prompted to select agents (e.g., Claude Code), scope (Global or project-level), and method (Symlink or copy).
How do you use this agent?
After installation, open Claude Code and run the skill command, e.g., /paper2code https://arxiv.org/abs/1706.03762. You can also specify a framework (--framework jax), mode (--mode full or --mode educational), or use a bare arXiv ID (/paper2code 1706.03762).
What are this agent's strengths and limitations?
- Citation anchoring: every code line references paper sections for verifiability.
- Ambiguity audit: explicitly marks unspecified and partially specified choices, preventing hallucination.
- Honest uncertainty:
[UNSPECIFIED]comments list common alternatives. - Appendix mining: treats appendices, footnotes, and figure captions as first-class sources.
- Includes a pedagogical notebook for learning and verification.
- Platform-specific: only supports Claude Code.
- No correctness guarantee: if the paper is wrong, the code is wrong.
- Does not implement baselines or standard components; requires external dependencies.
- Does not download datasets; provides a skeleton with TODOs.
- No training infrastructure like distributed training or experiment tracking.
How does this agent compare with similar options?
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| paper2code: Paper-to-Code Skill This agent | 36 · Major gaps | Agent plugin / skillFree + model costs | ★ 1.5k | 6mo ago | Python | Claude Code |
| Trace Generative Optimizer | 67 · Some gaps | Library / SDKFree + model costs | ★ 762 | 1y ago | Python | OpenAI API · Claude API |
| Weco Autoresearch | 63 · Some gaps | CLIFree + model costs | ★ 117 | 2d ago | Python | Codex · Claude Code · OpenAI API |
| AWorld Agent Harness | 48 · Major gaps | CLIFree + model costs | ★ 1.2k | 5d ago | Python | — |
How does FollowAgents rate this agent?
Why each dimension lost points
Evidence: The repository contains only README and LICENSE, no code or executable files, so there are no permission requests, data flows, or external effects. Deductions: Without an actual implementation, least privilege, user confirmation, data flow transparency, sensitive data handling, dependency security, external effects, rollback, and source attribution cannot be assessed; all scored 0.
Evidence: README is internally consistent, describing a clear workflow and design principles. Deductions: No actual code or tests, dependency availability is only based on npx and Python packages mentioned in README, not verified; failure messages are not described, hence score 1.
Evidence: README clearly identifies target audience (ML practitioners) and multiple usage scenarios (different modes, frameworks). Deductions: Capability boundaries are stated (what it won't do), but environment fit only mentions Claude Code, not other agents; trigger precision has examples but not verified.
Evidence: Information architecture is clear, install notes exist, examples and FAQ sections are present, known limitations are explicit, license is MIT. Deductions: Naming stability not mentioned, versioning/changelog missing, maintenance responsibility only implied by contribution guide, not explicit.
Evidence: Output usability is described in detail (generated file structure), marginal value is clear (addressing ambiguity in paper implementations). Deductions: Cost-benefit not quantified, no actual run evidence, hence score 1.
Evidence: Claim traceability is supported by citation anchoring mechanism, fact-inference separation is shown via UNSPECIFIED tags. Deductions: Cross-source corroboration only mentions official code, no actual verification, hence score 1.
- Not found in source: least-privilege scopingGrant only what the task needs: a dedicated account or read-only token, scoped to specific directories and repos.
- Not found in source: confirmation before actingTurn on (or add) a confirmation step before it acts, and try it in a sandbox or test environment before real data.
- Not found in source: data-flow disclosureWatch which external services it contacts (proxy or firewall logs) and keep sensitive data out until you know where it goes.
- Not found in source: sensitive-data handlingUse dedicated, low-privilege, revocable API keys — never production credentials — and keep secrets out of logs.
- Not found in source: dependency securityPin versions and run a dependency audit (npm audit, pip-audit) before installing; prefer running it in a container.
- Not found in source: disclosed external effectsEstablish which external systems it writes to, sends to or changes, and verify with test accounts or repos before production.
- Not found in source: rollback or recovery pathBack up first, or work on a git branch or snapshot, so its changes can be undone.
- Not found in source: verifiable attributionInstall from the official repo or registry and check the publisher and URL to avoid look-alike packages.
- The repository contains only README and LICENSE, no actual code or tests, so functionality cannot be verified.
- Installation command relies on npx skills add, which may involve external services; use with caution.
- Generated code accuracy and security are unverified; review before use.