re:factory
Describe what you want — a multi-agent pipeline researches, plans, builds, and evolves software automatically.
- Source repo
- boston-harness-group/remote-factory
- Stars
- ★ 75
- Last updated
- 1d ago
- License
- MIT
- Primary language
- Python
- FA score
- 71/100 · Some gaps
At a glance
- How it runs
- Works with
- Portable with changesCodex · Claude Code
- Cost
- Free software; you pay for model usage
- Setup effort
- Medium · a few setup steps
- You'll need
- Typical use
- A founder or researcher with an idea but no code, using
factory ceo "distributed eval runner" --mode designto refine a raw idea into a buildable spec - Not a fit if
- Teams without an authenticated Claude Code installation
- Developers who only want one-shot code completion, not orchestrated multi-agent loops
- Restricted environments that cannot run Python 3.11+ with uv
- Website
- akashgit.github.io/remote-factory ↗ set on GitHub
- Maintenance
- Commits in the last 30 dayslast commit 2026-11-19
- Source review
- 71/100 · Some gaps
What does this agent do, and when should you use it?
re:factory (repository boston-harness-group/remote-factory) is a domain-agnostic multi-agent software design and evolution harness driven by the factory CLI. It uses a four-layer architecture: a Python CLI, a workflow graph engine of typed Pydantic nodes (AgentNode, GateNode, ForkNode, and more), a CEO orchestrator, and eight Claude Code subprocess specialist agents (Researcher, Strategist, Builder, Health Checker, Code Reviewer, Adversarial Tester, Archivist, Failure Analyst). Workflows run in two formats from the same graph: headless deterministic DAG execution and SKILL.md prose playbooks the CEO follows interactively. All state stays local (.factory/ and ~/.factory/), and every experiment is kept or reverted based on a composite score across Hygiene, Growth, and Project tiers. Built-in modes include design, create, meta, and outer-loop, plus community benchmark workflows such as swebench and featurebench; the outer loop evolves workflow topologies via MAP-Elites. It installs as a global CLI and also ships as a bundled Claude Code plugin.
factory ceo "idea" --mode design starts a design flow: 3 parallel researchers study existing patterns and best practices, the Strategist synthesizes a structured plan, and after your approval the Builder implements and QA verifies. The design-v2 mode applies inference-time scaling with dynamic Research/Strategy/QA Directors that generate N/M/K approaches in parallel and synthesize them, maintaining an append-only User Intent Ledger. Create mode (--mode create --focus "...") researches existing patterns then implements a new workflow: Pydantic graph definition, SKILL.md, CLI wiring, and tests; a colon prefix like "build: ..." updates an existing mode instead. The outer loop runs factory outer-loop calibrate --benchmark featurebench --test-command "pytest tests/ -v" to mutate and evaluate workflow DAGs on real benchmark instances, selecting for test pass rates via MAP-Elites. Focus mode builds exactly one thing, targeting a backlog item or GitHub issues (factory ceo ~/my-project --focus 42). Evaluation is driven by a three-tier composite score, and factory discover auto-detects your project's language and framework on first run to generate the eval profile. Workflow graphs are exported to SKILL.md and SKILL.annotations.yaml through a verified pipeline (templatize → review agent → guard → split), with a CI regression test guarding against drift. LangFuse LLM tracing and ACE self-improvement are also supported.
- A founder or researcher with an idea but no code, using
factory ceo "distributed eval runner" --mode designto refine a raw idea into a buildable spec - An engineer maintaining an existing repo, using
--mode design --focus "issue #42"or--focus "auth layer"so agents study the backlog and iterate on improvements - A benchmarking researcher running community-contributed workflows like swebench, featurebench, or terminalbench to evaluate agent performance
- A tooling author using create mode to generate custom factory modes (e.g. a PR validation pipeline) with SKILL.md and tests produced automatically
- An automation team evolving workflow topologies with outer-loop + MAP-Elites against benchmarks to find higher pass-rate agent orchestration structures
- A Claude Code user installing the plugin to gain the
/factory:implementcommand and namespaced subagents likefactory:ceo
Key setup facts
Pulled from the install and usage notes below. The project README has the current values.
- Install / run
uv tool install git+https://github.com/akashgit/remote-factory.git
- Ports
3000- Environment variables
LANGFUSE_HOSTLANGFUSE_BASE_URLLANGFUSE_PUBLIC_KEYLANGFUSE_SECRET_KEY- Runtime
- Python 3.11+
- Install notes cover
- Docker
How do you install or deploy this agent?
Prerequisites: Python 3.11+, uv, and Claude Code (installed and authenticated).
uv tool install git+https://github.com/akashgit/remote-factory.gitAlternatively install as a Claude Code plugin (from inside Claude Code):
/plugin marketplace add akashgit/remote-factory#plugins
/plugin install factory@remote-factory
/reload-pluginsIf you only want standalone subagent prompts:
factory install # Install all 9 agents to ~/.claude/agents/
factory install --runner codex # Or install Codex TOML agents to ~/.codex/agents/Optional LangFuse LLM tracing requires Docker or Podman:
scripts/langfuse-setup start
export LANGFUSE_HOST=http://localhost:3000
export LANGFUSE_BASE_URL=http://localhost:3000
export LANGFUSE_PUBLIC_KEY=pk-lf-dev-local-key
export LANGFUSE_SECRET_KEY=sk-lf-dev-local-key
export TELEMETRY_PLATFORM=langfuseSee docs/setup.md in the repository for full authentication and environment variable details.
How do you use this agent?
After installation, use the factory CLI. Optionally run factory discover first to auto-detect your project's language and framework and generate the eval profile.
Design from an idea or a spec file:
factory ceo "my idea" --mode design
factory ceo ~/ideas/weather-dashboard.md --mode designImprove an existing project, targeting a topic or GitHub issues:
factory ceo /path/to/project --mode design --focus "auth layer"
factory ceo /path/to/project --focus 42
factory ceo /path/to/project --focus '42 and 43'The inference-time scaling design variant:
factory ceo ~/my-project --mode design-v2Create or update factory modes:
factory ceo /path/to/factory --mode create --focus "a mode that validates PRs with multi-stage checks"
factory ceo /path/to/factory --mode create --focus "build: add a code review gate after the builder"Run the evolutionary outer loop:
factory outer-loop calibrate ~/my-factory \
--benchmark featurebench \
--population-size 3 \
--project-dir /path/to/benchmark-instance \
--test-command "pytest tests/ -v"
factory ceo ~/my-factory --mode outer-loop --headlessContinuous, headless, and self-improvement runs:
factory run <path> --loop --interval 1800
factory tmux <path> --loop
factory ceo <path> --mode metaWith the plugin installed, invoke /factory:implement or subagents like factory:ceo directly from Claude Code. All state is written to project-local .factory/ (add to .gitignore) and global ~/.factory/.
What are this agent's strengths and limitations?
- Workflows are typed Pydantic DAGs (AgentNode/GateNode/ForkNode/JoinNode) that execute headlessly and export as SKILL.md interactive playbooks from the same graph, making modes programmatically extensible
- outer-loop evolves workflow topologies with MAP-Elites against real benchmarks, and a three-tier composite score (Hygiene/Growth/Project) automatically keeps or reverts experiments
- Verified skill generation includes a structural diff guard and a CI regression test (test_annotations_match_source) that prevents drift between graph definitions and exported skills
- Fully local state (.factory/ and ~/.factory/), local LangFuse tracing, and documented container/Kubernetes runtimes support auditing and private deployment
- The core pipeline depends on an authenticated Claude Code installation; the Codex path only installs standalone subagent prompts (--runner codex), while heavy lifting still requires the factory CLI
- A broad install surface: Python 3.11+, uv, Claude Code authentication, plus Docker/Podman and several environment variables if you want LangFuse tracing
- Persistent LLM calls (design, build, QA, outer loop) incur real Claude usage costs; the README states no usage limits
- Outer-loop calibration requires providing a real benchmark instance (e.g. a featurebench project directory and test command), adding setup cost for evolutionary use
How does this agent compare with similar options?
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| re:factory This agent | 71 · Some gaps | CLIFree + model costs | ★ 75 | 1d ago | Python | Codex · Claude Code |
| FrontierAgent | 66 · Some gaps | CLIFree + model costs | ★ 5.4k | today | Python | OpenAI API |
| AWorld Agent Harness | 48 · Major gaps | CLIFree + model costs | ★ 1.2k | 12d ago | Python | — |
| Vigil Multi-Agent Terminal Orchestrator | 68 · Some gaps | Desktop appFree + model costs | ★ 29 | 2d ago | Swift | Codex · Claude Code |
How does FollowAgents rate this agent?
Why each dimension lost points
Evidence shows explicit scope controls (factory.md Scope/Modifiable, Guards, research-mode Mutable/Fixed Surfaces with leakage fingerprinting), keep/revert rollback on experiment branches with git worktree isolation, and exceptionally detailed data-flow documentation (four-layer architecture, state machine, per-node reads/writes). Deductions: dependencies include non-mainstream packages (mempalace, graphifyy) with no provenance, a supply-chain risk; README publishes LangFuse dev credentials and a login password and encourages persisting them in ~/.bashrc; factory install writes 9 agents globally into ~/.claude/agents/; --auto-approve lets the CEO act as the user at gates and --headless/--loop enable unattended operation, weakening human confirmation; --clean-pr pushes PRs to external repos as an external effect gated only by a flag.
CLAUDE.md documents test fixtures, coverage, version fallback, and crash-resilient checkpoints; gates implement PROCEED/HALT branching with artifact verification scripts and hook logs. Deductions: documentation drift — AGENTS.md says three layers while README/CLAUDE.md say four; specialist role counts vary (7/8/9) across files; CLAUDE.md mentions 10 modes not all listed in README, creating ambiguity about which description runtime behavior follows.
Developer-scenario coverage is excellent: design/create/focus/research/outer-loop/meta modes, community benchmark workflows, container/Kubernetes runtimes, dual Claude Code plugin and Codex paths — environment fit is well evidenced. Deductions: capability boundaries are honestly disclosed in only one place (the outer-loop E2E finding that single-builder wins on simple tasks); overall failure modes and limits of autonomous code modification are under-documented; skill triggering relies on description text, with disable-model-invocation mitigating misfires only partially.
Information architecture is strong (layered docs, mkdocs site, CLI quick reference, abundant examples), full MIT LICENSE present with consistent author attribution, version derived via hatch-vcs from git tags with detailed explanation. Deductions: no CHANGELOG; naming drift (three vs four layers, role counts, plugin name 'factory' vs repo 'remote-factory'); install is a global uv tool install and docs advise persisting credential-bearing env vars into shell config; maintenance path is GitHub Issues only with no stated commitment.
Output usability is supported: per-experiment directories with hypothesis/diff/verdict, performance reports, a dashboard, and clean-pr mode for upstream contribution; marginal value over single-agent coding tools lies in structured experiment loops plus topology evolution and benchmark comparison. Deductions: high operating cost (parallel agents, Opus calls, 120-minute benchmark CI) with Cost Budget scoped only to research mode; the headline claim that autonomous evolution beats baselines lacks published reproducible numbers.
Most claims trace to concrete module paths (definitions.py, evaluator.py, overfit.py), the E2E finding is labeled as a finding rather than asserted fact, and the benchmark CI workflow is re-runnable. Deductions: benchmark tables publish no scores or links; narratives (8 SKILL files, role rosters) cannot be fully cross-checked within the sample; test bodies are not in the packet, so the claimed CI regression test cannot be verified here.
- Dependencies include non-mainstream packages (mempalace, graphifyy); audit their provenance and code before installing.
- LangFuse dev credentials/password in the README are local defaults only; never reuse them on any externally reachable instance or copy them into shared shell configs.
- With --auto-approve and --headless/--loop, keep/revert decisions run without human review; run only inside isolated containers/sandboxes or disposable repos.
- factory install writes agents globally to ~/.claude/agents/ or ~/.codex/agents/, affecting all sessions on the host — mind least privilege.
- Outside --clean-pr workflows the tool may push branches/PRs to external GitHub repos; verify target repo and credential scope before running.
- Documentation is inconsistent on layer count and specialist role roster; defer to the current source in the repository.
FAQ
Do I need a Claude subscription?
Does my data leave my machine?
How does the outer loop evaluate workflows?
What if the built-in workflows are not enough?
factory ceo /path/to/factory --mode create --focus "description" researches existing patterns and generates a new workflow's Pydantic graph definition, SKILL.md, CLI wiring, and tests. The "mode: change" format updates an existing mode. You can also edit factory/workflow/definitions.py manually and run factory workflow export-skills.Will skill files drift if I change workflow definitions?
Is re:factory free?
Is re:factory open source?
Is re:factory safe to use?
Is re:factory still maintained?
Who makes re:factory?
What is re:factory's official website?
Thanks — we use this to improve the page.