RecursiveMAS
A recursive latent-state framework for training and evaluating collaborating role-specific models.
- Source repo
- RecursiveMAS/RecursiveMAS
- Stars
- ★ 985
- Last updated
- 1mo ago
- License
- MIT
- Primary language
- Python
- FA score
- 31/100 · Major gaps
At a glance
- How it runs
- Works with
- Portable with changesOpenAI API (Partial support)
- Cost
- Free software; you pay for model usage
- Setup effort
- High · needs real infrastructure
- You'll need
- Typical use
- A researcher reproduces sequential math collaboration by training role-specific inner modules, training an outer module, and evaluating on Math500.
- Not a fit if
- Teams wanting a documented hosted chat product
- Users without CUDA GPUs or willing to download named Hugging Face checkpoints
- Source review
- 31/100 · Major gaps 8 safety controls not found
What does this agent do, and when should you use it?
RecursiveMAS is a research implementation that treats multi-agent collaboration as one recursive computation, with lightweight RecursiveLink modules exchanging and refining latent states across heterogeneous roles. The repository contains an inference and downstream-evaluation pipeline in inference/ plus inner- and outer-loop training scripts in train/. It releases sequential, mixture, distillation, and deliberation collaboration styles, together with corresponding Hugging Face reference checkpoints and training datasets. Its primary entry points are inference/run.py, train/train_inner.py, and train/train_outer.py, producing evaluations across math, science, medicine, code generation, and search QA benchmarks. It fits teams investigating or reproducing recursive multi-model collaboration, rather than buyers seeking a documented hosted chat product or general-purpose agent service.
train/train_inner.py trains each role's inner RecursiveLink while keeping the base model frozen and training a small ln_res_adapter. train/train_outer.py then connects the role-specific models for a selected collaboration style and trains the cross-agent outer RecursiveLink through recursion. For inference, inference/run.py loads a released reference system or locally supplied checkpoints selected with --style, --dataset, and optional --ckpt_override arguments, then evaluates it. Documented benchmarks are math500, gpqa, medqa, mbppplus, aime25, aime26, livecodebench, bamboogle, and hotpotqa, with accuracy, test-pass rate, pass@k, EM, or LLM-as-Judge metrics as applicable. Deliberation runs on bamboogle and hotpotqa can have a Tool-Caller query a real search API, while open-ended grading can use an OpenAI-compatible judge API.
- A researcher reproduces sequential math collaboration by training role-specific inner modules, training an outer module, and evaluating on Math500.
- An ML team uses released sequential_light or sequential_scaled reference checkpoints to test recursive role collaboration on math, science, or code benchmarks.
- A developer trains separate math, code, science, and summarizer roles for the mixture style before training its outer links.
- A team evaluating expert–learner coordination trains a distillation-style setup with the documented Distillation-Math or Distillation-Code datasets.
- A retrieval-QA researcher runs deliberation on bamboogle or hotpotqa and supplies a search API key for the Tool-Caller.
How do you install or deploy this agent?
Create and activate a Python 3.10 environment, then install the repository requirements from the project root:
conda create -n recursivemas python=3.10 -y
conda activate recursivemas
pip install -r requirements.txtFor deliberation runs on search datasets, place a search API key in a plain-text file and provide it with --tavily_keys_file. For LLM judging of open-ended answers, set API_KEY, API_BASE_URL, and API_MODEL.
How do you use this agent?
Run a released reference system, for example:
python inference/run.py \
--style sequential_scaled \
--dataset math500 \
--device cudaFor local training outputs, add --ckpt_override entries such as planner=train/ckpts/seq_light/planner_math and outer=train/ckpts/seq_light/outer_math. The documented workflow first uses train/train_inner.py for role-specific inner modules, then train/train_outer.py for outer links using the matching collaboration style and dataset. In Python, import load_mas_system from system_loader and call load_mas_system(style="sequential_light", device="cuda", trust_remote_code=True).
What are this agent's strengths and limitations?
- It makes inter-role coordination a trainable RecursiveLink mechanism, with separate within-role inner and cross-role outer training stages.
- It releases four concrete collaboration patterns—sequential, mixture, distillation, and deliberation—rather than a single orchestration example.
- It documents an experimental path from training datasets and scripts through reference checkpoints to downstream evaluation.
- Its stated benchmark coverage spans math, graduate science, medical QA, code generation, and open-domain search QA.
- The documented examples use CUDA, while released model combinations depend on named Qwen, Llama, Gemma, DeepSeek, and BioMistral models and Hugging Face assets.
- The repository says full paper reproduction requires task-specific data and configurations matched to each collaboration style; reference checkpoints are not a universal replacement.
- Deliberation on bamboogle and hotpotqa needs a real search API and key, and open-ended grading requires a separately configured OpenAI-compatible judge endpoint.
- There is no documented native integration with ChatGPT, Codex, Claude, or MCP; the OpenAI-compatible API path is documented only for judging.
How does this agent compare with similar options?
The repository acknowledges vLLM, ARPO, and TextGrad as open-source foundations, but does not provide a feature comparison with those projects or other multi-agent frameworks.
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| RecursiveMAS This agent | 31 · Major gaps | CLIFree + model costs | ★ 985 | 1mo ago | Python | — |
| Plexe: Build ML Models from Natural Language | 41 · Major gaps | CLIFree + model costs | ★ 2.6k | 8mo ago | Python | OpenAI API · Claude API |
| AutoResearch | 52 · Major gaps | CLIFree + model costs | ★ 99k | 7mo ago | Python | Codex · Claude Code |
| Tongyi DeepResearch | 49 · Major gaps | CLIFree + model costs | ★ 20k | 8mo ago | Python | — |
How does FollowAgents rate this agent?
Why each dimension lost points
Evidence shows: the repository provides no documentation on permission management, user confirmation, data flow transparency, sensitive data handling, dependency security, external effects, rollback, or source attribution. All trust-related criteria are absent, hence score 0.
Evidence shows: README and code structure are consistent, providing clear directory and commands, self-consistency scores 2. Dependency availability: requirements.txt lists pinned versions but no availability guarantees or mirrors, scores 1. Failure messages: no documentation on error handling or failure prompts, scores 0.
Evidence shows: README identifies target audience (researchers) and multiple usage scenarios (inference, training, evaluation), scores 2. Capability boundaries: no explicit statement of what the system can and cannot do, scores 1. Trigger precision: provides specific commands and parameters but no trigger conditions, scores 1. Environment fit: provides environment setup and dependencies but no compatibility notes for different hardware or OS, scores 1.
Evidence shows: information architecture is clear with directory and instructions, scores 2. Install notes: provides conda and pip installation steps, scores 2. Naming stability: no version history or naming conventions, scores 1. Examples and FAQ: provides multiple example commands but no FAQ, scores 2. Known limitations: not explicitly listed, scores 1. License: MIT license complete, scores 3. Versioning changelog: none, scores 0. Maintenance responsibility: no explicit maintainer but provides contact email, scores 1.
Evidence shows: output usability: provides inference outputs and evaluation metrics but no output format specification, scores 1. Marginal value: provides multiple collaboration patterns and benchmarks, research value, scores 2. Cost benefit: no performance or cost data, scores 1.
Evidence shows: claim traceability: provides paper and dataset links but no specific experiment configurations, scores 1. Cross-source corroboration: provides external coverage and demos but no independent verification, scores 1. Fact-inference separation: not explicitly distinguished, scores 1.
- Not found in source: least-privilege scopingGrant only what the task needs: a dedicated account or read-only token, scoped to specific directories and repos.
- Not found in source: confirmation before actingTurn on (or add) a confirmation step before it acts, and try it in a sandbox or test environment before real data.
- Not found in source: data-flow disclosureWatch which external services it contacts (proxy or firewall logs) and keep sensitive data out until you know where it goes.
- Not found in source: sensitive-data handlingUse dedicated, low-privilege, revocable API keys — never production credentials — and keep secrets out of logs.
- Not found in source: dependency securityPin versions and run a dependency audit (npm audit, pip-audit) before installing; prefer running it in a container.
- Not found in source: disclosed external effectsEstablish which external systems it writes to, sends to or changes, and verify with test accounts or repos before production.
- Not found in source: rollback or recovery pathBack up first, or work on a git branch or snapshot, so its changes can be undone.
- Not found in source: verifiable attributionInstall from the official repo or registry and check the publisher and URL to avoid look-alike packages.
- The repository provides no security or permission management documentation; assess risks before use.
- Dependencies are pinned but no security audit is provided; check for known vulnerabilities.
- No rollback or recovery mechanisms are provided; proceed with caution when updating or modifying.
- Maintenance responsibility is unclear; long-term support is uncertain.