AgentVerse
A Python framework for collaborative LLM task solving and configurable multi-agent simulation.
- Source repo
- OpenBMB/AgentVerse
- Stars
- ★ 5.2k
- Last updated
- 2y ago
- License
- Apache-2.0
- Primary language
- JavaScript
- FA score
- 28/100 · Major gaps
At a glance
- How it runs
- Works with
- Portable with changesOpenAI API
- Cost
- Free software; you pay for model usage
- Setup effort
- Medium · a few setup steps
- You'll need
- Typical use
- A researcher studying interactions among LLM roles such as a professor and students can run the NLP Classroom simulation.
- Not a fit if
- Users who want a stable simulation framework while the code is being refactored
- Users who need tool use but will not deploy BMTools or an XAgent ToolServer
- Source review
- 28/100 · Major gaps 8 safety controls not found
What does this agent do, and when should you use it?
AgentVerse is a Python framework for deploying multiple LLM-based agents through two frameworks: task-solving and simulation. Its task-solving workflow can run benchmark experiments or a configured single problem, with documented examples for HumanEval and brainstorming. Its simulation workflow launches configured multi-agent environments through CLI or a local GUI, illustrated by a nine-player NLP classroom. The documented model paths include OpenAI API, Azure OpenAI, vLLM, and FastChat-hosted local models, each requiring the relevant configuration and runtime setup. Tool-enabled workloads add external deployment requirements, so the full feature set is not a self-contained local installation.
After installation, it exposes agentverse-simulation, agentverse-simulation-gui, agentverse-benchmark, and agentverse-tasksolving. Simulation commands load a task such as simulation/nlp_classroom_9players and start a multi-agent environment; the GUI command serves the classroom demo locally at http://127.0.0.1:7860/. Task-solving can load a task configuration and dataset path, for example tasksolving/humaneval/gpt-3.5 with data/humaneval/test.jsonl for HumanEval. Model calls can use OPENAI_API_KEY, Azure OpenAI through AZURE_OPENAI_API_KEY and AZURE_OPENAI_API_BASE, vLLM through VLLM_API_KEY and VLLM_API_BASE, or FastChat local models configured with llm_type: local.
- A researcher studying interactions among LLM roles such as a professor and students can run the NLP Classroom simulation.
- An engineering team prototyping role-based code generation can use the documented Software Design scenario with a code writer, tester, and reviewer.
- An evaluation engineer running multi-agent experiments on HumanEval can use
agentverse-benchmarkwith the supplied task and dataset-path pattern. - A team serving larger inference workloads from an existing vLLM server can configure AgentVerse to use the server model name.
- A user with FastChat-compatible local LLaMA or Vicuna models can follow the documented local-server and
llm_type: localworkflow.
How do you install or deploy this agent?
Python 3.9+ is required.
git clone https://github.com/OpenBMB/AgentVerse.git --depth 1
cd AgentVerse
pip install -e .
export OPENAI_API_KEY="your_api_key_here"Alternatively, run pip install -U agentverse. Local-model use additionally requires pip install -r requirements_local.txt.
How do you use this agent?
Configure OPENAI_API_KEY, then run a simulation:
agentverse-simulation --task simulation/nlp_classroom_9playersFor the local GUI:
agentverse-simulation-gui --task simulation/nlp_classroom_9playersOpen http://127.0.0.1:7860/ after the server starts. A documented HumanEval invocation is:
agentverse-benchmark --task tasksolving/humaneval/gpt-3.5 --dataset_path data/humaneval/test.jsonl --overwriteWhat are this agent's strengths and limitations?
- It covers both collaborative task solving and multi-agent simulation, with distinct CLI entry points for each workflow.
- It includes a locally served GUI route for a classroom simulation, rather than only batch commands.
- The documented integration paths include OpenAI API, Azure OpenAI, vLLM, and FastChat-hosted local models.
- It provides a concrete HumanEval benchmark command and a separate command pattern for configured single-problem runs.
- The standard examples require an OpenAI API key; Azure, vLLM, and local-model routes require additional environment variables or configuration changes.
- Tool-enabled simulations require BMTools, while tool-using task-solving cases require an XAgent ToolServer deployment.
- The README says the simulation code is being refactored and directs simulation-only users seeking a stable version to the
release-0.1branch. - Its roadmap still lists documentation and more sophisticated conversation-history memory as unfinished work.
How does this agent compare with similar options?
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| AgentVerse This agent | 28 · Major gaps | CLIFree + model costs | ★ 5.2k | 2y ago | JavaScript | OpenAI API |
| Lagent Lightweight Agent Framework | 43 · Major gaps | Library / SDKFree + model costs | ★ 2.3k | 6mo ago | Python | OpenAI API |
| ChatArena | 32 · Major gaps | Library / SDKFree + model costs | ★ 1.6k | 1y ago | Python | OpenAI API |
| Hello-Agents | 54 · Major gaps | Web appFree + model costs | ★ 86k | 23d ago | Python | OpenAI API |
How does FollowAgents rate this agent?
Why each dimension lost points
Evidence shows: The repository does not provide explicit mechanisms for least privilege, user confirmation, data flow transparency, sensitive data handling, dependency security, external effects, rollback, or source attribution. README only instructs users to export API keys without explaining data usage or storage. Dependencies are pinned but no security audit is provided. Therefore all trust criteria score 0.
Evidence shows: README and config examples are consistent across task-solving and simulation frameworks, but no error handling or failure messages are documented. Dependencies are listed in requirements.txt but availability is not verified. Thus self-consistency scores 1, dependency availability scores 1, and failure messages score 0.
Evidence shows: README describes multiple application scenarios (e.g., software development and simulation) and provides guidance for customizing environments. However, capability boundaries are not clearly defined, trigger precision (e.g., commands and configs) is partially explained, and environment fit (e.g., local models and vLLM) is mentioned but not detailed. Therefore audience and scenarios score 2, capability boundaries score 1, trigger precision score 1, and environment fit score 1.
Evidence shows: README is well-structured with installation instructions, examples, and configuration guides. However, versioning/changelog and maintenance responsibility are not clearly documented. License is Apache-2.0, but known limitations are not detailed. Thus information architecture scores 2, install notes score 2, naming stability scores 1, examples and FAQ score 2, known limitations score 1, license scores 2, versioning/changelog scores 1, and maintenance responsibility scores 1.
Evidence shows: The framework provides multiple output formats (e.g., CLI and GUI), but it is not clear how outputs are consumed or validated. Marginal value is high due to unique multi-agent collaboration and simulation features. Cost-benefit is not detailed but relies on external APIs and local models. Therefore output usability scores 1, marginal value scores 2, and cost-benefit scores 1.
Evidence shows: Claims in README (e.g., paper and demos) are traceable to external sources, but no internal verification is provided. Cross-source corroboration is limited because only README and config files are available. Facts and inferences are not clearly separated. Therefore claim traceability scores 1, cross-source corroboration scores 1, and fact-inference separation scores 0.
- Not found in source: least-privilege scopingGrant only what the task needs: a dedicated account or read-only token, scoped to specific directories and repos.
- Not found in source: confirmation before actingTurn on (or add) a confirmation step before it acts, and try it in a sandbox or test environment before real data.
- Not found in source: data-flow disclosureWatch which external services it contacts (proxy or firewall logs) and keep sensitive data out until you know where it goes.
- Not found in source: sensitive-data handlingUse dedicated, low-privilege, revocable API keys — never production credentials — and keep secrets out of logs.
- Not found in source: dependency securityPin versions and run a dependency audit (npm audit, pip-audit) before installing; prefer running it in a container.
- Not found in source: disclosed external effectsEstablish which external systems it writes to, sends to or changes, and verify with test accounts or repos before production.
- Not found in source: rollback or recovery pathBack up first, or work on a git branch or snapshot, so its changes can be undone.
- Not found in source: verifiable attributionInstall from the official repo or registry and check the publisher and URL to avoid look-alike packages.
- No permission management or user confirmation mechanisms are provided, potentially executing high-privilege operations by default.
- Dependencies are pinned but no security audit is provided, posing known vulnerability risks.
- Insufficient documentation on data flow and sensitive data handling in README.
- Lack of versioning/changelog and maintenance responsibility, unclear update path.
Recent releases
Latest entries from the project's GitHub Releases. Notes are quoted from upstream as written, not rewritten or tested by FollowAgents.
-
Release notes excerpt
What's Changed * fix: fix a bug about updating kwargs by @minleminzui in https://github.com/OpenBMB/AgentVerse/pull/62 * add simulation ui params by @JetSquirrel in https://github.com/OpenBMB/AgentVerse/pull/63 * Modification by @1rubbishyuan in https://github.com/OpenBMB/AgentVerse/pull/65 * 56 to do update the readme by @yushengsu-thu in https://github.com/OpenBMB/AgentVerse/pull/66 * doc: modify README.md by @ASL-r in https://github.com/OpenBMB/AgentVerse/pull/67 * support local llms by @cheesewafer in https://github.com/OpenBMB/AgentVerse/pull/68 …View on GitHub ↗ -
Release notes excerpt
What's Changed * fix: arrange output parser classes by @minleminzui in https://github.com/OpenBMB/AgentVerse/pull/59 Full Changelog: https://github.com/OpenBMB/AgentVerse/compare/v0.1.5...v0.1.8View on GitHub ↗ -
Release notes excerpt
What's Changed * Zqm dev by @chanchimin in https://github.com/OpenBMB/AgentVerse/pull/4 * add: agentverse for database diagnosis by @zhouxh19 in https://github.com/OpenBMB/AgentVerse/pull/5 * Remove langchain, add several database and prisoner dilemma showcases by @chenweize1998 in https://github.com/OpenBMB/AgentVerse/pull/9 * Sde team remove langchain by @libowen2121 in https://github.com/OpenBMB/AgentVerse/pull/12 * add phaser ui example by @chenweize1998 in https://github.com/OpenBMB/AgentVerse/pull/16 …View on GitHub ↗
FAQ
Must I use OpenAI models?
What is the quickest first run?
OPENAI_API_KEY, and run agentverse-simulation --task simulation/nlp_classroom_9players.Can I inspect a simulation in a browser?
agentverse-simulation-gui --task simulation/nlp_classroom_9players, then open http://127.0.0.1:7860/.