AgentVerse
A Python framework for collaborative LLM task solving and configurable multi-agent simulation.
Per-dimension scores and reasoning
Evidence shows: The repository does not provide explicit mechanisms for least privilege, user confirmation, data flow transparency, sensitive data handling, dependency security, external effects, rollback, or source attribution. README only instructs users to export API keys without explaining data usage or storage. Dependencies are pinned but no security audit is provided. Therefore all trust criteria score 0.
Evidence shows: README and config examples are consistent across task-solving and simulation frameworks, but no error handling or failure messages are documented. Dependencies are listed in requirements.txt but availability is not verified. Thus self-consistency scores 1, dependency availability scores 1, and failure messages score 0.
Evidence shows: README describes multiple application scenarios (e.g., software development and simulation) and provides guidance for customizing environments. However, capability boundaries are not clearly defined, trigger precision (e.g., commands and configs) is partially explained, and environment fit (e.g., local models and vLLM) is mentioned but not detailed. Therefore audience and scenarios score 2, capability boundaries score 1, trigger precision score 1, and environment fit score 1.
Evidence shows: README is well-structured with installation instructions, examples, and configuration guides. However, versioning/changelog and maintenance responsibility are not clearly documented. License is Apache-2.0, but known limitations are not detailed. Thus information architecture scores 2, install notes score 2, naming stability scores 1, examples and FAQ score 2, known limitations score 1, license scores 2, versioning/changelog scores 1, and maintenance responsibility scores 1.
Evidence shows: The framework provides multiple output formats (e.g., CLI and GUI), but it is not clear how outputs are consumed or validated. Marginal value is high due to unique multi-agent collaboration and simulation features. Cost-benefit is not detailed but relies on external APIs and local models. Therefore output usability scores 1, marginal value scores 2, and cost-benefit scores 1.
Evidence shows: Claims in README (e.g., paper and demos) are traceable to external sources, but no internal verification is provided. Cross-source corroboration is limited because only README and config files are available. Facts and inferences are not clearly separated. Therefore claim traceability scores 1, cross-source corroboration scores 1, and fact-inference separation scores 0.
- No permission management or user confirmation mechanisms are provided, potentially executing high-privilege operations by default.
- Dependencies are pinned but no security audit is provided, posing known vulnerability risks.
- Insufficient documentation on data flow and sensitive data handling in README.
- Lack of versioning/changelog and maintenance responsibility, unclear update path.
What does this agent do, and when should you use it?
AgentVerse is a Python framework for deploying multiple LLM-based agents through two frameworks: task-solving and simulation. Its task-solving workflow can run benchmark experiments or a configured single problem, with documented examples for HumanEval and brainstorming. Its simulation workflow launches configured multi-agent environments through CLI or a local GUI, illustrated by a nine-player NLP classroom. The documented model paths include OpenAI API, Azure OpenAI, vLLM, and FastChat-hosted local models, each requiring the relevant configuration and runtime setup. Tool-enabled workloads add external deployment requirements, so the full feature set is not a self-contained local installation.
After installation, it exposes agentverse-simulation, agentverse-simulation-gui, agentverse-benchmark, and agentverse-tasksolving. Simulation commands load a task such as simulation/nlp_classroom_9players and start a multi-agent environment; the GUI command serves the classroom demo locally at http://127.0.0.1:7860/. Task-solving can load a task configuration and dataset path, for example tasksolving/humaneval/gpt-3.5 with data/humaneval/test.jsonl for HumanEval. Model calls can use OPENAI_API_KEY, Azure OpenAI through AZURE_OPENAI_API_KEY and AZURE_OPENAI_API_BASE, vLLM through VLLM_API_KEY and VLLM_API_BASE, or FastChat local models configured with llm_type: local.
- A researcher studying interactions among LLM roles such as a professor and students can run the NLP Classroom simulation.
- An engineering team prototyping role-based code generation can use the documented Software Design scenario with a code writer, tester, and reviewer.
- An evaluation engineer running multi-agent experiments on HumanEval can use
agentverse-benchmarkwith the supplied task and dataset-path pattern. - A team serving larger inference workloads from an existing vLLM server can configure AgentVerse to use the server model name.
- A user with FastChat-compatible local LLaMA or Vicuna models can follow the documented local-server and
llm_type: localworkflow.
What are this agent's strengths and limitations?
- It covers both collaborative task solving and multi-agent simulation, with distinct CLI entry points for each workflow.
- It includes a locally served GUI route for a classroom simulation, rather than only batch commands.
- The documented integration paths include OpenAI API, Azure OpenAI, vLLM, and FastChat-hosted local models.
- It provides a concrete HumanEval benchmark command and a separate command pattern for configured single-problem runs.
- The standard examples require an OpenAI API key; Azure, vLLM, and local-model routes require additional environment variables or configuration changes.
- Tool-enabled simulations require BMTools, while tool-using task-solving cases require an XAgent ToolServer deployment.
- The README says the simulation code is being refactored and directs simulation-only users seeking a stable version to the
release-0.1branch. - Its roadmap still lists documentation and more sophisticated conversation-history memory as unfinished work.
How do you install or deploy this agent?
Python 3.9+ is required.
git clone https://github.com/OpenBMB/AgentVerse.git --depth 1
cd AgentVerse
pip install -e .
export OPENAI_API_KEY="your_api_key_here"Alternatively, run pip install -U agentverse. Local-model use additionally requires pip install -r requirements_local.txt.
How do you use this agent?
Configure OPENAI_API_KEY, then run a simulation:
agentverse-simulation --task simulation/nlp_classroom_9playersFor the local GUI:
agentverse-simulation-gui --task simulation/nlp_classroom_9playersOpen http://127.0.0.1:7860/ after the server starts. A documented HumanEval invocation is:
agentverse-benchmark --task tasksolving/humaneval/gpt-3.5 --dataset_path data/humaneval/test.jsonl --overwriteFAQ
Must I use OpenAI models?
What is the quickest first run?
OPENAI_API_KEY, and run agentverse-simulation --task simulation/nlp_classroom_9players.Can I inspect a simulation in a browser?
agentverse-simulation-gui --task simulation/nlp_classroom_9players, then open http://127.0.0.1:7860/.