ChatArena
A Python framework for building and running multi-agent language-game experiments with LLM players.
- Source repo
- Farama-Foundation/ChatArena
- Stars
- ★ 1.6k
- Last updated
- 1y ago
- License
- Apache-2.0
- Primary language
- Python
- FA score
- 32/100 · Major gaps
At a glance
- How it runs
- Works with
- Universal · cross-platformOpenAI API
- Cost
- Free software; you pay for model usage
- Setup effort
- Low · running in minutes
- You'll need
- Typical use
- A multi-agent researcher needs a turn-based framework for testing communication, collaboration, or social interaction among LLM players.
- Not a fit if
- Teams needing ongoing maintenance or upstream support; the project was deprecated on Aug 11, 2025
- Users wanting all backends and environments out of the box; Anthropic, Cohere, Hugging Face and PettingZoo need optional extras
- Teams unwilling to depend on an OpenAI API key or pay per-use model costs
- Source review
- 32/100 · Major gaps 8 safety controls not found
What does this agent do, and when should you use it?
ChatArena is a Python library for studying autonomous LLM agents and their social interactions through multi-agent language games. Its core model consists of Arena, Environment, Language Backend, and Player: Arena manages the game loop, configuration loading, data storage, and HCI utilities such as a Web UI and CLI. Environments hold game state, apply transition logic, and render natural-language observations that players use to act. The repository includes conversation, moderator-led conversation, Chameleon, and PettingZoo-backed chess and tic-tac-toe environments, with JSON examples for running games. The project was declared deprecated on August 11, 2025, with no additional updates or support planned, so it is better suited to reproduction, experimentation, or self-maintained extensions than to a dependency requiring active upstream support.
A user can create an Arena from a JSON file with Arena.from_config("examples/nlp-classroom-3players.json") and advance the game with arena.run(num_steps=10). Arena coordinates Players and an Environment in the main loop; the Environment manages state, transitions, observations, terminal conditions, and rewards through mechanisms including step, reset, get_observation, is_terminal, and get_rewards. A Player receives a natural-language observation and, by default, queries a Language Backend before returning its response as an action. The project also exposes arena.launch_cli() for interactive use and a local Gradio Web UI launched with gradio app.py. New environments are created by extending Environment, assigning type_name, and adding the class to ALL_ENVIRONMENTS.
- A multi-agent researcher needs a turn-based framework for testing communication, collaboration, or social interaction among LLM players.
- A prompt engineer wants to run a configured multi-player interaction, such as examples/nlp-classroom-3players.json, without writing a game loop.
- A teaching or prototyping team wants to use the Chameleon social-deduction game to inspect reasoning and voting under asymmetric information.
- An experimenter wants to contrast LLM-moderated rock-paper-scissors or tic-tac-toe with PettingZoo environments driven by hard-coded rules.
- A Python developer wants to implement a new language game with custom state, observations, rewards, and step logic.
How do you install or deploy this agent?
The README body lists Python >= 3.7 as the requirement, while its badge says Python 3.9+. Install the core package:
pip install chatarenaInstall all supported backend dependencies:
pip install chatarena[all_backends]Install all environment dependencies:
pip install chatarena[all_envs]Install all optional dependencies:
pip install chatarena[all]For GPT-3.5-turbo or GPT-4 agents, set the optional credential:
export OPENAI_API_KEY="your_api_key_here"How do you use this agent?
Create and run a game from the repository example configuration:
arena = Arena.from_config("examples/nlp-classroom-3players.json")
arena.run(num_steps=10)Launch the interactive CLI:
arena.launch_cli()Launch the local Web UI:
pip install chatarena[gradio]
git clone https://github.com/chatarena/chatarena.git
cd chatarena
gradio app.pyThe README says the demo server can then be accessed in a browser on port 8080.
What are this agent's strengths and limitations?
- It separates Arena, Environment, Language Backend, and Player, allowing game rules, loops, prompt formatting, and player behavior to be customized independently.
- It provides a Python API, interactive CLI, and Gradio Web UI for both configured experiments and local demonstrations.
- Its included environments span open-ended conversation, moderator-controlled games, social deduction, and PettingZoo-backed rule-based games.
- JSON configuration and the documented Environment extension path support new language games with custom state, rewards, and observations.
- The project is explicitly deprecated and no further updates or support are planned, leaving adopters responsible for maintenance and compatibility.
- Using GPT-3.5-turbo or GPT-4 requires an OpenAI API key, creating an external-service dependency and potential usage cost.
- Advanced backends and environments are not in the default installation; features involving Anthropic, Cohere, Hugging Face, or PettingZoo require optional extras.
- The local Web UI workflow requires cloning the repository and installing the Gradio extra, rather than using only the core package.
How does this agent compare with similar options?
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| ChatArena This agent | 32 · Major gaps | Library / SDKFree + model costs | ★ 1.6k | 1y ago | Python | OpenAI API |
| AgentVerse | 28 · Major gaps | CLIFree + model costs | ★ 5.2k | 2y ago | JavaScript | OpenAI API |
| VirtualHome | 40 · Major gaps | Library / SDKFree | ★ 645 | 2y ago | Python | — |
| IR-SIM: Lightweight Robot Simulator | 43 · Major gaps | Library / SDKFree | ★ 1.2k | 5d ago | Python | — |
How does FollowAgents rate this agent?
Why each dimension lost points
Evidence shows: The repository provides no mechanisms for permission management, user confirmation, data flow transparency, sensitive data handling, dependency security, external effects, rollback, or source attribution. README only mentions the need for an OpenAI API key but does not explain its usage or storage. pyproject.toml lists dependencies but lacks security audits or vulnerability scanning. Therefore, all trust criteria score 0.
Evidence shows: README and test files provide basic consistency, but there are inconsistencies such as pyproject.toml declaring MIT license while LICENSE file is Apache-2.0. Dependency versions are pinned, but availability guarantees are not provided. Test files include skip conditions but lack detailed failure messages. Therefore, self-consistency, dependency availability, and failure messages each score 1.
Evidence shows: README describes multiple environments and scenarios, such as conversation and games, suitable for research. However, capability boundaries are not clearly defined, trigger conditions (e.g., environment configuration) are not detailed, and environment fit (e.g., Python version) is mentioned but not comprehensive. Therefore, audience and scenarios score 2, others score 1.
Evidence shows: README provides installation instructions, examples, and architecture diagram, with clear information architecture. However, naming stability is not explicit, known limitations are only mentioned as project deprecation, versioning changelog is missing, and maintenance responsibility is unclear. License is Apache-2.0, but pyproject.toml incorrectly declares MIT. Therefore, information architecture, install notes, examples, and license score 2, others score 1.
Evidence shows: README provides clear output examples and use cases, indicating good output usability. Marginal value is high due to multiple environments and interfaces. However, cost-benefit is not detailed, such as API costs or computational resources. Therefore, output usability and marginal value score 2, cost-benefit scores 1.
Evidence shows: Claims in README are partially supported by code examples, but no external verification or cross-source corroboration is provided. Facts and inferences are not clearly separated, e.g., the deprecation claim lacks evidence. Therefore, all criteria score 1.
- Not found in source: least-privilege scopingGrant only what the task needs: a dedicated account or read-only token, scoped to specific directories and repos.
- Not found in source: confirmation before actingTurn on (or add) a confirmation step before it acts, and try it in a sandbox or test environment before real data.
- Not found in source: data-flow disclosureWatch which external services it contacts (proxy or firewall logs) and keep sensitive data out until you know where it goes.
- Not found in source: sensitive-data handlingUse dedicated, low-privilege, revocable API keys — never production credentials — and keep secrets out of logs.
- Not found in source: dependency securityPin versions and run a dependency audit (npm audit, pip-audit) before installing; prefer running it in a container.
- Not found in source: disclosed external effectsEstablish which external systems it writes to, sends to or changes, and verify with test accounts or repos before production.
- Not found in source: rollback or recovery pathBack up first, or work on a git branch or snapshot, so its changes can be undone.
- Not found in source: verifiable attributionInstall from the official repo or registry and check the publisher and URL to avoid look-alike packages.
- The project is deprecated and no longer maintained; use with caution.
- License declaration in pyproject.toml is inconsistent with LICENSE file (MIT vs Apache-2.0).
- Dependencies are pinned but no security audit is provided; potential known vulnerabilities.
- API key handling is not documented, posing a leak risk.
Recent releases
Latest entries from the project's GitHub Releases. Notes are quoted from upstream as written, not rewritten or tested by FollowAgents.
-
Release notes excerpt
ChatArena 0.1.18 This is a hotfix release, fixing a minor issue with a kwarg in the Umshini environments, which caused issues in certain use cases. Apologies for the small and frequent releases, unfortunately there's no way around it currently. Bug Fixes * Umshini: fix minor typo with player_names kwarg by @elliottower in https://github.com/Farama-Foundation/chatarena/pull/117 Full Changelog: https://github.com/Farama-Foundation/chatarena/compare/v0.1.17...v0.1.18View on GitHub ↗ -
Release notes excerpt
ChatArena 0.1.17 This is a minor release, including a minor enhancement to the Umshini environments, and adding registration functionality for all environments and intelligence backends--allowing the use of Arena.from_config with custom environments. New Features and Improvements * Umshini: add classification of vulnerability category (e.g., prompt injection) for successful attacks by @elliottower in https://github.com/Farama-Foundation/chatarena/pull/114 …View on GitHub ↗ -
Release notes excerpt
ChatArena 0.1.16 This is a very minor release fixing some issues and adding slight enhancements to the Umshini environments. Bug Fixes * Umshini: Fix bug where explanation is not initialized by @andrewtanJS in https://github.com/Farama-Foundation/chatarena/pull/111 * Umshini: add asymmetric (fixed role) versions of deception and content moderation envs by @elliottower in https://github.com/Farama-Foundation/chatarena/pull/112 Full Changelog: https://github.com/Farama-Foundation/chatarena/compare/v0.1.15...v0.1.16View on GitHub ↗