ChatArena
A Python framework for building and running multi-agent language-game experiments with LLM players.
Per-dimension scores and reasoning
Evidence shows: The repository provides no mechanisms for permission management, user confirmation, data flow transparency, sensitive data handling, dependency security, external effects, rollback, or source attribution. README only mentions the need for an OpenAI API key but does not explain its usage or storage. pyproject.toml lists dependencies but lacks security audits or vulnerability scanning. Therefore, all trust criteria score 0.
Evidence shows: README and test files provide basic consistency, but there are inconsistencies such as pyproject.toml declaring MIT license while LICENSE file is Apache-2.0. Dependency versions are pinned, but availability guarantees are not provided. Test files include skip conditions but lack detailed failure messages. Therefore, self-consistency, dependency availability, and failure messages each score 1.
Evidence shows: README describes multiple environments and scenarios, such as conversation and games, suitable for research. However, capability boundaries are not clearly defined, trigger conditions (e.g., environment configuration) are not detailed, and environment fit (e.g., Python version) is mentioned but not comprehensive. Therefore, audience and scenarios score 2, others score 1.
Evidence shows: README provides installation instructions, examples, and architecture diagram, with clear information architecture. However, naming stability is not explicit, known limitations are only mentioned as project deprecation, versioning changelog is missing, and maintenance responsibility is unclear. License is Apache-2.0, but pyproject.toml incorrectly declares MIT. Therefore, information architecture, install notes, examples, and license score 2, others score 1.
Evidence shows: README provides clear output examples and use cases, indicating good output usability. Marginal value is high due to multiple environments and interfaces. However, cost-benefit is not detailed, such as API costs or computational resources. Therefore, output usability and marginal value score 2, cost-benefit scores 1.
Evidence shows: Claims in README are partially supported by code examples, but no external verification or cross-source corroboration is provided. Facts and inferences are not clearly separated, e.g., the deprecation claim lacks evidence. Therefore, all criteria score 1.
- The project is deprecated and no longer maintained; use with caution.
- License declaration in pyproject.toml is inconsistent with LICENSE file (MIT vs Apache-2.0).
- Dependencies are pinned but no security audit is provided; potential known vulnerabilities.
- API key handling is not documented, posing a leak risk.
What does this agent do, and when should you use it?
ChatArena is a Python library for studying autonomous LLM agents and their social interactions through multi-agent language games. Its core model consists of Arena, Environment, Language Backend, and Player: Arena manages the game loop, configuration loading, data storage, and HCI utilities such as a Web UI and CLI. Environments hold game state, apply transition logic, and render natural-language observations that players use to act. The repository includes conversation, moderator-led conversation, Chameleon, and PettingZoo-backed chess and tic-tac-toe environments, with JSON examples for running games. The project was declared deprecated on August 11, 2025, with no additional updates or support planned, so it is better suited to reproduction, experimentation, or self-maintained extensions than to a dependency requiring active upstream support.
A user can create an Arena from a JSON file with Arena.from_config("examples/nlp-classroom-3players.json") and advance the game with arena.run(num_steps=10). Arena coordinates Players and an Environment in the main loop; the Environment manages state, transitions, observations, terminal conditions, and rewards through mechanisms including step, reset, get_observation, is_terminal, and get_rewards. A Player receives a natural-language observation and, by default, queries a Language Backend before returning its response as an action. The project also exposes arena.launch_cli() for interactive use and a local Gradio Web UI launched with gradio app.py. New environments are created by extending Environment, assigning type_name, and adding the class to ALL_ENVIRONMENTS.
- A multi-agent researcher needs a turn-based framework for testing communication, collaboration, or social interaction among LLM players.
- A prompt engineer wants to run a configured multi-player interaction, such as examples/nlp-classroom-3players.json, without writing a game loop.
- A teaching or prototyping team wants to use the Chameleon social-deduction game to inspect reasoning and voting under asymmetric information.
- An experimenter wants to contrast LLM-moderated rock-paper-scissors or tic-tac-toe with PettingZoo environments driven by hard-coded rules.
- A Python developer wants to implement a new language game with custom state, observations, rewards, and step logic.
What are this agent's strengths and limitations?
- It separates Arena, Environment, Language Backend, and Player, allowing game rules, loops, prompt formatting, and player behavior to be customized independently.
- It provides a Python API, interactive CLI, and Gradio Web UI for both configured experiments and local demonstrations.
- Its included environments span open-ended conversation, moderator-controlled games, social deduction, and PettingZoo-backed rule-based games.
- JSON configuration and the documented Environment extension path support new language games with custom state, rewards, and observations.
- The project is explicitly deprecated and no further updates or support are planned, leaving adopters responsible for maintenance and compatibility.
- Using GPT-3.5-turbo or GPT-4 requires an OpenAI API key, creating an external-service dependency and potential usage cost.
- Advanced backends and environments are not in the default installation; features involving Anthropic, Cohere, Hugging Face, or PettingZoo require optional extras.
- The local Web UI workflow requires cloning the repository and installing the Gradio extra, rather than using only the core package.
How do you install or deploy this agent?
The README body lists Python >= 3.7 as the requirement, while its badge says Python 3.9+. Install the core package:
pip install chatarenaInstall all supported backend dependencies:
pip install chatarena[all_backends]Install all environment dependencies:
pip install chatarena[all_envs]Install all optional dependencies:
pip install chatarena[all]For GPT-3.5-turbo or GPT-4 agents, set the optional credential:
export OPENAI_API_KEY="your_api_key_here"How do you use this agent?
Create and run a game from the repository example configuration:
arena = Arena.from_config("examples/nlp-classroom-3players.json")
arena.run(num_steps=10)Launch the interactive CLI:
arena.launch_cli()Launch the local Web UI:
pip install chatarena[gradio]
git clone https://github.com/chatarena/chatarena.git
cd chatarena
gradio app.pyThe README says the demo server can then be accessed in a browser on port 8080.