LLM-in-Sandbox

What does this agent do, and when should you use it?

LLM-in-Sandbox is an open-source research framework (Apache-2.0) from RUC AI Box at Renmin University of China, accompanying the paper 'Computer Environments Elicit General Agentic Intelligence in LLMs'. Its core idea: place any LLM inside a code sandbox with basic computer functionalities such as a file system and command execution, turning it into a general agent that outperforms standalone LLMs across diverse non-code domains. The software ships as a CLI (llm-in-sandbox run) and is backend-agnostic, connecting to OpenAI, Anthropic, and self-hosted OpenAI-compatible servers like vLLM and SGLang. Execution happens in the Docker image cdx123/llm-in-sandbox:v0.1 (~400 MB, auto-pulled on first run) for security through isolation. Each run writes a timestamped folder under ./output containing answer files and a trajectory.json execution history. The repository also includes a v0.2.0 benchmark module for reproducing paper results, and a companion repo, LLM-in-Sandbox-RL, releasing RL code, models, data, and training logs.

You submit a task via llm-in-sandbox run with a --query and model configuration; the framework launches a Docker sandbox (default image cdx123/llm-in-sandbox:v0.1) and hands the task to the selected LLM, which iteratively executes commands and reads/writes files inside the sandbox for up to --max_steps (default 100) conversation turns. You can mount an input folder via --input_dir; results go to a timestamped folder under --output_dir (default ./output) containing files/answer.txt, produced files, and trajectory.json. The file system acts as long-term memory, with up to 8× token savings reported in the paper. Prompt templates are switched via --prompt_config (default ./config/general.yaml), and the llm_in_sandbox/benchmark/ module reproduces paper evaluations, evaluates any LLM, or adds custom tasks. llm-in-sandbox build builds a custom image from docker/Dockerfile.

1. Researchers reproducing the paper's evaluation showing sandboxing boosts LLM performance on chemistry, physics, math, biomedicine, long-context understanding, and instruction following
2. Teams running self-hosted models (e.g., Qwen3-Coder-30B, DeepSeek-V3.2 on vLLM or SGLang) who want general agentic capability without training
3. Engineers who want LLM code and file execution inside Docker isolation rather than unrestricted setups like OpenClaw
4. Researchers running reinforcement learning experiments using the open code, models, and wandb logs from LLM-in-Sandbox-RL
5. Evaluators benchmarking any LLM behind an OpenAI-compatible endpoint across domains via the built-in benchmark module
6. Agent developers looking for examples in non-coding tasks: scientific reasoning, travel planning, video production, music composition, poster design

How do you install or deploy this agent?

Prerequisites: Python 3.10+ and Docker. Install Docker if not present:

curl -fsSL https://get.docker.com -o get-docker.sh && sh get-docker.sh
dockerd > /var/log/dockerd.log 2>&1 &

Install via pip:

pip install llm-in-sandbox==0.2.0

Or from source:

git clone https://github.com/llm-in-sandbox/llm-in-sandbox.git
cd llm-in-sandbox
pip install -e .

The default Docker image (~400MB) is pulled automatically on first run. No local GPU is needed when using API services. Tested on Ubuntu 22.04 with Python 3.10.

How do you use this agent?

Option 1: cloud/API services (supply a base URL and API key):

llm-in-sandbox run \
    --query "write a hello world in python" \
    --llm_name "openai/gpt-5" \
    --llm_base_url "http://your-api-server/v1" \
    --api_key "your-api-key"

Option 2: self-hosted models. Start a vLLM server:

vllm serve Qwen/Qwen3-Coder-30B-A3B-Instruct \
    --served-model-name qwen3_coder \
    --enable-auto-tool-choice \
    --tool-call-parser qwen3_coder \
    --tensor-parallel-size 8  \
    --enable-prefix-caching

Then run the agent in a new terminal:

llm-in-sandbox run \
    --query "write a hello world in python" \
    --llm_name qwen3_coder \
    --llm_base_url "http://localhost:8000/v1"  \
    --temperature 0.7

SGLang serving DeepSeek-V3.2 is also documented (add --extra_body '{"chat_template_kwargs": {"thinking": True}}'). Common parameters: --input_dir (mount input files), --output_dir (default ./output), --max_steps (default 100), --prompt_config (default ./config/general.yaml). Run llm-in-sandbox run --help for all options. See llm_in_sandbox/benchmark/README.md to reproduce paper results.

What are this agent's strengths and limitations?

How does this agent compare with similar options?

Key facts side by side with the most closely related agents.

How does FollowAgents rate this agent?

Recent releases

Latest entries from the project's GitHub Releases. Notes are quoted from upstream as written, not rewritten or tested by FollowAgents.

1. v0.2.0 2026-02-11
   Release notes excerpt

   What's New Benchmark Module - Added benchmark framework to reproduce our paper results and evaluate any LLM/task - Support reward function to facilitate LLM-in-Sandbox-RL - Support for LLM-in-Sandbox and vanilla LLM modes - LLM-as-Judge evaluation Improvements - Restructured README and benchmark docs - Better error handling and Docker cleanup guidance - Clean action, improve observation PyPI - pip install llm-in-sandbox==0.2.0

   View on GitHub ↗

FAQ