CaveAgent
Turn LLMs into stateful runtime operators: inject, manipulate, and retrieve real Python objects instead of shuffling text.
- Source repo
- acodercat/cave-agent
- Stars
- ★ 205
- Last updated
- 1mo ago
- License
- MIT
- Primary language
- Python
- FA score
- 62/100 · Some gaps
At a glance
- How it runs
- Works with
- Universal · cross-platformOpenAI API · Claude API
- Cost
- Free software; you pay for model usage
- Setup effort
- Low · running in minutes
- You'll need
- Typical use
- A data analyst wants an LLM to query a database and produce renderable charts: inject a SQLAlchemy Engine and an ECharts config manager; the LLM runs SQL and returns real chart config objects for a web UI.
- Not a fit if
- Teams needing a built-in sandbox for untrusted code
- Projects still on Python 3.11 or older
- Teams requiring proven production maturity and third-party evaluations
- Source review
- 62/100 · Some gaps
What does this agent do, and when should you use it?
CaveAgent is an MIT-licensed open-source Python framework that gives LLM agents stateful runtime management. It breaks away from the text-in-text-out paradigm where tool interactions are limited to JSON primitives: any Python object—DataFrames, database connections, custom class instances—can be injected into a persistent runtime as first-class variables the LLM can manipulate across turns, without serialization. Two runtime backends are provided: the in-process IPythonRuntime (default, zero-copy direct access) and the process-isolated IPyKernelRuntime (crash-safe, objects injected via dill serialization). Function calling is code-based: the LLM writes and runs Python against injected objects. The framework includes multi-agent coordination, real-time event streaming, AST-based security rules, the Agent Skills open standard with an injection.py extension, multi-tier context compaction, large-output persistence, token budgets, and extensive API resilience mechanisms. Model support covers OpenAI-compatible endpoints and hundreds of providers through LiteLLM.
After installing, create an agent with CaveAgent(model, runtime=runtime). Developers declare Variable entries (e.g., a SQLAlchemy engine, a DataFrame), Function entries (Python functions), and Type entries (class schemas) on an IPythonRuntime or IPyKernelRuntime; the LLM generates and executes Python code that calls these objects directly, and results stay in the runtime for retrieval via await runtime.retrieve("name") as native Python types. Runs stream typed frozen-dataclass events (TextEvent, CodeEvent, ExecutionResultEvent, ThinkingChunkEvent, StoppedEvent) that can be matched and rendered, with renderers like TerminalRenderer provided. In multi-agent mode, an orchestrator's runtime injects sub-agents as first-class variables and drives them. Security is handled by an AST-based SecurityChecker with ImportRule, FunctionRule, AttributeRule and RegexRule—documented as advisory hardening, not a sandbox. Context compaction runs in tiers: microcompact clears old execution results instantly, full compact uses LLM summarization with a dual-phase prompt, CJK-aware token estimation, incremental re-compaction, and batch folding for oversized inputs.
- A data analyst wants an LLM to query a database and produce renderable charts: inject a SQLAlchemy Engine and an ECharts config manager; the LLM runs SQL and returns real chart config objects for a web UI.
- Conversational applications that need object state across turns: injected variables persist without being pushed into the context window or re-serialized each turn.
- Smart-home or IoT control: inject device class instances with methods (e.g., Light, Thermostat); the LLM calls methods directly and retrieves objects with updated state.
- Multi-agent data pipelines: an orchestrator injects cleaner and analyzer sub-agents as first-class variables, each sub-agent with its own runtime.
- Sandboxed workflows running untrusted code: use IPyKernelRuntime process isolation—kernel crashes leave the host alive, and you just reset the kernel.
- Long or cost-sensitive sessions: context compaction, cumulative token budgets (max_total_tokens, etc.), and large-output persistence keep context and spend bounded.
How do you install or deploy this agent?
Requires Python 3.12+. Install everything: pip install 'cave-agent[all]'. Or pick extras: OpenAI support via pip install 'cave-agent[openai]'; 100+ providers via pip install 'cave-agent[litellm]'; process-isolated kernel runtime via pip install 'cave-agent[ipykernel]'. You need an API key for whichever LLM provider you use.
How do you use this agent?
1) Build a model: from cave_agent.models import LiteLLMModel; model = LiteLLMModel(model_id="model-id", api_key="your-api-key", custom_llm_provider="openai"). 2) Build a runtime: from cave_agent.runtime import IPythonRuntime, Variable, Function; runtime = IPythonRuntime(variables=[Variable("secret", "!dlrow ,olleH", "A reversed message")], functions=[Function(reverse)]). 3) Run: agent = CaveAgent(model, runtime=runtime); response = await agent.run("Reverse the secret"); retrieve objects with await runtime.retrieve("secret"). Stream with async for event in agent.stream_events(...). Use the isolated backend via async with IPyKernelRuntime(...) as runtime. Key configuration includes max_steps (default 10), context_window (default 128000), max_total_tokens, and max_exec_timeout (IPyKernelRuntime only).
What are this agent's strengths and limitations?
- Object-level injection and retrieval is a concrete differentiator: the LLM manipulates real Python objects rather than JSON primitives, with state living in the runtime instead of the context window and zero serialization overhead in-process.
- Two runtime backends match trust levels: zero-copy IPythonRuntime for trusted environments, crash-isolated IPyKernelRuntime for untrusted code, with the kernel deferred until the first code execution.
- Strong engineering resilience: exponential-backoff retries for transient errors, no retry on billing errors, one-shot reactive compaction on context overflow, a stream idle-timeout watchdog, and up-to-3 continuation on truncated responses, with StopReason distinguishing model, runtime, and internal errors.
- Oversized execution output is not discarded: full text binds to runtime variables (_output_1, _output_2, …) that the model can slice, grep, or re-parse—no re-running expensive queries.
- Implements the Agent Skills open standard with an injection.py extension; skill metadata (~100 tokens) loads at startup and instructions load on-demand via progressive disclosure.
- SecurityChecker is advisory AST-based hardening, not a sandbox; the docs explicitly require a real isolation boundary (containers with seccomp/gVisor, OS resource limits) for genuinely untrusted code, which adopters must build themselves.
- The Python 3.12+ requirement is relatively strict, and core execution depends on embedded IPython; the isolated runtime adds IPyKernel and dill dependencies plus ~1s startup and serialization costs.
- max_exec_timeout requires a preemptible runtime and is only supported by IPyKernelRuntime—passing it to IPythonRuntime raises ValueError at construction.
- In process-isolated mode, injected objects are serialized via dill; local functions and closures work, but this carries performance and compatibility costs compared to in-process direct references.
- The project is young (arXiv paper dated 2026) with limited community blogs and third-party evaluations, so evidence of production maturity at scale is still thin.
How does this agent compare with similar options?
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| CaveAgent This agent | 62 · Some gaps | Library / SDKFree + model costs | ★ 205 | 1mo ago | Python | OpenAI API · Claude API |
| Agent Craft | 49 · Major gaps | CLIFree + model costs | ★ 521 | 9d ago | Python | OpenAI API |
| KodeAgent | 43 · Major gaps | Library / SDKFree + model costs | ★ 44 | 21d ago | Python | OpenAI API · Claude API |
| Langroid | 70 · Some gaps | Library / SDKFree + model costs | ★ 4.1k | today | Python | Claude Code · OpenAI API |
How does FollowAgents rate this agent?
Why each dimension lost points
Evidence shows the default IPythonRuntime executes LLM-generated code in the host process, and SecurityChecker is AST-only, with the README itself stating it is not a sandbox — a large external effect honestly disclosed, so external_effects is 1 and least_privilege 2 thanks to the process-isolated backend and deferred kernel start. No user-confirmation mechanism for dangerous operations appears (user_confirmation 1). No key redaction or data-classification guidance; examples pass api_key in plaintext (sensitive_data_handling 1). SHA-pinned CI actions and uv.lock give dependency_security 2. Rollback is only sporadically mentioned via reset()/kernel reset (1). MIT license, author attribution and arXiv link support source_attribution 2.
The claimed typed error hierarchy, retry/backoff, circuit breaker, and idle-timeout designs are corroborated across pyproject, CI, and test infrastructure (FakeModel/FailingModel, self-skipping live tests), giving self_consistency 2 rather than 3 because core src is absent from evidence, so implementation-vs-docs consistency cannot be statically confirmed. Extras splitting and locked installs support dependency_availability 2. StopReason-style failure semantics are clearly defined but documented-only (failure_messages 2).
Audiences (developers, internal tools, multi-tenant) and scenarios (trusted vs untrusted code) are explicitly separated with a runtime comparison table (audience_and_scenarios 2). capability_boundaries is 2 due to the explicit non-sandbox warning and budgets; on-demand skill activation and first-execution kernel spawn give trigger_precision 2; pinned Python 3.12+ and optional extras give environment_fit 2. Scores stop short of 3 because nothing was executed.
README is well structured with TOC, install, examples, features, configuration — information_architecture 3; per-extra install instructions are clear — install_notes 3; naming_stability 2, evidenced by a reasoned note preserving MessageRole's legacy behavior; examples are broad and file-referenced (examples_and_faq 2) but there is no FAQ; known_limitations 2 because boundaries (security, budget headroom) are proactively stated; the LICENSE file is complete — 3. Deductions: only a 0.8.0 Alpha version with no CHANGELOG or release notes (versioning_changelog 1) and no maintenance/governance commitment (maintenance_responsibility 1); the unverified publisher is not itself a deduction per the publisher rule.
Objects are injected/retrieved as native types and oversized outputs persist as runtime variables with actionable pointers — output_usability 2; object-level data flow versus JSON tool calls is a clear differentiated value (marginal_value 2); cost controls (token/time/cost budgets, multi-tier compaction, batched summarization) are well designed but unverified by execution, so cost_benefit 2.
README claims point to an arXiv paper, named example files, and concrete config fields, statically traceable (claim_traceability 2); README agrees with pyproject/CI/test comments, e.g., the note that the [all] extra fixed a past omission (cross_source_corroboration 2), though external claims (PyPI, paper content) cannot be checked; design rationale is mostly separated from fact statements in comments (fact_inference_separation 2). Overall confidence stays low because no code was executed.
- The default IPythonRuntime executes LLM-generated code in the host process; SecurityChecker is static AST hardening only and the README itself says it is not a sandbox — for untrusted code use IPyKernelRuntime plus a real isolation boundary (container/seccomp/gVisor).
- No user-confirmation mechanism exists: the LLM can directly manipulate injected sensitive objects such as DB connections; add an approval layer for sensitive use, and avoid the plaintext api_key pattern shown in examples by using environment variables.
- The project is at 0.8.0 Alpha with no CHANGELOG or maintenance commitment and an unverified publisher identity — assess supply-chain and upgrade risk before production adoption.
- This is a static source review; no reliability, cost, or security claim has been verified by execution.
Recent releases
Latest entries from the project's GitHub Releases. Notes are quoted from upstream as written, not rewritten or tested by FollowAgents.
-
v0.8.0 v0.8.0 — Hardened agent loop, typed failures, real recovery
Release notes excerpt
The 0.8.0 rewrite, hardened by fifteen review rounds: ~120 verified defects fixed, each pinned by a regression test. Highlights - One execution path — run() and streaming can no longer diverge; every run ends with a typed StoppedEvent - Typed failures — MODEL_ERROR / RUNTIME_ERROR / INTERNAL_ERROR are distinct: provider, runtime, or bug - Real recovery — output truncation, mid-stream failures, and prompt overflow resume instead of corrupting history …View on GitHub ↗ -
v0.7.5.0 v0.7.5 — Live reasoning, sturdier streaming & resilience
Release notes excerpt
Hardening pass across the model layer, runtime, and context handling, plus live reasoning support. Highlights - Live reasoning — reasoning models stream their thinking as THINKING_CHUNK events, rendered live in the terminal. - Sturdier streaming — typed StreamDelta contract, SSE connect-retry, an idle-timeout watchdog, and proper cleanup (aclose() + async context manager). - Typed errors — provider failures mapped to PromptTooLongError / ProviderBillingError; billing exhaustion no longer burns retries. …View on GitHub ↗ -
v0.7.3 v0.7.3 — Context Compaction & API Resilience
Release notes excerpt
What's New - Context Compaction — 3-tier strategy inspired by Claude Code: microcompact (clear old execution results) → LLM summarization (dual-phase analysis+summary prompt) → trim fallback, with circuit breaker protection. Replaces naive max_history truncation with token-aware context_window management. - API Resilience — Exponential backoff retry for 429/5xx/timeout errors (up to 5 attempts). Automatic …View on GitHub ↗