LycheeMemory
Lightweight, retrievable long-term memory infrastructure for LLM agent sessions.
What does this agent do, and when should you use it?
LycheeMemory is a self-hosted long-term memory framework for LLM agents with working, semantic, procedural-skill, and visual memory stores. It uses SQLite with FTS5 and LanceDB for storage and retrieval, and runs memory consolidation asynchronously after responses through a LangGraph pipeline. The project ships as a Python package and HTTP service, with an OpenAI-compatible Chat Completions API, an HTTP MCP endpoint, and plugins for OpenClaw, Claude Code, and Hermes. Its semantic layer combines typed, action-annotated MemoryRecords, hierarchical CompositeRecords, and retrieval feedback logs for agents that need durable, inspectable context across sessions. Deployment is a locally run Python service, while LLM and embedding calls require credentials for a compatible provider.
For each request, WMManager appends the session turn, checks a dual token-budget threshold, and produces compressed history plus recent raw turns. SearchCoordinator derives an ActionState and SearchPlan from the query and recent context, retrieves CompositeRecord trees, MemoryRecords, skills, and—when needed—raw episode turns; SynthesizerAgent scores candidates from 0 to 1 and produces background_context, provenance, and skill_reuse_plan; ReasoningAgent generates the final reply. After the reply, a background ConsolidatorAgent calls CompactSemanticEngine.ingest_conversation() to extract, decontextualize, and annotate MemoryRecords, then performs embedding-based Record Fusion and hierarchical consolidation. External hosts can mirror turns through /memory/append-turn, trigger consolidation through /memory/consolidate, or use /mcp tools including lychee_memory_smart_search, lychee_memory_search, lychee_memory_append_turn, and lychee_memory_consolidate.
- An OpenClaw team that wants user preferences, constraints, and prior tool-use knowledge to persist between sessions.
- A Python-agent developer adding durable memory while operating a local backend through lycheemem-cli.
- An MCP-capable client that needs one-call, compact background recall before composing a response.
- A support or operations agent that stores successful tool-use patterns as skills and retrieves them with HyDE.
- An agent workflow that handles screenshots, charts, or document images and needs text-to-image or similar-image memory retrieval.
What are this agent's strengths and limitations?
- Offers several integration surfaces in one project: Python package, HTTP MCP, OpenAI-compatible Chat Completions, and runtime plugins.
- Semantic memory is more than a flat vector store: MemoryRecords have seven types, action tags, usage statistics, and conflict-aware hierarchical Record Fusion.
- Retrieval combines ANN prefiltering and one LLM relevance judgment at the CompositeRecord level, optional tree expansion, and supplementary FTS, vector, and episode-turn recall.
- The 70% pre-compression and 90% blocking thresholds manage working context, while ConsolidatorAgent runs after the response without blocking it.
- You must operate a local Python service and configure LLM and embedding credentials; no managed deployment workflow is documented.
- The default Transformer reranker needs optional PyTorch/Transformers dependencies and loads a Hugging Face checkpoint on first use; unavailable dependencies fall back to baseline retrieval.
- Memory persists through SQLite, LanceDB, and the local filesystem, so adopters need to plan data paths, retention, and access control.
- Visual memory requires a VLM and a CLIP-style embedding model; the documentation does not provide cost, hardware, or production-scale capacity figures.
How do you install or deploy this agent?
Prerequisites: Python 3.9+ and an LLM API key for OpenAI, Gemini, or another litellm-compatible provider.
Install the core package:
pip install lycheemem
Install with the default Transformer reranker:
pip install "lycheemem[rerank]"
Create a .env file in the working directory with at least:
LLM_MODEL=openai/gpt-4o-mini
LLM_API_KEY=sk-...
EMBEDDING_MODEL=openai/text-embedding-3-small
EMBEDDING_DIM=1536
Start the service:
lycheemem-cli
The service defaults to http://localhost:8000, with interactive documentation at /docs.
How do you use this agent?
After starting the service, point an OpenAI-compatible SDK base_url at http://localhost:8000/v1 and call POST /v1/chat/completions with model, messages, and a stable session_id. For recall-only integration, send {"query":"...","top_k":5,"synthesize":true,"mode":"compact"} to POST /memory/smart-search and consume background_context. MCP clients can configure http://localhost:8000/mcp; after initialize, reuse the returned Mcp-Session-Id, with lychee_memory_smart_search in compact mode as the recommended path. External conversation hosts can write user or assistant turns to /memory/append-turn and consolidate the same session through /memory/consolidate.
How does this agent compare with similar options?
For OpenClaw, LycheeMemory provides a native plugin that automatically mirrors user and assistant turns and triggers boundary consolidation on /new, /reset, /stop, or session_end. The repository reports that, in its PinchBench evaluation using this plugin, it improved scores by about 6% versus OpenClaw native memory while reducing token use by about 71% and cost by about 55%; these are repository-reported evaluation results.