TeleMem: Long-Term and Multimodal Memory for Agentic AI
A high-performance drop-in replacement for Mem0 with semantic deduplication, per-character memory, and multimodal video reasoning for conversational AI.
- Source repo
- TeleAI-UAGI/telemem
- Stars
- ★ 494
- Last updated
- 3d ago
- License
- Apache-2.0
- Primary language
- Python
- FA score
- 57/100 · Major gaps
At a glance
- How it runs
- Works with
- Universal · cross-platformOpenAI APIClaude Code · Claude.ai (Partial support)
- Cost
- Free software; you pay for model usage
- Setup effort
- Medium · a few setup steps
- You'll need
- Typical use
- Role-play and companion AI developers who need isolated, per-character long-term memory profiles built automatically
- Not a fit if
- Teams wanting a single MiniMax API: MiniMax has no public embedding API, so a separate embedder is required
- Buyers needing independent third-party benchmarks: results come from the project's own ZH-4O dataset
- Source review
- 57/100 · Major gaps 1 safety controls not found
What does this agent do, and when should you use it?
TeleMem is an agent memory management layer that works as a high-performance drop-in replacement for Mem0 with one line of code (import telemem as mem0). Its pipeline runs character-aware summarization → LLM semantic clustering deduplication → buffered batch writes → dual FAISS + JSON storage → vector retrieval with reranking, and it automatically builds isolated memory profiles for every character in a dialogue. On the multimodal side, add_mm()/search_mm() implement a video → frame extraction → caption generation → vector database pipeline with ReAct-style multi-step video question answering. The project ships an MCP server (telemem-mcp) built on the official MCP Python SDK v2 that exposes eight memory tools for clients like Claude Desktop and Cursor, and can run fully locally (Qwen + Ollama + FAISS). On the ZH-4O Chinese long-dialogue benchmark it achieves 86.33% accuracy, about 19 points higher than Mem0.
TeleMem ingests dialogue messages via Memory.add(): it performs multi-perspective summarization (global event plus per-character views), vectorizes and retrieves similar memories, semantically merges them through an LLM when the write buffer flushes, and dual-writes to a FAISS index and JSON metadata. Memory.search() performs millisecond-level FAISS vector retrieval with reranking, similarity thresholds, and filters. add_mm() decodes a video into frames, generates clip captions with a VLM (e.g., Qwen3-Omni), and builds a vector database; search_mm() drives the MMCoreAgent through a THINK→ACTION→OBSERVATION loop using three tools — global_browse_tool, clip_search_tool, and frame_inspect_tool — for multi-step video QA. The telemem-mcp server supports stdio and Streamable HTTP transports and exposes eight tools: add_memory, search_memories, get_memories, get_memory, update_memory, delete_memory, delete_all_memories, and memory_history. It works with any OpenAI-compatible endpoint and ships configs for Ollama, DeepSeek, Moonshot (Kimi), and MiniMax, plus LangChain and LlamaIndex examples.
- Role-play and companion AI developers who need isolated, per-character long-term memory profiles built automatically
- Multi-NPC game or virtual-world builders: one add_batch() call gives each NPC a private memory profile plus a shared world-state
- Teams building long-conversation assistants (customer service, co-pilots) that must stay factually consistent over hundreds of turns while cutting LLM token cost
- Multimodal agent developers who need to store, retrieve, and reason over video content
- Claude Desktop/Cursor users who want to attach long-term memory to local agents via the MCP server (uvx telemem, zero install)
- Privacy-sensitive teams that want end-to-end local memory with Ollama + FAISS, with no data leaving the machine
How do you install or deploy this agent?
pip install telemem # core (text memory)
pip install "telemem[mcp]" # + MCP server
pip install "telemem[video]" # + video/multimodal pipeline
pip install "telemem[all]" # everything# Or dev environment (uv recommended)
uv sync --all-extras# Credentials for cloud providers:
export OPENAI_API_KEY="your-openai-api-key"
# Fully local option (Ollama) needs no API key: config/config.ollama.yamlHow do you use this agent?
import telemem as mem0
memory = mem0.Memory()messages = [{"role": "user", "content": "Jordan, did you take the subway to work again today?"},
{"role": "assistant", "content": "Yes, James. The subway is much faster."}
]
memory.add(messages=messages, user_id="Jordan")
results = memory.search("What transportation did Jordan use?", user_id="Jordan")for hit in results["results"]:
print(hit["memory"])# Local Qwen + FAISS config:
# TELEMEM_CONFIG=config/config.yaml python examples/quickstart.py
# MCP server: telemem-mcp or uvx telemem (stdio default)
# Multimodal: memory.add_mm(video_path, output_dir) then memory.search_mm(question, output_dir)What are this agent's strengths and limitations?
- Automatically builds isolated per-character memory profiles — aimed at role-play, NPCs, and multi-persona scenarios, a differentiator among open-source memory layers
- Full video memory pipeline (frame extraction → captions → vector DB) with ReAct-style multi-step video QA, beyond text-only memory
- 86.33% accuracy on ZH-4O, roughly 19 points above Mem0, with 2–3× faster batched writes
- Fully local by default (Qwen + FAISS): no cloud service, no paid tier, telemetry off by default
- mem0-compatible API plus an MCP server and LangChain/LlamaIndex examples keep migration and integration cheap
- Benchmark results use the project's own ZH-4O dataset and chosen configuration (Qwen3-8B), not independent third-party evaluation
- Semantic deduplication requires LLM calls; with external OpenAI-compatible providers you must configure both an LLM and a separate embedder (e.g., MiniMax has no public embedding API)
- Fully local operation requires you to provision Qwen-class models and a FAISS environment; the multimodal pipeline additionally needs a VLM and embedding service to generate captions and the vector DB
- The clip_secs parameter in add_mm() is reserved and read from config; the sample video requires pre-existing local artifacts to skip VLM calls
- Apache-2.0 licensing permits commercial use, but as a deep refactor of Mem0 you should track upstream Mem0 ecosystem changes over time
How does this agent compare with similar options?
The README includes a direct comparison table against Mem0: TeleMem supports per-character memory profiles, LLM-based semantic clustering deduplication, batched writes (2–3× faster), FAISS+JSON dual storage, and video multimodal memory, whereas Mem0 lacks multi-character separation, uses only vector-similarity filtering, writes streaming single entries, and handles only single image-to-text. In the reported benchmark, TeleMem (86.33%) outperforms Mem0 (70.20), MOOM (72.60), A-mem (73.78), Memobase (76.78), and RAG (62.45). Because TeleMem is mem0 API-compatible, existing Mem0 code can switch with one import line.
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| TeleMem: Long-Term and Multimodal Memory for Agentic AI This agent | 57 · Major gaps | Library / SDKFree + model costs | ★ 494 | 3d ago | Python | OpenAI API |
| Remnic Agent Memory | 85 · Good | CLIFree + model costs | ★ 227 | today | TypeScript | ChatGPT · Codex · Claude Code · OpenAI API |
| LycheeMemory | 52 · Major gaps | CLIFree + model costs | ★ 1.1k | 3mo ago | Python | Claude Code · OpenAI API |
| Compartment | 76 · Good | CLIFree | ★ 584 | 2d ago | Python | Codex · Claude Code |
How does FollowAgents rate this agent?
Why each dimension lost points
Evidence shows local-by-default operation (Qwen+FAISS), telemetry off by default, MCP destructive ops require explicit scope, and auditable storage layout — all positives. Deductions: requirements.txt contains 'httpx2==2.9.1' and 'httpcore2==2.9.1', non-standard names resembling typosquats of httpx/httpcore, plus implausible future-dated pins (certifi==2026.5.20, attrs==26.1.0), a concrete dependency-poisoning risk — dependency_security 0. The LICENSE appendix reads 'Copyright 2024 Memobase', mismatched with this project, indicating a copied, unedited license file — source_attribution 1. No confirmation flow for delete operations — user_confirmation 1. No rollback/recovery story beyond video caption checkpoints — rollback 1.
Evidence shows consistent API description between README and pyproject, mem0ai pinned to a tested minor range (>=2.0,<2.1), and an offline CI matrix (py3.10–3.12). Deductions: the suspicious lock entries could cause outright install failure or supply-chain compromise — dependency_availability 1; essentially no error-path/failure-message evidence in the provided files — failure_messages 1.
Evidence shows clear target scenarios (multi-character dialogue, companion/NPC, video QA), multi-provider configs (Ollama/DeepSeek/Kimi/MiniMax), Python version range, and extras — audience_and_scenarios 3. Deductions: capability boundaries only scattered (video pipeline needs a VLM; clip_secs is a reserved parameter), no systematic limitations section — capability_boundaries 2; trigger_precision and environment_fit scored 2 on documentation adequacy without executed verification.
Evidence shows complete repo structure, install paths (pip/uv/conda), bilingual docs, examples, MCP config, CI and docs workflows — information_architecture and install_notes 3. Deductions: LICENSE copyright holder reads 'Memobase', a concrete error — license 1; no dedicated known-limitations section — known_limitations 1; changelog exists only as links/news list, not shown in-source — versioning_changelog 2; maintainer is the unverified 'TeleAI-UAGI', so maintenance_responsibility rests on docs alone — 2.
Evidence shows mem0-compatible API (add/search returning {'results': [...]}) lowering migration cost, thorough parameter docs, and return-structure examples — output_usability 2. Deductions: the claimed edge over Mem0 (+19% accuracy, 2-3x write throughput) is self-asserted with no independently checkable evaluation artifacts in the provided sources — marginal_value and cost_benefit 2; reliance on mem0ai private helpers (_create_memory etc.) adds unpriced upstream fragility.
Evidence shows an arXiv paper, CITATION.cff, a baselines/ directory, and the ZH-4O dataset provenance, giving claim_traceability a basis. Deductions: figures like 86.33% cannot be re-checked from the provided files; the MCPVault 'verified' badge carries utm marketing parameters and several news items bear future dates (2026), suspicious corroboration signals — cross_source_corroboration 1; marketing phrasing ('唯一', '显著提升') is intermixed with factual statements — fact_inference_separation 1.
- Not found in source: dependency securityPin versions and run a dependency audit (npm audit, pip-audit) before installing; prefer running it in a container.
- Severe caution: requirements.txt lists httpx2==2.9.1 and httpcore2==2.9.1, likely typosquat-style package names, plus multiple future-dated pins that do not exist on PyPI (e.g., certifi==2026.5.20). Verify every dependency's authenticity and hash on PyPI before installing; otherwise there is a supply-chain poisoning / credential-exfiltration risk.
- The LICENSE file's copyright notice reads 'Copyright 2024 Memobase', inconsistent with this repository's ownership; clarify license provenance with upstream before relying on it.
- The README contains future-dated release announcements and a third-party 'verified' badge with marketing UTM parameters; the accuracy (86.33%) and performance claims are independently unverified — rerun the baselines/ evaluations before adoption.
- mem0 compatibility relies on mem0ai private helpers; upstream minor bumps can break functionality. Pin mem0ai==2.0.5 in production and run your own compatibility regressions.
- Conversation content is sent to the configured LLM/embedding provider unless the Ollama local config is used; assess outbound data flows for sensitive-data scenarios.
Recent releases
Latest entries from the project's GitHub Releases. Notes are quoted from upstream as written, not rewritten or tested by FollowAgents.
-
v1.10.0 v1.10.0 — DeepSeek Harness support
Release notes excerpt
TeleMem now connects directly to the official DeepSeek Harness through its Cordis MCP client. The opt-in patch launches the pinned TeleMem release with uvx and exposes all eight memory operations as mcp__telemem__* tools. Added - DeepSeek Harness patch at examples/deepseek-harness.cordis.yml, tested against the public 0.1.0-rc.5 MCP configuration contract. - Explicit forwarding for provider keys and TeleMem settings that DSH intentionally scrubs from MCP child processes. …View on GitHub ↗ -
v1.9.0 v1.9.0 — MCP SDK v2 / spec 2026-07-28
Release notes excerpt
TeleMem's MCP server is now built on the official MCP Python SDK v2 (mcp>=2.0.0,<3), implementing the current MCP specification (2026-07-28) while remaining compatible with legacy initialize-handshake clients. Changed - MCP server migrated to mcp.server.mcpserver.MCPServer — the v1 SDK's FastMCP import path was removed upstream; the server now declares server-level metadata (title, version, website_url, icon). …View on GitHub ↗ -
v1.8.0 v1.8.0 — the "claims = contracts" release
Release notes excerpt
Every documented argument of the public API now does exactly what the docs say. Full details in the [CHANGELOG](https://github.com/TeleAI-UAGI/telemem/blob/main/CHANGELOG.md). Fixed - Character memory extraction: the per-character prompt received the parse_messages helper function instead of the parsed current dialogue, so character profiles were built from a function repr rather than the conversation. - add() without a user_id now stores memories in the shared "events" scope (matching add_batch()), so search() can actually retrieve them. …View on GitHub ↗