TeleMem: Long-Term and Multimodal Memory for Agentic AI

What does this agent do, and when should you use it?

TeleMem is an agent memory management layer that works as a high-performance drop-in replacement for Mem0 with one line of code (import telemem as mem0). Its pipeline runs character-aware summarization → LLM semantic clustering deduplication → buffered batch writes → dual FAISS + JSON storage → vector retrieval with reranking, and it automatically builds isolated memory profiles for every character in a dialogue. On the multimodal side, add_mm()/search_mm() implement a video → frame extraction → caption generation → vector database pipeline with ReAct-style multi-step video question answering. The project ships an MCP server (telemem-mcp) built on the official MCP Python SDK v2 that exposes eight memory tools for clients like Claude Desktop and Cursor, and can run fully locally (Qwen + Ollama + FAISS). On the ZH-4O Chinese long-dialogue benchmark it achieves 86.33% accuracy, about 19 points higher than Mem0.

TeleMem ingests dialogue messages via Memory.add(): it performs multi-perspective summarization (global event plus per-character views), vectorizes and retrieves similar memories, semantically merges them through an LLM when the write buffer flushes, and dual-writes to a FAISS index and JSON metadata. Memory.search() performs millisecond-level FAISS vector retrieval with reranking, similarity thresholds, and filters. add_mm() decodes a video into frames, generates clip captions with a VLM (e.g., Qwen3-Omni), and builds a vector database; search_mm() drives the MMCoreAgent through a THINK→ACTION→OBSERVATION loop using three tools — global_browse_tool, clip_search_tool, and frame_inspect_tool — for multi-step video QA. The telemem-mcp server supports stdio and Streamable HTTP transports and exposes eight tools: add_memory, search_memories, get_memories, get_memory, update_memory, delete_memory, delete_all_memories, and memory_history. It works with any OpenAI-compatible endpoint and ships configs for Ollama, DeepSeek, Moonshot (Kimi), and MiniMax, plus LangChain and LlamaIndex examples.

1. Role-play and companion AI developers who need isolated, per-character long-term memory profiles built automatically
2. Multi-NPC game or virtual-world builders: one add_batch() call gives each NPC a private memory profile plus a shared world-state
3. Teams building long-conversation assistants (customer service, co-pilots) that must stay factually consistent over hundreds of turns while cutting LLM token cost
4. Multimodal agent developers who need to store, retrieve, and reason over video content
5. Claude Desktop/Cursor users who want to attach long-term memory to local agents via the MCP server (uvx telemem, zero install)
6. Privacy-sensitive teams that want end-to-end local memory with Ollama + FAISS, with no data leaving the machine

What are this agent's strengths and limitations?

How do you install or deploy this agent?

pip install telemem            # core (text memory)
pip install "telemem[mcp]"     # + MCP server
pip install "telemem[video]"   # + video/multimodal pipeline
pip install "telemem[all]"     # everything

# Or dev environment (uv recommended)
uv sync --all-extras

# Credentials for cloud providers:

export OPENAI_API_KEY="your-openai-api-key"
# Fully local option (Ollama) needs no API key: config/config.ollama.yaml

How do you use this agent?

import telemem as mem0
memory = mem0.Memory()

messages = [

{"role": "user", "content": "Jordan, did you take the subway to work again today?"},
{"role": "assistant", "content": "Yes, James. The subway is much faster."}
]

memory.add(messages=messages, user_id="Jordan")
results = memory.search("What transportation did Jordan use?", user_id="Jordan")

for hit in results["results"]:

print(hit["memory"])

# Local Qwen + FAISS config:

# TELEMEM_CONFIG=config/config.yaml python examples/quickstart.py
# MCP server: telemem-mcp or uvx telemem (stdio default)
# Multimodal: memory.add_mm(video_path, output_dir) then memory.search_mm(question, output_dir)

How does this agent compare with similar options?

FAQ