MineContext: Proactive Context-Aware AI Partner
Capture your digital context, proactively surface insights, todos, and summaries to bring clarity and efficiency to your workflow.
Evidence shows local-first storage and support for local models, but no fine-grained permission control or explicit least-privilege principle. User confirmation requires manual enabling of screen recording and API key input, but lacks confirmation for sensitive operations like data deletion. Data flow transparency is partially described in README but lacks details on API key and screenshot handling. Sensitive data handling mentions local storage but no encryption. Dependency security lacks audit or vulnerability scanning info. External effects include screen capture and external API calls but scope not fully clarified. Rollback mechanisms absent. Source attribution for third-party code not specified.
Self-consistency is mostly aligned between README and pyproject.toml, but version numbers differ (README mentions v0.1.8, pyproject.toml 0.1.0). Dependency availability lists dependencies but lacks lock files or version ranges. Failure messages documentation absent.
Audience and scenarios clearly defined in README. Capability boundaries listed with priorities and completion status, but limitations of incomplete features not detailed. Trigger precision described for automated tasks and screenshot intervals, but detailed conditions missing. Environment fit supports multiple platforms but OS-specific differences not explained.
Information architecture is clear with TOC and architecture sections. Install notes are detailed. Naming stability is consistent but version numbers inconsistent. Examples provided but FAQ missing. Known limitations not explicitly listed. License is Apache-2.0. Versioning and changelog absent. Maintenance responsibility not clearly stated.
Output usability described for summaries, todos, etc., but no example outputs. Marginal value compared with ChatGPT Pulse and Dayflow, highlighting advantages. Cost-benefit mentions using own API key but lacks detailed cost analysis.
Claim traceability weak; feature claims lack code references. Cross-source corroboration absent. Fact vs inference separation partially present but not explicit.
- Version inconsistency: README mentions v0.1.8, pyproject.toml 0.1.0, may cause confusion.
- Dependency security lacks audit or vulnerability scanning info; recommend checking dependency versions.
- Screen capture and API key handling flow not detailed; recommend reviewing data flow.
- No rollback or data recovery mechanism provided; use with caution.
What does this agent do, and when should you use it?
MineContext is an open-source, proactive context-aware AI partner from Volcano Engine. It captures your digital context via screen screenshots and content understanding, then uses a context engineering framework to proactively deliver high-value information such as daily/weekly summaries, to-dos, and activity records. Built with a local-first architecture, all data is stored locally by default (in ~/Library/Application Support/MineContext/Data), and it supports custom model services via the OpenAI API protocol, including fully local models for enhanced privacy. The project provides an Electron + React + TypeScript desktop app and a FastAPI backend, with modular components for context capture, processing, storage, and consumption. It integrates Vision-Language Models (VLM) and embedding models, and offers a backend debugging interface at http://localhost:1733 for monitoring token usage and adjusting automated tasks. Aimed at knowledge workers, content creators, lifelong learners, and project managers.
MineContext captures screen screenshots via the Electron app, sends them to a VLM (such as Doubao-Seed-1.6-flash) to understand the visual content, and uses an embedding model (such as Doubao-embedding-vision) to generate vector representations stored in a local vector database (ChromaDB) for similarity search. The backend, built with FastAPI, manages the pipeline through managers like CaptureManager, ProcessorManager, and ConsumptionManager, orchestrating capture, processing, and consumption. The frontend allows users to configure the capture region, view dashboards, chat with AI (Chat with AI), and receive automated summaries, todos, and activity records. It supports multiple model providers (OpenAI, Doubao, custom) via the OpenAI API protocol, and can interface with local models like those from LMStudio. The backend server runs on 127.0.0.1:8765 by default and can be started with opencontext start, with options to specify config file and port (e.g., --port 1733).
- A researcher analyzing vast amounts of information: MineContext automatically captures screen content, generates daily summaries and to-dos, improving information processing efficiency.
- A writer seeking inspiration: MineContext surfaces relevant context from previous work, assisting in content creation and reducing creative block.
- A student building a knowledge system: MineContext records learning materials and provides summaries, helping manage and connect information effectively.
- A project manager integrating multi-source information: MineContext captures meeting and document context, aiding alignment and decision-making.
- A privacy-conscious user wanting local AI: MineContext supports local models via the OpenAI API protocol, ensuring data never leaves the device.
What are this agent's strengths and limitations?
- Proactive delivery: automatically generates and pushes daily/weekly summaries, todos, and activity records to the homepage.
- Local-first privacy: data is stored locally by default, supports fully local models for zero data leakage.
- Open source and customizable: Apache 2.0 license allows full inspection and modification.
- Cost-effective: use your own API key, avoiding subscription fees and controlling spending.
- Multi-modal context: captures screen screenshots and integrates VLM and embeddings for rich context understanding.
- Requires screen recording permission: potential privacy concerns; only available on desktop.
- Setup complexity: requires API key configuration, model activation, and backend installation; may be time-consuming.
- Dependency on external model providers: while local models are supported, default recommendations are Volcano Engine services.
- Limited platform support: only macOS and Windows installers are provided; Linux support is not documented.
- Early-stage features: many planned context sources (e.g., file upload, meeting records) are not yet implemented, limiting applicability.
How do you install or deploy this agent?
- Download the latest release from GitHub Releases (e.g., v0.1.8) and install on macOS or Windows. 2. After launching, enter your API key; the backend environment will be installed automatically (may take ~2 minutes). 3. For screen capture, grant screen recording permission in system settings. 4. For backend debugging, visit http://localhost:1733. Alternatively, to build from source, install Node.js, pnpm, Python 3, and uv, then run:
git clone https://github.com/volcengine/MineContext.git,cd MineContext,uv sync,source .venv/bin/activate,./build.sh,cd frontend,pnpm install, andpnpm devfor development.
How do you use this agent?
- After launching, go to 'Screen Monitor' to enable screen sharing permission, then restart the app. 2. Configure the screen capture area in Settings, then click 'Start Recording' to begin capturing screenshots. 3. Wait for the context to be collected; MineContext will generate to-dos, prompts, summaries, and activities automatically. 4. Use 'Chat with AI' for proactive Q&A. 5. For backend, run
opencontext startwith optional flags:--config path/to/config.yaml,--port 1733. Configuration example is inconfig/config.yamlwhere you set VLM and embedding model providers, API keys, and models.
How does this agent compare with similar options?
Compared to ChatGPT Pulse, MineContext offers broader digital context capture via screenshots, local-first data storage, and a wider range of proactive insights (summaries, todos, activities), while being open source and more cost-effective. Compared to Dayflow, MineContext provides richer insights, context-aware Q&A, and content creation capabilities, whereas Dayflow focuses solely on activity logging.