Sapphire Blue AI
A self-hosted, customizable voice agent framework with long-term memory, a plugin ecosystem and real control over real systems.
- Source repo
- ddxfish/sapphire
- Stars
- ★ 328
- Last updated
- today
- License
- AGPL-3.0
- Primary language
- Python
- FA score
- 53/100 · Major gaps
At a glance
- How it runs
- Works with
- Universal · cross-platformOpenAI API · Claude APIClaude.ai (Partial support)
- Cost
- Free software; you pay for model usage
- Setup effort
- High · needs real infrastructure
- You'll need
- Typical use
- Self-hosters who want a voice assistant that remembers them, wakes them in the morning, and drives Home Assistant lights and thermostats.
- Not a fit if
- Users who want a zero-config assistant with no LLM or tool setup
- Anyone uncomfortable with an AI running shell commands and sending email autonomously
- Mac users who refuse Docker — Mac is Docker-only
- Source review
- 53/100 · Major gaps
What does this agent do, and when should you use it?
Sapphire is a self-hosted agentic framework whose main surface is a local web UI at https://localhost:8073, with wake word, STT, TTS and hands-free conversation mode. It wraps an LLM with assembled prompts, Personas, Toolsets and semantic vector memory to give the assistant a persistent personality, and the AI can write, validate and install its own tools at runtime. Plugins and daemons connect it to Discord, Telegram, email, Twilio phone calls, Google Calendar, Home Assistant, SSH, Bitcoin and MCP servers, while heartbeats and scheduled tasks make it run autonomously. Installation is manual conda + pip or Docker, the latter without wake word but isolated from the host. Data lives in the local user/ directory, and cloud LLMs are optional alongside local Ollama/LM Studio endpoints.
Sapphire launches a Python web service (port 8073) and calls the configured LLM provider — Claude, GPT, Gemini, Fireworks or any OpenAI/Anthropic-compatible endpoint — with prompts assembled from swappable sections. It detects the wake word with openWakeWord, transcribes speech with Faster Whisper, and synthesizes voice with Kokoro (or ElevenLabs), supporting streaming speech and barge-in during conversation mode. The memory layer performs semantic vector search over 100K+ labeled entries; the Mind Palace librarian sorts memories nightly, and the Goals system keeps a timestamped progress journal. The AI can write and install new tools at runtime via Tool Maker, read/edit/search/run commands through the Code Harness, and execute remote commands via the SSH plugin with safety blacklists. The plugin system supplies hooks, tools, voice commands, providers, daemons, apps and themes installable from GitHub in one click. Daemons and webhooks listen to email, Discord and Telegram events and auto-respond, and heartbeat mode handles morning greetings, alarms and random check-ins.
- Self-hosters who want a voice assistant that remembers them, wakes them in the morning, and drives Home Assistant lights and thermostats.
- Developers who want an extensible agent platform and add Discord/Telegram auto-reply, email handling and scheduled tasks through plugins.
- Researchers who want a portable voice work companion via wireless mic and speaker, dictating questions while the assistant spawns background agents to investigate.
- Privacy-focused users enabling the Privacy Vault so chats are encrypted, models are forced to private ones, and traffic routes through a SOCKS proxy.
- Creators building RPG-style AI stories and games in the Game Room / Story Engine with custom personas and voices.
- Operators using the SSH plugin and Code Harness to read, edit and run remote commands under a controlled blacklist.
How do you install or deploy this agent?
The easiest path is the Sapphire Launcher GUI (download the .exe or .AppImage), which installs git, conda and Sapphire for you. Manual install needs conda with Python 3.11 and git:
conda create -n sapphire python=3.11 -y
conda activate sapphire
git clone https://github.com/ddxfish/sapphire.git
cd sapphire
pip install -r requirements.txt
python main.pyOn Linux also install system packages and accept the conda Terms of Service (required since July 2025):
sudo apt-get install libportaudio2 git
conda tos accept --override-channels --channel https://repo.anaconda.com/pkgs/main
conda tos accept --override-channels --channel https://repo.anaconda.com/pkgs/r
conda tos accept --override-channels --channel https://repo.anaconda.com/pkgs/msys2Docker alternative (web UI only, no wake word):
mkdir ~/sapphire && cd ~/sapphire
curl -fsSL https://raw.githubusercontent.com/ddxfish/sapphire/main/docker-compose.yml -o docker-compose.yml
docker compose up -dRequirements: Ubuntu 22.04+ or Windows 11+ (Mac is Docker-only), 16GB+ RAM with TTS/STT, and an Nvidia GPU is recommended.
How do you use this agent?
Open https://localhost:8073; the setup wizard walks you through LLM configuration on first run (cloud API keys, or local Ollama/LM Studio endpoints). Then pick a Provider in the Chat sidebar, choose or create a Toolset, and create a Persona that bundles prompt, toolset, voice and memory scopes. Say "hey sapphire" to start a voice conversation.
Updating from the repository:
git pull
pip install -r requirements.txtUse Settings → Help to search the docs, Schedule → Events to install email/Discord/Telegram daemons, the plugin store for one-click community plugins, and Import/Export to share personas, prompts and toolsets as JSON. All data lives under user/ and can be backed up and restored from inside the app.
What are this agent's strengths and limitations?
- Broad plugin surface: hooks, tools, voice commands, providers, daemons, apps and themes, plus a Tool Maker that lets the AI write and install its own tools at runtime.
- Local-first privacy design: local runtime by default, Privacy Vault encrypted chats with forced private models, and app-wide SOCKS proxy routing.
- Deep memory and autonomy: semantic vector search over 100K+ entries, nightly Mind Palace librarian, Goals journal, heartbeats and scheduled tasks.
- Complete multimodal interaction: wake word, STT, TTS, streaming speech with barge-in, 3D GLB avatar, and the same chat opened from a phone browser.
- Flexible deployment: manual conda install, Docker (amd64/arm64) and a GUI Launcher.
- High safety bar: the AI can run shell commands, send email, control smart home devices and act autonomously, so users must carefully restrict toolsets.
- Heavy setup: conda, Python 3.11, 16GB+ RAM, Nvidia GPU recommended for local TTS/STT; Mac is Docker-only and Docker mode has no wake word.
- Solo-developer project: the README states it is maintained by one developer and core PRs are accepted only for single bugs, so ecosystem maturity depends on the community.
- Plugin trust is on the user: plugins install from GitHub in one click with only trust-level hints in the UI, with no evidence of enforced review.
- AGPL-3.0 licensing: modified deployments offered as a service must publish source changes, which is unfriendly to closed-source commercial use.
How does this agent compare with similar options?
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| Sapphire Blue AI This agent | 53 · Major gaps | Web appFree + model costs | ★ 328 | today | Python | OpenAI API · Claude API |
| Iva Personal Assistant | 85 · Good | Chat botFree + model costs | ★ 285 | 2d ago | TypeScript | ChatGPT · Codex |
| CyberVerse Digital Human Assistant | 33 · Major gaps | Self-hosted serviceFree + model costs | ★ 1.8k | 15d ago | Python | — |
| Ankita Desktop AI Workspace | 54 · Major gaps | Desktop appFree + model costs | ★ 14 | 4d ago | JavaScript | ChatGPT · OpenAI API |
How does FollowAgents rate this agent?
Why each dimension lost points
README explicitly warns Sapphire can run shell commands, send email, control smart home, and write its own tools, fully autonomous under schedules; docs say dangerous integrations require explicit opt-in but no per-action confirmation mechanism is shown, so least_privilege and user_confirmation score 1. PRIVACY.md details vault, private chats, local-only model/tool enforcement, encryption, and backup gaps, earning data_flow_transparency and sensitive_data_handling 2; self-disclosed gaps include plaintext chat names and old plaintext backups. requirements.txt has upper bounds and comments but no lockfile, hashes, or vulnerability scanning, so dependency_security is 1. external_effects scores 1 because real side effects (shell, email, smart home, Bitcoin) are only warned about in docs. rollback scores 2 for backup/restore, chat repair, and vault quarantine. source_attribution scores 1: README acknowledges openWakeWord, Faster Whisper, and Kokoro but not all dependencies or plugin provenance.
Docs are internally consistent on vault, private chats, backups, updates, and dependency caps, so self_consistency is 2. dependency_availability is 2: conda/pip/Docker paths and version caps are documented, but no lockfile or offline mirror is provided. TROUBLESHOOTING.md gives concrete errors and fixes for startup, audio, LLM, plugins, vault, and database, so failure_messages is 2.
README and INSTALLATION.md cover local/cloud LLM, Docker, voice, plugins, and multiple platforms, so audience_and_scenarios is 2. capability_boundaries is 2: private-mode refusal rules for network tools and cloud STT/TTS are documented. trigger_precision is 1: wake word, scheduled tasks, and daemons are documented, but docs admit sensitivity, VAD, and cron timezone pitfalls and no precise trigger contract is given. environment_fit is 2: Ubuntu/Windows/Docker/Mac are covered, but 16GB+ RAM, conda, and optional GPU are required.
Documentation has a clear table of contents and cross-links, so information_architecture is 2. INSTALLATION.md covers Launcher, manual, Docker, systemd, update, uninstall, and password reset, so install_notes is 3. naming_stability is 1: 1.x→2.0 migration, branch switching, and plugin signing changes exist with no stable API/naming commitment. examples_and_faq is 2: Quick Start, Mastery Guide, and Troubleshooting exist, but full FAQ and sample outputs are missing. known_limitations is 2: PRIVACY.md and README warnings are fairly thorough. license is 3: full AGPL-3.0 text is present. versioning_changelog is 1: only a link to docs/CHANGELOG.md is shown, not its content. maintenance_responsibility is 1: solo developer, unverified publisher, no governance or support commitment.
output_usability is 2: Web UI, voice, plugins, and import/export exist, but no concrete output samples are shown. marginal_value is 2: a self-hosted agent framework is differentiated, but the feature surface is broad and complex. cost_benefit is 1: 16GB+ RAM, conda, optional GPU, and many service integrations make cost high and benefit configuration-dependent.
claim_traceability is 1: most features are asserted in README without corresponding code or tests in the evidence. cross_source_corroboration is 1: only README, INSTALLATION, TROUBLESHOOTING, and PRIVACY partially corroborate install and privacy behavior. fact_inference_separation is 1: docs mix marketing language with factual claims without clear separation.
- Sapphire can run shell commands, send email, control smart home devices, and write its own tools; under schedules it runs unsupervised, so restrict toolsets and egress before deployment.
- Private mode depends on the vault; without a vault, chats are neither encrypted nor hidden, chat names are plaintext at rest, and old backups may retain plaintext history.
- Dependencies have upper bounds only, with no lockfile or hashes; plugins install from GitHub in one click and involve signing, so supply-chain risk must be assessed independently.
- Publisher is unverified and maintenance is by a solo developer; audit code and update paths before production use.