Local OpenAI LLM (Home Assistant Custom Integration)
Connect Home Assistant to any OpenAI-compatible LLM service — local or cloud — for Assist conversations, AI tasks, and retrieval-augmented generation.
- Source repo
- skye-harris/hass_local_openai_llm
- Stars
- ★ 302
- Last updated
- 2d ago
- License
- Apache-2.0
- Primary language
- Python
- FA score
- 66/100 · Some gaps
At a glance
- How it runs
- Works with
- Universal · cross-platformOpenAI API
- Cost
- Free software; you pay for model usage
- Setup effort
- Medium · a few setup steps
- You'll need
- Typical use
- Homelab users running llama.cpp, vLLM, or Ollama locally who want Home Assistant's Assist to answer voice commands with a local model instead of a cloud service.
- Not a fit if
- Users who don't run Home Assistant: it's an HA custom integration, not standalone
- Anyone wanting turnkey behavior: RAG and date/time injection are experimental and need per-model testing
- Teams unwilling to self-host Weaviate: RAG is local-only and Weaviate Cloud is unsupported
- Source review
- 66/100 · Some gaps
What does this agent do, and when should you use it?
Local OpenAI LLM is a Home Assistant custom integration by skye-harris, forked from the official OpenRouter integration, that connects to any OpenAI-compatible LLM service including llama.cpp, vLLM, Ollama, LM Studio, LocalAI, OpenRouter, DeepSeek, Google Gemini, and Scaleway. It provides Assist Conversation Agents (with streamed responses, TTS streaming, and parallel tool calling) and AI Task entities supporting text, image inputs, and image generation. The integration adds experimental Weaviate-backed Retrieval Augmented Generation, date/time context injection, <think> tag stripping, emoji stripping, history trimming, and temperature controls. Installation is via HACS, and all configuration happens in the Home Assistant UI, with the API key optional for local servers that don't require one. It suits anyone already running Home Assistant who wants to drive their home assistant with self-hosted or diverse inference backends.
The integration registers a 'Local OpenAI LLM' integration in Home Assistant and sends streaming chat completion requests to a user-configured OpenAI-compatible server URL (typically ending in /v1). It exposes two entity types: Conversation Agents for Assist (supporting TTS streaming, tool calling with parallel calls, history trimming, temperature control, and an 'always continue conversation' voice mode) and AI Task entities for text and image generation (images use the Images API spec, tested with StableDiffusion.cpp). Server-type-specific options include llama.cpp settings (enable_thinking, Slot ID pinning, use-loaded-model routing, sampling parameters like Top-P/Min-P/Top-K/Repeat Penalty/Presence Penalty), vLLM thinking token budget, DeepSeek Cloud reasoning effort, and LocalAI chat template args via the metadata field. It also supports chat_template_kwargs, custom HTTP headers, extra request body parameters, automatic <think> tag stripping, and experimental RAG: user messages are first queried against a Weaviate vector database (default object class Homeassistant, max 2 results, score threshold 0.9, hybrid alpha 0.5) with matched content injected into the current conversation, plus a local_openai.add_to_weaviate service to add data.
- Homelab users running llama.cpp, vLLM, or Ollama locally who want Home Assistant's Assist to answer voice commands with a local model instead of a cloud service.
- Users of local inference servers without authentication — the integration lets you leave the API key blank during setup.
- Users who want their assistant grounded in private knowledge and are willing to self-host Weaviate, feeding data in via the add_to_weaviate service.
- Users of reasoning models (e.g., Qwen3) who want to control thinking behavior — toggling enable_thinking or capping the vLLM thinking token budget.
- Users who want AI Task entities to generate or analyze images, provided their image server supports the Images API (e.g., StableDiffusion.cpp).
- Advanced users who need per-backend tuning of sampling parameters, chat template arguments, or custom HTTP headers.
How do you install or deploy this agent?
Install via HACS (recommended): 1) Make sure HACS is installed. 2) Use the README's HACS repository button, or add https://github.com/skye-harris/hass_local_openai_llm as a custom repository of type Integration in HACS. 3) Install 'Local OpenAI LLM' from HACS. 4) Restart Home Assistant. Manual install: copy the local_openai folder from the latest release into the custom_components folder of your config directory, then restart Home Assistant.
How do you use this agent?
1) Go to Settings → Devices & Services, click Add Integration, search for 'Local OpenAI LLM', and follow the wizard. 2) The Server URL must be a fully qualified URL to an OpenAI-compatible API (typically ending in /v1); the API key can be left blank. 3) Use at least a 10k context size on your inference server and limit history length and exposed entities to avoid context overflow — context size must be configured on the inference server directly, as OpenAI-compatible APIs don't expose it. 4) Enable tool calling in your inference engine (e.g., vLLM tool calling, llama.cpp function calling); parallel tool calling additionally requires model and server support. 5) Use the configured Conversation Agent in Assist, or create AI Task entities for text/image generation. 6) Optional: deploy the docker-compose.yml in the repo's weaviate directory (includes a management WebApp on port 9090), then fill in the Weaviate address and API key ('homeassistant' for the supplied compose file) in the LLM Server entity to enable RAG.
What are this agent's strengths and limitations?
- Works with many OpenAI-compatible backends (llama.cpp, vLLM, Ollama, LM Studio, LocalAI, OpenRouter, DeepSeek, Gemini, Scaleway, etc.) with dedicated tuning for llama.cpp, vLLM, LocalAI, and DeepSeek.
- Compared to the official OpenRouter integration it forked from, it adds streamed responses, TTS streaming, optional API key, image inputs/generation, parallel tool calling, <think> stripping, and history trimming.
- Experimental Weaviate RAG is built in, with a bundled docker-compose, a purpose-built WebApp manager, and a local_openai.add_to_weaviate service action.
- Fine-grained llama.cpp control (Slot ID prompt-cache pinning, use-loaded-model routing, Top-P/Min-P/Top-K/penalty sampling) suits latency-sensitive local deployments.
- Bound to the Home Assistant ecosystem: you must already run Home Assistant (HACS recommended); it cannot be used as a standalone agent.
- Context size cannot be set via the OpenAI-compatible API — it must be configured on the inference server — and Assist tooling/entity definitions need at least ~10k context, which smaller models may overflow.
- Several features are explicitly experimental (RAG, date/time injection role) and behave differently across models, requiring manual testing.
- RAG requires self-hosting and maintaining Weaviate (local self-hosted only; Weaviate Cloud is unsupported), adding operational overhead.
- Parallel tool calling, thinking controls, and image generation depend on model and server version support, so capability varies by backend.
How does this agent compare with similar options?
This integration is an explicit fork of Home Assistant's official OpenRouter integration (by @joostlek). Unlike the original, it removes the OpenRouter lock-in, supports any OpenAI-compatible server including local deployments, and adds streaming responses, TTS streaming, optional API key, image input/generation, parallel tool calling, and RAG. If you only use OpenRouter's cloud, the official integration suffices; for local backends or the added features, this fork is the better fit.
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| Local OpenAI LLM (Home Assistant Custom Integration) This agent | 66 · Some gaps | Self-hosted serviceFree + model costs | ★ 302 | 2d ago | Python | OpenAI API |
| GOD Agent Society Control Room | 60 · Some gaps | CLIFree + model costs | ★ 1.2k | 4d ago | Python | OpenAI API |
| Archestra | 71 · Some gaps | Self-hosted serviceFreemium | ★ 4.4k | today | TypeScript | Codex · Claude Code · OpenAI API · Claude API |
| AgenticOS Team Agent Workspace | 68 · Some gaps | Self-hosted serviceFree + model costs | ★ 118 | today | Python | OpenAI API · Claude API |
How does FollowAgents rate this agent?
Why each dimension lost points
Evidence shows a standard Home Assistant custom integration whose privilege scope follows the HA Assist/LLM APIs; no malicious behavior found. Deductions: API key and conversation-data handling are described only at README level; Weaviate writes user messages to a vector store with no concrete safeguards discussed for potentially sensitive home context; no CHANGELOG or signed releases, rollback relies only on normal HACS mechanics; no evidence of user-confirmation mechanisms. Source attribution is clear (fork of the OpenRouter integration with original author credited).
pyproject configures strict mypy, pytest coverage, and socket-disabled tests, supporting self-consistency; README clearly states external prerequisites (context size, tool calling support, parallel tool calling). Deductions: no evidence of error handling or user-facing failure messages in the provided files; tests cover only config_flow helpers, not the core request path.
Audience (HA users of self-hosted or multi-provider OpenAI-compatible services) and scenarios are well described, with 10+ server types individually documented. Deductions: many advanced features are flagged experimental or server-dependent, capability boundaries partly defer to user trial-and-error (e.g., date/time injection role), and trigger precision for Assist tool calls is only indirectly described.
Documentation is well organized: HACS/manual install, configuration notes, per-server options, experimental features, acknowledgements; limitations are thorough; Apache-2.0 LICENSE is complete. Deductions: no CHANGELOG file and versioning only points to GitHub releases; no explicit stability or upgrade-path commitments; maintenance is concentrated in a single author, creating sustainability uncertainty.
README is highly usable with concrete configuration guidance; marginal value over the fork origin is clear (streaming, TTS, image input/generation, RAG, multi-server support). Deductions: cost/benefit is unquantified; RAG requires running Weaviate plus a bundled management WebApp, increasing deployment cost while its benefit rests on the author's own model test list without independent verification.
README change lists and per-server configuration partially corroborate the config_flow tests (request body denylist, Weaviate class creation, temperature fields); external links to vLLM/llama.cpp/LocalAI docs allow cross-checking. Deductions: RAG testing conclusions are author-asserted only, and the core runtime code is not present in the evidence, so most behavioral claims cannot be verified by static review; facts and inferences are largely kept separate (experimental/dependency caveats are labeled).
- This static review is based only on README, LICENSE, build configs, and config_flow tests; the core runtime code is not in evidence and no behavioral claim has been execution-verified.
- Publisher identity is unverified; verify repository and release consistency before installing.
- Conversation content and potentially sensitive home context are sent to the configured LLM server, and RAG mode additionally writes user messages to a Weaviate vector store; tighten exposed entities and history length accordingly.
- RAG is experimental and its effectiveness rests only on the author's own testing; validate in low-risk scenarios first.
- No CHANGELOG exists; upgrade cautiously via HACS and prepare a rollback path.
Recent releases
Latest entries from the project's GitHub Releases. Notes are quoted from upstream as written, not rewritten or tested by FollowAgents.
-
Release notes excerpt
What's Changed * Emit an event with metadata when inference is run, that can be used to trigger automations by @skye-harris in https://github.com/skye-harris/hass_local_openai_llm/pull/131 * Resolve deprecation alert in HA 2026.10, bumped min HA version in hacs.json to 2026.10 Full Changelog: https://github.com/skye-harris/hass_local_openai_llm/compare/1.12.1...1.13.0View on GitHub ↗ -
Release notes excerpt
What's Changed * Changes/fixes to message history setting by @skye-harris in https://github.com/skye-harris/hass_local_openai_llm/pull/130 Full Changelog: https://github.com/skye-harris/hass_local_openai_llm/compare/1.12.0...1.12.1View on GitHub ↗ -
Release notes excerpt
What's Changed * Allow model temperature to be unset and use server default by @geryduyck in https://github.com/skye-harris/hass_local_openai_llm/pull/123 * Add use loaded model option for llama.cpp, allowing agents to use another model if their configured model is not currently loaded by @skye-harris in https://github.com/skye-harris/hass_local_openai_llm/pull/125 * HA 2026.9 Compatibility: use probatio instead of voluptous for schema conversion by @skye-harris in https://github.com/skye-harris/hass_local_openai_llm/pull/122 New Contributors …View on GitHub ↗