OM1 Robotics AI Runtime
A modular multimodal AI runtime that turns voice, vision and sensor input into real robot actions.
- Source repo
- OpenMind/OM1
- Stars
- ★ 3k
- Last updated
- 7d ago
- License
- MIT
- Primary language
- Go
- FA score
- 48/100 · Major gaps
At a glance
- How it runs
- Works with
- Universal · cross-platform
- Cost
- Free tier plus a paid hosted plan
- Setup effort
- Medium · a few setup steps
- You'll need
- Typical use
- Robotics teams wanting to add multimodal conversation and autonomous navigation to a humanoid or quadruped.
- Not a fit if
- Teams that need fully offline operation without the OpenMind cloud
- Windows-only developers unwilling to adapt the runtime themselves
- Hardware newcomers who want zero-configuration out-of-the-box use
- Source review
- 48/100 · Major gaps 1 safety controls not found
What does this agent do, and when should you use it?
OM1 is OpenMind's modular AI runtime for deploying multimodal AI agents across digital environments and physical robots, including humanoids, phone apps, quadrupeds such as the TurtleBot 4, and simulators like Gazebo and Isaac Sim. The core runtime has moved from Python to Go for lower latency, better concurrency, a smaller memory footprint on edge devices, and single-binary deployment with the Zenoh C library bundled alongside; the Python branch is deprecated and unmaintained. It ingests web data, social media, camera feeds and LIDAR, and executes actions through plugins that bridge ROS2, Zenoh, CycloneDDS, USB, serial and websockets. Pre-configured endpoints cover LLM and VLM providers such as OpenAI, xAI, DeepSeek, Anthropic, Meta, Gemini, NearAI and local Ollama, plus TTS and ASR, while json5 config files combine inputs and actions into agent behaviors. A Prometheus and Grafana stack is included for real-time pipeline metrics like LLM and ASR latency.
OM1 defines an agent as a json5 configuration (for example config/conversation.json5) that pairs inputs with actions. At runtime it reads camera, microphone, LIDAR, web and social inputs, calls the configured LLM/VLM endpoints, and dispatches actions through plugins under plugins/actions/. The Unitree Go2 connector registered as unitree_go2_autonomy/move, for instance, translates high-level goals into cmd_vel velocity commands published over Zenoh. On the hardware side it speaks to a robot's high-level SDK (HAL) over ROS2, CycloneDDS, Zenoh, USB, serial or websockets; BrainPack adds navigation, SLAM, auto charging and face detection/anonymization. The agent launches from the CLI with a -config flag, and a Prometheus/Grafana monitoring stack can be started with docker-compose.
- Robotics teams wanting to add multimodal conversation and autonomous navigation to a humanoid or quadruped.
- Researchers prototyping Unitree Go2 or G1 behaviors in Gazebo or NVIDIA Isaac Sim before touching hardware.
- Hardware vendors that already expose a high-level action-command HAL and need a runtime to translate LLM output into motion.
- Developers building a webcam-and-microphone conversation demo to exercise the input–LLM–action pipeline.
- Engineers using Prometheus and Grafana to watch LLM and ASR latency and diagnose real-time bottlenecks.
- Teams standardizing TurtleBot 4, phone apps and other form factors on one agent configuration.
How do you install or deploy this agent?
Install system dependencies first: on macOS run brew install portaudio ffmpeg; on Linux run the apt command below. The fastest path is to download a pre-built binary for linux-amd64, linux-arm64, darwin-arm64 or darwin-amd64 from the Releases page — the archive already bundles config/, knowledge_base/ and libzenohc. To build from source you need Go 1.25.0+ and make, then run the build commands.
How do you use this agent?
Get an API key from the OpenMind Portal, export it as OM_API_KEY, then launch the conversation agent, which uses your webcam and microphone for voice interaction. Set LD_LIBRARY_PATH on Linux and DYLD_LIBRARY_PATH on macOS before running; on macOS clear the quarantine flag if Gatekeeper blocks the binary. Optionally start the monitoring stack and open the OM1 Latency Monitoring dashboard at http://localhost:3000 (default admin/admin).
What are this agent's strengths and limitations?
- The Go runtime offers low latency, efficient concurrency and a small edge-device footprint, shipped as a single binary.
- A plugin hardware layer spanning ROS2, Zenoh, CycloneDDS, USB, serial and websockets makes it adaptable to different robot HALs.
- Pre-configured endpoints for OpenAI, Anthropic, Gemini, DeepSeek, local Ollama and more, plus TTS/ASR, keep provider choice flexible.
- An included Prometheus + Grafana stack exposes LLM and ASR latency out of the box.
- The BrainPack line is documented as full-autonomy support for Unitree Go2, Deep Robotics M20 Pro, Unitree G1 and LimX Tron, covering navigation, SLAM, auto charging and face anonymization.
- The Go runtime covers the core pipeline but several Python-era capabilities are still under active development.
- The Python version is deprecated and no longer maintained, so existing Python projects face a migration or self-maintenance burden.
- The default quickstart depends on an OpenMind Portal API key and the OMCU billing system, with a free plan of 50 OMCU per month.
- You must supply a HAL that accepts high-level action commands (often requiring RL, simulation and custom VLA work); OM1 does not provide low-level locomotion control.
- Recommended platforms are Nvidia Thor, Jetson, Mac and Ubuntu — Windows and Raspberry Pi are described only as expected to run.
How does this agent compare with similar options?
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| OM1 Robotics AI Runtime This agent | 48 · Major gaps | CLIFreemium | ★ 3k | 7d ago | Go | — |
| RAI — Embodied AI Agent Framework for Robotics | 55 · Major gaps | FrameworkFree + model costs | ★ 603 | 8d ago | Python | — |
| Blades | 23 · Major gaps | FrameworkFree + model costs | ★ 816 | 5mo ago | Go | OpenAI API |
| Stately Agent | 80 · Good | Library / SDKFree + model costs | ★ 481 | today | TypeScript | — |
How does FollowAgents rate this agent?
Why each dimension lost points
This is a robot runtime that drives real hardware (Unitree Go2/G1, LimX, etc.) and calls cloud LLM/ASR/TTS services. The docs show a game controller takes precedence over AI physical actions (unitree_go2_quadruped_configurations.md), which is the only visible human-override mechanism; there is no confirmation, approval, or least-privilege design for LLM-generated actions. The MCP example executes `npx -y` over stdio and carries a GitHub PAT, a broad permission surface. On data flow, docs admit data goes to at least three cloud LLMs but do not disclose which sensor data goes where. Sensitive-data handling is only a one-line 'Face Detection and Anonymization' claim with no implementation evidence. Dependency security is limited to pinned versions in go.mod with no audit/vulnerability-scanning evidence. Rollback is entirely absent (no action undo, no safe-stop). Source attribution has MIT and a copyright line, but publisher identity is unverified. Deductions: least_privilege 1 for arbitrary MCP command execution and unsandboxed hardware actions; user_confirmation 1 for controller-only override; data_flow_transparency 1 for only generic multi-LLM mention; sensitive_data_handling 1 for a single anonymization claim; dependency_security 1 for pinning without audit; external_effects 1 for high-impact physical actions with thin safeguards; rollback 0 for no evidence; source_attribution 2 for clear LICENSE and copyright.
Config docs are consistent on single/multi-mode, version checking (v1.1.0, major/minor mismatch behavior), and connector resolution (name+connector), and they corroborate go.mod and the CI build flow, so self-consistency is 2. Dependency availability is 2: go.mod pins versions, CI caches zenoh-c and bundles libzenohc, and prebuilt binaries plus INSTALL.txt are provided. Failure messages are 1: there is a troubleshooting doc and version-mismatch logging, but the troubleshooting table is very generic ('check settings', 'restart') and the simulator troubleshooting doc still references Python 3.10/uv, inconsistent with the Go runtime.
Audience and scenarios cover developers, robot hardware integration, simulators (Gazebo/Isaac Sim), and cloud simulation, scoring 2. Capability boundaries are explicit: OM1 assumes a high-level hardware SDK, requires you to build your own HAL, the Python runtime is deprecated, and some Go capabilities are still in development, scoring 2. Trigger precision is 1: mode switching relies on keyword matching (trigger_keywords) with cooldowns, a coarse mechanism with no evidence of false-trigger safeguards. Environment fit is 2: recommended platforms are listed, 'should run' elsewhere is noted, and macOS/Linux install differences are documented.
Information architecture is 2: docs/SUMMARY.md, GitBook config, and a topic-organized tree. Install notes are 3: system deps, prebuilt binaries, source build, API key, Grafana, and troubleshooting are all detailed. Naming stability is 1: there is a Python-to-Go migration, a deprecated Python branch, and config version v1.1.0 coupled to runtime version, so naming and structure are still evolving. Examples and FAQ are 2: config examples, a developer cookbook, and simulator guides exist, but there is no consolidated FAQ. Known limitations are 1: only scattered mentions (Python deprecated, some capabilities in development, platforms 'should' work) with no systematic limitations list. License is 3: full MIT text. Versioning/changelog is 1: config version checking and CI-generated release notes exist, but there is no in-repo CHANGELOG. Maintenance responsibility is 1: CONTRIBUTING is referenced and CI exists, but publisher identity is unverified and there is no explicit maintainer/SLA statement.
Output usability is 2: runnable configs, prebuilt binaries, verification steps, and a monitoring stack make ordinary use feasible. Marginal value is 2: as a modular robot HAL/runtime integrating multiple LLMs, middleware, and simulators it adds genuine value. Cost-benefit is 1: it requires an OpenMind API key with OMCU billing, depends on cloud LLMs, and has a high hardware/simulation barrier with a limited free tier.
Claim traceability is 1: README makes many capability claims (full autonomy, SLAM, auto charging, face anonymization) that point to external docs and the BrainPack repo, with no corresponding implementation evidence in this repository. Cross-source corroboration is 1: README, config docs, go.mod, and CI corroborate each other on build and configuration, but the core autonomy claims cannot be cross-verified from the given files. Fact/inference separation is 1: docs generally distinguish 'supported/in development/should run', but some capability statements are absolute and unqualified.
- Not found in source: rollback or recovery pathBack up first, or work on a git branch or snapshot, so its changes can be undone.
- The runtime can directly drive physical robots (Unitree Go2/G1, etc.); LLM-generated actions lack confirmation or approval, relying only on game-controller override, so physical safety must be assessed before deployment.
- The MCP example executes `npx -y` over stdio and carries a GitHub PAT, a broad permission surface; review and scope MCP servers and credentials.
- README claims of full autonomy, SLAM, auto charging, and face anonymization point mainly to external docs and the BrainPack repo with no implementation evidence here; do not infer these are present.
- No evidence of rollback/undo for physical actions; design your own safe-stop and recovery procedures.
- Publisher identity is unverified; maintenance responsibility and update path are unclear.
- Simulator troubleshooting docs still reference Python 3.10/uv, inconsistent with the Go runtime, and may mislead troubleshooting.