OM1 Robotics AI Runtime

A modular multimodal AI runtime that turns voice, vision and sensor input into real robot actions.

Source repo
OpenMind/OM1
Stars
★ 3k
Last updated
7d ago
License
MIT
Primary language
Go

At a glance

How it runs
CLI
Works with
Universal · cross-platform
Cost
Free tier plus a paid hosted plan
Setup effort
Medium · a few setup steps
You'll need
Go 1.25.0+ (source builds)makeportaudioffmpegOpenMind API key (OM_API_KEY)Zenoh C library (libzenohc, bundled in release)Shell / CLINetwork accessLocal filesystem
Typical use
Robotics teams wanting to add multimodal conversation and autonomous navigation to a humanoid or quadruped.
Not a fit if
  • Teams that need fully offline operation without the OpenMind cloud
  • Windows-only developers unwilling to adapt the runtime themselves
  • Hardware newcomers who want zero-configuration out-of-the-box use

What does this agent do, and when should you use it?

OM1 is OpenMind's modular AI runtime for deploying multimodal AI agents across digital environments and physical robots, including humanoids, phone apps, quadrupeds such as the TurtleBot 4, and simulators like Gazebo and Isaac Sim. The core runtime has moved from Python to Go for lower latency, better concurrency, a smaller memory footprint on edge devices, and single-binary deployment with the Zenoh C library bundled alongside; the Python branch is deprecated and unmaintained. It ingests web data, social media, camera feeds and LIDAR, and executes actions through plugins that bridge ROS2, Zenoh, CycloneDDS, USB, serial and websockets. Pre-configured endpoints cover LLM and VLM providers such as OpenAI, xAI, DeepSeek, Anthropic, Meta, Gemini, NearAI and local Ollama, plus TTS and ASR, while json5 config files combine inputs and actions into agent behaviors. A Prometheus and Grafana stack is included for real-time pipeline metrics like LLM and ASR latency.

OM1 defines an agent as a json5 configuration (for example config/conversation.json5) that pairs inputs with actions. At runtime it reads camera, microphone, LIDAR, web and social inputs, calls the configured LLM/VLM endpoints, and dispatches actions through plugins under plugins/actions/. The Unitree Go2 connector registered as unitree_go2_autonomy/move, for instance, translates high-level goals into cmd_vel velocity commands published over Zenoh. On the hardware side it speaks to a robot's high-level SDK (HAL) over ROS2, CycloneDDS, Zenoh, USB, serial or websockets; BrainPack adds navigation, SLAM, auto charging and face detection/anonymization. The agent launches from the CLI with a -config flag, and a Prometheus/Grafana monitoring stack can be started with docker-compose.

  1. Robotics teams wanting to add multimodal conversation and autonomous navigation to a humanoid or quadruped.
  2. Researchers prototyping Unitree Go2 or G1 behaviors in Gazebo or NVIDIA Isaac Sim before touching hardware.
  3. Hardware vendors that already expose a high-level action-command HAL and need a runtime to translate LLM output into motion.
  4. Developers building a webcam-and-microphone conversation demo to exercise the input–LLM–action pipeline.
  5. Engineers using Prometheus and Grafana to watch LLM and ASR latency and diagnose real-time bottlenecks.
  6. Teams standardizing TurtleBot 4, phone apps and other form factors on one agent configuration.

How do you install or deploy this agent?

Install system dependencies first: on macOS run brew install portaudio ffmpeg; on Linux run the apt command below. The fastest path is to download a pre-built binary for linux-amd64, linux-arm64, darwin-arm64 or darwin-amd64 from the Releases page — the archive already bundles config/, knowledge_base/ and libzenohc. To build from source you need Go 1.25.0+ and make, then run the build commands.

How do you use this agent?

Get an API key from the OpenMind Portal, export it as OM_API_KEY, then launch the conversation agent, which uses your webcam and microphone for voice interaction. Set LD_LIBRARY_PATH on Linux and DYLD_LIBRARY_PATH on macOS before running; on macOS clear the quarantine flag if Gatekeeper blocks the binary. Optionally start the monitoring stack and open the OM1 Latency Monitoring dashboard at http://localhost:3000 (default admin/admin).

What are this agent's strengths and limitations?

Pros
  • The Go runtime offers low latency, efficient concurrency and a small edge-device footprint, shipped as a single binary.
  • A plugin hardware layer spanning ROS2, Zenoh, CycloneDDS, USB, serial and websockets makes it adaptable to different robot HALs.
  • Pre-configured endpoints for OpenAI, Anthropic, Gemini, DeepSeek, local Ollama and more, plus TTS/ASR, keep provider choice flexible.
  • An included Prometheus + Grafana stack exposes LLM and ASR latency out of the box.
  • The BrainPack line is documented as full-autonomy support for Unitree Go2, Deep Robotics M20 Pro, Unitree G1 and LimX Tron, covering navigation, SLAM, auto charging and face anonymization.
Limitations
  • The Go runtime covers the core pipeline but several Python-era capabilities are still under active development.
  • The Python version is deprecated and no longer maintained, so existing Python projects face a migration or self-maintenance burden.
  • The default quickstart depends on an OpenMind Portal API key and the OMCU billing system, with a free plan of 50 OMCU per month.
  • You must supply a HAL that accepts high-level action commands (often requiring RL, simulation and custom VLA work); OM1 does not provide low-level locomotion control.
  • Recommended platforms are Nvidia Thor, Jetson, Mac and Ubuntu — Windows and Raspberry Pi are described only as expected to run.

How does this agent compare with similar options?

Key facts side by side with the most closely related agents.

Agent Source review Form / cost Stars Updated Language Full support on
OM1 Robotics AI Runtime This agent 48 · Major gaps CLIFreemium ★ 3k 7d ago Go —
RAI — Embodied AI Agent Framework for Robotics 55 · Major gaps FrameworkFree + model costs ★ 603 8d ago Python —
Blades 23 · Major gaps FrameworkFree + model costs ★ 816 5mo ago Go OpenAI API
Stately Agent 80 · Good Library / SDKFree + model costs ★ 481 today TypeScript —

How does FollowAgents rate this agent?

FollowAgents source review · FARS-2.1
Major gaps
48/ 100 5-point scale 2.4 / 5
Trust 10/29
Reliability 8/14
Adaptability 10/18
Convention 10/18
Effectiveness 7/13
Verifiability 3/8
Why each dimension lost points
Trust10 / 29 · 1.7/5

This is a robot runtime that drives real hardware (Unitree Go2/G1, LimX, etc.) and calls cloud LLM/ASR/TTS services. The docs show a game controller takes precedence over AI physical actions (unitree_go2_quadruped_configurations.md), which is the only visible human-override mechanism; there is no confirmation, approval, or least-privilege design for LLM-generated actions. The MCP example executes `npx -y` over stdio and carries a GitHub PAT, a broad permission surface. On data flow, docs admit data goes to at least three cloud LLMs but do not disclose which sensor data goes where. Sensitive-data handling is only a one-line 'Face Detection and Anonymization' claim with no implementation evidence. Dependency security is limited to pinned versions in go.mod with no audit/vulnerability-scanning evidence. Rollback is entirely absent (no action undo, no safe-stop). Source attribution has MIT and a copyright line, but publisher identity is unverified. Deductions: least_privilege 1 for arbitrary MCP command execution and unsandboxed hardware actions; user_confirmation 1 for controller-only override; data_flow_transparency 1 for only generic multi-LLM mention; sensitive_data_handling 1 for a single anonymization claim; dependency_security 1 for pinning without audit; external_effects 1 for high-impact physical actions with thin safeguards; rollback 0 for no evidence; source_attribution 2 for clear LICENSE and copyright.

Reliability8 / 14 · 2.9/5

Config docs are consistent on single/multi-mode, version checking (v1.1.0, major/minor mismatch behavior), and connector resolution (name+connector), and they corroborate go.mod and the CI build flow, so self-consistency is 2. Dependency availability is 2: go.mod pins versions, CI caches zenoh-c and bundles libzenohc, and prebuilt binaries plus INSTALL.txt are provided. Failure messages are 1: there is a troubleshooting doc and version-mismatch logging, but the troubleshooting table is very generic ('check settings', 'restart') and the simulator troubleshooting doc still references Python 3.10/uv, inconsistent with the Go runtime.

Adaptability10 / 18 · 2.8/5

Audience and scenarios cover developers, robot hardware integration, simulators (Gazebo/Isaac Sim), and cloud simulation, scoring 2. Capability boundaries are explicit: OM1 assumes a high-level hardware SDK, requires you to build your own HAL, the Python runtime is deprecated, and some Go capabilities are still in development, scoring 2. Trigger precision is 1: mode switching relies on keyword matching (trigger_keywords) with cooldowns, a coarse mechanism with no evidence of false-trigger safeguards. Environment fit is 2: recommended platforms are listed, 'should run' elsewhere is noted, and macOS/Linux install differences are documented.

Convention10 / 18 · 2.8/5

Information architecture is 2: docs/SUMMARY.md, GitBook config, and a topic-organized tree. Install notes are 3: system deps, prebuilt binaries, source build, API key, Grafana, and troubleshooting are all detailed. Naming stability is 1: there is a Python-to-Go migration, a deprecated Python branch, and config version v1.1.0 coupled to runtime version, so naming and structure are still evolving. Examples and FAQ are 2: config examples, a developer cookbook, and simulator guides exist, but there is no consolidated FAQ. Known limitations are 1: only scattered mentions (Python deprecated, some capabilities in development, platforms 'should' work) with no systematic limitations list. License is 3: full MIT text. Versioning/changelog is 1: config version checking and CI-generated release notes exist, but there is no in-repo CHANGELOG. Maintenance responsibility is 1: CONTRIBUTING is referenced and CI exists, but publisher identity is unverified and there is no explicit maintainer/SLA statement.

Effectiveness7 / 13 · 2.7/5

Output usability is 2: runnable configs, prebuilt binaries, verification steps, and a monitoring stack make ordinary use feasible. Marginal value is 2: as a modular robot HAL/runtime integrating multiple LLMs, middleware, and simulators it adds genuine value. Cost-benefit is 1: it requires an OpenMind API key with OMCU billing, depends on cloud LLMs, and has a high hardware/simulation barrier with a limited free tier.

Verifiability3 / 8 · 1.9/5

Claim traceability is 1: README makes many capability claims (full autonomy, SLAM, auto charging, face anonymization) that point to external docs and the BrainPack repo, with no corresponding implementation evidence in this repository. Cross-source corroboration is 1: README, config docs, go.mod, and CI corroborate each other on build and configuration, but the core autonomy claims cannot be cross-verified from the given files. Fact/inference separation is 1: docs generally distinguish 'supported/in development/should run', but some capability statements are absolute and unqualified.

Risks and how to mitigate them
  • Not found in source: rollback or recovery pathBack up first, or work on a git branch or snapshot, so its changes can be undone.
  • The runtime can directly drive physical robots (Unitree Go2/G1, etc.); LLM-generated actions lack confirmation or approval, relying only on game-controller override, so physical safety must be assessed before deployment.
  • The MCP example executes `npx -y` over stdio and carries a GitHub PAT, a broad permission surface; review and scope MCP servers and credentials.
  • README claims of full autonomy, SLAM, auto charging, and face anonymization point mainly to external docs and the BrainPack repo with no implementation evidence here; do not infer these are present.
  • No evidence of rollback/undo for physical actions; design your own safe-stop and recovery procedures.
  • Publisher identity is unverified; maintenance responsibility and update path are unclear.
  • Simulator troubleshooting docs still reference Python 3.10/uv, inconsistent with the Go runtime, and may mislead troubleshooting.
Evidence confidence: Low Reviewed Oct 20, 2026 Reviewed revision 3589af1214a6
See the full review method →

FAQ

Is OM1 free to use?
The code is MIT-licensed and can be self-hosted at no license cost. The quickstart uses an OpenMind platform API key billed in OMCU, with 50 OMCU free per month and paid upgrades via the portal; using local models like Ollama can reduce external calls.
Can I still use the Python version?
It runs, but OpenMind marks it deprecated and unmaintained and recommends Go for new development. Some capabilities from the Python branch are still under development in the Go runtime.
Does OM1 control my robot directly?
OM1 assumes your robot provides a high-level HAL that accepts commands like backflip, run, move(0.37, 0, 0), reached over ROS2, Zenoh, CycloneDDS, USB, serial or websockets. Without a suitable HAL you must build one with reinforcement learning, simulation and custom VLAs.
Do I need a GPU?
Not strictly, but Nvidia Thor and Jetson AGX Orin are the recommended development platforms, and Isaac Sim requires an NVIDIA GPU with CUDA. Generic Linux and Mac hosts also work, depending on whether you use cloud or local model endpoints.
What if the metrics port is taken?
If port 9090 reports 'Address already in use', stop the conflicting process (for example another Prometheus) or free the port before starting the stack.
View on GitHub ↗ Install ↓

Compare agents like this one

The same FARS review applied across the shortlist this agent qualifies for.

Related agents