PhyAgentOS — Session-Centered Runtime for Embodied Intelligence

What does this agent do, and when should you use it?

PhyAgentOS is a self-evolving embodied AI operating system built on agentic workflows, introducing Cognitive-Physical Decoupling and a Session-Centered Runtime. It separates planning (Track A: Planner, Critic, Memory) from execution (Track B: WatchdogSupervisor, SessionRunner, SkillRuntime, TargetSessionHandle). The system decouples via TargetAdapter, PolicyAdapter, and ActionBridge, so adding a new robot requires only a ~100-line Target Adapter with zero changes to the scheduling layer. It supports three target kinds (debug, simulation, real_robot) and dual skill runtimes (PolicySkillRuntime for policy loops, BuiltinSkillRuntime for built-in loops like benchmarks). The project defines a comprehensive protocol file matrix (AGENTS.md, SOUL.md, TARGETS.md, SESSIONS.md, etc.) to inject context and log state. Safety is enforced through four layers: Critic validation, Strict Preflight, Target-side SafetyGuard, and Operator Override. It is designed for both simulation (LIBERO, RoboCasa) and real-robot deployment (Franka, Go2, XLeRobot, AgileX PIPER). The repository includes a full TUI (paos tui) and CLI agent, with a 5-minute quick start.

PhyAgentOS provides paos CLI commands (paos onboard, paos agent, paos tui) to initialize a workspace and start interactive sessions. The execution pipeline is: WatchdogSupervisor schedules SessionRunner, which invokes a SkillRuntime (PolicySkillRuntime or BuiltinSkillRuntime) and communicates with the target via TargetSessionHandle. It loads protocol files (AGENTS.md, SOUL.md, USER.md, TOOLS.md, SKILLS.md, etc.) into the agent's system prompt and writes SESSIONS.md with session state. It composes AdapterPlan to wire target and policy adapters, eliminating combinatorial explosion. For benchmarking, it can run LIBERO benchmark via a remote TargetWS server (libero/server.py) and policy servers (e.g., native_openpi_server.py for OpenPI PI0.5). The system records state, actions, and perception results in Markdown + YAML files for full auditability. It also includes a SessionVerifier and failure recovery server.

1. Robotics researchers: plug in a new robot (e.g., Franka, Go2) by implementing a Target Adapter (~100 lines) without changing the scheduling layer.
2. Benchmark teams: run policy evaluations on LIBERO or RoboCasa simulation suites with episode replanning and verification.
3. Safety-critical deployments: leverage four-layer safety (Critic, Preflight, SafetyGuard, Operator Override) for real-robot tasks.
4. Multi-robot coordination: use Fleet Mode to manage shared and per-robot workspaces with priority-based scheduling.
5. Developers validating protocols: use the debug target (echo/mock/dry-run) for hardware-less pipeline testing.

How do you install or deploy this agent?

git clone https://github.com/PhyAgentOS/PhyAgentOS.git && cd PhyAgentOS
pip install -e .            # Python ≥ 3.11
pip install -e ".[dev]"     # Dev dependencies

How do you use this agent?

paos onboard              # Initialize workspace
paos agent                # CLI chat
paos tui                  # full-screen TUI

Optional: Connect runtime services. Example: evaluate PI0.5 on LIBERO spatial suite using three terminals:

1. Start LIBERO TargetWS: python PhyAgentOS/runtime/targets/remote/libero/server.py --host 0.0.0.0 --port 9002 --camera-height 256 --camera-width 256 --max-steps 300 --num-steps-wait 10 --control-mode relative --seed 7
2. Start policy server: python -m PhyAgentOS.runtime.policy.openpi.native_openpi_server --policy-config pi05_libero --checkpoint-dir gs://openpi-assets/checkpoints/pi05_libero --host 0.0.0.0 --port 8000
3. Run agent: paos agent --workspace ~/.PhyAgentOS/workspace -m "Evaluate PI0.5 on LIBERO suite libero_spatial..."

What are this agent's strengths and limitations?

How does this agent compare with similar options?

Key facts side by side with the most closely related agents.

How does FollowAgents rate this agent?

Recent releases

Latest entries from the project's GitHub Releases. Notes are quoted from upstream as written, not rewritten or tested by FollowAgents.

1. v1.1.0 2026-10-09
   Release notes excerpt

   Provider management, Skill Runtime and governed interactions Changes since v1.0.0 Added - Centralized provider management with configuration, testing, authentication, model discovery, saved model selection and per-session runtime overrides. - Direct OpenAI Responses API support, including tool calls, multimodal input, reasoning effort and integration with the Forge verifier. - Requesty, Cheaper Inference and API Route as OpenAI-compatible gateways. - directory_tar_gz Forge Node packages for executables with bundled runtime …

   View on GitHub ↗
2. v1.0.0 2026-09-05
   Release notes excerpt

   PhyAgentOS v1.0.0 Release artifacts - phyagentos_ai-1.0.0-py3-none-any.whl — Python wheel - phyagentos_ai-1.0.0.tar.gz — Python source distribution - SHA256SUMS — SHA-256 checksums for both packages Validation Built and verified by [GitHub Actions](https://github.com/PhyAgentOS/PhyAgentOS-core/actions/runs/33950485241) in an Ubuntu 20.04 container: - Python 3.11 and 3.12 tests, Ruff checks, and bytecode compilation - Node 20 bridge build - Package metadata validation with twine check - Clean-environment wheel installation - paos --version and paos --help smoke tests …

   View on GitHub ↗

FAQ