PhyAgentOS — Session-Centered Runtime for Embodied Intelligence
Cognitive-physical decoupling with a session-centered runtime: one codebase, any hardware, with multi-layer safety and full auditability.
Evidence shows safety layers (Critic validation, Preflight, SafetyGuard, Operator Override) but no implementation details or least-privilege evidence. User confirmation mechanism not explicit. Data flow transparency is good with Markdown+YAML audit logs, but sensitive data handling not specified. Dependency security lacks vulnerability scanning or lock files. External effects (e.g., real robot operations) have safety layers but unverified. Rollback not explicit. Source attribution has MIT license and contributors, but publisher unverified.
Self-consistency is good, README aligns with project structure, but no code-level verification. Dependency availability lacks lock files or mirrors. Failure messages not exemplified.
Audience clear (developers, users), scenarios cover debug, simulation, real robot. Capability boundaries via TARGETS.md and SKILLRUNTIME.md, but trigger precision not detailed. Environment fit has Python version and dependencies, but no cross-platform tests.
Information architecture clear with protocol file matrix and project structure. Install notes detailed. Naming stable with version numbers. Examples and FAQ have quick start and docs links. Known limitations not explicitly listed. License MIT. Versioning changelog table. Maintenance responsibility has contributing guide, but publisher unverified.
Output usability has TUI and CLI, but no actual output examples. Marginal value has innovations like cognitive-physical decoupling. Cost-benefit lacks performance or resource data.
Claims have tech report and website links, but no concrete evidence. Cross-source corroboration insufficient. Fact-inference separation not explicit.
- Publisher identity unverified; exercise caution with dependencies.
- Real robot operations involve safety risks; require rigorous testing and approval.
- Many dependencies; check supply chain security.
What does this agent do, and when should you use it?
PhyAgentOS is a self-evolving embodied AI operating system built on agentic workflows, introducing Cognitive-Physical Decoupling and a Session-Centered Runtime. It separates planning (Track A: Planner, Critic, Memory) from execution (Track B: WatchdogSupervisor, SessionRunner, SkillRuntime, TargetSessionHandle). The system decouples via TargetAdapter, PolicyAdapter, and ActionBridge, so adding a new robot requires only a ~100-line Target Adapter with zero changes to the scheduling layer. It supports three target kinds (debug, simulation, real_robot) and dual skill runtimes (PolicySkillRuntime for policy loops, BuiltinSkillRuntime for built-in loops like benchmarks). The project defines a comprehensive protocol file matrix (AGENTS.md, SOUL.md, TARGETS.md, SESSIONS.md, etc.) to inject context and log state. Safety is enforced through four layers: Critic validation, Strict Preflight, Target-side SafetyGuard, and Operator Override. It is designed for both simulation (LIBERO, RoboCasa) and real-robot deployment (Franka, Go2, XLeRobot, AgileX PIPER). The repository includes a full TUI (`paos tui`) and CLI agent, with a 5-minute quick start.
PhyAgentOS provides paos CLI commands (paos onboard, paos agent, paos tui) to initialize a workspace and start interactive sessions. The execution pipeline is: WatchdogSupervisor schedules SessionRunner, which invokes a SkillRuntime (PolicySkillRuntime or BuiltinSkillRuntime) and communicates with the target via TargetSessionHandle. It loads protocol files (AGENTS.md, SOUL.md, USER.md, TOOLS.md, SKILLS.md, etc.) into the agent's system prompt and writes SESSIONS.md with session state. It composes AdapterPlan to wire target and policy adapters, eliminating combinatorial explosion. For benchmarking, it can run LIBERO benchmark via a remote TargetWS server (libero/server.py) and policy servers (e.g., native_openpi_server.py for OpenPI PI0.5). The system records state, actions, and perception results in Markdown + YAML files for full auditability. It also includes a SessionVerifier and failure recovery server.
- Robotics researchers: plug in a new robot (e.g., Franka, Go2) by implementing a Target Adapter (~100 lines) without changing the scheduling layer.
- Benchmark teams: run policy evaluations on LIBERO or RoboCasa simulation suites with episode replanning and verification.
- Safety-critical deployments: leverage four-layer safety (Critic, Preflight, SafetyGuard, Operator Override) for real-robot tasks.
- Multi-robot coordination: use Fleet Mode to manage shared and per-robot workspaces with priority-based scheduling.
- Developers validating protocols: use the debug target (echo/mock/dry-run) for hardware-less pipeline testing.
What are this agent's strengths and limitations?
- Cognitive-physical decoupling: new hardware requires only a ~100-line Target Adapter, no changes to scheduler.
- Four-layer safety (Critic, Preflight, SafetyGuard, Operator Override) suitable for real-robot deployment.
- Full auditability: state, actions, and perception results logged in Markdown/YAML, traceable and reproducible.
- Built-in TUI (
paos tui) with tiling chat, live status/logs, and theme management.
- Project is early-stage (v0.1.x), API may change; watch version updates.
- Quick-start LIBERO example requires multiple services (conda envs, remote servers) with complex setup.
- Real-robot support (e.g., Franka, Go2) may require additional adaptation; detailed configuration docs are sparse.
- arXiv tech report link (2607.16636) appears placeholder and may not be available yet.
How do you install or deploy this agent?
git clone https://github.com/PhyAgentOS/PhyAgentOS.git && cd PhyAgentOS
pip install -e . # Python ≥ 3.11
pip install -e ".[dev]" # Dev dependenciesHow do you use this agent?
paos onboard # Initialize workspace
paos agent # CLI chat
paos tui # full-screen TUIOptional: Connect runtime services. Example: evaluate PI0.5 on LIBERO spatial suite using three terminals:
- Start LIBERO TargetWS:
python PhyAgentOS/runtime/targets/remote/libero/server.py --host 0.0.0.0 --port 9002 --camera-height 256 --camera-width 256 --max-steps 300 --num-steps-wait 10 --control-mode relative --seed 7 - Start policy server:
python -m PhyAgentOS.runtime.policy.openpi.native_openpi_server --policy-config pi05_libero --checkpoint-dir gs://openpi-assets/checkpoints/pi05_libero --host 0.0.0.0 --port 8000 - Run agent:
paos agent --workspace ~/.PhyAgentOS/workspace -m "Evaluate PI0.5 on LIBERO suite libero_spatial..."