CyberVerse Digital Human Assistant
A self-hosted, voice-first framework for realtime character agents with memory, RAG, and optional digital-human video.
- Source repo
- Lynpoint/CyberVerse
- Stars
- ★ 1.7k
- Last updated
- 13d ago
- License
- GPL-3.0
- Primary language
- Python
- FA score
- 33/100 · Major gaps
At a glance
- Works with
- Universal · cross-platform
- You'll need
- Typical use
- A developer building a self-hosted voice assistant can disable avatar inference with
inference.avatar.enabled: falseand operate CyberVerse as a realtime voice-only system. - Main limitation
- Base deployment requires Node, Go, Conda, Python, FFmpeg, and several system audio libraries, making the runtime setup relatively involved.
- Source review
- 33/100 · Major gaps 8 safety controls not found
What does this agent do, and when should you use it?
CyberVerse is a self-hosted realtime digital-human agent framework built around WebRTC and voice interaction. Its foreground PersonaAgent maintains conversation flow, handles interruptions and context switches, while background SubAgents take on longer work such as search, research, material organization, summarization, and HTML report generation. Character conversation history is persisted on local disk, and imported knowledge bases, documents, and biographical material are indexed for retrieval-augmented generation. Its plugin-oriented stack makes brain, voice, hearing, tools, memory, and face replaceable, with provider keys and endpoints configurable at `/settings`. It can run as a voice-only assistant or optionally deliver a realtime video persona through local FlashHead or LiveAct models and listed cloud digital-human services.
Runtime behavior is controlled through config/cyberverse.yaml; omni, LLM, embedding, TTS, and ASR definitions are discovered from infra/config/*_models/ and can be overridden under config/*_models/. The system handles realtime voice interaction, calls configured ASR, LLM/omni, and TTS services, saves and reloads character history, and retrieves indexed character material for responses; background SubAgents can perform search, research, organization, summarization, and HTML report generation asynchronously. With inference.avatar.enabled: true, the Python inference service can drive FlashHead or LiveAct according to config/avatar_models/; Baidu Xiling is embedded as an H5 iframe in the frontend and uses sendAudioData / AUDIO_STREAM_RENDER for audio delivery. Deployment starts the Python inference service, Go API server, and frontend with make inference, make server, and make frontend, and exposes health checking at GET /api/v1/health.
- A developer building a self-hosted voice assistant can disable avatar inference with
inference.avatar.enabled: falseand operate CyberVerse as a realtime voice-only system. - A character-driven application team can import biographies, documents, or knowledge bases for each persona and use persisted history plus RAG to maintain continuity across sessions.
- A product team seeking video-call-style interaction can use FlashHead or LiveAct when it has the documented CUDA-capable GPU environment, weights, and model configuration.
- A research-oriented assistant can delegate search, material organization, summarization, or HTML report generation to a SubAgent while the user continues the foreground voice conversation.
- A deployer without a local avatar GPU can use the documented cloud digital-human paths, including Baidu Xiling, Xunfei Digital Human, or Vidu S1.
How do you install or deploy this agent?
Prerequisites are Node 18+, Go 1.25, Conda, Python 3.10+, FFmpeg, libopus-dev, libopusfile-dev, libsoxr-dev, and pkg-config.
git clone https://github.com/dsd2077/CyberVerse.git
cd CyberVerse
conda create -n cyberverse python=3.10
conda activate cyberverse
cp -r infra/config configPopulate config/env with at least one documented model credential, such as DASHSCOPE_API_KEY, or both DOUBAO_ACCESS_TOKEN and DOUBAO_APP_ID. Edit config/cyberverse.yaml for voice-only operation:
inference:
avatar:
enabled: falseThen install project and voice-agent dependencies:
make setup
pip install -e ".[all]"How do you use this agent?
Start the services in three terminals:
conda activate cyberverse
make inferencemake servermake frontendVerify the API:
curl -s http://localhost:8080/api/v1/healthOpen http://localhost:5173 in a browser. For local digital-human video, install the documented CUDA 12.8+ PyTorch environment and model weights, set inference.avatar.enabled to true, and configure model paths in config/avatar_models/flash_head.yaml or config/avatar_models/live_act.yaml.
What are this agent's strengths and limitations?
- The documented PersonaAgent/SubAgent split keeps long-running work asynchronous while the foreground voice conversation can continue.
- Per-character local history persistence and indexed imported material provide a concrete path to session continuity and persona-grounded RAG.
- A single deployment can switch between voice-only operation and optional digital-human video using local FlashHead or LiveAct models or listed cloud services.
- Provider and model combinations are configurable through built-in model directories, local overrides, and
/settings; the LiteLLM plugin supplies a unified path to many LLM providers.
- Base deployment requires Node, Go, Conda, Python, FFmpeg, and several system audio libraries, making the runtime setup relatively involved.
- Local digital-human video adds CUDA 12.8+, PyTorch 2.8, model-weight, and supported-GPU requirements; realtime performance varies with model, resolution, and available GPUs.
- Default voice functionality depends on user-configured realtime voice, omni, LLM, TTS, and ASR providers, and the documented runtime cost depends on those choices.
- Remote direct streaming depends on browser access to the embedded TURN server over
8443/TCP; cloud security groups, firewalls, or NAT restrictions can cause ICE failures or publish timeouts.
How does this agent compare with similar options?
Key facts side by side with the most closely related agents.
| Agent | Source review | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|
| CyberVerse Digital Human Assistant This agent | 33 · Major gaps | ★ 1.7k | 13d ago | Python | — |
| PyGPT Desktop Assistant | 49 · Major gaps | ★ 1.9k | 4d ago | Python | OpenAI API · Claude API |
| AgentCall join-meeting Skill | 46 · Major gaps | ★ 157 | 8d ago | Python | Codex · Claude Code |
| Octop Self-Hosted Assistant | 67 · Some gaps | ★ 4.8k | today | Python | Codex · Claude Code · OpenAI API |
How does FollowAgents rate this agent?
Why each dimension lost points
Evidence shows: README mentions API keys stored in config/env and Web UI /settings, but no details on least privilege, user confirmation, data flow transparency, or sensitive data handling. No evidence of dependency security audits or vulnerability scanning. External effects (e.g., network calls, GPU usage) not explicitly stated. No rollback mechanism. Source attribution only via acknowledgments and links in README, but publisher unverified. Therefore all trust criteria scored 0.
Evidence shows: README and pyproject.toml are largely consistent in description, but lack detailed error handling documentation. Dependencies are listed in pyproject.toml, but availability not verified. README provides some troubleshooting (e.g., RTP and ICE connection), but not comprehensive. Thus self-consistency, dependency availability, and failure messages each scored 1.
Evidence shows: README clearly identifies target audience (developers) and scenarios (voice-first AI agents, digital humans). Capability boundaries are explained through feature list and configuration options. Trigger precision (e.g., RTP calculation) has concrete examples but not precise. Environment fit (e.g., GPU requirements, cloud images) is documented. Therefore audience and scenarios, capability boundaries, environment fit scored 2, trigger precision scored 1.
Evidence shows: README provides clear information architecture (sections, features, quick start). Install notes are detailed (steps, dependencies). Naming stability not explicitly stated. Examples and FAQ exist (e.g., RTP troubleshooting). Known limitations partially mentioned (e.g., GPU requirements), but not comprehensive. License is GPL-3.0, but versioning and changelog missing. Maintenance responsibility indicated via community links and PR welcome, but not explicit. Thus information architecture, install notes, examples and FAQ, license scored 2, naming stability, known limitations, maintenance responsibility scored 1, versioning and changelog scored 0.
Evidence shows: Output usability is described via demos and feature descriptions, but no actual output examples. Marginal value is evident through unique features (digital humans, multi-agent). Cost-benefit is addressed via cloud images and optional GPU, but no cost estimates. Therefore output usability and marginal value scored 2, cost-benefit scored 1.
Evidence shows: Claims in README (e.g., performance numbers) lack verifiable tests or benchmarks. Cross-source corroboration is limited (e.g., external model links). Fact and inference not clearly separated. Therefore all criteria scored 1.
- Not found in source: least-privilege scopingGrant only what the task needs: a dedicated account or read-only token, scoped to specific directories and repos.
- Not found in source: confirmation before actingTurn on (or add) a confirmation step before it acts, and try it in a sandbox or test environment before real data.
- Not found in source: data-flow disclosureWatch which external services it contacts (proxy or firewall logs) and keep sensitive data out until you know where it goes.
- Not found in source: sensitive-data handlingUse dedicated, low-privilege, revocable API keys — never production credentials — and keep secrets out of logs.
- Not found in source: dependency securityPin versions and run a dependency audit (npm audit, pip-audit) before installing; prefer running it in a container.
- Not found in source: disclosed external effectsEstablish which external systems it writes to, sends to or changes, and verify with test accounts or repos before production.
- Not found in source: rollback or recovery pathBack up first, or work on a git branch or snapshot, so its changes can be undone.
- Not found in source: verifiable attributionInstall from the official repo or registry and check the publisher and URL to avoid look-alike packages.
- Publisher identity unverified; treat source with caution.
- API key management lacks security details; review configuration and permissions.
- Dependencies not security-audited; potential vulnerabilities.
- Performance claims (e.g., FPS) lack benchmarks; verify independently.
- Versioning and changelog missing; update tracking difficult.
FAQ
Does voice-only operation require a local avatar GPU?
inference.avatar.enabled: false runs CyberVerse as a voice-only assistant. Runtime cost then depends on the configured realtime voice, omni, LLM, TTS, and ASR providers.Which local video-avatar models are documented?
config/avatar_models/.How can I tell whether avatar inference is too slow for realtime playback?
elapsed / (frames / fps). An RTP above 1 means generation is slower than playback and can lead to lag or stutter.What should I check if a remote page loads but audio or video never connects?
streaming_mode: direct with the embedded TURN server, verify that the browser can reach the server on 8443/TCP. The documented causes include cloud security groups, firewalls, and NAT restrictions; SSH port forwarding is a documented workaround.