CyberVerse Digital Human Assistant
A self-hosted, voice-first framework for realtime character agents with memory, RAG, and optional digital-human video.
- Source repo
- Lynpoint/CyberVerse
- Stars
- ★ 1.9k
- Last updated
- 18d ago
- License
- GPL-3.0
- Primary language
- Python
- FA score
- 49/100 · Major gaps
At a glance
- How it runs
- Works with
- Universal · cross-platform
- Cost
- Free software; you pay for model usage
- Setup effort
- High · needs real infrastructure
- You'll need
- Typical use
- A developer building a self-hosted voice assistant can disable avatar inference with
inference.avatar.enabled: falseand operate CyberVerse as a realtime voice-only system. - Not a fit if
- Users wanting a quick trial: needs Node, Go, Conda, Python, FFmpeg and audio libs
- Teams without a CUDA 12.8+ GPU that still want realtime digital-human video
- Source review
- 49/100 · Major gaps 1 safety controls not found
What does this agent do, and when should you use it?
CyberVerse is a self-hosted realtime digital-human agent framework built around WebRTC and voice interaction. Its foreground PersonaAgent maintains conversation flow, handles interruptions and context switches, while background SubAgents take on longer work such as search, research, material organization, summarization, and HTML report generation. Character conversation history is persisted on local disk, and imported knowledge bases, documents, and biographical material are indexed for retrieval-augmented generation. Its plugin-oriented stack makes brain, voice, hearing, tools, memory, and face replaceable, with provider keys and endpoints configurable at /settings. It can run as a voice-only assistant or optionally deliver a realtime video persona through local FlashHead or LiveAct models and listed cloud digital-human services.
Runtime behavior is controlled through config/cyberverse.yaml; omni, LLM, embedding, TTS, and ASR definitions are discovered from infra/config/*_models/ and can be overridden under config/*_models/. The system handles realtime voice interaction, calls configured ASR, LLM/omni, and TTS services, saves and reloads character history, and retrieves indexed character material for responses; background SubAgents can perform search, research, organization, summarization, and HTML report generation asynchronously. With inference.avatar.enabled: true, the Python inference service can drive FlashHead or LiveAct according to config/avatar_models/; Baidu Xiling is embedded as an H5 iframe in the frontend and uses sendAudioData / AUDIO_STREAM_RENDER for audio delivery. Deployment starts the Python inference service, Go API server, and frontend with make inference, make server, and make frontend, and exposes health checking at GET /api/v1/health.
- A developer building a self-hosted voice assistant can disable avatar inference with
inference.avatar.enabled: falseand operate CyberVerse as a realtime voice-only system. - A character-driven application team can import biographies, documents, or knowledge bases for each persona and use persisted history plus RAG to maintain continuity across sessions.
- A product team seeking video-call-style interaction can use FlashHead or LiveAct when it has the documented CUDA-capable GPU environment, weights, and model configuration.
- A research-oriented assistant can delegate search, material organization, summarization, or HTML report generation to a SubAgent while the user continues the foreground voice conversation.
- A deployer without a local avatar GPU can use the documented cloud digital-human paths, including Baidu Xiling, Xunfei Digital Human, or Vidu S1.
How do you install or deploy this agent?
Prerequisites are Node 18+, Go 1.25, Conda, Python 3.10+, FFmpeg, libopus-dev, libopusfile-dev, libsoxr-dev, and pkg-config.
git clone https://github.com/dsd2077/CyberVerse.git
cd CyberVerse
conda create -n cyberverse python=3.10
conda activate cyberverse
cp -r infra/config configPopulate config/env with at least one documented model credential, such as DASHSCOPE_API_KEY, or both DOUBAO_ACCESS_TOKEN and DOUBAO_APP_ID. Edit config/cyberverse.yaml for voice-only operation:
inference:
avatar:
enabled: falseThen install project and voice-agent dependencies:
make setup
pip install -e ".[all]"How do you use this agent?
Start the services in three terminals:
conda activate cyberverse
make inferencemake servermake frontendVerify the API:
curl -s http://localhost:8080/api/v1/healthOpen http://localhost:5173 in a browser. For local digital-human video, install the documented CUDA 12.8+ PyTorch environment and model weights, set inference.avatar.enabled to true, and configure model paths in config/avatar_models/flash_head.yaml or config/avatar_models/live_act.yaml.
What are this agent's strengths and limitations?
- The documented PersonaAgent/SubAgent split keeps long-running work asynchronous while the foreground voice conversation can continue.
- Per-character local history persistence and indexed imported material provide a concrete path to session continuity and persona-grounded RAG.
- A single deployment can switch between voice-only operation and optional digital-human video using local FlashHead or LiveAct models or listed cloud services.
- Provider and model combinations are configurable through built-in model directories, local overrides, and
/settings; the LiteLLM plugin supplies a unified path to many LLM providers.
- Base deployment requires Node, Go, Conda, Python, FFmpeg, and several system audio libraries, making the runtime setup relatively involved.
- Local digital-human video adds CUDA 12.8+, PyTorch 2.8, model-weight, and supported-GPU requirements; realtime performance varies with model, resolution, and available GPUs.
- Default voice functionality depends on user-configured realtime voice, omni, LLM, TTS, and ASR providers, and the documented runtime cost depends on those choices.
- Remote direct streaming depends on browser access to the embedded TURN server over
8443/TCP; cloud security groups, firewalls, or NAT restrictions can cause ICE failures or publish timeouts.
How does this agent compare with similar options?
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| CyberVerse Digital Human Assistant This agent | 49 · Major gaps | Self-hosted serviceFree + model costs | ★ 1.9k | 18d ago | Python | — |
| Sapphire Blue AI | 69 · Some gaps | Web appFree + model costs | ★ 330 | 1d ago | Python | OpenAI API · Claude API |
| PyGPT Desktop Assistant | 56 · Major gaps | Desktop appFree + model costs | ★ 2k | 1mo ago | Python | OpenAI API · Claude API |
| AgentCall join-meeting Skill | 51 · Major gaps | Agent plugin / skillFreemium | ★ 184 | 15d ago | Python | Codex · Claude Code |
How does FollowAgents rate this agent?
Why each dimension lost points
The README gives partial data-flow evidence: providers are configurable, API keys have documented locations, and character histories are said to persist on local disk. Least privilege, sensitive-data protections, confirmation before tool effects, and controls for external side effects are not established; rollback is not shown. It names some model and dependency sources and the license is identified as GPL-3.0, supporting a score of 2 for source attribution. Dependency security and maintenance responsibility remain unclear.
The README supplies setup steps, a health check, performance diagnostics, and troubleshooting guidance; the server code logs plugin registration and initialization failures. These support ordinary deployment and some failure visibility. The sampled files do not fully corroborate runtime and dependency requirements, and the code shows exception logging without a consistent user-facing recovery path, so scores remain low.
The materials target self-hosted operators and describe voice-only, GPU avatar, cloud avatar, and multiple model configurations, with concrete GPU, networking, and cost conditions. That supports ordinary scenario and environment fit. Capability boundaries and mode selection are only partly explained, while trigger conditions and interaction permission policies are not sufficiently described.
The README has substantial feature, setup, configuration, FAQ, troubleshooting, and limitation sections, and the GPL-3.0 text is supplied, justifying full marks for license. Setup is concrete but spans multiple runtimes, external weights, and optional dependencies, leaving some version and configuration choices to the operator. Naming, release history, maintenance responsibility, and update path are weakly documented, so those criteria are discounted.
The stated combination of real-time voice interaction, persona memory, RAG, tools, and background tasks has clear intended utility. The documentation also provides actionable setup, diagnostics, and performance adjustments. Deployment complexity, GPU needs, and external service costs are only partly characterized, so cost-benefit does not earn full marks.
The README gives concrete hardware figures, setup steps, RTP diagnostics, and dependency references; sampled source supports some claims about plugin initialization and failure logging, and documented claims are generally distinguishable from inference. The file sample provides limited cross-source support for product claims and cannot establish the full advertised capability, so scores are partial.
- Not found in source: rollback or recovery pathBack up first, or work on a git branch or snapshot, so its changes can be undone.
- Deployment involves multiple runtimes, optional GPU model weights, and external services; estimate hardware and service costs for the chosen mode, and verify credential storage and access protections.
- Examples reference digital-human assets, cloud services, and external model weights. The materials do not fully specify asset-use limits, data-retention policies, or confirmation controls for tool side effects.
Maintain this project? Add the FARS badge to your README
[](https://followagents.com/en/agents/cyberverse)<a href="https://followagents.com/en/agents/cyberverse"><img src="https://followagents.com/badge/cyberverse.svg" alt="FARS review"></a>Recent releases
Latest entries from the project's GitHub Releases. Notes are quoted from upstream as written, not rewritten or tested by FollowAgents.
-
v0.1.0 CyberVerse v0.1.0
Release notes excerpt
Full Changelog: https://github.com/dsd2077/CyberVerse/commits/v0.1.0View on GitHub ↗
FAQ
Does voice-only operation require a local avatar GPU?
inference.avatar.enabled: false runs CyberVerse as a voice-only assistant. Runtime cost then depends on the configured realtime voice, omni, LLM, TTS, and ASR providers.Which local video-avatar models are documented?
config/avatar_models/.How can I tell whether avatar inference is too slow for realtime playback?
elapsed / (frames / fps). An RTP above 1 means generation is slower than playback and can lead to lag or stutter.What should I check if a remote page loads but audio or video never connects?
streaming_mode: direct with the embedded TURN server, verify that the browser can reach the server on 8443/TCP. The documented causes include cloud security groups, firewalls, and NAT restrictions; SSH port forwarding is a documented workaround.Thanks — we use this to improve the page.