Future AGI
One platform to trace, evaluate, simulate, protect, and improve production AI applications.
Per-dimension scores and reasoning
Evidence shows: README explicitly states that self-hosted instances collect telemetry (registration collects admin emails and domains, heartbeat collects anonymous usage counts) and provides an opt-out option (FUTURE_AGI_TELEMETRY_DISABLED=1), but even when disabled, a minimal census ping is sent once, demonstrating data flow transparency (2). Sensitive data handling: README claims no trace data, prompts, or API keys are collected, but no code-level evidence is provided, so deduct (2). Dependency security: CI workflows use third-party actions (e.g., actions/checkout@v4), but no dependency vulnerability scanning or SBOM evidence is present, so deduct (1). External effects: telemetry data exfiltration is an external effect, but opt-out is provided, so 1. Rollback: README mentions production deployment uses ./deploy/setup.sh to generate secrets and pin image versions, but no rollback mechanism documentation, so deduct (1). Source attribution: README lists multiple sub-repos and SDKs, but no explicit source verification mechanism, so 2. Least privilege: no permission model documentation found, but telemetry opt-out indicates some control, so 1. User confirmation: telemetry opt-out requires user action, but no explicit confirmation prompt on first run, so 1.
Self-consistency: README claims about features, architecture, and performance are largely consistent with repo structure, but some inconsistencies exist (e.g., claims 50+ frameworks but no full list), so 2. Dependency availability: README provides Docker Compose and pip/npm installation, but no availability guarantees, so 2. Failure messages: no clear error handling documentation or user-friendly failure message examples found, so 1.
Audience and scenarios: README lists multiple use cases (customer support, voice agents, internal tools, etc.) with explanations, so 2. Capability boundaries: README clearly lists core features (simulate, evaluate, protect, monitor, optimize), but does not detail limitations of each, so 2. Trigger precision: no documentation on trigger conditions or precise control features, so 1. Environment fit: README provides multiple deployment options (Docker Compose, production overlay, Kubernetes coming soon) and runtime requirements, so 2.
Information architecture: README is well-structured with sections for quickstart, features, deployment, architecture, SDKs, comparison, roadmap, etc., so 2. Install notes: README provides detailed installation steps (Docker Compose, pip/npm) and prerequisites, so 2. Naming stability: no documentation on naming conventions or API stability, so 1. Examples and FAQ: README provides code examples (Python/TypeScript), but no FAQ section, so 2. Known limitations: README mentions 'nightly release, expect rough edges', but does not list specific limitations, so 1. License: LICENSE file is Apache 2.0 and README clearly states, so 3. Versioning and changelog: README links to Changelog, but no content provided, so 1. Maintenance responsibility: README provides community support channels (Discord, GitHub Discussions) and security reporting process, so 2.
Output usability: README provides clear output examples (code snippets, architecture diagram), but no detailed output format documentation, so 2. Marginal value: README claims all-in-one platform replacing multiple tools, but no detailed comparison data, so 2. Cost-benefit: README mentions free cloud tier and self-hosting, but no detailed cost analysis, so 1.
Claim traceability: Performance claims (e.g., 29k req/s) lack reproducible benchmark details, so 1. Cross-source corroboration: no third-party verification or independent test results, so 1. Fact-inference separation: README mixes facts (feature lists) and inferences ('self-improving') without clear distinction, so 1.
- Telemetry data collection: Self-hosted instances collect telemetry by default, including admin emails and domains, and even when disabled, a census ping is sent once. Assess privacy implications.
- Dependency security: No dependency vulnerability scanning or SBOM provided; recommend security audit before deployment.
- Performance claims: Performance figures (e.g., 29k req/s) lack reproducible benchmark details; treat with caution.
What does this agent do, and when should you use it?
Future AGI is a self-hostable platform for evaluating, observing, and improving LLM and AI-agent applications. Its platform includes tracer, agentic_eval, simulate, and model_hub, with Agent Command Center serving as an OpenAI-compatible gateway. Applications send telemetry through traceAI OpenTelemetry instrumentation, while the platform exposes span graphs, latency, token-cost views, and dashboards and can feed production traces into evaluation and optimization work. The stated stack is Django/Channels, a Go gateway, and a React/Vite frontend backed by PostgreSQL, ClickHouse, Redis, RabbitMQ, and Temporal. It offers Docker Compose self-hosting and a managed Cloud option, but the repository labels the release as nightly software for early testing.
In Python, developers can call register(project_name="my-agent") and OpenAIInstrumentor().instrument(); in TypeScript, they can call register({ projectName: "my-agent" }) and new OpenAIInstrumentation().instrument() to add tracing around existing OpenAI calls. traceAI emits OpenTelemetry data, and tracer ingests OTLP data to build span graphs; PostgreSQL stores metadata while ClickHouse stores spans and time-series data. agentic_eval exposes 50+ metrics through evaluate(), including groundedness, hallucination, tool-use correctness, PII, tone, and custom rubrics. simulate runs multi-turn scenarios using personas, adversarial inputs, and edge cases. Agent Command Center provides an OpenAI-compatible HTTP gateway with routing, semantic caching, virtual keys, MCP, and A2A, while scanners can run inline at the gateway or through an SDK. The optimization section lists GEPA, PromptWizard, ProTeGi, Bayesian, Meta-Prompt, and Random algorithms.
- A production support-agent team that needs OpenTelemetry traces, span views, latency, token-cost data, and evaluations of live calls.
- A RAG or answer-generation team that wants to measure groundedness and hallucination before or after release.
- A voice-agent team using LiveKit, VAPI, Retell, or Pipecat that needs multi-turn persona and adversarial simulations before launch.
- A platform team that needs one OpenAI-compatible gateway for multiple model providers, routing, semantic caching, and virtual keys.
- A team operating against PII, jailbreak, or prompt-injection risk that wants scanners deployed at the gateway or in an SDK.
- A prompt-engineering team that wants to turn production traces into data for experiments with the six listed optimization algorithms.
What are this agent's strengths and limitations?
- Combines OTLP tracing, evaluations, simulations, guardrails, prompt optimization, and an OpenAI-compatible gateway in one documented platform.
- Documents 50+ evaluation metrics and three evaluation approaches: LLM-as-judge, heuristic, and ML.
- Provides a self-host path based on published images and Docker Compose, plus a production setup script that generates secrets and pins image tags.
- The gateway lists 100+ providers, 15 routing strategies, semantic caching, virtual keys, MCP, and A2A.
- Uses OpenTelemetry tracing and states support for instrumentation across 50+ AI frameworks.
- The README explicitly calls this a nightly release for early testing and says users should expect rough edges while a stable version is pending.
- The full self-hosted deployment carries operational dependencies across Docker Compose, PostgreSQL, ClickHouse, Redis, RabbitMQ, and Temporal.
- The supplied examples show instrumentation and an OpenAI call, but not client initialization, credential setup, or initial self-hosted user provisioning.
- Self-hosted deployments collect telemetry by default; disabling it still sends one minimal census ping after first start, so complete silence requires network-edge blocking.
- Official Kubernetes and Helm support is marked as coming soon; current cloud deployment guidance is to run Docker Compose on a VM.
How do you install or deploy this agent?
For self-hosting, install Docker Desktop or Docker Engine with Docker Compose, then run:
git clone https://github.com/future-agi/future-agi.git
cd future-agi
./bin/installOn Windows PowerShell, run ./bin\install.ps1. Open http://localhost:3000 when installation completes. For production, run ./deploy/setup.sh; the repository says it generates required secrets, pins image versions, pulls images, and starts the stack. The documented runtime also names Python 3.11+, Go 1.23+, and Node 20+.
How do you use this agent?
For Python tracing, import register from fi_instrumentation and OpenAIInstrumentor from traceai_openai, call register(project_name="my-agent"), then call OpenAIInstrumentor().instrument() before the existing client.chat.completions.create(...) call. For TypeScript, use @traceai/fi-core and @traceai/openai, call register({ projectName: "my-agent" }), and instantiate new OpenAIInstrumentation().instrument() before calling the OpenAI client. The supplied material does not document client construction, provider credential configuration, or first-admin setup for a self-hosted instance.
How does this agent compare with similar options?
The README compares Future AGI with Langfuse, Phoenix, Braintrust, and Helicone. Its table presents Future AGI as covering open-source and self-hosted deployment, OpenTelemetry tracing, evaluations, agent simulation, voice-agent evaluation, a built-in gateway, guardrails, prompt optimization, prompt management, datasets and experiments, and a no-code evaluation builder; it identifies the core license as Apache 2.0.
FAQ
Can it run fully offline?
FUTURE_AGI_TELEMETRY_DISABLED=1 still permits one minimal census ping; complete network silence requires blocking it at the network edge.