PandaProbe
An engineering platform for tracing, evaluating, monitoring, and debugging AI agents.
What does this agent do, and when should you use it?
PandaProbe is an open-source agent engineering platform from Chirpz AI for collaborative tracing, evaluation, monitoring, and debugging. Its documented stack includes a Next.js dashboard, FastAPI application server, Celery workers and scheduler, PostgreSQL, and Redis. Data-plane clients submit trace or evaluation work with X-API-Key and X-Project-Name, while Redis queues that work for background processing. Workers persist traces and spans to PostgreSQL; evaluation workers use a LiteLLM-backed LLM-as-a-judge flow to store a verdict and score. Teams can use PandaProbe Cloud or self-host the service with Docker.
A client sends POST /traces or POST /evaluations to the FastAPI API. The service resolves organization and project context through the Identity Service, enqueues ingestion or evaluation work in Redis, and returns 202 Accepted. A Celery Worker consumes the job, persists traces and spans in PostgreSQL, or calls the LiteLLM LLM Engine for an LLM-as-a-judge verdict and score before persisting the evaluation result. GET /traces and GET /sessions return filtered, paginated data. Management-plane requests use a Bearer token authenticated through the Auth Service with Supabase or Firebase.
- A LangGraph engineering team that needs centralized trace and span records to investigate agent execution issues.
- A CrewAI team that wants to enqueue evaluations and retain LLM-as-a-judge scores for its multi-agent workflows.
- A team using the Claude Agent SDK that needs one service for observing and evaluating agent runs.
- An organization with internal deployment requirements that wants to run the dashboard, API, workers, PostgreSQL, and Redis under Docker.
- A platform team that needs project-scoped agent data and can identify data-plane requests with X-API-Key and X-Project-Name.
What are this agent's strengths and limitations?
- Trace ingestion and evaluations are queued through Redis and handled by Celery, allowing callers to receive 202 Accepted after submission.
- The documented architecture covers trace/span persistence, trace and session querying, and stored LLM-as-a-judge verdicts and scores.
- It offers both a managed cloud option and a documented Docker self-hosting path with named service components.
- The repository explicitly names integrations with LangGraph, CrewAI, and the Claude Agent SDK.
- Self-hosting requires Docker and a multi-service stack comprising Next.js, FastAPI, Celery, PostgreSQL, and Redis.
- Evaluation depends on a LiteLLM-backed LLM-as-a-judge flow, but the supplied material does not specify supported model providers, model configuration, or cost controls.
- Data-plane writes require X-API-Key and X-Project-Name, while management requests require a Bearer token validated through Supabase or Firebase; credential issuance is not documented here.
- The supplied material does not include client SDK initialization, complete API payloads, or integration configuration examples.
How do you install or deploy this agent?
Docker must be installed and running. Run:
git clone https://github.com/chirpz-ai/pandaprobe.git
cd pandaprobe
./start.sh
After startup, open the dashboard at http://localhost:3000 and the API reference at http://localhost:8000/scalar.
How do you use this agent?
Start the service, then open http://localhost:3000 for the dashboard. For the data plane, provide X-API-Key and X-Project-Name and submit traces to POST /traces or evaluations to POST /evaluations; both are processed asynchronously and return 202 Accepted. Query filtered, paginated records through GET /traces and GET /sessions. Management-plane requests require a Bearer token. The supplied material does not document API-key provisioning, project creation, or copyable request bodies.