Designing Multi-Agent Systems (PicoAgents)
Build LLM-enabled multi-agent systems from scratch: a book plus a fully runnable teaching framework covering everything from a single agent to autonomous orchestration.
- Source repo
- victordibia/designing-multiagent-systems
- Stars
- ★ 1.4k
- Last updated
- 1mo ago
- License
- Apache-2.0
- Primary language
- Python
- FA score
- 59/100 · Major gaps
At a glance
- How it runs
- Works with
- Universal · cross-platformOpenAI API · Claude API
- Cost
- Free software; you pay for model usage
- Setup effort
- Medium · a few setup steps
- You'll need
- Typical use
- Engineers who want to truly understand how multi-agent systems work internally rather than treating a framework as a black box — the from-scratch teaching path suits systematic learning.
- Not a fit if
- Teams that want a ready-made agent product rather than building agents in code
- Users who want everything working out of the box without optional extras like persist or otel
- Source review
- 59/100 · Major gaps
What does this agent do, and when should you use it?
This is the official code repository for the book 'Designing Multi-Agent Systems' by Victor Dibia, centered on PicoAgents — a full-featured multi-agent framework built entirely from scratch for teaching purposes. The repository is organized by book chapter, covering agent fundamentals (tools, memory, streaming, middleware), Playwright-based computer-use agents, a type-safe workflow engine, three autonomous orchestration modes (GroupChat, LLM-driven, plan-based), and an LLM-as-judge evaluation framework. It ships a FastAPI+SSE backend with a React Web UI (launched via 'picoagents ui') that auto-discovers local agents and provides streaming chat, a debug rail, run history, an MCP Playground, and an evaluation dashboard. The model layer supports OpenAI, Azure OpenAI, Anthropic, GitHub Models, and any OpenAI-compatible local endpoint such as Ollama or vLLM. Core patterns are deliberately framework-agnostic, and the repo includes equivalent implementations in LangGraph, Microsoft Agent Framework, and Google ADK for easy migration.
The repository has two parts. First, the framework source (picoagents/): a core Agent implementation (_agent.py with streaming, tool calling, memory, middleware, and human-in-the-loop approvals), a _computer_use browser-automation module (navigate/click/type/scroll/extract tools), a DAG-based workflow engine, three orchestrators (round-robin, LLM-driven speaker selection, Magentic One-style plan-based), 15+ built-in tools, 9 termination conditions, an eval module (LLM-as-judge, reference-based matching, composite scoring), and model clients for OpenAI, Azure, and Anthropic. Second, 50+ runnable examples in examples/ organized by chapter, including a FastAPI+SSE agent UX app, a production workflow analyzing 5,000+ companies (two-stage filtering for ~90% cost reduction, checkpointing and resumability), and a complete software-engineering agent. After installation, 'picoagents ui' launches a Web UI that auto-discovers agents, orchestrators, and workflows in a directory, plus an MCP Playground for connecting to MCP servers and inspecting JSON-RPC traffic.
- Engineers who want to truly understand how multi-agent systems work internally rather than treating a framework as a black box — the from-scratch teaching path suits systematic learning.
- Teams evaluating LangGraph, AutoGen/Agent Framework, and Google ADK, needing side-by-side implementations of the same patterns for comparison.
- Building production-grade agents that need streaming output, tool-approval loops (human-in-the-loop), and OpenTelemetry observability.
- Building computer-use agents that drive a browser via multimodal vision models to navigate, click, and extract content.
- Data teams batch-analyzing thousands of unstructured records with LLMs, requiring cost optimization and checkpoint/resume.
- Engineering teams setting up agent evaluation pipelines (LLM-as-judge, reference validation, batch runs, metrics).
How do you install or deploy this agent?
Requires Python and an OpenAI API key. Local install:
git clone https://github.com/victordibia/designing-multiagent-systems.git
cd designing-multiagent-systems/picoagents
python -m venv venv && source venv/bin/activate
pip install -e . # basic install
pip install -e ".[all]" # includes web, mcp, computer-use, examples extras
export OPENAI_API_KEY="your-key"Alternatively, use the GitHub Codespaces badge in the README for a preconfigured browser environment (60 free hours/month), or run the per-chapter Colab notebooks with no installation.
How do you use this agent?
1) Run your first agent: python examples/agents/basic-agent.py; 2) Launch the Web UI: picoagents ui (or picoagents ui --dir ./examples) — it auto-discovers agents, orchestrators, and workflows and provides streaming chat, a live debug rail, run history, the MCP Playground, and an evaluation dashboard; 3) Create an agent via Python API: from picoagents import Agent, OpenAIChatCompletionClient, then Agent(name, instructions, model_client, tools=[...]) and await agent.run(...); 4) Switch providers by swapping the client class or base_url (e.g., Ollama via OpenAIChatCompletionClient(model="llama3.2", base_url="http://localhost:11434/v1")); 5) Computer use: python examples/agents/computer_use.py; orchestration: python examples/orchestration/round-robin.py.
What are this agent's strengths and limitations?
- Complete, tested, from-scratch implementations: every abstraction (agent loop, tools, memory, orchestration, evaluation) is transparent and readable — rare teaching-grade source code.
- Framework-agnostic patterns with direct equivalents provided for LangGraph, Microsoft Agent Framework, and Google ADK, avoiding lock-in and outdated-API issues.
- Model clients cover OpenAI, Azure, Anthropic, GitHub Models (free tier), and any OpenAI-compatible local endpoint, keeping switching costs low.
- Documented production practices: ~90% LLM cost reduction via two-stage filtering, checkpointable/resumable workflows, and a think tool with a 54% performance gain.
- Ready-to-use Web UI and CLI (picoagents ui) with auto-discovery, SSE streaming, MCP Playground, and an evaluation dashboard.
- Examples require API keys (OpenAI by default) and incur LLM costs; most examples cannot run without credentials.
- PicoAgents is a teaching framework — production adoption requires assessing its long-term maintenance and community maturity; the book itself must be purchased separately.
- Browser automation depends on Playwright and an extra install (pip install -e ".[computer-use]"); persist, otel, dev, and frameworks extras are not included in [all] and need separate installation.
- Production observability features (OpenTelemetry, run/eval persistence) require optional extras and are absent from the default install.
How does this agent compare with similar options?
The repository itself provides task-equivalent implementations comparing PicoAgents with LangGraph, Microsoft Agent Framework, and Google ADK, concluding the core patterns transfer across all of them; PicoAgents differentiates by being built from scratch for learning, while production deployments typically move to those mature frameworks.
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| Designing Multi-Agent Systems (PicoAgents) This agent | 59 · Major gaps | Library / SDKFree + model costs | ★ 1.4k | 1mo ago | Python | OpenAI API · Claude API |
| HUD | 63 · Some gaps | CLIFree + model costs | ★ 315 | 1d ago | Python | OpenAI API · Claude API |
| PandaProbe | 41 · Major gaps | Self-hosted serviceFreemium | ★ 792 | 18d ago | Python | — |
| CLI Agent Orchestrator | 65 · Some gaps | CLIFree + model costs | ★ 1.5k | 1d ago | Python | Codex · Claude Code |
How does FollowAgents rate this agent?
Why each dimension lost points
Evidence: README references approval examples (approval_example.py), middleware, termination conditions, and human-in-the-loop, indicating tool-call confirmation is designed in; streaming events (TaskStart/ToolCallEvent etc.) visible in tests make data flow reasonably transparent — 2. least_privilege only 1: no per-tool/agent permission scoping or least-privilege defaults documented. user_confirmation only 1: approval example exists but its mandatory/default behavior is not shown. sensitive_data_handling only 1: API keys via env vars only, no leakage/storage guidance. dependency_security only 1: sensible extras split but no pinning, audit, or vulnerability policy. external_effects only 1: computer-use browser automation and SWE agent create real side effects with no documented default safeguards or consent flows in evidence. rollback only 1: checkpointing mentioned only for yc_analysis, insufficient as framework-level rollback. source_attribution 2: author, book, and code path attribution clear; publisher unverified so no bonus.
self_consistency 2: README matches directory tree and test references; mock-client tests are internally consistent, but test_agent_as_tool_strategies.py contains a confusing comment block where an assertion expectation was 'fixed' after the fact, showing loose test discipline — not 3. dependency_availability 2: OpenAI/Azure/Anthropic/GitHub Models plus local OpenAI-compatible endpoints give good availability paths. failure_messages 1: 9 termination conditions described, but actual error message quality and exception handling are not evidenced in these files.
audience_and_scenarios 3: clearly targeted at learners and developers, with chapter-mapped scenarios (basic agents, orchestration, evaluation, production case studies). capability_boundaries 2: teaching scope, from-scratch implementation, and framework comparisons are stated plainly. trigger_precision 1: this is a framework/tutorial repo, not an AGENTS.md-manifest agent; no trigger or invocation-timing specification exists — deduction is for absent evidence. environment_fit 2: Colab, Codespaces, and local venv plus optional extras cover common environments, though Windows-specific notes are absent.
information_architecture 3: full directory tree and chapter-to-code mapping tables make the repo navigable. install_notes 3: step-by-step clone/venv/extras/API-key instructions. naming_stability 2: consistent naming (picoagents, chapter-organized examples) but no long-term naming commitment evidenced. examples_and_faq 3: 50+ examples, notebooks, code_along progression, and framework comparisons; no FAQ, but example density justifies the score. known_limitations 1: no stated limitations (e.g., explicit warning that teaching implementations are not production-hardened beyond scattered hints). license 3: full Apache-2.0 text in repo. versioning_changelog 1: no version number or changelog evidence. maintenance_responsibility 2: single named author with a companion book implying sustained investment, but no explicit maintenance commitment or update-path document.
output_usability 2: structured outputs (Pydantic), streaming events, Web UI, and evaluation dashboard make outputs usable. marginal_value 2: a from-scratch educational framework plus cross-framework comparisons (LangGraph, Google ADK, MS Agent Framework) is differentiated, though production readiness is unverified by execution. cost_benefit 2: claimed optimizations (two-stage filtering for 90% cost reduction) are plausible but the numbers are unproven in the source files, so no 3.
claim_traceability 2: nearly every feature claim links to a code path, statically traceable. cross_source_corroboration 2: README, tests, and directory structure corroborate each other (_agent.py, AgentAsTool appear in tests), but only a subset of files was available, limiting full cross-checking. fact_inference_separation 1: quantitative claims ('54% performance gain', '90% LLM cost reduction', 'production-ready') are mixed with factual statements without cited sources or methodology — a concrete deduction.
- Conservative static review with low confidence: no code was executed; tests were read, not run, so runnability and correctness are unverified.
- Computer-use and SWE agents have real external side effects (browser actions, code edits); run in a sandbox and add your own confirmation mechanisms before use.
- Quantitative claims (90% cost reduction, 54% performance gain) lack supporting evaluation evidence in the source files; do not take them at face value.
- No version numbers, changelog, or stated limitations; teaching implementations should not be deployed to production without hardening.
- Dependencies are not pinned; review third-party packages pulled in by extras before installing.
- API keys are handled via environment variables only per the docs; when using the Web UI or MCP Playground, ensure keys and JSON-RPC traffic are not inadvertently exposed.
Recent releases
Latest entries from the project's GitHub Releases. Notes are quoted from upstream as written, not rewritten or tested by FollowAgents.
-
v0.4.0 v0.4.0: Hooks, Compaction, Context Tools, Benchmarks
Release notes excerpt
What's New Deterministic Loop Hooks - BaseStartHook / BaseEndHook for injecting instructions before the first LLM call and resuming the loop when the agent would stop - PlanningHook (forces task planning) and CompletionCheckHook (verifies todo completion before allowing stop) - Composable TerminationCondition with | / & operators Context Compaction Strategies - CompactionStrategy protocol called before each LLM call …View on GitHub ↗ -
v0.3.2 v0.3.2 - Fix AgentResponse print output
Release notes excerpt
What's Changed Fixed - Fixed AgentResponse.__str__() to display full conversation content instead of only metadata. Previously, printing a response would only show usage statistics; now it shows all messages followed by usage stats. Changed - Improved user experience when printing agent responses - users can now simply print(response) to see the complete conversation history. Installation ```bash pip install -U picoagents ``` Full Changelog: https://github.com/victordibia/designing-multiagent-systems/compare/v0.3.1...v0.3.2View on GitHub ↗ -
Release notes excerpt
PicoAgents v0.3.1 Added - Anthropic Claude model client with full API support - GitHub Models integration example - Agent-as-tool result strategies for flexible output handling - Context engineering examples Changed - Updated model client tests for better coverage and reliability - Improved workflow integration tests with proper Pydantic model handling - Enhanced WebUI workflow view components Fixed - Fixed Anthropic model name in tests (claude-3-5-haiku-20241022) - Fixed CancellationToken import path in workflow tests …View on GitHub ↗