AI Gateway Labs
Hands-on labs for building an enterprise-grade AI gateway over models, MCP tools, and agents using Azure API Management and Microsoft Foundry.
Evidence shows labs deploy Bicep infrastructure into the user's own Azure subscription and require Contributor + RBAC Administrator roles — broader than least privilege; labs create billable resources with confirmation/rollback only implied by cleanup notebooks, no enforced mechanism; data flow through APIM is documented, giving adequate transparency; secrets are handled via .env and OAuth labs but no rotation/leak guidance; dependencies are partly pinned (mcp==1.21.2, semantic-kernel==1.33.0) and CodeQL scanning is configured, which is adequate; copyright Microsoft, MIT license clear.
The layered design of a root environment plus per-lab pinned pyprojects is coherent and conflict-aware; most root dependencies are unpinned, limiting reproducibility though per-lab pinning partially compensates; nbchecks.py in CI enforces notebooks without stored outputs — a verifiable quality control; failure messaging inside notebooks cannot be assessed from static evidence, hence the deduction.
Audience (engineers learning/evaluating AI gateway patterns) and scenarios (models, tools, agents, MCP, A2A) are clearly laid out; however, as a lab collection rather than a reusable agent product, capability boundaries and trigger precision are not stated; environment fit is good (uv, Python 3.12, Codespaces, per-lab environments).
Information architecture is clear (labs/tools/skills split, table navigation, doc links); install notes are concrete and executable (uv sync, kernel selection); naming is stable; examples are abundant (30+ labs, videos, skills); MIT license file is complete — full marks; but there is no CHANGELOG, no known-limitations document, and the root version is fixed at 0.1.0 with no release process, hence deductions; CONTRIBUTING.md exists and maintenance points to the Azure-Samples org.
Output is runnable notebooks + Bicep + policies with clear marginal value for learning gateway patterns, plus a mock server lowering the barrier; but every lab requires deploying real billable resources into the user's subscription and no per-lab cost estimate is provided, hence the deduction.
Key README claims (gateway capabilities, Well-Architected alignment) link to Microsoft Learn docs and videos, making them traceable; static evidence (README, CI config, dependency manifest) corroborates each other; facts are mostly separable from marketing wording (e.g., 'enterprise-grade'), though some promotional phrasing remains unproven.
- Labs require subscription-level Contributor + RBAC Administrator roles and deploy billable Azure resources; verify quota and cost caps before running.
- Most root dependencies are unpinned; reproducibility relies on per-lab pyprojects. Audit and pin dependencies before any production use.
- Secrets are managed via .env with no rotation or leak-detection guidance in the repository; do not place real production credentials in lab environments.
- This is a lab/teaching collection, not a reusable agent product; the publisher is unverified by the FollowAgents registry and should be treated as unknown.
What does this agent do, and when should you use it?
AI Gateway Labs is an MIT-licensed Azure-Samples repository containing 30+ hands-on labs delivered as Jupyter Notebooks, each paired with Bicep infrastructure templates and Azure API Management (APIM) policy XML. The labs span three themes: model governance (backend pool load balancing, token rate limiting, semantic caching, model routing, FinOps quota management), tool integration (Model Context Protocol, MCP client authorization, function calling, realtime audio with MCP), and agent orchestration (Foundry Agent Service, OpenAI Agents SDK, Gemini MCP agents, A2A-enabled agents). The deployment boundary is your own Azure subscription: after cloning the repo and syncing the Python environment with uv, each notebook drives Azure CLI and Bicep deployments of APIM instances and model backends, then issues live inference requests against them. The repository also ships Copilot Agent Skills (lab-creator, apim-bicep, apim-policies, etc.) for AI-assisted creation of new labs, plus developer tools including a mock OpenAI server and OAuth test client. It is best understood as an evaluation and enablement toolkit for teams deciding whether to adopt APIM as their AI gateway — not a ready-to-install product.
The repository is a collection of executable learning and evaluation assets. Each lab (e.g., labs/backend-pool-load-balancing/backend-pool-load-balancing.ipynb) uses your configured Azure subscription credentials to deploy an APIM instance and Azure OpenAI / Foundry model backends via Bicep, then walks through notebook steps that configure and exercise APIM policies — token rate limiting, semantic caching, model routing, content safety, OAuth — and finally emits real inference calls whose token metrics, logs, and traces you inspect. The tools/ folder adds tracing, streaming, rate-limit testing, an OpenAI API mock server, and an OAuth client notebook. Copilot skills (lab-creator, apim-bicep, apim-terraform, apim-policies, apim-kql, mcp-builder) generate complete new lab skeletons — notebook, Bicep, policy XML, README, and cleanup script — from prompts in VS Code.
- A platform engineering team evaluating APIM as the gateway for production AI workloads can run the backend pool load balancing and model routing labs to validate multi-model high availability.
- A FinOps owner building AI budget controls can use the FinOps Framework lab to test automated token quotas and rate limiting.
- A developer adding MCP tools to an agent can work through the MCP and MCP client authorization labs to learn OAuth credential management at the gateway layer.
- An architect deciding whether semantic caching pays off can run the semantic-caching lab and compare cost and latency with vector-similarity caching.
- An enterprise aligning to the Azure Well-Architected Framework can adopt labs mapped to the Security, Reliability, Performance, Operations, and Cost pillars.
- A team that wants to author its own gateway labs can use the bundled Copilot skills (e.g., lab-creator) in VS Code to scaffold notebooks, Bicep, and policies from a prompt.
What are this agent's strengths and limitations?
- Maintained by the official Azure-Samples organization, with labs designed around all five Azure Well-Architected Framework pillars.
- Unusually broad coverage: 30+ labs spanning model governance (limiting, caching, routing) through MCP/A2A agents across OpenAI Agents SDK, Autogen, and Gemini.
- Each lab is a runnable end-to-end asset — notebook plus Bicep template plus APIM policy XML — deployable to your own subscription for real verification.
- Bundled Copilot Agent Skills, an enterprise e-Book, a workshop, and conference videos support team-wide enablement.
- Tightly coupled to the Azure ecosystem — APIM, Azure OpenAI, Foundry, and Bicep are all required; the pattern cannot be reproduced on AWS, GCP, or fully on-premises.
- Requires a paid Azure subscription with Contributor and RBAC Administrator permissions; labs incur deployment and inference costs that the source does not quantify.
- It is a samples/lab repository, not an installable product — no production packaging (Docker image, Helm chart) exists, so adoption requires your own engineering effort.
- The full lab path involves several toolchain moving parts (Python, uv, Azure CLI, VS Code), creating initial setup friction.
How do you install or deploy this agent?
Prerequisites: Python 3.12+, the uv package manager (Linux/macOS: curl -LsSf https://astral.sh/uv/install.sh | sh; Windows: powershell -c "irm https://astral.sh/uv/install.ps1 | iex"), VS Code with the Jupyter extension, an Azure subscription with Contributor and RBAC Administrator roles, and an authenticated Azure CLI. Install: git clone https://github.com/Azure-Samples/AI-Gateway.git && cd AI-Gateway && uv sync && uv pip install -r pyproject.toml && code . Alternatively, open directly in GitHub Codespaces.
How do you use this agent?
Open any lab notebook (e.g., labs/token-rate-limiting/token-rate-limiting.ipynb) in VS Code, select the .venv interpreter created by uv sync as the Jupyter kernel, and execute cells step by step: deploy the Bicep template to create APIM and model backends, run inference tests against the APIM endpoint, then review token metrics and logs in Azure. Run each lab's cleanup notebook afterwards to release resources.