Agent Learning Kit

What does this agent do, and when should you use it?

Agent Learning Kit is Future AGI's local-first SDK and CLI for testing, simulating, red-teaming, and optimizing AI agents. It collapses three engines — simulate, evals, and optimize — onto one public developer surface, with red teaming running on the simulate and evals engines rather than as a fourth engine. The Python package is agent-learning-kit, the namespace is fi.alk, the CLI is agent-learn, and a TypeScript package ships as @future-agi/agent-learning-kit. Its loop is simulate a workflow, evaluate the behavior and runtime evidence, optimize the weak layer, promote the result into a replayable artifact, then prove release readiness with local gates. The documented quickstart runs fully offline with no API key or network; only platform-backed evaluation requires AGENT_LEARNING_API_KEY. The README states the v1 bar is the executable environment_10x_robustness release gate, so the shipped contract is locally verifiable rather than cloud-dependent.

It reads agent source, run manifests, evaluation suites, optimization manifests, and red-team manifests, executes tasks inside local worlds, and writes JSON artifacts. Concrete operations include driving a single SimulationSpec through a SimulationRunner as the plug-and-play simulation surface; evaluating prompts, task outputs, runtime contracts, traces, memory, retrieval, safety, and robustness evidence; searching over prompts, agents, framework adapters, worlds, multi-agent interactions, memory layers, workflows, and red-team scenarios; and emitting agent-learning.release-proof.v1 evidence via release-check and release-proof. The bundled harness under src/fi/alk/harness/ reads what an agent verifiably is, builds a real world its tools act on, and writes test scenarios that are each proved before being kept, driven as a conversation in a terminal or on a web page served at localhost:8777. Adapter coverage spans LangChain, LangGraph, LlamaIndex, AutoGen, CrewAI, LiveKit, Pipecat, Browser Use, MCP, A2A, and custom orchestration objects, with runtime-simulated coverage for PydanticAI and handoff-transcript promotion for OpenAI Agents.

1. A developer building a LangChain or LangGraph agent who wants offline task simulation and regression replay before pushing to CI.
2. A team that needs repeatable tests for multi-agent coordination and handoff logic instead of manual trial runs.
3. A security engineer red-teaming agents with corpora, campaigns, and adaptive loops without sending data to a third-party cloud.
4. A release engineer who needs the local gate evidence that agent-learn release-proof writes as sign-off material.
5. A team tuning retrieval or memory layers that wants evaluation results for both inside the same artifact model.
6. An integration team that wants voice, realtime, browser/CUA, workflow, lifecycle, and protocol traces evaluated together.

How do you install or deploy this agent?

Install from PyPI:

pip install agent-learning-kit

Optional extras:

pip install "agent-learning-kit[livekit]"
pip install "agent-learning-kit[nli]"
pip install "agent-learning-kit[all]"

To develop against source:

git clone https://github.com/future-agi/agent-learning-kit
cd agent-learning-kit
uv sync          # or: pip install -e .

The TypeScript package (npm publishing lands at the v1 launch; today build from typescript/agent-learning-kit):

pnpm add @future-agi/agent-learning-kit

How do you use this agent?

Start with the local doctor to confirm the environment:

agent-learn doctor

Evaluate a suite:

agent-learn eval examples/eval_suite.json \
  --output artifacts/eval.json

Simulate a run manifest:

AGENT_LEARNING_RUN_EXAMPLE_KEY=offline-demo-key \
  agent-learn run examples/run_manifest.json \
  --no-eval \
  --output artifacts/run.json

Optimize an agent workflow:

AGENT_LEARNING_OPTIMIZE_EXAMPLE_KEY=offline-demo-key \
  agent-learn optimize examples/optimization_manifest.json \
  --output artifacts/optimization.json

Run a red-team campaign:

AGENT_LEARNING_REDTEAM_EXAMPLE_KEY=offline-demo-key \
  agent-learn redteam examples/redteam_manifest.json \
  --output artifacts/redteam.json

Cut local release proof:

agent-learn release-check --project-root .
agent-learn release-proof \
  --project-root . \
  --output /tmp/agent-learning-release-proof.json \
  --quiet

For Future AGI platform-backed evaluation, set AGENT_LEARNING_API_KEY (it takes precedence over the FUTURE_AGI_API_KEY and FI_API_KEY aliases) or call configure from code:

from fi.alk import configure

configure(api_key="...")

On the TypeScript side, import the evaluator surfaces:

import { Evaluator } from "@future-agi/agent-learning-kit";
import { LocalEvaluator } from "@future-agi/agent-learning-kit/evals/local";

What are this agent's strengths and limitations?

How does this agent compare with similar options?

Key facts side by side with the most closely related agents.

How does FollowAgents rate this agent?

Recent releases

Latest entries from the project's GitHub Releases. Notes are quoted from upstream as written, not rewritten or tested by FollowAgents.

1. ts-v0.2.0 @future-agi/ai-evaluation (TypeScript) 0.2.0 2026-04-28
   Release notes excerpt

   TypeScript SDK release. See CHANGELOG.md for full notes.

   View on GitHub ↗
2. py-v1.1.0 ai-evaluation (Python) 1.1.0 2026-04-23
   Release notes excerpt

   Python SDK release. See CHANGELOG.md for full notes.

   View on GitHub ↗
3. v1.0.0 2026-02-28
   Release notes excerpt

   Unified evaluate() API — One function for all evaluation needs. Local metrics, cloud templates, and LLM-as-judge through a single entrypoint. ```python …

   View on GitHub ↗

FAQ