Agent Lightning
Capture agent trajectories and train improved prompts or policy resources for existing AI agents.
- Source repo
- microsoft/agent-lightning
- Stars
- ★ 19k
- Last updated
- 1d ago
- License
- MIT
- Primary language
- Python
- FA score
- 40/100 · Major gaps
At a glance
- How it runs
- Works with
- Universal · cross-platformOpenAI API
- Cost
- Free software; you pay for model usage
- Setup effort
- High · needs real infrastructure
- You'll need
- Typical use
- A team with a LangChain customer-support or workflow agent that wants to preserve its existing framework while training from prompts, tool calls, and rewards.
- Not a fit if
- Individual developers without GPU or Kubernetes resources
- Users who want a ready chat product, not a training pipeline
- Source review
- 40/100 · Major gaps 8 safety controls not found
What does this agent do, and when should you use it?
Agent Lightning is a training framework for optimizing AI agents, not a standalone conversational agent product. It is presented as compatible with LangChain, OpenAI Agent SDK, AutoGen, CrewAI, Microsoft Agent Framework, and Python OpenAI implementations without an agent framework. A running agent can emit events with the lightweight agl.emit_xxx() helper or use a tracer to collect prompts, tool calls, and rewards as structured spans. LightningStore keeps tasks, resources, and traces synchronized, while an algorithm reads spans and posts resources such as refined prompt templates or policy weights. The Trainer streams datasets to runners, moves resources between the store and algorithm, and updates the inference engine when improvements arrive.
While an agent runs, Agent Lightning collects prompt, tool-call, and reward events through agl.emit_xxx() or a tracer and organizes them as structured spans. The spans flow to LightningStore alongside tasks and resources. A selected or custom algorithm reads those spans and performs training approaches named in the README—reinforcement learning, automatic prompt optimization, or supervised fine-tuning—then posts updated resources such as prompt templates or policy weights. Trainer streams datasets to runners, transfers resources between LightningStore and the algorithm, and updates the inference engine when improvements land; in a multi-agent system, one or more agents can be selected for optimization.
- A team with a LangChain customer-support or workflow agent that wants to preserve its existing framework while training from prompts, tool calls, and rewards.
- A developer using OpenAI Agent SDK or Python OpenAI who wants to connect existing agent trajectories to a reinforcement-learning workflow.
- A team operating an AutoGen, CrewAI, or Microsoft Agent Framework system with several roles and wanting to optimize only selected agents.
- A researcher with evaluable tasks and reward signals who wants to experiment with reinforcement learning, automatic prompt optimization, or supervised fine-tuning over structured spans.
- An agent-platform team that needs improved prompt templates or policy weights returned to an inference engine after training.
How do you install or deploy this agent?
Install with: pip install agentlightning. The README also provides a nightly pre-release command: pip install --upgrade --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ --pre agentlightning. The supplied material does not specify a Python version, credential requirements, or a post-install initialization command.
How do you use this agent?
The supplied material describes keeping the existing agent running and either emitting events with agl.emit_xxx() or allowing a tracer to collect prompts, tool calls, and rewards; LightningStore, an algorithm, and Trainer then form the training loop. It does not provide a copyable minimal Python example, the exact agl.emit_xxx() signature, initialization for Tracer, LightningStore, or Trainer, credential configuration, or a first runnable invocation. A reliable end-to-end launch command therefore cannot be derived from the supplied evidence.
What are this agent's strengths and limitations?
- Explicitly presented as working alongside several existing agent frameworks, including LangChain, OpenAI Agent SDK, AutoGen, CrewAI, and Microsoft Agent Framework.
- Uses
agl.emit_xxx()or a tracer to turn prompts, tool calls, and rewards into structured spans for training. - Can selectively optimize one or more agents in a multi-agent system instead of requiring a full-system rewrite.
- The documented loop connects LightningStore, algorithms, Trainer, runners, and the inference engine through explicit resource flow.
- The supplied material lacks a minimal runnable example, key-object construction details, and exact API signatures, so initial integration requires additional documentation or examples.
- Useful training depends on available tasks, rewards, and trajectories; the README does not explain reward design, outcome evaluation, or training-failure diagnosis.
- Despite the near-zero-code-change positioning, adoption still requires integrating
agl.emit_xxx()or a tracer and feeding training outputs back into the inference engine. - Python-version support, model-provider configuration, hardware needs, and training cost are not specified.
How does this agent compare with similar options?
It is positioned as a training and optimization layer for agents built with LangChain, OpenAI Agent SDK, AutoGen, CrewAI, or Microsoft Agent Framework, rather than as a replacement application-building framework for those systems.
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| Agent Lightning This agent | 40 · Major gaps | CLIFree + model costs | ★ 19k | 1d ago | Python | OpenAI API |
| ART Agent Reinforcement Trainer | 34 · Major gaps | Library / SDKFreemium | ★ 11k | today | Python | — |
| AgentFlow | 31 · Major gaps | CLIFree + model costs | ★ 2.1k | 9mo ago | Python | OpenAI API |
| Station Research Environment | 60 · Some gaps | CLIFree + model costs | ★ 211 | 23d ago | Python | — |
How does FollowAgents rate this agent?
Why each dimension lost points
Evidence shows the repository is published by Microsoft official organization, but no specific implementation or documentation for permission management, user confirmation, data flow transparency, sensitive data handling, dependency security, external effects, rollback, or source attribution was found. Therefore all trust criteria score 0.
Self-consistency: README, pyproject.toml, and test files are largely consistent in functional description, but no deep code verification. Dependency availability: pyproject.toml lists many dependencies but lacks lock file or version range details, posing dependency drift risk. Failure messages: Test files include error handling tests, but no user-facing error message documentation.
Audience and scenarios: README clearly targets AI agent developers, supporting multiple frameworks. Capability boundaries: Documentation describes core features but lacks explicit limitations. Trigger precision: No detailed trigger conditions or configuration instructions. Environment fit: pyproject.toml specifies Python version and platform constraints, but no complete system requirements.
Information architecture: README is well-structured with installation, features, architecture. Install notes: Provides pip install command and documentation link. Naming stability: Project name and version are clear. Examples and FAQ: Provides examples directory and community projects, but no FAQ. Known limitations: Not explicitly listed. License: MIT license file present. Versioning and changelog: No changelog provided. Maintenance responsibility: Contribution guide and CLA instructions exist.
Output usability: Documentation and examples provide usage guidance. Marginal value: Project claims zero code change, but no benchmark data. Cost-benefit: No performance or resource consumption data.
Claim traceability: README cites papers and blogs, but lacks implementation details. Cross-source corroboration: External articles and community projects exist, but no independent verification. Fact-inference separation: Documentation mixes facts and marketing claims without clear distinction.
- Not found in source: least-privilege scopingGrant only what the task needs: a dedicated account or read-only token, scoped to specific directories and repos.
- Not found in source: confirmation before actingTurn on (or add) a confirmation step before it acts, and try it in a sandbox or test environment before real data.
- Not found in source: data-flow disclosureWatch which external services it contacts (proxy or firewall logs) and keep sensitive data out until you know where it goes.
- Not found in source: sensitive-data handlingUse dedicated, low-privilege, revocable API keys — never production credentials — and keep secrets out of logs.
- Not found in source: dependency securityPin versions and run a dependency audit (npm audit, pip-audit) before installing; prefer running it in a container.
- Not found in source: disclosed external effectsEstablish which external systems it writes to, sends to or changes, and verify with test accounts or repos before production.
- Not found in source: rollback or recovery pathBack up first, or work on a git branch or snapshot, so its changes can be undone.
- Not found in source: verifiable attributionInstall from the official repo or registry and check the publisher and URL to avoid look-alike packages.
- No specific implementation for permission management, data flow transparency, or sensitive data handling; evaluate before use.
- Large dependency list without version locking poses supply chain risk.
- Documentation contains marketing claims lacking independent verification.
Recent releases
Latest entries from the project's GitHub Releases. Notes are quoted from upstream as written, not rewritten or tested by FollowAgents.
-
v1.0.3 Agent Lightning v1.0.3
Release notes excerpt
Highlights - Score Centering for low-precision RL: Added Score Centering with top-K behavior-probability capture and a runnable Qwen3.5-9B math training example. (#651, #677). - Lower-memory MoE routing replay: Reduced peak host-memory usage when decoding routed experts by processing one rollout at a time, while preserving the resulting training tensors (#636). Features - Added INT8 weight/activation rollouts with an INT4 Attention KV cache, weight synchronization, and model preparation and precision-verification tools for the math training example (#651). …View on GitHub ↗ -
v1.0.2 Agent Lightning v1.0.2
Release notes excerpt
Highlights - MoE coding-agent training: Added Qwen3.5-35B-A3B training with CISPO and R3 expert-routing replay, plus a runnable example and guide (#601, #607). In the published pure-RL experiment, SWE-bench Verified performance rose from 47.8% to 61.6% after 1,792 training examples (#608). - Multimodal training: Image inputs now flow from rollout traces into VERL training batches, fixing a case where the model saw images during rollout but not during training. A multimodal QA example is included (#560). Features …View on GitHub ↗ -
v1.0.1 Agent Lightning v1.0.1
Release notes excerpt
Agent Lightning v1.0.1 marks the first official release of the Agent Lightning Skill, which helps coding agents optimize other AI agents. Provide an editable agent and a benchmark, and the skill guides systematic improvements to prompts, tools, workflows, models, and reasoning settings—balancing accuracy, cost, latency, and reliability through measured iteration. Install it for Claude Code, Codex, or GitHub Copilot: ```bash gh skill install microsoft/agent-lightning agent-lightning --agent <agent> ``` …View on GitHub ↗