Oumi

An open, end-to-end platform for fine-tuning, evaluating, and deploying foundation models from 10M to 400B+ parameters.

Source repo
oumi-ai/oumi
Stars
★ 9.4k
Last updated
1d ago
License
Apache-2.0
Primary language
Python

At a glance

How it runs
CLILibrary / SDK
Works with
Universal · cross-platformClaude Code · OpenAI APIChatGPT · Claude API (Partial support)
Cost
Free software; you pay for model usage
Setup effort
High · needs real infrastructure
You'll need
PythonPyTorchDocker (alternative install path)Shell / CLINetwork accessLocal filesystemMCP Server
Typical use
A team that wants to fine-tune Qwen3 or a Gemma model for its own domain and would rather start from an existing YAML recipe than write a training script.
Not a fit if
  • Users who only want a chat UI and no config files or CLI
  • Teams with no GPU access doing only light experimentation
  • Production buyers who need a commercial SLA and vendor support
Website
oumi.ai ↗ set on GitHub
Maintenance
Commits in the last 30 dayslast commit 2026-11-20latest release v0.9 (2026-09-16)+1.1 ★/day over 7 days
Source review
63/100 · Some gaps

What does this agent do, and when should you use it?

Oumi is a fully open-source platform that covers the full lifecycle of foundation models: data synthesis and curation, pre-training and fine-tuning, evaluation, inference, and deployment. It is driven by a single oumi CLI plus YAML recipes, so you rarely have to write your own training loop or data pipeline. Training supports SFT, LoRA, QLoRA, DPO, and GRPO, scaling through FSDP, DeepSpeed, and DDP; inference integrates vLLM and SGLang, and models can be pushed to dedicated endpoints with oumi deploy. Beyond text models it handles vision-language models and agentic, tool-using models: you can define executable tool environments (database, HTTP endpoint, lookup, simulated) and run verl GRPO reinforcement learning over them. Evaluation spans standard benchmarks plus multi-criteria RubricJudge and LLM-as-a-Judge. It runs from a laptop up to clusters and clouds via oumi launch (AWS, Azure, GCP, Lambda, Modal, Slurm), and oumi-mcp exposes an MCP server for Claude and Cursor.

Oumi centers on the oumi command-line interface. You author or reuse a YAML config from configs/recipes and run oumi train -c <config> to launch training (SFT, LoRA, QLoRA, DPO, GRPO on PyTorch, optionally with FSDP/DeepSpeed/DDP), oumi evaluate -c <config> to score a model on standard benchmarks or custom judges, and oumi infer -c <config> --interactive for interactive inference through engines such as vLLM, SGLang, Fireworks, or OpenRouter. On the data side it reads raw corpora and uses LLM judges, including the multi-criteria RubricJudge, to synthesize and filter training data, including multi-turn tool-use conversations. For agentic training it manages executable tool environments (database, HTTP endpoint, lookup, simulated) and applies verl GRPO over them. For delivery, oumi deploy publishes models to dedicated inference endpoints on Fireworks and Parasail, oumi launch up submits jobs to cloud platforms, and oumi-mcp runs an MCP server consumable by Claude and Cursor. oumi analyze covers analysis, and the configs/recipes tree ships ready-made configurations for Qwen, Gemma, Llama, DeepSeek, GLM, Phi, OLMo, gpt-oss, Falcon, and more.

  1. A team that wants to fine-tune Qwen3 or a Gemma model for its own domain and would rather start from an existing YAML recipe than write a training script.
  2. A research group running reproducible GRPO reinforcement-learning experiments across multiple GPUs or a cloud cluster, needing FSDP or DeepSpeed.
  3. A developer training a tool-calling agent model who needs to construct database, HTTP, or lookup tool environments and do reinforcement learning over them.
  4. A data team that must synthesize, filter, and curate training data at scale with LLM-as-a-Judge or RubricJudge, including vision-language datasets.
  5. An engineering team deploying a fine-tuned model locally through vLLM/SGLang, or publishing it to a managed endpoint with oumi deploy.
  6. Developers already working inside an AI-assisted IDE such as Claude or Cursor who want to drive Oumi workflows through the oumi-mcp MCP server.

Key setup facts

Pulled from the install and usage notes below. The project README has the current values.

Install / run
  • uv pip install oumi
  • uv pip install 'oumi[gpu]'
  • uv pip install git+https://github.com/oumi-ai/oumi.git
  • docker pull ghcr.io/oumi-ai/oumi:latest
Config files
configs/recipes/smollm/sft/135m/quickstart_train.yaml configs/recipes/smollm/evaluation/135m/quickstart_eval.yaml configs/recipes/smollm/inference/135m_infer.yaml configs/recipes/smollm/sft/135m/quickstart_gcp_job.yaml
Install notes cover
Docker

How do you install or deploy this agent?

The README recommends pip (uv):

# Basic installation
uv pip install oumi

# With GPU support
uv pip install 'oumi[gpu]'

# Latest development version
uv pip install git+https://github.com/oumi-ai/oumi.git

Docker and an experimental one-line script are also documented:

docker pull ghcr.io/oumi-ai/oumi:latest
docker run --gpus all -it ghcr.io/oumi-ai/oumi:latest oumi --help
curl -LsSf https://oumi.ai/install.sh | bash

Oumi is in beta and under active development: core features are stable, but some advanced features may change. More installation options are in the official installation guide.

How do you use this agent?

Pick a YAML config under configs/recipes and drive it with the oumi CLI:

# Training
oumi train -c configs/recipes/smollm/sft/135m/quickstart_train.yaml

# Evaluation
oumi evaluate -c configs/recipes/smollm/evaluation/135m/quickstart_eval.yaml

# Inference
oumi infer -c configs/recipes/smollm/inference/135m_infer.yaml --interactive

To run remotely, submit with oumi launch; to train inside the container, mount your working directory and pass --config:

oumi launch up -c configs/recipes/smollm/sft/135m/quickstart_gcp_job.yaml --resources.cloud aws
docker run --gpus all -v $(pwd):/workspace -it ghcr.io/oumi-ai/oumi:latest \
    oumi train --config /workspace/my_config.yaml

What are this agent's strengths and limitations?

Pros
  • One CLI plus YAML recipes covers the whole pipeline — training, evaluation, inference, data synthesis, and deployment — with no custom training loop.
  • Broad training method coverage: SFT, LoRA, QLoRA, DPO, and GRPO, with native FSDP, DeepSpeed, and DDP support for scaling.
  • Agentic focus backed by concrete components: executable tool environments (database, HTTP endpoint, lookup, simulated) and verl GRPO reinforcement learning over them.
  • Evaluation is first-class, combining standard benchmarks with multi-criteria RubricJudge and LLM-as-a-Judge.
  • Multiple delivery paths: vLLM/SGLang inference, oumi deploy to dedicated endpoints, oumi-mcp for Claude and Cursor, and oumi launch to AWS, Azure, GCP, Lambda, Modal, or Slurm.
  • Apache-2.0 licensed and fully open source, with no vendor lock-in.
Limitations
  • Oumi is explicitly in beta, and the README warns that some advanced features may change as the platform evolves.
  • Full capability needs GPU hardware and a heavy Python/PyTorch environment, so the barrier to entry is higher than with lightweight tools.
  • Cloud job submission and dedicated endpoint deployment depend on external accounts and quotas (AWS, Azure, GCP, Lambda, Modal, Slurm, Fireworks, Parasail).
  • Many supported model weights ship under custom licenses (Llama, Gemma, Qwen and others), so commercial use requires checking each license.
  • The README's news list contains dates that run into 2026, so release history should be verified against the actual release pages.

How does this agent compare with similar options?

Key facts side by side with the most closely related agents.

Agent Source review Form / cost Stars Updated Language Full support on
Oumi This agent 63 · Some gaps CLIFree + model costs ★ 9.4k 1d ago Python Claude Code · OpenAI API
DeepEval 69 · Some gaps CLIFree + model costs ★ 20k 1d ago Python OpenAI API · Claude API
AI Agents Projects & Tutorials Insufficient evidence Library / SDKFree + model costs ★ 3k 14d ago Jupyter Notebook OpenAI API · Claude API
LLM-in-Sandbox 47 · Major gaps CLIFree + model costs ★ 249 3mo ago Python OpenAI API · Claude API

How does FollowAgents rate this agent?

FollowAgents source review · FARS-2.1
Some gaps
63/ 100 5-point scale 3.2 / 5
Trust 12/29
Reliability 6/14
Adaptability 14/18
Convention 14/18
Effectiveness 12/13
Verifiability 5/8
Why each dimension lost points
Trust12 / 29 · 2.1/5

The code exposes CLI, cloud job launch, deployment, and MCP interfaces, indicating local, network, and cloud effects. The quick installer pipes curl output to a shell, while the packet does not explain the script contents, granular permissions, pre-execution confirmation, or a general recovery path; least privilege, confirmation, data-flow transparency, sensitive-data handling, external effects, and rollback therefore score 1. Dependencies have many version bounds and comments documenting compatibility constraints, supporting a 2 for dependency security. The Apache license and source copyright headers provide attribution, but publisher identity remains unknown as stipulated, so source attribution scores 2.

Reliability6 / 14 · 2.1/5

The README claims production-grade reliability while also describing the project as beta. The installation guide requires Python 3.9+, whereas pyproject.toml requires 3.10 through 3.14; this concrete inconsistency limits self-consistency to 1. Dependencies have numerous constraints and optional groups, but the packet provides no lockfile or installation-success evidence, so availability scores 1. Troubleshooting gives concrete symptoms and remedies for Windows, Intel Macs, OOM, cloud mounts, and inference, earning 2 for failure messages.

Adaptability14 / 18 · 3.9/5

Documentation covers researchers and developers across local machines, clusters, and multiple cloud platforms, with training, evaluation, inference, and agent tool environments; audience and scenarios score 3. The README says configurations are examples rather than an exhaustive support list and notes that advanced features may change, but the packet does not systematically define boundaries, so capability boundaries score 2. CLI entry points, command help, and configuration examples help users identify applicable tasks, earning 2 for trigger precision. The docs explicitly exclude Intel Macs, recommend WSL for Windows, and describe GPU and cloud-specific optional dependencies, earning 2 for environment fit.

Convention14 / 18 · 3.9/5

The README, installation guide, quickstart, troubleshooting page, and project metadata form a clear information structure, scoring 3. Installation methods, Python requirements, optional dependencies, and a verification command are documented, but the quick installer script's contents are not shown, so install notes score 2. CLI and package names consistently center on Oumi/oumi, with some variation between OSS wording, project naming, and command presentation; naming stability scores 2. Tutorials, recipes, FAQ material, and concrete troubleshooting cases are extensive, scoring 3. The packet states beta status, Intel Mac limitations, Windows guidance, and some dependency compatibility issues, but coverage is not comprehensive, so known limitations score 2. A full Apache-2.0 license is included, scoring 3. The README has dated release and feature updates, but the sample mixes very recent future-dated entries with older updates and shows no revision policy, so versioning and changelog score 2. Security reporting and contribution paths are described, but maintenance responsibility is attributed mainly to a team/community without a verifiable named contact, scoring 2.

Effectiveness12 / 13 · 4.6/5

A unified CLI, configuration overrides, extensive recipes, and examples for training, evaluation, inference, and cloud jobs make outputs directly usable, scoring 3. The packet supports substantial added value through integrated workflows for open-model fine-tuning, tool environments, RL, evaluation, and deployment, scoring 3. Cost-benefit scores 2: the docs warn that cloud services can be expensive and advise tearing down clusters, and provide a small-model quickstart, but no systematic resource or cost estimates.

Verifiability5 / 8 · 3.1/5

README feature claims are supported in part by CLI registration, configuration paths, package metadata, and documentation examples, but broad model-compatibility and production-quality claims lack itemized evidence in this sample; claim traceability scores 2. The README, quickstart, installation guide, pyproject, and CLI entry points corroborate one another in several areas, while also revealing a Python-version mismatch, earning 2 for cross-source corroboration. The material distinguishes beta status, an experimental installer, examples, and exhaustive support lists, but does not consistently separate promotional capability claims from demonstrated facts, so fact-inference separation scores 2.

Risks and how to mitigate them
  • The quick-install command pipes a remote script directly to a shell; the reviewed packet does not include that script, so inspect its source and actions before use.
  • The installation guide's Python 3.9+ requirement conflicts with project metadata requiring 3.10 through 3.14; choosing an environment from the metadata is safer.
  • Cloud launch and deployment may incur charges and involve credentials or file mounts; the quickstart explicitly warns about cloud costs and that cluster teardown deletes remote data.
Evidence confidence: Low Reviewed Nov 08, 2026 Reviewed revision 4c0aa76443dd New commits since this review; the score may not cover them
See the full review method →
Maintain this project? Add the FARS badge to your README

The badge shows the current review score and verdict and links back to this page. It refreshes about once an hour after a re-review.

FARS

[![FARS review](https://followagents.com/badge/oumi.svg)](https://followagents.com/en/agents/oumi)
<a href="https://followagents.com/en/agents/oumi"><img src="https://followagents.com/badge/oumi.svg" alt="FARS review"></a>

Recent releases

Latest entries from the project's GitHub Releases. Notes are quoted from upstream as written, not rewritten or tested by FollowAgents.

  1. v0.9
    Release notes excerpt
    Oumi v0.9 Release Notes This release makes Oumi an end-to-end stack for agentic, tool-using models: you can now train on tool-calling data, execute tool calls against real environments (databases, HTTP endpoints, deterministic lookups, and model-simulated tools), synthesize multi-turn tool conversations, and run GRPO/RL over any of those environments with verl. It also adds a multi-criteria RubricJudge, partial-failure-tolerant inference/synthesis/judging, a Modal GPU launcher and SkyPilot-routed Slurm, and new Gemma 4 recipes. Highlights 🛠️ Tool-use training …
    View on GitHub ↗
  2. v0.8
    Release notes excerpt
    Oumi v0.8 Release Notes This release lands the new oumi deploy CLI for shipping models to dedicated inference endpoints, an oumi-mcp server for MCP-capable assistants, batch-API parity across hosted providers, and a major dependency push to Transformersv5 / TRL 0.30+ / vLLM` 0.20+. Highlights oumi deploy — new CLI for dedicated inference endpoints …
    View on GitHub ↗
  3. v0.7
    Release notes excerpt
    Oumi 0.7 Release ✨ Highlights This release brings major platform upgrades (Python 3.14, PyTorch 2.9), new inference engines, rule-based evaluation judges, and significant CLI/documentation improvements. --- 🚀 New Features Inference - Fireworks inference engine - New backend for Fireworks AI (#2158) ```yaml Fireworks example (set FIREWORKS_API_KEY env var) model: model_name: "accounts/fireworks/models/llama4-maverick-instruct-basic" engine: FIREWORKS ``` ```bash oumi infer -i -c configs/recipes/llama4/inference/maverick_instruct_fireworks_infer.yaml ``` …
    View on GitHub ↗

FAQ

Do I need a GPU to use Oumi?
Training and GPU-backed inference need one; the install docs offer uv pip install 'oumi[gpu]' and a Docker path with --gpus all. Inference routed to commercial APIs can run without local GPU.
Does Oumi itself cost money?
Oumi is released under Apache-2.0 at no cost. You still pay for any commercial model APIs you call (OpenAI, Anthropic, Vertex AI, Together, Fireworks) and for cloud compute such as AWS, Azure, or GCP.
Can it train tool-calling agents?
Yes. The README documents tool-calling SFT/DPO, executable tool environments (database, HTTP endpoint, lookup, simulated), GRPO reinforcement learning via verl over those environments, and synthesis of multi-turn tool-use conversations.
How do I serve a model in production?
Use vLLM or SGLang for inference, or publish to dedicated inference endpoints on Fireworks and Parasail with oumi deploy. Cluster jobs are submitted with oumi launch up to AWS, Azure, GCP, or Lambda.
Which models are supported?
The README lists Qwen (Qwen3, Qwen3.5, Qwen3-VL, Qwen3-Coder, QwQ), Gemma 3/4, Llama 3.x/4, DeepSeek R1, GLM, Phi, OLMo 3, gpt-oss, and Falcon, and notes that the Transformers integration lets you use many more architectures.
Is Oumi open source?
Yes. The source code is on GitHub (oumi-ai/oumi) under the Apache-2.0 licence.
Is Oumi safe to use?
FollowAgents' FARS review (Nov 08, 2026) scored Oumi 12/29 on trust and safety. None of the eight safety controls the review checks for was missing outright. The repository has changed since that review.
Is Oumi still maintained?
Yes. The latest commit on GitHub is from 2026-11-20, within the last 30 days. The latest release is v0.9 (2026-09-16).
Who makes Oumi?
Oumi is published on GitHub by the account oumi-ai. FollowAgents has not verified which person or company is behind that account.
What is Oumi's official website?
The project's GitHub repository lists https://oumi.ai as its website. That link is set by the repository owner; the source code itself is on GitHub (oumi-ai/oumi).
Was this page helpful?
View on GitHub ↗ Install ↓

Compare agents like this one

The same FARS review applied across the shortlist this agent qualifies for.

Related concepts