ActiveHarness for Ruby
Orchestrate observable, cost-tracked LLM workflows with retries and fallbacks in Ruby applications.
- Source repo
- the-teacher/active_harness
- Stars
- ★ 111
- Last updated
- 29d ago
- License
- MIT
- Primary language
- Ruby
- FA score
- 63/100 · Some gaps
At a glance
- How it runs
- Works with
- Universal · cross-platformOpenAI API · Claude API
- Cost
- Free software; you pay for model usage
- Setup effort
- Medium · a few setup steps
- You'll need
- Typical use
- Rails teams combining deterministic Ruby work, such as fetching a web page, with model-based extraction in one Pipeline.
- Not a fit if
- Teams that require a stable, fixed API
- Maintenance
- Commits in the last 30 dayslast commit 2026-10-23
- Source review
- 63/100 · Some gaps
What does this agent do, and when should you use it?
ActiveHarness is an LLM framework for Ruby and Rails, and it also works in plain Ruby. Its components include Prompt, Request, Pipeline, Tribunal, Memory, Tool, and Agent; since v0.3.0, Request means a single model call, while Agent means a tool-using loop. Requests can define model chains, retries, fallbacks, and lifecycle hooks, while pipelines combine sequential Ruby steps and model calls. Tribunals run requests in parallel and produce a Verdict based on agreement. The framework includes token, cost, and timing tracking, event tracing, and SSE streaming. The README labels the API a work in progress and says it may change between versions.
Install the active_harness gem in a Ruby or Rails project and organize AI code under app/ai. Configure Request objects with model calls, chains, fallbacks, retries, and hooks; use Pipeline to run Ruby operations and Requests in sequence or compose nested workflows; use Tribunal to run multiple Requests concurrently and produce a Verdict. An Agent can ask a model, execute only its listed Tools, check a GoalCheck, and stop at configured limits. Memory can store conversation history in JSON, SQLite, or PostgreSQL and inject it into prompts. During execution, the framework can track tokens, cost, and time, emit OpenTelemetry traces, and stream output over SSE.
- Rails teams combining deterministic Ruby work, such as fetching a web page, with model-based extraction in one Pipeline.
- Ruby developers who want a configured backup model and retries when a model call fails.
- Application teams that need parallel model responses and a Verdict based on majority, unanimity, or custom rules.
- Developers building tool-using loops where the model may call only explicitly listed actions and a GoalCheck evaluates completion.
- Ruby applications that need to preserve conversation history and reuse it in later prompts.
- Teams that need token cost, execution timing, and trace visibility across requests, pipelines, and tribunals.
How do you install or deploy this agent?
The README says the gem works with Rails and plain Ruby and documents Rails generators, but the supplied material does not include the gem installation command, required Ruby version, complete provider configuration, or credential names. The Rails setup generator is:
rails g active_harness:installIt creates a working example FilesAgent with tools, a goal check, and tracing concerns. The README also lists rails g active_harness:agent, :tool, and :goal. Plain Ruby installation and initialization steps are not specified in the supplied material.
How do you use this agent?
Build on the example created by the Rails install generator: define an Agent, specify its Request, allowed Tools, and Goal, then call it:
class FilesAgent < ActiveHarness::Agent
max_turns 8
request FilesRequest
tools LsTool, ReadFileTool
goal AnswerPresentGoal
end
agent = FilesAgent.call(input: "What is in the storage directory?")
agent.output
agent.stop_reason
puts agent.traceModel access requires a configured provider. The README documents tool calling with OpenAI-compatible providers and Anthropic, but the supplied material does not list credential names or a complete configuration example. Retries, fallbacks, tracing, and cost controls can be configured in Requests, Pipelines, or Tribunals as needed.
What are this agent's strengths and limitations?
- Pipelines combine ordinary Ruby operations and LLM Requests, keeping deterministic work and model tasks in one workflow.
- Requests support model chains, automatic fallbacks, retries, and lifecycle hooks for explicit failure handling.
- Tribunals run requests in parallel and support unanimous, majority, or custom Verdict strategies.
- Includes cost and timing tracking, OpenTelemetry tracing, SSE streaming, and JSON, SQLite, or PostgreSQL conversation memory.
- Agents can call only explicitly listed Tools; Tools support typed parameters, ordered validation steps, and default-deny behavior.
- The project is explicitly a work in progress, and its API may change between versions.
- The Agent abstraction is a first iteration; the v0.3.0 rename from ActiveHarness::Agent to ActiveHarness::Request may require code updates.
- Model use requires provider configuration and network access; the supplied material does not specify a Ruby version or credential setup.
- The documented tool-calling scope names OpenAI-compatible providers and Anthropic, so support for tool calling through every provider is not established.
How does this agent compare with similar options?
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| ActiveHarness for Ruby This agent | 63 · Some gaps | FrameworkFree + model costs | ★ 111 | 29d ago | Ruby | OpenAI API · Claude API |
| Adaline Gateway | 54 · Major gaps | Library / SDKFree + model costs | ★ 608 | 3mo ago | TypeScript | OpenAI API · Claude API |
| Instructor | 64 · Some gaps | Library / SDKFree + model costs | ★ 14k | 2d ago | Python | OpenAI API · Claude API |
| Agent Adapter | 72 · Some gaps | Library / SDKFree + model costs | ★ 13 | 3d ago | Python | OpenAI API · Claude API |
How does FollowAgents rate this agent?
Why each dimension lost points
The docs show agents can call only explicitly listed tools, with default-deny behavior, parameter steps, and safe refusals; the sample file reader is limited to storage/, supporting meaningful least-privilege and external-effect controls. API keys can be configured through ENV or Rails credentials, and tracing can record calls. However, there is no general per-action user-confirmation flow or complete account of data movement among prompts, memory, logs, and providers, nor a sensitive-data retention policy. Expected errors and safe messages are described, but dependency security evidence is thin. The installer changes application files and routes; it says existing files are skipped, but gives no undo procedure. Attribution appears in the README and license, while publisher identity and maintenance/update path remain unclear.
The README says the single-call abstraction was renamed from ActiveHarness::Agent to Request and that Agent now means the tool-calling loop. The supplied CLAUDE.md still says version 0.2.x and describes the older product model, creating a conflict with the README's v0.3.0 naming note. The docs identify concurrent-ruby for parallel tribunals and disclose that pricing comes from models.dev and returns an empty list if fetching and caching fail. Retries, fallbacks, stop errors, and partial failures are described, though this static evidence does not establish runtime behavior.
The intended users and scenarios are concrete: Rails and plain Ruby, requests, pipelines, tribunals, agents, tools, memory, streaming, and multiple providers. The docs specify agent tool lists and turn/cost limits, as well as image and audio provider scope. Broad extension points such as custom providers and external LLM backends remain, and their permission boundaries are not fully discussed. The README presents a clear purpose, and execution is initiated through user-defined classes and DSL calls rather than an evidenced implicit trigger. Rails autoloading, generators, and plain Ruby setup are all documented.
The README provides a clear structure, concept index, and feature-organized docs. Installation, Rails/plain Ruby configuration, generators, and many examples are covered; the MIT license text is complete. The README labels the project work in progress and warns that the API may change, but the supplied material has no full changelog or clear version-support policy. The naming migration also conflicts with the older version description in CLAUDE.md. Attribution is visible in the README and license, but a responsible maintainer and update channel are not clearly identified. Examples cover common workflows, though no dedicated FAQ is shown.
The framework combines retries, fallbacks, cost tracking, event hooks, pipelines, and constrained tools in a Ruby/Rails DSL, potentially reducing custom infrastructure work. The docs provide concrete structures and usage examples. The value claims are mainly self-described, without comparative evaluation or external validation. Multiple providers, parallel tribunals, and retries can add spend and configuration complexity; the supplied cost-control material does not fully quantify those tradeoffs, so these criteria receive moderate scores.
The README and installation/configuration docs tie many capability claims to API examples, settings, and stated limits. The token usage doc also says some providers omit usage and streaming calls return nil usage. There is some corroboration across materials, but the supplied CLAUDE.md conflicts with the README on version and abstraction naming, reducing consistency; promotional outcome claims lack external evidence. The docs generally distinguish examples from configuration facts, but they cannot establish implementation behavior or results.
- This is a static review with low confidence; no code was executed, so it does not establish safety, correctness, or test quality.
- CLAUDE.md's 0.2.x version and older Agent naming conflict with the README's v0.3.0 note; verify the actual API and version history for the target revision.
- Before deployment, review tool implementations and sensitive-data handling in prompts, memory, and tracing, and inspect the routes and files written by the installer.
FAQ
Is ActiveHarness::Request an autonomous agent?
Can a workflow combine Ruby code with model calls?
Which providers support tool calling?
How can I track model-call costs and execution?
What should I know before adopting it?
Is ActiveHarness for Ruby open source?
Is ActiveHarness for Ruby safe to use?
Is ActiveHarness for Ruby still maintained?
Who makes ActiveHarness for Ruby?
Thanks — we use this to improve the page.