Adala
Train iterative data-labeling and processing skills from ground-truth datasets.
- Source repo
- HumanSignal/Adala
- Stars
- ★ 1.6k
- Last updated
- 5d ago
- License
- Apache-2.0
- Primary language
- Python
- FA score
- 42/100 · Major gaps
At a glance
- Works with
- Portable with changesOpenAI API
- You'll need
- Typical use
- A data scientist with a labeled text table wants to train and run sentiment classification in a notebook.
- Main limitation
- The examples require an OpenAI or OpenRouter API key, so execution depends on external model services and network access.
- Source review
- 42/100 · Major gaps 2 safety controls not found
What does this agent do, and when should you use it?
Adala is a Python framework for data-processing agents, with a stated emphasis on data-labeling tasks. An Agent learns and applies skills in an environment defined by a ground-truth dataset; the README uses StaticEnvironment and describes a runtime as an LLM. Its quickstart combines ClassificationSkill, OpenAIChatRuntime, and pandas DataFrames, then calls agent.learn() before agent.run() produces predictions for new rows. The documented examples cover classification, summarization, question answering, translation, text generation, skill sequences, ontology inference, and GSM8K math reasoning. The evidenced interface is a Python-library and notebook workflow; CLI, REST API, and vision/multimodal skills remain roadmap items.
A user passes a labeled pandas DataFrame to StaticEnvironment(df=train_df), then defines a ClassificationSkill with a name, instructions, labels, input_template, and output_template. Agent associates that skill with OpenAIChatRuntime instances in runtimes and teacher_runtimes; the quickstart uses gpt-4o and calls agent.learn(learning_iterations=3, accuracy_threshold=0.95). It then reads a new DataFrame through agent.run(test_df) and returns predictions. The same OpenAIChatRuntime can be configured with base_url=https://openrouter.ai/api/v1 and provider='Custom' to call anthropic/claude-3.5-haiku through OpenRouter.
- A data scientist with a labeled text table wants to train and run sentiment classification in a notebook.
- An AI engineer needs modular skills such as ClassificationSkill in a pandas DataFrame processing pipeline.
- A researcher wants one framework for experiments in classification, summarization, question answering, translation, or sequences of skills.
- A team with ground-truth examples wants to control learning using iteration counts and an accuracy threshold.
- A developer using OpenRouter wants to run the documented Claude 3.5 Haiku configuration through an OpenAI-compatible endpoint.
How do you install or deploy this agent?
Install with pip install adala. To install the repository version: pip install git+https://github.com/HumanSignal/Adala.git. For development: git clone https://github.com/HumanSignal/Adala.git, change into Adala/, then run poetry install. Before the OpenAI quickstart, set export OPENAI_API_KEY='your-openai-api-key'; for the OpenRouter example, set export OPENROUTER_API_KEY='your-openrouter-api-key'.
How do you use this agent?
Import Agent, StaticEnvironment, ClassificationSkill, and OpenAIChatRuntime. Create a StaticEnvironment from a training DataFrame with text and sentiment columns, then construct Agent(environment=..., skills=ClassificationSkill(name='sentiment', instructions='Label text as positive, negative or neutral.', labels=['Positive','Negative','Neutral'], input_template='Text: {text}', output_template='Sentiment: {sentiment}'), runtimes={'openai': OpenAIChatRuntime(model='gpt-4o')}, teacher_runtimes={'default': OpenAIChatRuntime(model='gpt-4o')}, default_runtime='openai'). Run agent.learn(learning_iterations=3, accuracy_threshold=0.95), then generate output with predictions = agent.run(test_df) for a new DataFrame containing text. For OpenRouter, set the runtime base_url to https://openrouter.ai/api/v1, use anthropic/claude-3.5-haiku, and provide OPENROUTER_API_KEY.
What are this agent's strengths and limitations?
- Ground-truth data explicitly defines the agent environment, anchoring the learning flow in labeled examples.
- Skills, execution runtimes, and teacher runtimes are configured separately; the README describes a skill being deployable across multiple runtimes.
- The documented pandas and notebook workflow is paired with examples for classification, summarization, question answering, translation, and text generation.
- The README provides both an OpenAI configuration and a concrete OpenRouter configuration for Claude 3.5 Haiku via an OpenAI-compatible API.
- The examples require an OpenAI or OpenRouter API key, so execution depends on external model services and network access.
- Only OpenAIChatRuntime and an OpenRouter configuration are evidenced; support scope and trade-offs for other providers are not documented.
- CLI utilities, REST API access, and vision/multimodal skills are roadmap items rather than established capabilities.
- Quality depends on the supplied ground-truth dataset; the material provides no dataset-size guidance, benchmark results, or failure-recovery procedure.
How does this agent compare with similar options?
The README presents direct use of OpenAIChatRuntime alongside an OpenRouter route to Claude 3.5 Haiku through an OpenAI-compatible API. The former uses OPENAI_API_KEY with model='gpt-4o'; the latter requires a custom base_url, provider='Custom', and OPENROUTER_API_KEY.
Key facts side by side with the most closely related agents.
| Agent | Source review | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|
| Adala This agent | 42 · Major gaps | ★ 1.6k | 5d ago | Python | OpenAI API |
| EDSL (Expected Parrot Domain-Specific Language) | 63 · Some gaps | ★ 497 | 5d ago | Python | OpenAI API · Claude API |
| K-Dense BYOK | 49 · Major gaps | ★ 1.2k | 12d ago | TypeScript | ChatGPT · Codex · Claude.ai |
| MiroFlow Research Agent | 53 · Major gaps | ★ 3.1k | 7mo ago | Python | — |
How does FollowAgents rate this agent?
Why each dimension lost points
Evidence shows: framework requires user API keys (e.g., OPENAI_API_KEY), but no least-privilege or sandboxing; no user confirmation step; data flow transparency limited to documentation; sensitive data handling not specified; many dependencies, some from direct GitHub links, posing supply chain risk; external effects not specified; no rollback mechanism; source attribution only via author info. Deductions: lack of security mechanisms and transparency.
Evidence shows: README and pyproject.toml are consistent, but version numbers differ (README doesn't mention version, pyproject is 0.0.4dev); dependencies are pinned, but some from direct GitHub links, availability risk; failure messages not documented. Deductions: dependency availability risk, opaque failure handling.
Evidence shows: README identifies target users (AI engineers, ML researchers, etc.) and use cases; capability boundaries via skill types; trigger precision not specified; environment fit supports Python 3.10+ and multiple LLM providers. Deductions: trigger precision not specified.
Evidence shows: README structure clear, includes installation, quickstart, examples; install notes detailed; naming stable (adala); examples abundant; known limitations not specified; license Apache-2.0; version number exists but no changelog; maintenance responsibility not specified. Deductions: missing known limitations and changelog.
Evidence shows: output is DataFrame, directly usable; marginal value high, provides autonomous learning and multi-skill support; cost-benefit not quantified. Deductions: cost-benefit not quantified.
Evidence shows: README claims partially supported by examples, but no independent verification; test cassettes exist but not independently run; facts and inferences not clearly separated. Deductions: lack of independent verification and fact/inference separation.
- Not found in source: confirmation before actingTurn on (or add) a confirmation step before it acts, and try it in a sandbox or test environment before real data.
- Not found in source: rollback or recovery pathBack up first, or work on a git branch or snapshot, so its changes can be undone.
- Dependencies include packages from direct GitHub links, posing supply chain risk; review dependency sources.
- Framework requires user API keys but lacks least-privilege or sandboxing; use with caution.
- Version number inconsistent with README and no changelog; be aware of compatibility on upgrade.
FAQ
Which credentials are required?
OPENAI_API_KEY. The documented OpenRouter route for Claude 3.5 Haiku requires OPENROUTER_API_KEY.Can it process unlabeled data?
agent.run(test_df) to generate predictions. The learning environment itself is defined by a ground-truth dataset.