Adala
Train iterative data-labeling and processing skills from ground-truth datasets.
- Source repo
- HumanSignal/Adala
- Stars
- ★ 1.6k
- Last updated
- 2d ago
- License
- Apache-2.0
- Primary language
- Python
- FA score
- 46/100 · Major gaps
At a glance
- How it runs
- Works with
- Portable with changesOpenAI API
- Cost
- Free software; you pay for model usage
- Setup effort
- Low · running in minutes
- You'll need
- Typical use
- A data scientist with a labeled text table wants to train and run sentiment classification in a notebook.
- Not a fit if
- Users needing a ready CLI or REST API; both are still roadmap items
- Teams needing image or multimodal labeling; vision skills are not implemented
- Source review
- 46/100 · Major gaps 1 safety controls not found
What does this agent do, and when should you use it?
Adala is a Python framework for data-processing agents, with a stated emphasis on data-labeling tasks. An Agent learns and applies skills in an environment defined by a ground-truth dataset; the README uses StaticEnvironment and describes a runtime as an LLM. Its quickstart combines ClassificationSkill, OpenAIChatRuntime, and pandas DataFrames, then calls agent.learn() before agent.run() produces predictions for new rows. The documented examples cover classification, summarization, question answering, translation, text generation, skill sequences, ontology inference, and GSM8K math reasoning. The evidenced interface is a Python-library and notebook workflow; CLI, REST API, and vision/multimodal skills remain roadmap items.
A user passes a labeled pandas DataFrame to StaticEnvironment(df=train_df), then defines a ClassificationSkill with a name, instructions, labels, input_template, and output_template. Agent associates that skill with OpenAIChatRuntime instances in runtimes and teacher_runtimes; the quickstart uses gpt-4o and calls agent.learn(learning_iterations=3, accuracy_threshold=0.95). It then reads a new DataFrame through agent.run(test_df) and returns predictions. The same OpenAIChatRuntime can be configured with base_url=https://openrouter.ai/api/v1 and provider='Custom' to call anthropic/claude-3.5-haiku through OpenRouter.
- A data scientist with a labeled text table wants to train and run sentiment classification in a notebook.
- An AI engineer needs modular skills such as ClassificationSkill in a pandas DataFrame processing pipeline.
- A researcher wants one framework for experiments in classification, summarization, question answering, translation, or sequences of skills.
- A team with ground-truth examples wants to control learning using iteration counts and an accuracy threshold.
- A developer using OpenRouter wants to run the documented Claude 3.5 Haiku configuration through an OpenAI-compatible endpoint.
How do you install or deploy this agent?
Install with pip install adala. To install the repository version: pip install git+https://github.com/HumanSignal/Adala.git. For development: git clone https://github.com/HumanSignal/Adala.git, change into Adala/, then run poetry install. Before the OpenAI quickstart, set export OPENAI_API_KEY='your-openai-api-key'; for the OpenRouter example, set export OPENROUTER_API_KEY='your-openrouter-api-key'.
How do you use this agent?
Import Agent, StaticEnvironment, ClassificationSkill, and OpenAIChatRuntime. Create a StaticEnvironment from a training DataFrame with text and sentiment columns, then construct Agent(environment=..., skills=ClassificationSkill(name='sentiment', instructions='Label text as positive, negative or neutral.', labels=['Positive','Negative','Neutral'], input_template='Text: {text}', output_template='Sentiment: {sentiment}'), runtimes={'openai': OpenAIChatRuntime(model='gpt-4o')}, teacher_runtimes={'default': OpenAIChatRuntime(model='gpt-4o')}, default_runtime='openai'). Run agent.learn(learning_iterations=3, accuracy_threshold=0.95), then generate output with predictions = agent.run(test_df) for a new DataFrame containing text. For OpenRouter, set the runtime base_url to https://openrouter.ai/api/v1, use anthropic/claude-3.5-haiku, and provide OPENROUTER_API_KEY.
What are this agent's strengths and limitations?
- Ground-truth data explicitly defines the agent environment, anchoring the learning flow in labeled examples.
- Skills, execution runtimes, and teacher runtimes are configured separately; the README describes a skill being deployable across multiple runtimes.
- The documented pandas and notebook workflow is paired with examples for classification, summarization, question answering, translation, and text generation.
- The README provides both an OpenAI configuration and a concrete OpenRouter configuration for Claude 3.5 Haiku via an OpenAI-compatible API.
- The examples require an OpenAI or OpenRouter API key, so execution depends on external model services and network access.
- Only OpenAIChatRuntime and an OpenRouter configuration are evidenced; support scope and trade-offs for other providers are not documented.
- CLI utilities, REST API access, and vision/multimodal skills are roadmap items rather than established capabilities.
- Quality depends on the supplied ground-truth dataset; the material provides no dataset-size guidance, benchmark results, or failure-recovery procedure.
How does this agent compare with similar options?
The README presents direct use of OpenAIChatRuntime alongside an OpenRouter route to Claude 3.5 Haiku through an OpenAI-compatible API. The former uses OPENAI_API_KEY with model='gpt-4o'; the latter requires a custom base_url, provider='Custom', and OPENROUTER_API_KEY.
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| Adala This agent | 46 · Major gaps | Library / SDKFree + model costs | ★ 1.6k | 2d ago | Python | OpenAI API |
| EDSL (Expected Parrot Domain-Specific Language) | 60 · Some gaps | Library / SDKFree + model costs | ★ 515 | today | Python | OpenAI API · Claude API |
| K-Dense BYOK | 73 · Some gaps | Desktop appFree + model costs | ★ 1.4k | 23d ago | TypeScript | ChatGPT · Codex · Claude.ai |
| Potato Annotation Platform | 67 · Some gaps | CLIFree | ★ 443 | 3d ago | Python | OpenAI API · Claude API |
How does FollowAgents rate this agent?
Why each dimension lost points
The README shows API key setup through environment variables and explicitly demonstrates learn and run calls; server/app.py also shows CORS configured for all origins, methods, and headers. External LLM data flow is visible in the README and recorded requests, but the packet does not sufficiently explain retention, sensitive data policy, or controls for individual external effects. Evidence for user confirmation and rollback is limited. The dependency manifest lists many broad version ranges and a remote SDK archive, with no visible security review evidence. pyproject includes an author field, but publisher identity remains unknown under the prompt, so attribution receives only limited credit.
The README's self-learning and reliability claims lack supporting implementation details or metrics in this packet, and the model and SDK versions shown in recorded tests differ from the current dependency declarations, so self-consistency scores low. Dependencies include a mixture of ranges, pins, and a remote archive, but no lockfile or availability guarantees are shown. server/app.py returns HTTP errors and explanatory messages for connection validation failures, supporting ordinary diagnosis; broader recovery paths are not shown. No score is reduced for matters that require execution to verify.
The README identifies AI engineers, researchers, data scientists, and educators as audiences, and gives classification, summarization, question-answering, and translation scenarios. Skills, environments, and runtimes are configurable, but capability boundaries are sparsely documented; claims about autonomous learning and output reliability are broad without clear scope or failure conditions. Python, pandas, OpenAI, and OpenAI-compatible service examples show some environment fit; Python version information differs between the README and pyproject.
The README has a clear overview, installation, prerequisites, quickstart, examples, roadmap, and support sections. Installation steps are concise, though the main dependency set includes substantial server components and deployment configuration is limited. Naming and interface examples are recognizable, but the README's gpt-4o differs from gpt-3.5-turbo and gpt-4o-mini in recorded tests. Examples and support entry points are substantial; the roadmap gives some limitations, but there is no systematic limitations section or FAQ. The LICENSE presents the Apache 2.0 terms in full and merits full marks. The version field is 0.0.4dev, but no changelog is shown; the author field provides a limited maintenance clue while ongoing responsibility and update ownership remain unclear.
The framework offers reusable combinations of skills, environments, and runtimes, with structured classification output in the examples, supporting data-labeling use. The README also shows tabular results and varied tasks. Iterative learning requires external LLM calls and may incur cost and latency, but the packet provides no cost controls, budget guidance, or quantified value, so cost-benefit receives a low score.
Some claims can be compared across the README examples, pyproject dependencies, server/app.py, and recorded tests; the recordings show input text in requests sent to OpenAI, supporting the external data-flow assessment. Many claims about reliability, trustworthiness, and autonomous learning have no accompanying measurement evidence, and recordings do not establish execution results for the reviewed revision. The documentation does not consistently distinguish claims from evidence, and the packet is a sample, so only partial credit is warranted.
- Not found in source: rollback or recovery pathBack up first, or work on a git branch or snapshot, so its changes can be undone.
- Examples send input text to the configured external LLM service; confirm the provider, data handling terms, and organizational policy before submitting sensitive data.
- server/app.py allows wildcard CORS origins, methods, and headers while allowing credentials; tighten this configuration for the deployment environment.
- The dependency manifest includes broad version ranges and a remote SDK archive; this static packet does not establish dependency security status or execution results.
Maintain this project? Add the FARS badge to your README
[](https://followagents.com/en/agents/adala)<a href="https://followagents.com/en/agents/adala"><img src="https://followagents.com/badge/adala.svg" alt="FARS review"></a>Recent releases
Latest entries from the project's GitHub Releases. Notes are quoted from upstream as written, not rewritten or tested by FollowAgents.
-
0.0.4 Adala v0.0.4 - Introducing a New Code Environment and Enhanced Automated Prompt Engineering
Release notes excerpt
Major updates * New Prompt Engineering Algorithm: The upgraded prompt engineering algorithm implements the [PE2 method](https://arxiv.org/abs/2311.05661), which significantly enhances the quality of created prompts. * [Benchmark Update](https://github.com/HumanSignal/Adala/blob/master/examples/gsm8k_test.ipynb): Demonstrates a considerable advancement over the zero-shot setup, with a 20% improvement demonstrated in the GSM-8k benchmark. …View on GitHub ↗ -
0.0.3 Adala v0.0.3 – Extending agent’s feedback and control; core skills update; parallel execution, and more.
Release notes excerpt
Adala 0.0.3 introduces key updates enhancing AI agent feedback and control. Notable features include the extension of feedback collection to prediction methods and new console and web environments for improved feedback integration. We've decoupled Runtimes from Skills, allowing for more flexible skill definitions and controlled outputs. This update also refines basic skill classes into three core classes: TransformSkill, AnalysisSkill, and SynthesisSkill, enabling better composition and class inheritance. Additionally, ParallelSkillSet allows for concurrent skill execution, increasing effi …View on GitHub ↗ -
0.0.2 Adala v0.0.2 – Introducing Skill Sets!
Release notes excerpt
Focusing on Complex Tasks We're introducing skill sets, a concept focused on nuanced and sophisticated data processing that reflects the complex nature of real-world problem-solving. In practical scenarios, skills are rarely isolated and often require a multifaceted approach where the output of one skill feeds into the input of another. …View on GitHub ↗
FAQ
Which credentials are required?
OPENAI_API_KEY. The documented OpenRouter route for Claude 3.5 Haiku requires OPENROUTER_API_KEY.Can it process unlabeled data?
agent.run(test_df) to generate predictions. The learning environment itself is defined by a ground-truth dataset.Does it already expose a server, REST API, or CLI?
Which tasks are documented?
Thanks — we use this to improve the page.