EDSL (Expected Parrot Domain-Specific Language)
A Python DSL for designing, running, and analyzing AI-powered surveys and experiments with large numbers of AI agents and LLMs, simulating social science and market research.
Data flows are documented (remote cache, key management pages) and the MIT license names three authors; however, remote inference and automatic upload of results to the Expected Parrot server appear to be defaults with no explicit user-confirmation or opt-out explanation in the README (tests reveal disable_remote_inference flags), so deducted. Survey data goes to a third-party server; sensitive-data handling is a single 'securely manage keys' claim — thin evidence. Dependencies are range-pinned and modularized as optional extras, but no lockfile or audit evidence. The remote cache aids reproducibility but there is no documented way to revoke or delete uploaded results.
Unit tests under tests/agents cover serialization, custom exceptions, JSONL round-trips, etc.; custom exception classes (AgentNameError, etc.) show designed failure messages; CI validates builds and imports across three OSes and four Python versions. Deducted because this is a static review with no executed tests, and the self-hosted coopr test job only triggers on commit messages containing 'coopr', leaving coverage questionable.
Audience and scenarios are well described (social science, market research, data labeling) with examples across question types and models; the README explicitly warns that AI-simulated responses reflect statistical patterns, not real demographic opinions — good boundary disclosure. However, README states Python 3.9–3.13 while pyproject requires >=3.10,<3.14 — a self-contradiction in environment requirements, deducted.
MIT license with named copyright holders; install instructions, docs links, and a starter tutorial are present; examples include reproducible outputs with Coop links, earning high marks. Deducted for no visible CHANGELOG in the evidence and a development version string (1.0.8.dev1); maintenance responsibility relies on external sites and Discord with thin in-repo governance.
Outputs are structured datasets with built-in select/visualization/HTML export (tests verify to_html/save_html/JSONL) — usability evidence is strong. Marginal value over direct LLM API use lies in declarative question types, piping, caching, and the collaboration platform, but the core value depends on a proprietary server ecosystem; cost-benefit documentation mentions free credits and usage tracking only, so deducted.
Each feature example in the README links to a Coop artifact for traceability; README, pyproject, and tests corroborate each other (except the Python version mismatch); docs clearly separate 'AI statistical simulation' from 'real human data' — good fact/inference separation. Deducted because all shown outputs are static and cannot be verified without execution.
- Default behavior appears to upload survey data and prompts to the Expected Parrot remote server and cache them; verify how to disable remote inference/caching and how to delete uploaded data before use.
- README claims Python 3.9 support but pyproject requires >=3.10; treat pyproject as authoritative.
- Current version is 1.0.8.dev1 (development) with no in-repo changelog; use caution in production.
- Agent-simulated answers are statistical patterns, not evidence of real opinions, as the authors themselves state.
- Publisher identity is unverified, and the core workflow depends on the proprietary platform ecosystem.
What does this agent do, and when should you use it?
EDSL is an open-source (MIT) Python domain-specific language from Expected Parrot for running computational social science and market research at scale with AI agents and LLMs. Developers build surveys from declarative question types (QuestionMultipleChoice, QuestionFreeText, QuestionLinearScale, etc.), define agent personas with Agent/AgentList, parameterize prompts with ScenarioList, and call multiple models at once via Model/ModelList (e.g., gpt-4o and gemini-1.5-flash). Survey objects support piping and skip/stop logic for complex data-labeling flows. Results are returned as specified datasets with built-in analysis, visualization, and selection methods such as select(). API calls are cached automatically; when run remotely, results are stored with verified prompts and timestamps in a universal remote cache on the Expected Parrot server that anyone can replicate for free. Users can run surveys locally or on the Expected Parrot server, and share workflows and results on the Coop platform.
After installing edsl via pip, users declare question objects in Python (QuestionMultipleChoice, QuestionFreeText, QuestionList, QuestionLinearScale, etc.), import data sources like CSV, PDF, and PNG as scenarios with methods such as ScenarioList.from_list, and inject them into prompts via {{ scenario.field }} placeholders. Agents are constructed with Agent(traits=...) and combined with questions or surveys using q.by(al).run() or survey.run(). Survey supports piping (e.g., {{ color.answer }}) and skip/stop rules. Outputs are queried with results.select("persona", "example"). LLM access uses your own API keys or a single Expected Parrot key covering all available models; with an account, runs are stored in the remote cache for exact replication, and workflows can be shared on Coop.
- Social science researchers: prototype and pre-test surveys with trait-based AI agents before recruiting real respondents.
- Market research teams: run the same questionnaire across multiple LLMs (e.g., gpt-4o vs. gemini-1.5-flash) and compare responses side by side.
- Data-labeling engineers: build multi-step labeling flows using answer piping plus skip and stop rules.
- LLM evaluators: batch-compare how different models answer identical prompts through one interface.
- Research teams: publish survey code and results on Coop so anyone can retrieve the exact outputs for free.
- Synthetic-data generators: produce structured simulated response datasets by parameterizing prompts with scenarios.
What are this agent's strengths and limitations?
- Declarative question types guarantee consistent result formats without hand-writing JSON schemas.
- Automatic caching of API calls; remote results carry verified prompts and timestamps so anyone can replicate exact outputs free, with no setup or API keys.
- One unified interface runs many models and many agents simultaneously for direct comparison.
- Built-in piping plus skip/stop logic enables rich multi-step data-labeling flows.
- An Expected Parrot key unlocks all available models with centralized team key, expense, and usage management.
- The full experience (remote cache, Coop) depends on the Expected Parrot account and server, creating platform coupling.
- Requires LLM API keys and incurs model-call costs that scale with model choice and survey volume.
- Agent responses reflect statistical patterns from training data, not real demographic opinions; human validation is required for measuring actual attitudes.
- Only Python 3.9–3.13 runtimes are supported.
- README gives no concrete guidance on integrating results with external data stacks (pandas, SQL); evaluate this yourself.
How do you install or deploy this agent?
- Ensure Python 3.9–3.13. 2. Run
pip install edsl. 3. Provide LLM API keys: use your own keys (e.g., OpenAI) or create an account at expectedparrot.com to get a single Expected Parrot key that accesses all available models. 4. Create an account at expectedparrot.com/login to enable remote caching and Coop sharing.
How do you use this agent?
Minimal example: python\nfrom edsl import QuestionMultipleChoice\nq = QuestionMultipleChoice(\n question_name="example",\n question_text="How do you feel today?",\n question_options=["Bad", "OK", "Good"]\n)\nresults = q.run()\nresults.select("example")\n For multi-agent and multi-model runs, build AgentList personas and ModelList models, then execute with q.by(al).by(ml).run(). Start with the official starter tutorial; surveys can run locally or remotely (logged in) to gain caching and replication.
How does this agent compare with similar options?
The README names no direct competitors, but its positioning (a Python DSL for simulating surveys with LLM agents) contrasts with traditional human survey platforms and manual crowdsourced labeling: EDSL delivers zero-marginal-cost pre-testing via AI-simulated responses, while explicitly requiring validation with real human data for actual attitudes and behaviors.