XScientist
Turn one idea into a Git-like research history that is inspectable, reproducible, and reversible — evidence you can audit end to end.
- Source repo
- smileformylove/XScientist
- Stars
- ★ 125
- Last updated
- 1d ago
- License
- Apache-2.0
- Primary language
- Python
- FA score
- 61/100 · Some gaps
At a glance
- How it runs
- Works with
- Universal · cross-platformOpenAI API · Claude API
- Cost
- Free software; you pay for model usage
- Setup effort
- Medium · a few setup steps
- You'll need
- Typical use
- A researcher with no model or API key who wants to record an idea as a falsifiable, versioned research start (explore, zero cost)
- Not a fit if
- Teams needing instantly verified scientific claims: it is alpha software whose outputs must pass evidence and review gates
- Users unwilling to run a Docker-isolated executor: generated experiment code never runs on the host
- Source review
- 61/100 · Some gaps
What does this agent do, and when should you use it?
XScientist is a local-first autonomous research system and open scientific protocol delivered as a CLI (the xscientist command) plus a Python SDK. It externalizes question framing, hypotheses, decisions, experiment attempts, evidence, objections, and gates as typed objects with content hashes, and versions them through a Git-like Research VCS that supports commit, diff, branch, merge, and revert of scientific meaning. It works with no model or API key at all (the explore command), with local Ollama models, or with hosted OpenAI, Anthropic, Zhipu, Bedrock, Vertex, and other OpenAI-compatible endpoints. Generated experiment code never runs silently on the host — it must execute in a configured Docker-isolated executor, and autonomous runs can be capped with --max-cost-usd. A completed run is only presented as a verified scientific claim when the trace, replay, and verify audit gates actually pass; otherwise output is labeled exploratory or a draft. The released version is 0.1.4 on PyPI; the main branch additionally carries unreleased protocol work that requires a source install with a pinned commit for reproducibility.
Users record an idea, expected observation, disconfirming result, and first test through xscientist explore's plain-language questions; the system forms a falsifiable hypothesis and versions the state as idea saved, falsifiable, or planned. xscientist start runs an autonomous study: proposing rival and null hypotheses, ranking experiments by expected information value, executing bounded experiments in the Docker executor while retaining failed and negative results, scanning anomalies and contradictions, running independent review roles, and stopping at hard gates. xscientist record, research decide, and research experiment write results as immutable, content-hashed evidence objects in checkpoints; history list/diff/rollback and audit --level trace/replay/verify provide inspection and reproduction. provider list/test/explain manage models and credential sources; research bundle/export/fsck, benchmark autoresearch, the FAR-inspired research opportunity funnel, and the belief-context projection (research belief) expose deeper protocol surfaces.
- A researcher with no model or API key who wants to record an idea as a falsifiable, versioned research start (explore, zero cost)
- Users running end-to-end autonomous studies on a local Ollama model (e.g. gemma3) with no hosted API cost
- Researchers with a hosted API key running a publication-oriented workflow via --autopilot publication --max-cost-usd 10
- Educators or reviewers demonstrating a complete but deliberately contested evidence history with the $0.00 demo
- Teams needing a fully auditable literature-to-open-problem path — candidate pool, attempts, independent judgment, and resource allocation (opportunity funnel commands)
- CI/scripts that pass question, provider, model, and budget explicitly with --non-interactive
How do you install or deploy this agent?
Released 0.1.4 needs Python 3.10+ and Git. Install from PyPI:
python -m pip install "xscientist==0.1.4"For the unreleased protocol work on main, install from source — pin a commit for repeatable research:
python -m pip install "xscientist @ git+https://github.com/smileformylove/XScientist.git@main"For model features, install a provider extra, e.g. local Ollama:
python -m pip install \
"xscientist[research,openai-compatible] @ git+https://github.com/smileformylove/XScientist.git@main"Or a hosted service (extras: openai, anthropic, zhipu, bedrock, vertex, openai-compatible):
python -m pip install \
"xscientist[research,openai] @ git+https://github.com/smileformylove/XScientist.git@main"
export OPENAI_API_KEY="..."Model-backed experiments additionally require Docker.
How do you use this agent?
Zero-cost start (no model, key, Docker, or network call):
xscientist explore ./my-study
xscientist status ./my-studyOr record an idea non-interactively:
xscientist explore ./my-study \
--idea "Does daily walking improve sleep quality?" \
--expect "Daily walking improves a preregistered sleep score." \
--disprove "The score is unchanged or worse." \
--test "Compare walking and usual-activity periods." \
--non-interactiveSee the complete contested evidence history demo:
xscientist demo ./first-study --autopilot --open
xscientist status ./first-studyUse a local Ollama model:
ollama pull gemma3
ollama ls
xscientist provider list
xscientist start ./my-study --prepare-only
xscientist start ./my-studyUse a hosted model with a cost cap:
xscientist start ./hosted-study --prepare-only
xscientist start ./hosted-study --max-cost-usd 10Prepare the isolated executor and check readiness before model-backed experiments:
xscientist executor prepare --workspace ./ood-study
xscientist provider check --workspace ./ood-study --max-cost-usd 10
xscientist doctor --workspace ./ood-study --deepInspect and reproduce:
xscientist history list ./first-study
xscientist history diff ./first-study
xscientist audit ./first-study --level trace
xscientist audit ./first-study --level replay
xscientist audit ./first-study --level verifyLong-running background studies:
xscientist runs list --workspace ./ood-study
xscientist runs watch RUN_ID --workspace ./ood-studyIf GLM-5.3 resolves as the execution model, a separate --judgment-model (e.g. openai/gpt-4.1) is required for scientific-judgment roles, or start fails closed with a recovery command.
What are this agent's strengths and limitations?
- Research VCS versions scientific meaning — hypotheses, evidence, objections — as typed, hash-checked objects with diff, branch, and revert, not an opaque chat transcript
- The one-way trace → replay → verify audit ladder separates traceable, reproducible, and independently verified claims; failed and negative results stay visible
- Model-agnostic: Ollama local models plus OpenAI, Anthropic, Zhipu, Bedrock, Vertex, and OpenAI-compatible endpoints, with credentials kept in a protected .env or process environment
- Generated code runs only in a configured Docker-isolated executor, with --max-cost-usd caps and fail-closed behavior on unknown model pricing
- Explicitly alpha research software, not an oracle; autonomous runs may incur paid model costs
- A version boundary exists between the PyPI release (0.1.4) and unreleased protocol work on main; repeatable research requires pinning a source commit
- Model-backed experiments require Docker and a version-matched executor image; atomic plot publication needs POSIX fcntl and fails with no fallback on unsupported platforms
- The publication autopilot explicitly promises no manuscript completion, scientific verification, venue submission, or acceptance; full autonomous use requires learning profiles (balanced/discovery/publication) and gate concepts
How does this agent compare with similar options?
The README's system-comparison matrix names full research agents such as ScientistOne, AI Scientist v2, DeepScientist, and AI-Researcher, plus references like FAR (Find–Attempt–Recommend), JevGPT, and AutoResearchEval. XScientist's stated differentiation is its Git-like inspectable evidence history and trace/replay/verify audit gates rather than matching those systems' benchmark scores. The repository ships xscientist benchmark systems to generate a source-audited local comparison matrix, hard-coding official_comparable: false and score_claim_allowed: false.
Key facts side by side with the most closely related agents.
| Agent | Source review | Form / cost | Stars | Updated | Language | Full support on |
|---|---|---|---|---|---|---|
| XScientist This agent | 61 · Some gaps | CLIFree + model costs | ★ 125 | 1d ago | Python | OpenAI API · Claude API |
| Station Research Environment | 60 · Some gaps | CLIFree + model costs | ★ 198 | 4d ago | Python | — |
| Autonomous Agents Research Papers | 28 · Major gaps | Library / SDKFree | ★ 1.4k | 3mo ago | — | — |
| Wisp Science | 71 · Some gaps | Desktop appFree + model costs | ★ 1k | today | Rust | Codex · Claude Code · OpenAI API · Claude API |
How does FollowAgents rate this agent?
Why each dimension lost points
README describes Docker-isolated execution, keys only in process env or a protected .env, a read-only provider explain command, content hashing, a rollback boundary, cost caps and fail-closed design, but beyond a small test subset no implementation code is visible, so most safety claims are assertions; least_privilege, user_confirmation and rollback get 1. CI actions are SHA-pinned with constraints files but no lockfile or dependency audit, so dependency_security is 1. source_attribution is notably weak: the LICENSE copyright reads 'Copyright 2020 Rémi Louf' (apparently copied from another project), pyproject authors are 'AI Scientist Contributors', and the MIT component and THIRD_PARTY_NOTICES are referenced but not shown.
Version boundary (published 0.1.4 vs unreleased main protocol work) is stated consistently and dev builds identify as 0.1.5.dev0 to prevent executor-image misuse; extras are well partitioned with constraints; failure behavior includes structured errors, non-zero exit codes and copyable repair commands. Implementation is not visible, so 2 not 3.
Paths for keyless beginners, local Ollama users, hosted-model users and CI/scripts are thoroughly mapped (3). Capability boundaries (alpha status, no paper/acceptance promise, POSIX fcntl limitation, Python 3.10+, Docker prerequisite) and trigger precision (JSON action contracts, required markers) are documented, but some paths depend on an unreleased main build, so 2.
Documentation structure, install instructions, version-boundary labeling, bilingual links are complete, so information_architecture and install_notes get 3. License scores only 1: 'Apache-2.0 AND MIT' is claimed but only an Apache text with a foreign copyright holder is shown and the MIT text is missing. Maintenance responsibility rests with an unverified anonymous publisher with no stated maintainers or update commitment. No FAQ; CHANGELOG exists only as a link.
Outputs are structured JSON with next_steps contracts and status/history, automatable; cost caps, a $0 demo and --prepare-only readiness checks lower trial cost, giving 2 across the board. Actual effectiveness requires execution and cannot be credited in static review.
Typed objects, content hashes, checkpoint DAG and trajectory hashes are well described (2). But key claims (isolation, review gates, publication verification) lack corroborating code or independent evidence in the visible files and the arXiv badge cannot be checked, so corroboration is 1. Published/unreleased labeling and the intentionally contested demo distinguish fact from promise clearly, so fact_inference_separation is 2.
- The LICENSE copyright reads 'Copyright 2020 Rémi Louf', inconsistent with this project (AI Scientist Contributors); the license file appears copied from another project — verify the license chain and the missing MIT text before use.
- Most safety claims (Docker isolation, rollback, key protection, review gates) appear only in the README; no corresponding implementation was visible in this static review, so they must not be treated as verified.
- Protocol features on main are unreleased; pin PyPI 0.1.4 or a specific commit for production use, as the README itself recommends.
- Autonomous runs may incur paid model calls; set --max-cost-usd and understand the execution boundary for generated code.
- Publisher identity is unverified by the FollowAgents registry; maintenance sustainability and update path are unknown.
Recent releases
Latest entries from the project's GitHub Releases. Notes are quoted from upstream as written, not rewritten or tested by FollowAgents.
-
Release notes excerpt
[0.1.2] - 2026-08-17 Added - A one-command, provider-free xscientist demo that creates a deterministic Research VCS history with a failed attempt, supporting and refuting evidence, an independent rejection, a contested claim, and an offline DAG for $0. - A compact, read-only xscientist status view for scientific progress, automated run state, budget use, results, the evidence DAG, and the next valid action. - xscientist provider check now reports credential presence, client availability, live-verification scope, and optional cost-enforcement …View on GitHub ↗ -
Release notes excerpt
[0.1.1] - 2026-08-08 Added - Unified xscientist setup, doctor, and capability commands with task-aware readiness checks and exact opt-in installation guidance. - Deterministic xscientist research decide policy for checkpoint, fork, merge, hold, and release transitions, including stable decision identifiers and explicit trace requirements. - Payload-free long-term research technology trees spanning every local research line, with topology, frontier, cycle, and missing-reference views. - Stable scientific merge-conflict identifiers, actionable resolution …View on GitHub ↗ -
Release notes excerpt
0.1.0 - 2026-08-06 Added - Installable xscientist Python package, public CLI, SDK, and optional HTTP service. - Long-running research workflows, structured review/repair artifacts, and ARA protocol tooling. - Cross-version CI coverage for Python 3.10, 3.11, and 3.12. - Branch-aware whole-repository coverage regression checks. - Automated engineering, distribution inventory, isolated wheel, and release metadata checks. - GitHub issue/PR templates, dependency update policy, citation metadata, and an engineering maintenance guide. Changed …View on GitHub ↗