R&D-Agent

README requires Docker and instructs users to run docker without sudo; the Agent executes LLM-generated code inside containers, a high-privilege execution surface. No sandbox isolation, resource quotas, or human confirmation steps for generated code are described, so least_privilege and user_confirmation score 1. Data flow: only LLM API key and embedding service configuration is explained; there is no systematic disclosure of which third-party endpoints receive papers, financial reports, or Kaggle data, so data_flow_transparency and sensitive_data_handling score 1. Dependency security has Dependabot, CodeQL badges, and CI constraints, but requirements.txt is largely unpinned (only litellm>=1.73 and pydantic-ai-slim pinned), scoring 2. External effects (file writes, training runs, paid API calls) have no rollback or undo mechanism described, so rollback is 0. Source attribution is clear: MIT copyright to Microsoft, SECURITY.md provides MSRC reporting channel, source_attribution scores 2.

README and pyproject agree on Python >=3.10 (CI tests 3.10/3.11) and Linux-only, but the OS Independent classifier conflicts with the Linux-only statement, so self_consistency scores 1. Dependencies load dynamically from requirements.txt with a clear install path, dependency_availability scores 2. autorl_bench tests show failed evaluations raise RuntimeError instead of silently returning 0.0 and pass through an error field, giving concrete evidence for failure_messages, scoring 2.

README lists quant, data science, Kaggle, paper reproduction, and LLM fine-tuning scenarios, audience_and_scenarios scores 2. However, input boundaries, required data formats, cost ceilings, and failure modes per scenario are not systematically defined, so capability_boundaries scores 1. Triggers are CLI subcommands (rdagent fin_quant etc.) with parameters and prerequisites scattered across docs, trigger_precision scores 1. Environment fit is Linux + Docker only with no Windows/macOS guidance, environment_fit scores 1.

README is well structured with news, quick start, configuration, run, and UI monitoring, information_architecture scores 2. Install steps (conda, pip, make dev, health_check) are concrete, install_notes scores 2. Naming stability: many CLI subcommands and scenario names, some marked experimental/deprecated, naming_stability scores 1. Examples are rich (multiple demo commands and video links), examples_and_faq scores 2. Known limitations mention only Linux-only and Web UI not supporting data_science, known_limitations scores 1. LICENSE is full MIT text, license scores 3. Version is generated dynamically via setuptools-scm with no explicit CHANGELOG, versioning_changelog scores 1. Maintenance responsibility has Microsoft org ownership, SECURITY.md, CI, and issue links, maintenance_responsibility scores 2.

Outputs are code, factors, models, and logs that users must evaluate and integrate themselves, output_usability scores 1. The project targets automated R&D with papers and MLE-bench results supporting its value proposition, but static files cannot verify actual gains, marginal_value scores 1. Cost: README mentions RD-Agent(Q) under $10, but overall LLM call and training costs are not quantified, cost_benefit scores 1.

README cites arXiv papers, a tech report, and MLE-bench detailed run links, so claim_traceability has external pointers but no corresponding data files in the repo, scoring 1. Cross-source corroboration: MLE-bench numbers can be compared against papers/external links, but in-repo tests and docs do not reproduce them, cross_source_corroboration scores 1. Fact/inference separation: README mixes benchmark results with vision statements without marking which are reproduced facts, fact_inference_separation scores 1.