Deep Researcher Agent
An autonomous operations loop for planning, running, monitoring, and iterating deep-learning experiments on GPUs.
What does this agent do, and when should you use it?
Deep Researcher Agent is a 24/7 automation framework for deep-learning experiments, not a general-purpose coding assistant. Its core combines the THINK→EXECUTE→REFLECT loop in core.loop, Leader-Worker dispatch, zero-LLM monitoring in core.monitor, and a two-tier memory design. A project is driven by PROJECT_BRIEF.md and records operational state in MEMORY_LOG.md, state.json, experiments.jsonl, DEAD_ENDS.md, and INSIGHTS.md. Experiments can run locally, on one SSH-connected host, or through Slurm; the controller remains local in the remote modes. It ships as Claude Code slash commands and Codex local skills, with Anthropic-compatible, OpenAI-compatible, and limited CLI subscription provider modes.
On each cycle, the framework reads PROJECT_BRIEF.md, HUMAN_DIRECTIVE.md, MEMORY_LOG.md, and the experiment ledger, then has the Leader plan a next experiment and optionally dispatch an Idea, Code, or Writing worker. The Code worker can read and modify workspace code and configuration, perform a dry-run, and call launch_experiment to start training. During training, core.monitor polls process state, logs, and nvidia-smi without LLM calls. After a run ends, REFLECT parses logs and metrics, appends the hypothesis, metrics, and outcome to workspace/experiments.jsonl, and updates state, memory, and journals; the Slurm backend obtains terminal states such as FAILED, TIMEOUT, and CANCELLED from sacct. Supporting commands include /experiment-status, /gpu-monitor, /progress-report, /obsidian-sync, /daily-papers, /paper-analyze, and /conf-search.
- A researcher with an existing PyTorch project who wants bounded, repeated ablations under explicit GPU, epoch, and search-space constraints.
- An ML engineer tuning optimizers, learning rates, or augmentation from the outcomes and logs of prior training runs.
- A user with one remote GPU host who wants the controller on a laptop while code operations, training, logging, and GPU checks run over SSH.
- A researcher on a Slurm GPU cluster who needs sbatch submission and sacct-based reporting of actual job completion states.
- A project lead who needs persistent records of hypotheses, metrics, failed directions, and durable experiment observations.
- A Claude Code or Codex user who wants local commands or skills for launching loops, checking status, and reporting progress.
What are this agent's strengths and limitations?
- Monitoring is explicitly designed to avoid LLM calls while training, using process, log, and GPU polling instead.
- Local, SSH, and Slurm backends cover workstation GPUs, a remote host, and cluster submission; Slurm records terminal outcomes through sacct.
- The experiments.jsonl ledger plus DEAD_ENDS.md and INSIGHTS.md preserve structured results and failed directions across cycles.
- Bounded two-tier memory and one-worker dispatch are concrete controls on context growth and concurrent LLM cost.
- Running real experiments requires Python 3.10+, an NVIDIA GPU, a project workspace, and runnable training commands; it does not supply datasets or research code.
- API operation depends on an Anthropic-compatible or OpenAI-compatible endpoint and credentials; subscription operation requires a locally installed and logged-in Claude or Codex CLI.
- The repository documents codex_cli only for the Leader/THINK path: its internal agent loop prevents the framework from recovering PIDs for experiments it launches.
- The default is one worker and experiment.max_parallel is 1, so it is not positioned as a large-scale parallel experiment scheduler.
How do you install or deploy this agent?
Provide Python 3.10+, at least one NVIDIA GPU, and either Anthropic/OpenAI-compatible API credentials or a logged-in Claude/Codex CLI. Run:
git clone https://github.com/Xiangyue-Zhang/auto-deep-researcher-24x7.git
cd auto-deep-researcher-24x7
pip install -r requirements.txt
python install.py
python -m core.loop --check
Create a project directory containing at least PROJECT_BRIEF.md. For API mode, set agent.provider to "anthropic" or "openai" in the project config.yaml, choose model, and set the corresponding credential environment variable.
How do you use this agent?
A minimal direct invocation is:
python -m core.loop --project /path/to/project --gpu 0 --max-cycles 5
Remove --max-cycles for continuous operation. In Claude Code, run /auto-experiment --project /path/to/project --gpu 0; the installer also provides matching Codex local skills. For remote execution, set execution.mode: "ssh" with ssh_host and remote_workspace. For Slurm, set execution.mode: "slurm" and provide ssh_host, remote_workspace, slurm_partition, and slurm_time. Check active work with /experiment-status and steer the next cycle with workspace/HUMAN_DIRECTIVE.md.
How does this agent compare with similar options?
The project compares itself with Claude Scholar, AI Scientist, OpenHands, and SWE-Agent, emphasizing continuous experiment runs, zero-cost monitoring during training, GPU management, and constant-size memory. Its own table also presents a narrower paper-writing and general-coding scope.