DeepSeek Harness for GitHub

Evidence shows strong least-privilege design: allow-write defaults false, natural-language-write is a separate default-off switch, strict permission profile, Docker isolation, worker never holds the real GITHUB_TOKEN or DeepSeek key, controller-side credential proxy, typed github.* exact-ID deferred mutations, and fork review checks out only the trusted base SHA with persist-credentials:false. Deductions: isolation:none and dsh-executable host execution remain trusted inputs that remove the OS boundary if misconfigured; in native mode bridge networking and read/write mounts are whole-worker capabilities rather than per-tool sandboxes, and the docs admit extension startup and background processes are outside ToolRuntime limits. User confirmation: writes require multiple explicit switches and validation, but the installer creates files, sets a secret, commits and pushes, with confirmation described only in README and no code-level evidence. Data-flow transparency is good: trusted control plane vs untrusted data is clearly separated, credential-routing inputs are flagged, CI logs are bounded and redacted. Sensitive data handling covers redaction, bounding, no credentials in argv, and Session excluding credentials. Dependency security is deducted: the repo's own audit doc reports a non-clean graph (12 affected entries, 5 high) with unresolved MCP OAuth and nested fflate copies, mitigated only by reachability arguments. External effects are gated by deferred writes and validation, but enabling write mode can create branches and PRs, and rollback relies only on branch/PR isolation with no explicit revert mechanism. Source attribution has MIT and THIRD_PARTY_NOTICES references, but publisher identity is unverified and attribution rests on self-description.

Self-consistency: docs broadly agree on command semantics, write gates and mode defaults, but DSH version 0.2.1-rc.2 in README/docs coexists with 0.2.0-rc.2 in the dependency audit doc, an inconsistency. Dependency availability: exact pins and image digests aid reproducibility, but the audit doc admits a non-clean audit with unresolved nested dependencies, and Node 24 plus Docker are heavy prerequisites. Failure messages: docs describe failure classification, redaction, step summary and result-json, but mostly as assertions without concrete error codes or message samples, so only a moderate score.

Audience and scenarios are broad: PR review, CI diagnosis, auto-fix, Issue-to-PR, general Q&A, and dispatch automation, with bilingual docs and multiple templates. Capability boundaries are thoroughly documented: native is explicitly not a sandbox, extensions are trusted worker code, Validation Integrity is not a formal proof, and PDF/Office are not parsed. Trigger precision is high: line-start @dsh, non-quoted non-code, one actionable mention, bot requests ignored, reserved first-word commands. Environment fit is deducted: heavy reliance on GitHub-hosted Linux runners, Docker and Node 24, self-hosted/corporate networks need extra segmentation, and Windows setup steps are long with PowerShell special cases.

Information architecture is clear: README plus a layered docs directory covering setup, usage, configuration, security, troubleshooting and architecture, with generated input tables. Install notes are detailed, including an interactive installer, manual setup and recovery guidance. Naming stability is deducted: installer 0.7.2, Action v0.12.0 and DSH 0.2.1-rc.2 coexist, and production use requires switching to a full commit SHA, making version-to-name mapping non-obvious. Examples and FAQ: several example templates and troubleshooting exist, but example file contents are not in this evidence and FAQ form is not prominent. Known limitations are a clear strength. License is full MIT text. Versioning/changelog: README references CHANGELOG but its content is absent and multiple version numbers coexist. Maintenance responsibility: stated as maintained by @Lixiaoyiao and a community non-official project, but the publisher is unverified and the maintenance commitment lacks external corroboration.

Output usability: schema-v1 structured results, sticky comments, high-confidence inline findings, step summary and outputs are designed for usability, but no actual output samples are in evidence (only live-run links), so quality cannot be statically confirmed. Marginal value: adds AI review and diagnosis on top of existing CI and human review, genuinely useful but crowded, with differentiation mainly in security boundaries rather than capability. Cost-benefit: requires Docker, Node 24, DeepSeek API quota and long timeouts (default 20 minutes, 3 turns), which may be heavy for small repos; no cost or token-budget estimates are provided.

Claim traceability: many claims cite doc sections, invariant IDs, advisory links and reproduction commands, which is good; however core security guarantees are mostly documentation assertions without corresponding code or test evidence in this material. Cross-source corroboration: README, SECURITY, configuration and usage largely agree, but the DSH version mismatch between the dependency audit doc and README weakens corroboration. Fact/inference separation: the dependency audit doc clearly distinguishes fixed, disputed and reachability-inferred items, which is good; but live runs and CI badges are external claims that static review cannot verify, so full marks are not given.