Meta-Harness

The README clearly attributes the paper, authors, independent community projects, and separate artifact repository, while the LICENSE supplies explicit copyright and MIT terms, so source_attribution earns full marks. It only broadly says the framework controls what is stored, retrieved, and shown and that proposer interactions should be logged, giving data_flow_transparency thin support without destinations, retention, or log-content details. Commands and tests demonstrate evaluation execution and file writes, so external_effects is minimally visible, but no effects inventory or boundary is supplied. The material documents no least-privilege design, sensitive-data controls, dependency-security practice, consequential-action confirmation, or rollback mechanism, so those criteria score zero. Unknown publisher identity is treated as unknown and is not used to infer risk or safety.

The README's account of the framework, two experiments, separate artifact, and Claude Code proposer assumption is internally coherent, and the shown verifier is consistent with task-output evaluation; self_consistency is therefore adequate. Setup shows uv sync and run commands but no displayed dependency locks, supported versions, or fallback strategy, so dependency_availability is thin. The verifier silently converts a missing file or malformed JSON into a zero reward, and the README merely invites generic problem reports; no actionable failure messages are evidenced.

Research use, text classification, Terminal-Bench 2, and onboarding for new domains are explicit, giving adequate audience-and-scenario coverage. The source defines the harness around a fixed base model, distinguishes framework, examples, and artifact, and explains the Claude Code default plus the wrapper needed for alternatives, so capability boundaries are reasonably clear. Triggering is largely an instruction to have a coding assistant read ONBOARDING.md and converse, without precise entry, selection, or stopping rules, making trigger_precision thin. Environment evidence is limited to uv, Python, bash, a Claude Code assumption, and absolute container paths in tests; platform, version, and resource compatibility are not documented.

The README has a strong structure covering contents, quick starts, onboarding, release status, and citation, so information_architecture earns full marks. Basic install commands and pointers to subdirectory guides are present, but the supplied evidence omits those detailed guides, prerequisites, and troubleshooting, preventing full install_notes credit. Names and entry points appear consistent, though no stability or compatibility policy exists. Two command examples and a new-domain flow are useful, but there is no FAQ. Known limitations earn full marks because the README explicitly says this is cleaned paper code, was not tested beyond confirming it runs, and currently assumes Claude Code. The complete MIT text earns full license credit. A single unversioned release note is not a changelog. Authors are identifiable and issue reports are invited, but maintainer roles, a support route, update policy, and maintenance commitment are absent, so maintenance responsibility is thin.

The quick starts offer runnable experiment paths, onboarding is expected to produce domain_spec.md, and the verifier emits a machine-readable reward, supporting ordinary output usability; complete output schemas and explanatory diagnostics are absent. Automated search over task-specific harnesses offers plausible marginal value over manual harness design and is illustrated in two domains, but no comparative results are included in the supplied files. The README says subdirectory documentation covers runtime, yet that evidence is not supplied, and there is no treatment of API expense, compute requirements, search budget, or benefit tradeoffs, so cost_benefit is only thinly addressed.

Core claims are traceable to a named paper, concrete example directories, a separate artifact, citation metadata, and an exact expected output in the verifier, yielding adequate claim_traceability without line-by-line mapping from results to code. Corroboration is limited because the supplied set contains only the README, license, and a very small verifier fragment rather than the paper text, full implementation, dependency files, or broad tests. Fact-inference separation is reasonably careful: the README distinguishes paper code, the cleaned release, independent community work, and prospective listings, and explicitly limits its testing claim. Broader claims about framework value still lack direct substantiation in the provided material.