Kane CLI
Natural-language web and mobile validation for coding agents, terminals, and CI pipelines.
Per-dimension scores and reasoning
Least privilege has some support: the release workflow grants only contents: read, mobile support is off by default, and runs can be bounded by step and time limits. However, browser automation inherently permits broad external actions, with no evidenced domain, action, or filesystem sandbox. User confirmation appears in selected flows such as requirements review and opening evidence, but ordinary natural-language runs do not document per-action confirmation for form submission, purchases, or other irreversible effects. Data-flow documentation identifies dashboard uploads, a local-only evidence-serving path, and explicit cloud sync, but does not fully describe server-side processing, retention, or telemetry. Secret variables can be masked, logout revokes OAuth tokens, and private vulnerability reporting is provided; credential storage, command-line access-key exposure, and complete redaction boundaries are not deeply documented. Dependency security is thin: major-version GitHub Actions and a security policy are shown, but there is no supplied dependency inventory, lock strategy, scanning, or supply-chain verification. External effects such as browser launch, run upload, cloud sync, and local deletion are generally disclosed, while website-side effects of autonomous actions lack systematic safeguards. Rollback is limited to logout, replayable caches, and local deletion, with no documented undo for website actions or uploads. Attribution is supported by repository and product naming, license copyright, and the security address, but TestMu AI, LambdaTest, and multiple domains coexist; publisher identity is externally unverified, so attribution is not fully conclusive.
The README is broadly consistent about commands, output, exit codes, and workflows, and the browser-check workflow demonstrates a concrete integration path. A deduction applies because SECURITY.md supports only 0.1.x while the README cites version 0.2.6. Dependency availability is adequately addressed through Node 18+, Chrome requirements and paths, platform restrictions, CI timeout controls, and mobile toolchains, but behavior during external service, authentication, or remote-grid outages is incomplete. Failure messaging is thoroughly specified through failed statuses, reason fields, a stable run_end event, exit codes, platform-specific Chrome remediation, and a workflow that distinguishes fatal posting failures from nonfatal crosspost failures.
Audience and scenario coverage is thorough across interactive CLI/TUI use, coding agents, CI, scheduled execution, remote grids, mobile testing, committed tests, and requirements assurance. Boundaries identify mobile platform constraints, Chrome requirements, flow-splitting guidance, the sole stable event, and Python-only code export; they do not sufficiently bound which website actions are safe or which interaction classes are unsupported. Triggers and invocation patterns are precise around --agent, URLs, store-as extraction, and one-objective guidance, but the instruction to use Kane for anything needing a real browser is overly broad and does not distinguish high-risk external actions. Environment fit is strong across macOS, Linux, Windows, Apple Silicon, CI, custom Chrome paths, existing CDP or Playwright endpoints, and layered variable sources.
Information architecture is clear, progressing from installation and first run through persistent tests, evidence, assurance, and command reference. Installation notes thoroughly cover npm, Homebrew, shell installation, version pinning, Chrome, platform paths, mobile prerequisites, and an update reference. Naming is mostly stable, but kane-cli, Kane CLI, TestMu AI, LambdaTest, testmuai.com, and lambdatest.com are mixed, reducing identity and endpoint consistency. Numerous concrete examples and troubleshooting links are present, but the supplied material does not include an actual FAQ or the complete troubleshooting text. Known limitations are explicit for platforms, browser requirements, event shapes, step scope, targets, and export language. Apache-2.0 metadata matches the full license and identifies the copyright holder. An update link, pinned-version example, and release-announcement workflow exist, but no actual changelog, compatibility policy, or migration history is supplied. Maintenance paths are suggested by the LambdaTest repository, TestMu AI branding, security email, Discord, and support documentation, though the rebrand and unverified publisher identity leave responsibility less than fully unified.
Output usability is thorough: stable run_end NDJSON, exit codes, extracted state, evidence packs, Result.md, cached replay, batch execution, and coverage reports are directly shaped for automation and review. Marginal value over direct browser tooling is plausibly supported by natural-language execution, agent integration, replayable Markdown, requirements linkage, and evidence packaging, but the evidence consists largely of the project's own claims without comparative measurements or independent results. Cost-benefit controls include reported credits, cache replay without further LLM cost, parallelism, timeouts, and step caps; pricing, typical consumption, performance benchmarks, and remote-service costs are absent.
Traceability is well designed on paper: requirement extraction cites sources, tests link acceptance criteria, design explain replays decisions, and evidence packs contain screenshots, logs, and failure records. The supplied source does not include implementation code or example artifacts that directly verify those claims. Cross-source corroboration is moderate because LICENSE, SECURITY.md, and the workflows support parts of the README concerning licensing, security contacts, releases, and integration, while most execution, safety, and effectiveness claims remain README-only. Fact/inference separation is partly explicit: proposals require review, coverage distinguishes proven from owed work, and stable run_end events are separated from unstable progress events. Marketing claims such as being a validation layer are not clearly labeled as unverified product assertions.
- Natural-language browser tasks can create real, irreversible effects on third-party sites. Add human confirmation and constrained test accounts for submissions, purchases, deletions, messages, or account changes.
- Runs are described as uploading results to a dashboard. Before using real credentials, personal data, or regulated data, verify server-side processing, retention, telemetry, log redaction, and deletion policies.
- Secret variables provide log masking, not complete credential security. Avoid placing access keys in shell history and verify local credential-file permissions and evidence-pack contents.
- SECURITY.md lists only 0.1.x as supported while the README references 0.2.6; confirm the currently supported release and security-update path before deployment.
- No dependency manifest, lockfile, or security-scan result is supplied, so third-party dependency and installer supply-chain status cannot be established statically.
What does this agent do, and when should you use it?
Kane CLI is TestMu AI's command-line validation layer for turning natural-language objectives into actions in a real Chrome browser or a supported mobile simulator or emulator. It supports both one-off runs and committed `_test.md` files whose unchanged, previously recorded steps can replay from cache. For scripts and coding agents, `--agent` emits NDJSON, with a stable `run_end` event carrying status, summaries, termination details, duration, and extracted `final_state` values. Each execution also seals a `.evidence` pack containing the test definition, results, step screenshots, browser logs, and failure records. Beyond execution, its assurance commands ingest requirements, extract cited use cases, design tests linked to acceptance criteria, and report proven versus outstanding coverage. Desktop web testing spans macOS, Linux, and Windows, while mobile testing is restricted to Apple Silicon Macs with local iOS or Android tooling.
A user gives kane-cli run a natural-language objective; the CLI launches local Google Chrome, connects through the Chrome DevTools Protocol, and performs browser operations such as navigation, clicking, form entry, assertions, and extraction. Values requested with the “store as” pattern are returned in run_end.final_state; --agent writes step events and the terminal run_end record as NDJSON, while exit codes 0, 1, 2, and 3 represent pass, test failure, setup or infrastructure error, and timeout or cancellation. Durable tests live in _test.md files executed by kane-cli testmd run; successful steps are recorded for cached replay, while @import, tags, and kane-cli testrun run support shared flows and parallel batches. Every run creates a .evidence pack that can be checked, viewed, or combined through kane-cli evidence validate, serve, and merge. The requirements workflow combines context ingest, context extract, context review, design tests, cover, and maintain reconcile to produce a reviewed, traceable, maintainable suite. Mobile objectives run with --target simulator or --target emulator and identify an app build or uploaded app ID through --app.
- A developer using Cursor, Claude Code, GitHub Copilot, or Codex asks the coding agent to open the real application and verify a newly implemented UI flow before committing.
- A QA engineer stores login, search, checkout, or smoke flows as reviewable
_test.mdfiles and replays their recordings during regression testing. - A CI maintainer runs end-to-end checks with
--agent --headless --timeoutand gates the pipeline on therun_endrecord or process exit code. - A product or test lead ingests a PRD, reviews source-cited use cases, designs tests tied to acceptance criteria, and inspects requirement coverage gaps.
- A team that needs execution evidence retains
.evidencefiles containing screenshots, console and network logs, and failure records as consolidated CI artifacts. - A mobile team working on an Apple Silicon Mac validates application flows against a local iOS Simulator or Android Emulator.
What are this agent's strengths and limitations?
- One CLI covers natural-language browser execution, cached Markdown tests, batch suites, evidence packaging, and requirements-to-coverage analysis.
--agentprovides machine-readable NDJSON and defines the stablerun_endevent as the supported automation boundary.- Successful
_test.mdsteps replay from recorded cache; only edited steps must be authored again, avoiding repeated LLM cost for unchanged work. - Evidence packs consolidate the test definition, step screenshots, step-attributed console and network logs, and failure records, including one pack for an entire batch.
- The assurance workflow links acceptance criteria and scenarios to individual tests, so coverage reflects requirements and execution evidence rather than raw test counts.
- Desktop web execution depends on a local Google Chrome installation; CI users must also configure headless execution and reliable CDP startup.
- Runs require TestMu AI authentication, and result uploads plus hosted test-management features introduce network and vendor dependencies.
- Mobile testing works only on Apple Silicon Macs and requires separate Xcode or Android Studio setup, a compatible virtual device, and managed Kane CLI tooling.
- The npm route requires Node.js 18 or newer; the Node-free Homebrew option is limited to macOS and Linux.
- Extraction requires explicit “store as” phrasing, or observed values will not appear in
final_state; integrations also should not depend on the unstable progress-event shape. - Local evidence can contain screenshots and browser logs, so adopters need their own controls for sensitive data and CI artifact retention.
How do you install or deploy this agent?
The recommended installation requires Node.js 18 or newer: npm install -g @testmuai/kane-cli. On macOS or Linux, brew install LambdaTest/kane/kane-cli needs no Node installation and installs Chrome through the google-chrome cask. A shell installer is also available: curl -fsSL https://raw.githubusercontent.com/LambdaTest/kane-cli/main/install.sh | sh. After npm or script installation, Google Chrome must be available at a standard path; set KANE_CLI_CHROME_PATH=/path/to/chrome for a custom location. Authenticate interactively with kane-cli login; in CI, use kane-cli login --username "$LT_USERNAME" --access-key "$LT_ACCESS_KEY" and inject those values from the CI secret store. Mobile setup additionally requires an Apple Silicon Mac plus Xcode or Android Studio with an arm64-v8a AVD; after signing in, run kane-cli doctor --install to install the managed testing components.
How do you use this agent?
A first web check is kane-cli run "Go to https://example.com and assert the page title contains 'Example'". For a coding agent or parser, run kane-cli run "Go to https://news.ycombinator.com and store the title of the top story as 'top_story'" --agent and consume the stable run_end event; extraction objectives must use the “store as” wording. In CI, use kane-cli run "<objective>" --agent --headless --timeout 120. Save durable flows as _test.md and run them with kane-cli testmd run <path>; execute a tagged batch with kane-cli testrun run --tags smoke --parallel 4. Serve a local pack with kane-cli evidence serve .testmuai/evidence/<execution-id>.evidence. A basic requirements workflow is kane-cli context ingest ./prd.md, kane-cli context extract, kane-cli context review, kane-cli design tests --use-case <ref>, then kane-cli cover. For mobile, use kane-cli run "<objective>" --target simulator --app ./MyApp.zip.
How does this agent compare with similar options?
Compared with writing Playwright, Puppeteer, or Selenium scripts directly, Kane CLI exposes natural-language objectives, _test.md suites, and a structured run_end result, while adding recorded replay, evidence packs, and requirements coverage. Its agent guidance explicitly tells coding agents to use Kane CLI instead of those frameworks for browser tasks; the tradeoff is adopting TestMu AI authentication, execution conventions, and output contracts.
FAQ
What account or credentials are required?
kane-cli login. Non-interactive environments can supply a TestMu AI/LambdaTest username and access key; the documented CI approach injects them from a secret store rather than committed configuration.How should automation determine whether a run succeeded?
--agent, parse stdout one line at a time, and treat only the event whose type is run_end as the stable terminal record. Also inspect the exit code: 0 means passed, 1 test failure, 2 setup or infrastructure error, and 3 timeout or cancellation.Are evidence packs automatically uploaded?
test_url; named or saved runs also create local .evidence files under .testmuai/evidence/. evidence serve uses a local-only server to expose a pack to the viewer without uploading that pack.Can it test mobile applications?
arm64-v8a AVD, and the managed tooling must be installed with kane-cli doctor --install.Is pricing or fully self-hosted operation documented?
run_end may report consumed credits and provides kane-cli balance, but it does not state prices. Browsers and mobile virtual devices run locally, and evidence packs can be served locally; authentication and test-manager integration still depend on TestMu AI.