ClickClickClick
Run Android and macOS UI tasks with selectable local or remote models.
Per-dimension scores and reasoning
Evidence shows: The repository provides no mechanisms for least privilege, user confirmation, data flow transparency, sensitive data handling, dependency security, external effects, rollback, or source attribution. All trust-related criteria are absent, hence score 0.
Evidence shows: Test files exist but cover only some executor functions, not core LLM interactions. Dependency list is complete but availability not verified. Error handling is partially shown in tests but not fully in code. Thus self-consistency, dependency availability, and failure messages each score 1.
Evidence shows: README provides usage scenarios for Raspberry Pi and macOS, but target audience is not explicit. Capability boundaries are partially shown in tests but not documented. Trigger precision is undefined. Environment fit is only partially described. Hence each scores 1.
Evidence shows: README and pyproject.toml provide basic installation instructions, but FAQ, known limitations, and changelog are missing. License is MIT but copyright attribution is inconsistent. Maintenance responsibility is unclear. Thus information architecture, install notes, naming stability, examples and FAQ, and maintenance responsibility score 1; known limitations and versioning/changelog score 0; license scores 2.
Evidence shows: Output usability is not specified, marginal value is not quantified, and cost-benefit is not analyzed. Hence each scores 1.
Evidence shows: Claims in README are partially consistent with code, but lack independent verification. Tests provide some corroboration but do not cover all features. Facts and inferences are not clearly separated. Hence each scores 1.
- The repository provides no security mechanisms such as least privilege, user confirmation, or data flow transparency, potentially allowing the LLM to perform arbitrary actions.
- Dependency list includes many unpinned packages, posing supply chain risks.
- Test coverage is incomplete, not covering core LLM interactions and error handling.
- Documentation lacks known limitations and changelog, and maintenance responsibility is unclear.
What does this agent do, and when should you use it?
ClickClickClick is an experimental framework for autonomous Android and computer UI use. It separates task handling into Planner, Finder, and Executor components, with OpenAI, Gemini, or Ollama selectable for planning and UI-element finding. The project documents a command-line workflow and a `POST /execute` REST API; its README table of contents also lists a Gradio web interface. API executions return a `result` object, while Android execution requires adb on the machine running the code. The project explicitly describes the current code as highly experimental and advises use at your own risk.
Configure model settings in config/models.yaml and export the keys named there, or run python main.py setup to choose a Planner and Finder interactively. Submit a task through python main.py run "<task>" or ./click3 run <task-prompt>, selecting --platform=android|osx, --planner-model=openai|gemini|ollama, --finder-model=gemini|openai|ollama, and --image-quality as needed. The POST /execute endpoint accepts task_prompt, platform, planner model, finder model, and image quality; it returns a result object on success, 400 for invalid parameters, and may return 500 for execution errors. Documented demonstrations include drafting a Gmail message, finding bus stops in Google Maps, and starting a 3+2 game on Lichess.
- An automation developer operating Android apps from natural-language tasks on a local machine with adb configured.
- A prototype team delegating browser tasks such as opening Google News to a chosen Gemini, OpenAI, or Ollama model combination.
- An internal-tools developer submitting Android or macOS tasks to the documented
POST /executeendpoint. - An experimenter testing basic navigation with local Ollama models while accepting slower responses and weaker element finding.
- An engineer evaluating different Planner and Finder model pairings for UI automation.
What are this agent's strengths and limitations?
- Planner and Finder can be selected independently from OpenAI, Gemini, and Ollama, allowing local and remote model combinations.
- It explicitly supports both Android and
osxexecution platforms. - The documented
POST /executeendpoint complements the CLI for programmatic integration. - An image-quality control can reduce processing overhead, with
--image-quality=45specifically recommended for Ollama use.
- The project explicitly labels the current code highly experimental, so behavior and interfaces may evolve in future commits.
- Android execution depends on adb being installed on the machine running the code.
- The README describes Ollama qwen3.5:4b as slow for planning, suitable only for basic navigation, and unreliable for Finder UI-element detection.
- Model keys and settings must be managed through
config/models.yamland exported environment variables.
How do you install or deploy this agent?
Clone https://github.com/instavm/clickclickclick and enter the directory. Create and activate a virtual environment with python3 -m venv venv and source venv/bin/activate; on Windows, use venv\Scripts\activate. Install dependencies with pip install -r requirements.txt. For Android tasks, install adb on the local machine that runs the code. Configure models in config/models.yaml and export the keys specified there. Local-model use additionally requires Ollama.
How do you use this agent?
Run python main.py setup and follow the prompts to select the Planner and Finder and provide required API keys. A first task can be run with python main.py run "Open Google news" --platform=android --planner-model=openai --finder-model=gemini. The documented local-Ollama example is python main.py run "Open Reddit" --platform=android --planner-model=ollama --finder-model=gemini --image-quality=45. To expose the API, start uvicorn api:app --reload, then send a JSON POST request containing task_prompt to http://127.0.0.1:8000/execute.
FAQ
Which model providers are supported?
What is required for Android tasks?
Can it be called as an HTTP service?
uvicorn api:app --reload and submit task parameters to POST /execute.Is local Ollama appropriate for UI-element finding?
How does the API report failures?
detail description.