ClickClickClick
Run Android and macOS UI tasks with selectable local or remote models.
What does this agent do, and when should you use it?
ClickClickClick is an experimental framework for autonomous Android and computer UI use. It separates task handling into Planner, Finder, and Executor components, with OpenAI, Gemini, or Ollama selectable for planning and UI-element finding. The project documents a command-line workflow and a `POST /execute` REST API; its README table of contents also lists a Gradio web interface. API executions return a `result` object, while Android execution requires adb on the machine running the code. The project explicitly describes the current code as highly experimental and advises use at your own risk.
Configure model settings in config/models.yaml and export the keys named there, or run python main.py setup to choose a Planner and Finder interactively. Submit a task through python main.py run "<task>" or ./click3 run <task-prompt>, selecting --platform=android|osx, --planner-model=openai|gemini|ollama, --finder-model=gemini|openai|ollama, and --image-quality as needed. The POST /execute endpoint accepts task_prompt, platform, planner model, finder model, and image quality; it returns a result object on success, 400 for invalid parameters, and may return 500 for execution errors. Documented demonstrations include drafting a Gmail message, finding bus stops in Google Maps, and starting a 3+2 game on Lichess.
- An automation developer operating Android apps from natural-language tasks on a local machine with adb configured.
- A prototype team delegating browser tasks such as opening Google News to a chosen Gemini, OpenAI, or Ollama model combination.
- An internal-tools developer submitting Android or macOS tasks to the documented `POST /execute` endpoint.
- An experimenter testing basic navigation with local Ollama models while accepting slower responses and weaker element finding.
- An engineer evaluating different Planner and Finder model pairings for UI automation.
What are this agent's strengths and limitations?
- Planner and Finder can be selected independently from OpenAI, Gemini, and Ollama, allowing local and remote model combinations.
- It explicitly supports both Android and `osx` execution platforms.
- The documented `POST /execute` endpoint complements the CLI for programmatic integration.
- An image-quality control can reduce processing overhead, with `--image-quality=45` specifically recommended for Ollama use.
- The project explicitly labels the current code highly experimental, so behavior and interfaces may evolve in future commits.
- Android execution depends on adb being installed on the machine running the code.
- The README describes Ollama qwen3.5:4b as slow for planning, suitable only for basic navigation, and unreliable for Finder UI-element detection.
- Model keys and settings must be managed through `config/models.yaml` and exported environment variables.
How do you install or deploy this agent?
Clone https://github.com/instavm/clickclickclick and enter the directory. Create and activate a virtual environment with python3 -m venv venv and source venv/bin/activate; on Windows, use venv\Scripts\activate. Install dependencies with pip install -r requirements.txt. For Android tasks, install adb on the local machine that runs the code. Configure models in config/models.yaml and export the keys specified there. Local-model use additionally requires Ollama.
How do you use this agent?
Run python main.py setup and follow the prompts to select the Planner and Finder and provide required API keys. A first task can be run with python main.py run "Open Google news" --platform=android --planner-model=openai --finder-model=gemini. The documented local-Ollama example is python main.py run "Open Reddit" --platform=android --planner-model=ollama --finder-model=gemini --image-quality=45. To expose the API, start uvicorn api:app --reload, then send a JSON POST request containing task_prompt to http://127.0.0.1:8000/execute.