FollowAgents AI AGENTS DIRECTORY

Agents tagged "web-crawler"

1 results
web-crawler ×
All Dev & Engineering Data & Analysis Productivity & Collaboration Writing & Content Design & Frontend Automation & Ops
Role
Any
Find agents for your role
All roles Software Engineer Frontend / Full-stack DevOps / SRE Data Analyst Data Scientist / ML Product Manager Designer Technical Writer Content & Marketing Researcher Founder / Indie Hacker Student / Learner
Publisher
Any
Filter by official publisher
All publishers OpenAI 6 Anthropic 1 Microsoft 23 Google 9 GitHub 2 Amazon Web Services 2 NVIDIA 4
Dev & Engineering

Spider - High-Performance Web Crawler and Scraper in Rust

Get web data for AI agents and LLMs with a concurrency-first, streaming crawler that renders JavaScript on demand.

48 · Major gaps ★ 2.8k · today