Web data developer tools
Every listing tagged web-data, A to Z: 6 tools across 1 category. Listed in alphabetical order, not ranked.
Apify
Web scraping and automation platform: thousands of ready-made scrapers (Actors) in the Apify Store plus your own, with proxies, storage and scheduling, and an MCP server so agents can run any Actor. Apify also maintains the open-source Crawlee library. Free plan with $5 of usage a month.
Context.dev
Web data API for AI agents: scrape any URL into LLM-ready Markdown, search the live web, crawl and map sites, monitor pages for changes and pull brand data, with SDKs, a CLI and an MCP server; free credits to start.
Exa
Web search API for AI: real-time search with page contents, deep multi-step search, answers with citations, monitors and an async research agent, plus a hosted MCP server. Pay as you go, with $10 in free credits every month.
Firecrawl
Web data API for AI agents: scrape any URL into clean Markdown or JSON, crawl and map whole sites, search the web, run browser sessions and monitor pages, with an MCP server. Document parsing too: PDFs, including scans with OCR, and Word, Excel and PowerPoint files to Markdown at 1 credit a PDF page. Open source under AGPL-3.0 with a self-host guide, or use the cloud with 1,000 free credits a month.
Pomerado
Turns any website, from consumer sites to legacy portals, into a hosted API or MCP server for your agents, faster and more reliable than browser agents; open source and backed by Y Combinator.
Tavily
Search API built for AI agents: search, extract, map, crawl and research endpoints that return clean, LLM-ready results, with SDKs, an MCP server and a CLI. 1,000 free credits a month, no card.