Web Data API to Scrape, Crawl, Map, and Search The Web

Decodo's Web Data API is a single API that scrapes pages, maps sites, crawls them, and returns search results, with proxies, JavaScript rendering, and anti-bot handling built in.

Try an example

Trusted by:

RTMP.IN logo

100+

ready-made templates

99.99%

success rate

200

requests per second

125M+

IPs worldwide

Up to 10K

free requests

Four web data endpoints, a single API key

Point our Web Data API at any website and get clean data, we'll handle the proxies, parsing, and retries.

  • LLM-ready output, no cleanup required

    Get structured Markdown or JSON stripped of nav bars, cookie banners, and ad clutter, ready to drop straight into a RAG pipeline or an LLM's context window.

    A dark user interface showing settings for PSP, Language, and Sound mode with current selections.
  • AI Integrations

    Native MCP Server support plus LangChain and LlamaIndex integrations, an agent calls Decodo directly as a tool, no custom wrapper code needed.

    Dark interface showing a glowing text box labeled "RESPONSE EXAMPLE JSON" with action buttons.
  • One API, four endpoints

    Scrape a page, map a site, crawl it end to end, or search the live web, all under a single key. Everything an agent needs to explore the web, not just fetch one URL.

    • AI agents: Give AI agents access to fresh web data for real-time research.
    • AI workflows: Bring web data into AI applications, RAG pipelines, and automated workflows.
    Start for free

Reach any target, from anywhere

4 ENDPOINTS, ANY URLMCP-NATIVE
  • A dark search engine window displays text results on a glowing purple background.

    /scrape

    Clean data from any URL

    Turn any webpage into structured output – HTML, JSON, CSV, or Markdown without building or maintaining your own parser.

    • Single-page extraction
    • JS rendering & anti-bot handling built in
    • HTML, JSON, CSV, or Markdown output
  • Glowing dark UI windows for web browser, shopping cart, and notes on a purple grid background.

    /map

    URL discovery for any site

    Provide a URL and get back every page on the site – crawled links plus sitemap in one call.

    • Up to 147K URLs per call
    • Combines links + sitemap
    • Filter by path or domain
  • A dark code editor displays intricate data structures against a vibrant purple grid background.

    /crawl

    Autonomous site crawling

    Point it at a site and get up to 10K pages scraped automatically – no manual link-following required.

    • Up to 10K pages per crawl
    • Automatic retry with JS rendering on failed pages
    • Markdown or HTML output, with callback support when done
  • Digital interface displaying data visualizations and a URL on a dark purple grid background.

    /search

    Real-time search results

    Query the live web and get back organic results – built for agents that need real-time context, not just a static page.

    • Google search results on demand
    • Structured, parsed output
    • Same API key as scrape, map, and crawl

One API key, four endpoints. Starting at $0.15/1K requests.

Start for free
Credit card offering "$5 FREE CREDIT" alongside a couponizer card and wallet icon.

Enough free credit to build something real

Start using Web Data API with $5 in free credit, up to 10K requests across all endpoints – /scrape, /map, /crawl, and /search. That's enough to map a real site, pull a working data sample, and test your integration end-to-end, not just send a couple of requests before you hit a wall.

Pricing stays flat as you scale. There are no hidden multipliers for JavaScript rendering or protected sites, so the price you test with is close to what you'll pay in production.

icon_check-circle

No card required

icon_check-circle

$5 free credit

icon_check-circle

No pricing multipliers

AI interface with neon accents showing a query and an AI agent response.

Plug live web data into any agent with MCP Server

Decodo ships an official MCP Server and CLI, so your agent can scrape, map, crawl, and search with a single config line. Decodo is also officially listed in GitHub's MCP Registry, which makes it easy to find and plug into your existing setup.

Once connected, Decodo's MCP Server gives your agent web search and scraping tools it can call on its own. It can run a Google search to find relevant sources, map a site to locate the right URLs, crawl the pages it needs, and scrape the details it's after. Results come back in formats agents handle well, like Markdown or parsed, structured data, so your model spends its context on the content instead of HTML.

icon_check-circle

MCP

icon_check-circle

CLI

icon_check-circle

Agent Skills

Popular targets for mapping, crawling, searching, and scraping

Skip the selectors and collect public data from search engines, eCommerce sites, and almost any other website online. Pick a target below, or start with our ready-made scraping templates.

  • All

Simple, usage-based pricing

Scale as your projects grow and only pay for successful requests.

10 req/s

$0

/month

Up to 5K crawls or 10K pages scraped

10 req/s

$19

/month

Up to 19K crawls or 38K pages scraped

25 req/s

$49

/month

Up to 65K crawls or 122K pages scraped

50 req/s

$99

/month

Up to 141K crawls or 330K pages scraped

100 req/s

$249

/month

Up to 383K crawls or 1.2M pages scraped

Need more?

Chat with us and we’ll find the best solution for you

With each plan, you access:

JavaScript rendering

99.99% success rate

HTML, Markdown, XHR, plus JSON, CSV, PNG where supported

One-click AI integrations

LLM-ready Markdown format

100+ pre-built templates

Single API key for all endpoints

/scrape, /search, /map, /crawl

24/7 tech support

14-day money-back

SSL Secure Payment

Your information is protected by 256-bit SSL

Frequently asked questions

What formats does the API return?

Neon 'FAQ' text with a magnifying glass and eye icon, plus a speech bubble.

Every /scrape call can return HTML, JSON, CSV, or Markdown – pick the format your agent or pipeline consumes best.

How does Decodo handle anti-bot protection?

Neon 'FAQ' text with a magnifying glass and eye icon, plus a speech bubble.

Requests route through a 125M+ IP residential and datacenter pool with automatic rotation, headless rendering, and fingerprint management – maintained by us, invisible to you.

Is there a free plan for the Web Data API?

Neon 'FAQ' text with a magnifying glass and eye icon, plus a speech bubble.

Yes. The starter plan includes up to 10K requests per month, all four endpoints, and every pre-built template. No credit card required.

How do AI agents connect?

Neon 'FAQ' text with a magnifying glass and eye icon, plus a speech bubble.

Decodo ships an official MCP Server, listed in GitHub's MCP Registry, plus integrations for LangChain, n8n, and OpenClaw. One API key covers everything.

What happens when a request fails?

Neon 'FAQ' text with a magnifying glass and eye icon, plus a speech bubble.

Failed requests are retried automatically across the proxy pool. You only pay for successful responses.

Start Collecting Web Data Today

Equip a single API key with four endpoints for each stage in your web data journey, whether it's for mapping, crawling, searching, or scraping.

© 2018-2026 decodo.com (formerly smartproxy.com). All Rights Reserved