News Scraper API

Media monitoring breaks when every publisher uses a different layout. Our news scraper API standardizes scraping news articles – headline, byline, date, and body text arrive as fields, not markup.

* This scraper is now a part of the Web Data API.

125M+

IPs worldwide

99.99%

success rate

200

requests per second

100+

ready-made templates

Free

starter plan

Be ahead of the news scraping game

Code block showing <!doctype html>, linked to cards labeled APP TITLES; UPDATE HISTORY; CONTACT DETAILS; PRICING

What is news scraper API?

What does a news scraper API return that an RSS feed does not? The article itself, read from the public page rather than a summary. Scraping news articles across a long publisher list means selector drift is the real maintenance cost, since every publisher runs its own template. Per article:

  • Headline, subheading, and byline
  • Publish and update timestamps
  • Section, tags, and category
  • Body text with paragraph structure
  • Canonical URL and image captions

Body text is where the legal line sits, so most pipelines analyze it internally and publish only the headline, link, and timestamp.

Dark window with bug icon and scraping UI showing Search URL https://ip.decodo.com/ and 'Start scraping' button

Collect news data

Filling a monitoring index or a training corpus takes volume, and the Web Data API supplies it from a single endpoint.

From request to storage:

1. Send one URL or a batch of article links

2. Render pages that load text through scripts

3. Route through 125M+ proxies spanning residential, mobile, ISP, and datacenter types

4. Take HTML or parsed JSON back

5. Pick from 195+ geo-locations for regional editions

6. Schedule recurring pulls per publisher

Built-in scraper

JavaScript rendering

Easy API integration

195+ geo-locations, including country-, state-, and city-level targeting

No CAPTCHAs or IP blocks

Scrape news with Python, Node.js, or cURL

Our news Scraper API works with every major programming language, so it plugs into the tools you already run.

import requests
url = "https://scraper-api.decodo.com/v2/scrape"
payload = {
"url": "https://news.google.com/",
"headless": "html"
}
headers = {
"accept": "application/json",
"content-type": "application/json",
"authorization": "Basic [YOUR_BASE64_ENCODED_CREDENTIALS]"
}
response = requests.post(url, json=payload, headers=headers)
print(response.text)

Get more from the news scraper API

JavaScript rendering, proxy management, and CAPTCHA handling ship in one API call, so news data arrives without blocks.

100% success rate

Dead links and empty responses cost nothing. Charges apply per article that returns actual content.

Advanced anti-bot bypass

Publisher defenses run from none to strict. Request signatures, header rotation, and TLS handling cover the strict end.

Real-time or on-demand results

Poll a breaking story every few minutes, or queue an overnight pass across yesterday's archive.

Bypassing-CAPTCHAs-icon

125M+ proxy pool

Editions differ between countries. Choose residential, mobile, ISP, or datacenter routes among 195+ locations to capture the version local readers get.

JavaScript rendering

Many outlets load body copy and embeds after first paint. We wait for the render, then return the complete text.

Easy integration

Feed the endpoint a URL list and read results in Python, Node.js, or cURL. Sample scripts ship with the docs.

What does Web Data API cost?

Choose a plan based on your data collection volume. All plans include the same powerful features – you only pay for what you use. Start with the free plan to test before committing.

10 req/s

$0

/month

Up to 5K crawls or 10K pages scraped

10 req/s

$19

/month

Up to 19K crawls or 38K pages scraped

25 req/s

$49

/month

Up to 65K crawls or 122K pages scraped

50 req/s

$99

/month

Up to 141K crawls or 330K pages scraped

100 req/s

$249

/month

Up to 383K crawls or 1.2M pages scraped

See full pricing details in the documentation.

With each plan, you access:

JavaScript rendering

99.99% success rate

HTML, Markdown, XHR, plus JSON, PNG where supported

One-click AI integrations

LLM-ready Markdown format

100+ pre-built templates

Single API key for all endpoints

/scrape, /search, /map, /crawl

24/7 tech support

14-day money-back

SSL Secure Payment

Your information is protected by 256-bit SSL

Why the scraping community chooses Decodo

135K+ developers, SEOs, and data teams rely on Decodo to collect public web data at scale – including from News. Whether you're shipping a side project or running production pipelines, our infrastructure is built to keep your News scraper running while you focus on what to do with the data.

Decodo

Manual data collection

Other APIs

125M+ residential, mobile, datacenter, and ISP proxies

Manage proxy rotation yourself

Limited proxy pools

Advanced browser fingerprinting

Build CAPTCHA solvers

Frequent CAPTCHA blocks

Only pay for successful requests

Handle retries manually

Pay for failed requests

100+ ready-made scraping templates

Maintenance overhead

Complex documentation

Data in JSON, CSV, HTML, and Markdown formats

Days to implement

Limited output formats

Why the scraping community chooses Decodo

135K+ developers, SEOs, and data teams rely on Decodo to collect public web data at scale – including from news. Side projects and production pipelines run on the same infrastructure, built to keep your news scraper up while you focus on the data itself.

Attentive service

The professional expertise of the Decodo solution has significantly boosted our business growth while enhancing overall efficiency and effectiveness.

Easy to get things done

Decodo provides great service with a simple setup and friendly support team.

A key to our work

Decodo enables us to develop and test applications in varied environments while supporting precise data collection for research and audience profiling.

Decodo-best-usability-award-2025-by-G2

Best Usability 2025

Awarded for the ease of use and fastest time to value for proxy and scraping solutions.

Decodo-Highest-User-Adoption-2025-award-by-G2

Best User Adoption 2025

Praised for the seamless onboarding experience and impactful engagement efforts.

Decodo-best-value-by-Proxyway-2025-award

Best Value 2025

Recognized for the 5th year in a row for top-tier proxy and scraping solutions.

Trusted by:

RTMP.IN logo

Decodo blog

Build knowledge on our solutions and improve your workflows with step-by-step guides, expert tips, and developer articles.

Most recent

What Is T3 Code? Setup, Features and Use Cases

What Is T3 Code? Setup, Features and Use Cases

T3 Code is a free, open-source app for managing AI coding agents like Claude Code, Codex, and Cursor from a single desktop, web, or mobile interface. Developed by Theo Browne's T3 Tools team, it has become one of GitHub's fast-growing developer projects in 2026. In this guide, we'll cover what T3 Code does, its main features, how to get started, and how it compares to other coding tools.

Most popular

Residential Proxy VS Datacenter Proxy — monitor icon opposite server stack on dark gradient background

Residential vs. Datacenter Proxies: Which Should You Choose?

Google Sheets Web Scraping: An Ultimate Guide for 2026

Two robots placing glowing stars to form a five-star rating against a brick wall

Manage Your Business Reputation with SERP Scraping API

Unlocked padlock being cut by dashed diagonal line over a browser search-results window on dark blue background

How to Scrape Google Without Getting Blocked

Glowing orange outline icon of a browser window with a magnifying glass, on a dark rounded square

What Is SERP Analysis and How To Do It in 2026?

API icon exchanging 'Request' and 'Response' between APP and BACKEND

What is an API?

real estate card showing Newport Beach, California and 23% Price growth of metros over skyline and bar chart on dark grid

How to Scrape Hotel Listings: Unlocking the Secrets

Browser window labeled 'Scraping tools' showing content panel, on a dark dotted background with a wireframe bug outline

What is Data Scraping? Definition and Best Techniques (2026)

Web Crawling vs Web Scraping: What’s the Difference?

Web scraping UI with 'Start scraping' button, URL field, JSON 'Response' preview and 'Live preview' globe on dark background

What Is Web Scraping? A Complete Guide to Its Uses and Best Practices

Python logo rising from a bowl beside a spoon in neon magenta on dark background

Beautiful Soup Web Scraping: How to Parse Scraped HTML with Python

Frequently asked questions

Is it legal to scrape news?

Headlines, metadata, and publish dates from public articles can normally be collected. Republishing full text is a different question, because article bodies are copyrighted. Most teams keep text for internal analysis, then surface short quotes with attribution. Check each publisher's Terms of Service, and treat author details as personal data.

How does the news scraper API bypass blocks and CAPTCHAs?

News sites span the whole defense spectrum. Every call picks an egress IP, applies a matching request signature, reorders headers, and negotiates TLS properly, with CAPTCHAs resolved in flight. For aggregator listings rather than article pages, reach for the Google News Scraper API.

What output formats does the news scraper API support?

Expect HTML first, and JSON where parsing covers the result format. CSV comes out of the dashboard. Large monitoring jobs tend to stream JSON into a search index. Our guide to scraping Google News results walks through one working pipeline.

Can I geo-target news requests to a specific country or city?

Yes. There are 195+ locations to target, with city-level and ZIP-level choices in parts of the network. Publishers often serve different editions, consent screens, or paywall prompts by country, and matching the origin keeps a corpus coherent.

How fast is the news scraper API and can it handle large-scale jobs?

Article volume is the constraint, not latency; concurrency follows the plan you pick. Responses land in seconds and thousands of URLs move together. Schedule per-publisher jobs, or trigger a pull when a feed updates.

Can I try the news scraper API for free?

Yes. News scraping falls inside the Web Data API starter plan, which opens without payment details. Point the quota at the publishers you actually monitor, since layout coverage varies between outlets far more than plan limits do.

News Scraper API for Your Data Needs

Gain access to real-time data at any scale without worrying about proxy setup or blocks.

14-day money-back option

© 2018-2026 decodo.com (formerly smartproxy.com). All Rights Reserved