urllib3 vs. Requests: Which Python HTTP Library to Use?

What urllib3 and Requests are (and how they relate)

Before the feature breakdown, let's clear up the naming mess.

Python's built-in urllib ships with the language. urllib2 was its Python 2 successor, now gone. urllib3 is a completely separate third-party library with an unfortunate name. They share a prefix, not a codebase.

urllib3 was built to fill the gaps urllib left open, adding connection pooling, thread safety, retry logic, and proxy support. Requests then wrapped urllib3 with an API that developers actually want to use, adding session management, automatic encoding, and built-in JSON handling.

When you install Requests, urllib3 comes along as a dependency. You likely already have both. The question is which API surface you write against.

For a quick definition of the Requests library, see the Python Requests glossary entry.

Installing both libraries and comparing basic syntax

Both libraries install through pip:

pip install urllib3
pip install requests

pip install urllib3
pip install requests

Running pip install requests pulls in urllib3 automatically as a dependency, so you get both with a single command.

The syntax differences show up immediately once you start making requests. The examples below all use httpbin.org as the test target.

GET requests

Requests:

import requests

response = requests.get("https://httpbin.org/get")
print(response.json())

import requests

response = requests.get("https://httpbin.org/get")
print(response.json())

urllib3:

import urllib3
import json

http = urllib3.PoolManager()
response = http.request("GET", "https://httpbin.org/get")
print(json.loads(response.data.decode("utf-8")))

import urllib3
import json

http = urllib3.PoolManager()
response = http.request("GET", "https://httpbin.org/get")
print(json.loads(response.data.decode("utf-8")))

Requests exposes a module-level get() function that returns a response object with built-in JSON parsing. urllib3 requires you to first create a PoolManager instance, then call http.request() with the HTTP method as an explicit string argument.

POST requests with JSON

Requests:

import requests

response = requests.post("https://httpbin.org/post", json={"key": "value"})
print(response.json())

import requests

response = requests.post("https://httpbin.org/post", json={"key": "value"})
print(response.json())

urllib3:

import urllib3
import json

http = urllib3.PoolManager()
encoded = json.dumps({"key": "value"}).encode("utf-8")

response = http.request(
    "POST",
    "https://httpbin.org/post",
    body=encoded,
    headers={"Content-Type": "application/json"}
)
print(json.loads(response.data.decode("utf-8")))

import urllib3
import json

http = urllib3.PoolManager()
encoded = json.dumps({"key": "value"}).encode("utf-8")

response = http.request(
    "POST",
    "https://httpbin.org/post",
    body=encoded,
    headers={"Content-Type": "application/json"}
)
print(json.loads(response.data.decode("utf-8")))

The json= parameter in Requests handles serialization, byte encoding, and the Content-Type header in one shot. With urllib3, you manage each step yourself.

Response handling

Response handling is where the day-to-day gap is most felt. Requests gives you convenience methods directly on the response object:

response.json()       # parsed JSON
response.text         # decoded string
response.content      # raw bytes
response.status_code  # HTTP status as int

response.json()       # parsed JSON
response.text         # decoded string
response.content      # raw bytes
response.status_code  # HTTP status as int

urllib3 returns response.data as raw bytes, so decoding and parsing are manual:

response.data                              # raw bytes
response.data.decode("utf-8")             # decoded string
json.loads(response.data.decode("utf-8")) # parsed JSON
response.status                            # HTTP status as int

response.data                              # raw bytes
response.data.decode("utf-8")             # decoded string
json.loads(response.data.decode("utf-8")) # parsed JSON
response.status                            # HTTP status as int

That extra manual work is the tradeoff for operating closer to the HTTP layer – which matters when you need fine-grained control over connections, retries, or pooling behavior.

For a deeper look at the Requests API, see our guide to mastering Python Requests.

Feature comparison

Connection pooling

Both libraries pool connections, but they expose different levels of control.

urllib3 lets you configure pool behavior directly through PoolManager parameters like maxsize (connections per host) and block (whether to queue or raise when the pool is full):

http = urllib3.PoolManager(maxsize=10, block=True)

http = urllib3.PoolManager(maxsize=10, block=True)

Requests inherits this same pooling through urllib3 under the hood but abstracts it away. You interact with it indirectly through Session objects and HTTPAdapter. For most projects, the abstraction is sufficient. When you need explicit per-host limits or hard resource caps on a scraping worker, urllib3 gives you the knobs.

Session and cookie handling

Requests has a clear advantage here. Its Session object persists cookies, headers, and authentication across requests automatically:

session = requests.Session()
session.get("https://httpbin.org/cookies/set?token=abc123")
response = session.get("https://httpbin.org/cookies")
print(response.json())  # token persists across calls

session = requests.Session()
session.get("https://httpbin.org/cookies/set?token=abc123")
response = session.get("https://httpbin.org/cookies")
print(response.json())  # token persists across calls

urllib3 manages session state manually – you read Set-Cookie headers from responses and pass them back as Cookie headers on subsequent requests. That works for a one-off authenticated call, but it becomes unwieldy fast in multi-step login flows or any workflow where state accumulates across requests.

Authentication

Both libraries support HTTP Basic Auth, but the ergonomics differ.

Requests provides dedicated auth classes that plug into the auth= parameter:

from requests.auth import HTTPBasicAuth

response = requests.get(
    "https://httpbin.org/basic-auth/user/pass",
    auth=HTTPBasicAuth("user", "pass")
)

from requests.auth import HTTPBasicAuth

response = requests.get(
    "https://httpbin.org/basic-auth/user/pass",
    auth=HTTPBasicAuth("user", "pass")
)

urllib3 handles it through a header utility:

from urllib3.util import make_headers

headers = make_headers(basic_auth="user:pass")
response = http.request("GET", "https://httpbin.org/basic-auth/user/pass", headers=headers)

from urllib3.util import make_headers

headers = make_headers(basic_auth="user:pass")
response = http.request("GET", "https://httpbin.org/basic-auth/user/pass", headers=headers)

The functional result is identical, but Requests pulls ahead on more complex auth schemes (digest, OAuth) because its auth= parameter accepts any callable that modifies the request, making custom auth handlers straightforward to plug in.

Retry logic

This is one area where both libraries use the same engine. urllib3's Retry class is the most configurable retry tool in Python's HTTP ecosystem, giving you control over total attempts, backoff timing, and exactly which status codes trigger a retry:

import urllib3
from urllib3.util.retry import Retry

retry = Retry(
    total=5,
    backoff_factor=1,
    status_forcelist=[429, 500, 502, 503, 504]
)
http = urllib3.PoolManager(retries=retry)

import urllib3
from urllib3.util.retry import Retry

retry = Retry(
    total=5,
    backoff_factor=1,
    status_forcelist=[429, 500, 502, 503, 504]
)
http = urllib3.PoolManager(retries=retry)

Requests borrows this same Retry class through HTTPAdapter, which you mount onto a Session:

import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

retry = Retry(total=5, backoff_factor=1, status_forcelist=[429, 503])
adapter = HTTPAdapter(max_retries=retry)

session = requests.Session()
session.mount("http://", adapter)
session.mount("https://", adapter)

import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

retry = Retry(total=5, backoff_factor=1, status_forcelist=[429, 503])
adapter = HTTPAdapter(max_retries=retry)

session = requests.Session()
session.mount("http://", adapter)
session.mount("https://", adapter)

The retry behavior is identical because the underlying code is the same. Requests just requires a few extra lines of wiring. This is the pattern you want for any scraper hitting rate-limited targets. See how to retry failed Python Requests for a full breakdown.

SSL/TLS

Requests simplifies SSL through verify=True/False for certificate validation and cert= for client certificates – covering the majority of use cases.

urllib3 gives you direct access to ssl.SSLContext, which means you can configure custom CA bundles, specific cipher suites, and client certificates at the connection level. If your infrastructure requires non-standard TLS configurations, urllib3 handles them without workarounds.

If you're running into SSL errors on the Requests side, our blog post on how to fix SSLError in Python Requests covers every common cause.

HTTP/2, HTTP/3, and async

Both libraries support HTTP/1.1 only and are synchronous by design.

For HTTP/2 support without switching ecosystems, the community forks niquests and urllib3-future add that capability while keeping the familiar APIs intact. For async, the alternatives section below covers where to go when blocking I/O becomes a bottleneck.

Ease of use and syntax

Requests was built around the idea of "HTTP for humans" – method-per-verb functions (requests.get, requests.post, requests.put), automatic parameter encoding, and minimal boilerplate. The goal was to make HTTP feel like a native part of Python rather than a protocol you have to negotiate with.

urllib3 takes the opposite stance, and deliberately so. Its explicitness mirrors Python's own design principle – explicit is better than implicit. PoolManager instantiation, manual header dicts, byte response decoding – none of it happens automatically, because urllib3 is designed to be composed into larger systems where that control matters. The extra code is the feature.

Take a task every scraper handles – fetching a page with a custom User-Agent and a timeout:

Requests:

import requests

response = requests.get(
    "https://httpbin.org/get",
    headers={"User-Agent": "MyBot/1.0"},
    timeout=10
)
print(response.text)

import requests

response = requests.get(
    "https://httpbin.org/get",
    headers={"User-Agent": "MyBot/1.0"},
    timeout=10
)
print(response.text)

urllib3:

import urllib3

http = urllib3.PoolManager()
response = http.request(
    "GET",
    "https://httpbin.org/get",
    headers={"User-Agent": "MyBot/1.0"},
    timeout=urllib3.Timeout(connect=10, read=10)
)
print(response.data.decode("utf-8"))

import urllib3

http = urllib3.PoolManager()
response = http.request(
    "GET",
    "https://httpbin.org/get",
    headers={"User-Agent": "MyBot/1.0"},
    timeout=urllib3.Timeout(connect=10, read=10)
)
print(response.data.decode("utf-8"))

The urllib3 version is longer, but look at what the extra lines reveal. urllib3.Timeout(connect=10, read=10) separates the connection timeout from the read timeout – two distinct network events that Requests collapses into a single integer. With Requests, timeout=10 applies the same limit to both phases. With urllib3, you can give the server 2 seconds to accept the connection and 30 seconds to stream back a large response. For scrapers hitting slow or inconsistent targets, that distinction matters.

Learning curve

Requests is the right starting point for beginners and covers most intermediate use cases without friction. The API maps directly to how developers already think about HTTP – you want to GET something, you call requests.get(). No setup, no instantiation, no decoding.

urllib3 rewards developers who already understand HTTP at the protocol level. If you know what a connection pool is and why it matters, urllib3's explicit configuration gives you exactly the control you'd want. If you're still building that mental model, the overhead gets in the way before it helps.

When urllib3's extra code pays off

• Building HTTP libraries or frameworks. When you're writing code that other developers will depend on, you want control over every layer. Abstracting those decisions away with Requests means being constrained by Requests' opinions, not just HTTP's.
• Custom connection pool strategies. urllib3 lets you configure per-host connection limits, pool blocking behavior, and connection reuse directly. Requests inherits pooling through urllib3 but gives you no direct access to those controls.
• Fine-grained SSL configuration. urllib3's direct access to ssl.SSLContext lets you set custom CA bundles, cipher suites, and client certificates at the connection level. Requests simplifies SSL to two parameters, which covers most cases but leaves no room for non-standard TLS setups.

For most scraping projects, none of these scenarios apply – and Requests is the faster, cleaner choice. For deeper coverage of the Requests API, see how to master Python Requests.

Comparing performance and benchmarks

For single requests, performance is nearly identical. Requests adds a thin abstraction layer on top of urllib3, but that overhead is measured in microseconds – network latency, which typically runs in the tens to hundreds of milliseconds, dominates every real-world measurement.

The question changes at scale, and it's not really about which library is faster. It's about connection pooling.

How connection pooling affects performance

Every HTTP request over a new TCP connection requires a handshake before any data moves. That handshake adds latency on every call – small for a single request, significant when you're making thousands. Connection pooling solves this by keeping established connections open and reusing them across requests, eliminating repeated handshake overhead.

urllib3 handles pooling through PoolManager. When you create one instance and reuse it, connections are held open and reused automatically. When you create a new PoolManager per request, you defeat the mechanism entirely. The benchmark below shows what that costs across 100 requests:

import urllib3
import time

http = urllib3.PoolManager(maxsize=10)
urls = ["https://httpbin.org/get"] * 100

# Pooled: connections reused across all 100 requests
start = time.time()
for url in urls:
    http.request("GET", url)
pooled_time = time.time() - start

# Unpooled: new connection per request, handshake every time
start = time.time()
for url in urls:
    urllib3.PoolManager().request("GET", url)
unpooled_time = time.time() - start

print(f"Pooled: {pooled_time:.2f}s | Unpooled: {unpooled_time:.2f}s")

import urllib3
import time

http = urllib3.PoolManager(maxsize=10)
urls = ["https://httpbin.org/get"] * 100

# Pooled: connections reused across all 100 requests
start = time.time()
for url in urls:
    http.request("GET", url)
pooled_time = time.time() - start

# Unpooled: new connection per request, handshake every time
start = time.time()
for url in urls:
    urllib3.PoolManager().request("GET", url)
unpooled_time = time.time() - start

print(f"Pooled: {pooled_time:.2f}s | Unpooled: {unpooled_time:.2f}s")

The pooled version wins by a meaningful margin – not because urllib3 is faster, but because TCP handshakes compound fast across hundreds of requests.

Where Requests fits into this

Requests without a Session object creates a new connection per request, putting it in the same position as the unpooled urllib3 example above. Add a Session and connection reuse kicks in, narrowing the gap with urllib3 significantly:

import requests

session = requests.Session()
for url in urls:
    session.get(url)  # connections reused across requests

import requests

session = requests.Session()
for url in urls:
    session.get(url)  # connections reused across requests

The practical takeaway: the performance difference between urllib3 and a properly configured Requests Session is negligible for most workloads. The difference between using a Session and not using one is not.

Memory and long-running scrapers

urllib3's PoolManager holds open connections in memory for the lifetime of the pool. For a script that makes 500 requests and exits, that's irrelevant. For a persistent scraping process running continuously, open connections accumulate and need to be accounted for in resource planning. Set maxsize explicitly rather than leaving it at the default, and close the pool when it's no longer needed:

http = urllib3.PoolManager(maxsize=5)
# ... scraping work ...
http.clear()  # releases all pooled connections

http = urllib3.PoolManager(maxsize=5)
# ... scraping work ...
http.clear()  # releases all pooled connections

What the real bottleneck is

For most scraping workloads, the HTTP library is not the constraint. Network I/O, target server rate limits, and anti-bot delays account for the overwhelming majority of scrape time. Optimizing the library before addressing those factors is the wrong order of operations.

See how to web scrape without getting blocked to learn where the real leverage is.

Web scraping use cases and proxy integration

Most scraping targets track and block IP addresses. Residential proxies route your requests through real peer devices, making traffic look like it comes from genuine users rather than a data center. Both libraries support proxy integration, but the approach differs – and most guides only show the Requests side. This section covers both.

Requests with proxies

Requests uses a proxies dict that maps protocol to proxy URL. Here's how to integrate Decodo residential proxies:

import requests

proxies = {
    "http": "http://USERNAME:PASSWORD@gate.decodo.com:7000",
    "https": "http://USERNAME:PASSWORD@gate.decodo.com:7000"
}

response = requests.get("https://httpbin.org/ip", proxies=proxies)
print(response.json())

import requests

proxies = {
    "http": "http://USERNAME:PASSWORD@gate.decodo.com:7000",
    "https": "http://USERNAME:PASSWORD@gate.decodo.com:7000"
}

response = requests.get("https://httpbin.org/ip", proxies=proxies)
print(response.json())

For any scraper making more than a handful of requests, attach the proxy config to a Session so it persists automatically across every call:

session = requests.Session()
session.proxies = proxies

response = session.get("https://httpbin.org/ip")

session = requests.Session()
session.proxies = proxies

response = session.get("https://httpbin.org/ip")

urllib3 with proxies

urllib3 uses ProxyManager instead of a dict – direct instantiation with the proxy URL as the first argument:

import urllib3
import json

proxy_url = "http://USERNAME:PASSWORD@gate.decodo.com:7000"
http = urllib3.ProxyManager(proxy_url)

response = http.request("GET", "https://httpbin.org/ip")
print(json.loads(response.data.decode("utf-8")))

import urllib3
import json

proxy_url = "http://USERNAME:PASSWORD@gate.decodo.com:7000"
http = urllib3.ProxyManager(proxy_url)

response = http.request("GET", "https://httpbin.org/ip")
print(json.loads(response.data.decode("utf-8")))

ProxyManager accepts proxy_headers for any custom headers your proxy provider requires, and takes the same Retry and Timeout objects as PoolManager. When you need pool-level control over proxy connections – per-host connection limits, custom timeout profiles per target – ProxyManager gives you that directly. Requests' proxies dict doesn't.

Session persistence for multi-step scrapes

Some targets require more than a single authenticated request. Login flows, paginated scrapes, and sites that track session state across pages all depend on cookies persisting between calls. Requests' Session handles this automatically:

import requests

session = requests.Session()
session.proxies = {
    "http": "http://USERNAME:PASSWORD@gate.decodo.com:7000",
    "https": "http://USERNAME:PASSWORD@gate.decodo.com:7000"
}

# Step 1: log in - session captures the auth cookie
session.post("https://example.com/login", data={"user": "me", "pass": "secret"})

# Step 2: access protected content - cookie sent automatically
response = session.get("https://example.com/dashboard")
print(response.text)

import requests

session = requests.Session()
session.proxies = {
    "http": "http://USERNAME:PASSWORD@gate.decodo.com:7000",
    "https": "http://USERNAME:PASSWORD@gate.decodo.com:7000"
}

# Step 1: log in - session captures the auth cookie
session.post("https://example.com/login", data={"user": "me", "pass": "secret"})

# Step 2: access protected content - cookie sent automatically
response = session.get("https://example.com/dashboard")
print(response.text)

Without a Session, every request starts fresh with no cookies. For multi-step scrapes, that means re-authenticating on every call or losing state between pages.

Retry logic for scraping

Rate limits and temporary server errors (429s and 503s) are routine when scraping at scale. The pattern below combines proxy rotation with automatic retry logic so your scraper recovers without manual intervention:

import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

retry = Retry(
    total=5,
    backoff_factor=1,
    status_forcelist=[429, 503]
)

proxies = {
    "http": "http://USERNAME:PASSWORD@gate.decodo.com:7000",
    "https": "http://USERNAME:PASSWORD@gate.decodo.com:7000"
}

session = requests.Session()
adapter = HTTPAdapter(max_retries=retry)
session.mount("http://", adapter)
session.mount("https://", adapter)
session.proxies = proxies

response = session.get("https://httpbin.org/ip")
print(response.json())

import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

retry = Retry(
    total=5,
    backoff_factor=1,
    status_forcelist=[429, 503]
)

proxies = {
    "http": "http://USERNAME:PASSWORD@gate.decodo.com:7000",
    "https": "http://USERNAME:PASSWORD@gate.decodo.com:7000"
}

session = requests.Session()
adapter = HTTPAdapter(max_retries=retry)
session.mount("http://", adapter)
session.mount("https://", adapter)
session.proxies = proxies

response = session.get("https://httpbin.org/ip")
print(response.json())

backoff_factor=1 means the scraper waits 1 second before the first retry, 2 seconds before the second, 4 before the third, and so on. This prevents hammering a rate-limited target and getting your IP banned faster than the proxy can rotate it.

Decodo residential proxies rotate IPs automatically across a real peer network, which is what keeps scrapers running against Google, Amazon, and other targets with active bot detection. Both libraries work with the same credentials and endpoint.

Which library to use for scraping

Requests is the default for most scraping projects. The Session object, automatic cookie handling, and clean proxy integration cover the vast majority of patterns – single-page scrapes, login flows, paginated crawls, API automation.

Reach for urllib3 directly when you're building a scraping framework that other code will sit on top of, or when you need pool-level proxy control that Requests' dict-based integration doesn't expose.

For more on why rotating IPs matter, see our blog post on what rotating proxies are.

When to consider HTTPX or aiohttp

If concurrent requests are the real requirement, urllib3 and Requests both hit the same wall: they're synchronous. A scraper making thousands of parallel requests will spend most of its time waiting.

HTTPX is the cleanest path forward for Requests users – near-identical API, native async via httpx.AsyncClient. aiohttp is more powerful for pure async workloads but carries a steeper learning curve.

If you need HTTP/2 without switching ecosystems entirely, the community forks niquests and urllib3-future add HTTP/2 and HTTP/3 support to the familiar Requests and urllib3 APIs respectively, without requiring a full migration.

For a full async breakdown, HTTPX vs. Requests vs. AIOHTTP shows that comparison. For a wider view of the Python HTTP library landscape, see best Python HTTP clients for web scraping.

Choose the right library for your use case

The sections above cover how each library works. This one tells you which to pick.

Use Requests when:

• You're building a scraper or automating API calls. Requests is the default for application-layer HTTP work. The Session object, automatic cookie handling, built-in JSON parsing, and clean proxy integration cover everything most scrapers need. There's no reason to add boilerplate that doesn't buy you anything.
• You're working on a team. Requests' API is intuitive enough that developers without deep HTTP knowledge can read, write, and debug it without a steep learning curve. urllib3 code requires understanding what a PoolManager is, why responses come back as bytes, and how to construct headers manually – all reasonable asks for senior engineers, harder for everyone else.
• You're prioritizing iteration speed. When you're prototyping, debugging, or shipping fast, fewer lines of code means fewer places for bugs to hide. Requests gets you to a working scraper faster than urllib3 in almost every scenario.
• You need session and cookie persistence. Login flows, multi-step scrapes, and any target that tracks state across requests all depend on cookies being passed automatically. Requests' Session handles this without any extra work on your part.

Use urllib3 when:

• You're building a library or framework other developers will import. When your code is infrastructure rather than an application, you want control over every HTTP decision – connection behavior, timeout granularity, header construction, retry logic. Requests makes those decisions for you, which is a liability when developers building on top of your library need different behavior.
• You need fine-grained SSL/TLS configuration. Requests covers verify=True/False and client certificates. If you need custom CA bundles, specific cipher suites, or ssl.SSLContext-level control, urllib3 gives you direct access without workarounds.
• You're managing connection pools explicitly. High-throughput infrastructure often requires per-host connection limits, precise pool blocking behavior, and tight memory management. urllib3's PoolManager exposes those controls directly. Requests inherits pooling from urllib3 but gives you no way to configure it at that level.
• Installing Requests isn't an option. Some constrained environments restrict third-party dependencies beyond what Requests pulls in. urllib3 is a lighter dependency footprint for cases where that matters.

Use both when:

urllib3 handles the transport layer of a framework you're building, while a Requests-based application sits on top as the consumer-facing HTTP interface. The HTTPAdapter retry pattern covered in the feature comparison section is the most common version of this – you're using urllib3's Retry class through Requests' API surface. Both libraries are present; each does what it's best at.

Async is the real fork in the road

If concurrent requests are the requirement, neither library is the right answer. Both are synchronous – they block the thread until the server responds. A scraper making 1,000 parallel requests through either library will spend most of its time waiting.

HTTPX is the cleanest migration path for Requests users who need async – near-identical API, native AsyncClient. If you want a full picture of the Python HTTP library landscape before deciding, best Python HTTP clients for web scraping covers all the major options side by side.

Final thoughts

Requests wraps urllib3 with a friendlier API. At its core, choosing between them is a question of how much control you need versus how much boilerplate you're willing to manage – and for most projects, the honest answer is that the control isn't worth the cost.

Requests covers the majority of HTTP work: scraping, API calls, login flows, multi-step workflows. Its Session object, automatic encoding, and clean proxy integration handle the patterns that come up constantly in real projects. urllib3 becomes relevant when you're building infrastructure that other code depends on – a library, a framework, a high-throughput pipeline where connection pool configuration and TLS control actually matter.

The library choice is a smaller decision than it first appears. Both sit on the same transport engine. Both support proxies, retry logic, SSL configuration, and connection pooling – just at different levels of abstraction. Where developers lose time is making the wrong call for their context and spending weeks refactoring, or under-configuring a Session and wondering why performance doesn't scale.

Add proxies regardless of which library you choose. Rotating residential IPs are what keeps scrapers running against targets that block data center traffic, enforce rate limits, or fingerprint request patterns. Decodo residential proxies work with both libraries out of the box – the integration code is in the scraping section above.