Best Rotating Proxy Services for Python Requests in 2026

By Elena Park · 2026-04-04 · 12 min read · Engineering

Five rotating proxy services that work flawlessly with Python's requests library. Code examples for each.

Short answer and the code that proves it

For rotating proxies with Python requests in 2026, Decodo is the best default: a 115M+ IP residential pool, standard gateway authentication that works with requests out of the box, and mid-tier pricing around $2.20-3.50/GB. Bright Data and Oxylabs are the picks for enterprise scale and hard targets, IPRoyal for the lowest cost of entry, Webshare for a free tier to validate your code.

Integration is three lines. You point the proxies dict at a rotating gateway and every request leaves from a different exit IP, with no rotation logic of your own: proxies = {"http": url, "https": url} where url is http://user:[email protected]:7000.

The engineering that matters is not rotation - the gateway handles that - it is sessions, retries, timeouts and error classification. That is where most Python scrapers lose their success rate.

Minimal working setup

Use a requests.Session even with rotating proxies. The session gives you connection pooling and consistent headers; the gateway still rotates the exit IP per request unless you pin a session token in the username.

Always set a timeout as a tuple: timeout=(5, 30) means five seconds to connect and thirty to read. A missing timeout is the single most common reason a scraper appears to hang forever, because a dead proxy hop never returns.

• s = requests.Session(); s.proxies = {"http": PROXY, "https": PROXY}
• s.headers.update({"User-Agent": UA, "Accept-Language": "en-US,en;q=0.9"})
• r = s.get(url, timeout=(5, 30)); r.raise_for_status()
• Verify the exit IP once at startup against an IP echo endpoint
• Read credentials from environment variables, never from the source file

Sticky sessions in requests

When a workflow needs one IP across several requests - a login, pagination that depends on server-side state, a checkout - encode a session token in the proxy username. Most providers accept user-session-abc123 or user-sessid-abc123 with a TTL of 1 to 30 minutes.

Generate one token per logical flow, build a dedicated Session for it, and discard both together. Reusing a single token across your entire crawl defeats the point of a rotating pool and concentrates your traffic on one IP until it is rate limited.

Country and city targeting uses the same mechanism, so switching geography in Python means changing a string, not adding a dependency.

Retries that help instead of hurting

Mount an HTTPAdapter with urllib3 Retry configured for the statuses that are genuinely transient, and let permanent failures fail fast. Retrying a 407 or a 404 wastes bandwidth you are billed for; retrying a 429 without backoff escalates a soft limit into a hard ban.

Use backoff_factor with jitter, cap total retries at three, and respect Retry-After when the server sends it. Count retries per target host so one broken site cannot consume the whole budget.

• Retry on 429, 500, 502, 503, 504 - transient
• Never retry 407 (bad credentials or allowlist) or 404 - permanent
• Treat repeated 403 as an identity problem: rotate session or upgrade proxy type, do not retry blindly
• backoff_factor plus random jitter; cap at 3 attempts
• Honour Retry-After headers when present

Concurrency without wrecking your success rate

requests is synchronous, so throughput comes from threads. A ThreadPoolExecutor with 10-50 workers is the practical range for most residential gateways; beyond that you hit provider concurrency limits and target-side rate limits at the same time and cannot tell them apart.

Ramp concurrency rather than starting at maximum, and instrument success rate per worker count so you can find the knee in the curve. If you need thousands of concurrent requests, move to httpx or aiohttp with async, but do that after your error handling is solid, not before.

Never share one Session across threads if you are also pinning sticky sessions - one Session per identity keeps the mapping between cookie jar and exit IP intact.

Provider comparison for Python workloads

Every provider here works with requests via the same gateway pattern, so the differences are pool quality, price and how much unblocking they do for you.

• Decodo - best default: 115M+ residential IPs, ~$2.20-3.50/GB, city and ASN targeting, clean docs for Python
• Bright Data - largest audited network, strongest compliance documentation, unblocker and SERP APIs alongside raw proxies
• Oxylabs - best when you want managed scraper APIs for hard targets instead of maintaining fingerprints in Python
• IPRoyal - cheapest credible residential at roughly $1.75/GB, pay-as-you-go with no minimum
• SOAX - best when part of the workload is mobile or needs carrier targeting
• Webshare - free tier for validating code before you spend anything

When requests is the wrong tool

If the data only exists after JavaScript runs, requests cannot see it no matter which proxy you use. Check first whether the page fetches JSON from an internal endpoint - calling that endpoint directly with requests is faster and cheaper than rendering.

If the target runs serious bot management, the blocker is your TLS fingerprint, not the IP. requests presents a JA3 hash that identifies the library, so you either swap in a client that mimics browser TLS, drive a real browser, or hand the page to a managed unblocking endpoint.

Rough decision rule: static HTML plus moderate protection means requests. Client-side rendering or aggressive bot management means a browser or a managed API.

Cost control in Python

Bandwidth is the bill, so shrink what you download. Send Accept-Encoding: gzip (requests handles decompression), use stream=True and abort on responses over a size threshold, and never fetch images or fonts you do not parse.

Cache by URL hash so retries and re-runs do not re-buy the same bytes, and deduplicate your frontier before dispatching. As a planning figure, 1 GB covers roughly 5,000-10,000 lightweight HTML pages or 1,500-3,000 image-heavy product pages.

Log bytes and status per request. Without that, you cannot tell an expensive scraper from a broken one.

Common mistakes

Nearly every failing Python proxy scraper we look at shares a small set of defects, and none of them are about the provider.

• No timeout, so dead proxy hops hang the worker indefinitely
• Retrying non-transient errors and paying for the bandwidth
• One sticky session token reused across the whole crawl
• Default requests headers, so the request is trivially identified
• Assuming a 403 means the proxy is bad rather than the fingerprint
• Hardcoded credentials committed to the repository
• No per-request logging of status, bytes and exit IP, so failures cannot be diagnosed

A checklist before you scale up

Prove the pipeline small before you buy volume: confirm the exit IP rotates, confirm sticky sessions hold, confirm retries behave under an injected 429, and confirm your parser handles a challenge page instead of silently storing garbage.

Then run a 48-hour parallel test on two providers against your real targets and compare cost per successful record. Aggregate success rates, including the ones we publish, are a starting point - your specific targets decide the winner.

If your target list includes heavily protected sites, budget for a managed endpoint on those specific URLs and keep requests for the easy majority. That hybrid is usually the cheapest configuration overall.

Frequently Asked Questions