How to use a proxy with Python Requests

Setup guide · Last revised Oct 19, 2026

The basic request

Set the environment variable PROXY_URL to your proxy’s address with its login, such as http://USERNAME:[email protected]:8000, then:

import os
import requests

PROXY = os.environ["PROXY_URL"]
proxies = {"http": PROXY, "https": PROXY}

r = requests.get("https://api.ipify.org", proxies=proxies, timeout=(10, 30))
print(r.status_code, r.text)

200 203.0.113.24

Both keys point at the same http:// address even for https:// pages: the https key names the pages it is used for, not the protocol spoken to the proxy. “Note that proxy URLs must include the scheme” (Requests, Proxies). Reading the address from an environment variable keeps the login out of your code; the documentation calls storing it “in an environment variable or a version-controlled file” a security risk, so keep that variable out of anything you commit or share.

timeout=(10, 30) waits 10 seconds to connect and 30 for the answer. Without it, a stuck exit can hold your script indefinitely.

Sessions: pass proxies on every call

The documentation’s warning: “Setting session.proxies may behave differently than expected. Values provided will be overwritten by environmental proxies”, so “explicitly specify the proxies argument on all individual requests”. We checked it: with an HTTPS_PROXY variable holding a wrong password and the right one in session.proxies, Requests used the variable and failed:

requests.exceptions.ProxyError: ... Tunnel connection failed: 407 Proxy Authentication Required

A leftover HTTPS_PROXY from another tool, a container or a CI setting is enough to send your scraper through the wrong proxy, or to a login that no longer works. Pass proxies= on each request and the variable cannot win.

SOCKS5

proxies = {"http": "socks5h://USERNAME:[email protected]:1080",
           "https": "socks5h://USERNAME:[email protected]:1080"}

This needs an extra package, python -m pip install 'requests[socks]'; without it Requests stops with InvalidSchema: Missing dependencies for SOCKS support. Use socks5h: with plain socks5 “the DNS resolution” happens “on the client, rather than on the proxy server” (Requests documentation), so your own network sees every site you look up (DNS leaks).

Retries that wait

A 429 or a 502 is often over in seconds. Let urllib3, which Requests runs on, retry with a growing wait instead of writing a loop:

from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

session = requests.Session()
retry = Retry(total=3, backoff_factor=1, status_forcelist=[429, 502, 503, 504])
session.mount("https://", HTTPAdapter(max_retries=retry))
r = session.get(url, proxies=proxies, timeout=(10, 30))

When every try fails it raises RetryError (our run against a page that always answers 429: too many 429 error responses, after about 7 seconds of waits). urllib3 honours a site’s Retry-After header by default, so a site that asks you to wait two minutes gets two minutes. Every retry is traffic on a plan billed by the GB; three is enough to ride out a blip without paying for a block. When the retries run out, the codes mean: 429, 504, 403.

The errors you will see

Next

How to test a proxy checks the address, location and speed before a long run. What a scraper’s traffic costs a month: one scraper for a month.