The Python Requests Library: GET, POST, and the Footguns Nobody Warns You About

requests is the library almost every Python project reaches for to talk to an API over HTTP. It's been the de facto standard for over a decade, sitting behind millions of real projects — from one-off scripts to production services — because it turns what would be several lines of urllib boilerplate into one readable call.

This isn't a reference dump of every method the library has. It's the calls you'll actually write, the response fields worth checking, and — because this is the part most tutorials skip — the defaults that work fine in a five-minute script and quietly break once that script becomes something a service depends on. Several of these are things I've personally had to debug in production: a hung worker thread, a "500" silently parsed as valid JSON, a retry storm that made an outage worse. They show up here because they're real, not because a checklist said to include them.

For the authoritative, exhaustive reference, the official Requests documentation is the right place to look up something this page doesn't cover. This page is the version aimed at getting the defaults right the first time.

Installing it

pip install requests

It's not part of the standard library, so it needs to be in your project's dependencies — add it to requirements.txt or pyproject.toml rather than relying on it being globally installed.

A basic GET request

import requests

response = requests.get("https://api.github.com/users/octocat")
print(response.status_code)
print(response.json())

requests.get() sends the request and blocks until a response comes back. response.status_code is the HTTP status (200, 404, 500, …), and .json() parses the response body as JSON into a Python dict — it raises requests.exceptions.JSONDecodeError if the body isn't actually JSON, which is worth knowing before you call it blindly on an error response that came back as HTML.

Query parameters, the right way

response = requests.get(
    "https://api.github.com/search/repositories",
    params={"q": "language:python", "sort": "stars"},
)
print(response.url)

Pass params= as a dict instead of building the query string by hand. requests handles the ? and & separators and URL-encodes each value — a search term with a space or an & in it won't silently corrupt the URL the way string concatenation would.

Sending JSON with POST

response = requests.post(
    "https://api.example.com/tickets",
    json={"title": "Login button misaligned", "priority": "low"},
)

json= does two things: serializes the dict to a JSON string, and sets the Content-Type: application/json header for you. That second part matters — many APIs decide how to parse the body based on that header, so passing the same dict via data= instead sends it as URL-encoded form data, and the server will likely reject it or misread it.

params= and json= aren't mutually exclusive, and combining them is a genuinely common, real pattern most references don't show:

response = requests.post(
    "https://api.example.com/tickets",
    params={"notify": "true"},
    json={"title": "Login button misaligned", "priority": "low"},
)

This sends ?notify=true in the URL and the ticket data as the JSON body in the same request — useful for an API that separates request metadata (should this trigger a notification?) from the actual payload.

Checking whether it actually worked

response = requests.get("https://api.example.com/users/999999")
response.raise_for_status()

A 404 or 500 response is still a successful HTTP exchange as far as requests is concerned — the request went out and something came back, so no exception is raised automatically. response.raise_for_status() turns a 4xx or 5xx status into a requests.exceptions.HTTPError you can catch, instead of code silently continuing to process an error page as if it were data.

One real, well-documented gap worth knowing about: raise_for_status()'s default error message doesn't include the response body. If an API returns a 400 with a specific, useful reason in the body ("email already registered"), the raw exception message won't show it — this is a known, tracked limitation, not a bug you're doing something wrong to hit. The practical fix is logging the body yourself before re-raising:

try:
    response.raise_for_status()
except requests.exceptions.HTTPError as err:
    print(f"Request failed: {err} — body: {response.text[:500]}")
    raise

Slicing to 500 characters keeps a genuinely huge error page from flooding your logs while still capturing the part that usually explains what went wrong.

The default nobody expects: no timeout

# This can hang forever if the server never responds
response = requests.get("https://api.example.com/data")

# This can't
response = requests.get("https://api.example.com/data", timeout=5)

requests has no default timeout. If the server accepts the connection but never sends a response — a hung process, a firewall silently dropping packets — your code waits indefinitely. In a script that's an annoyance; in a web server handling that request inside another request, it's one slow dependency taking down every worker thread that calls it. This is genuinely the single most common way a requests call takes down a production service, and it's also the one the library gives you zero warning about — pass timeout= on every call that talks to a service you don't fully control, including internal ones.

A timeout can also be a (connect, read) tuple — a shorter limit for establishing the connection, a longer one for waiting on the actual response body:

response = requests.get("https://api.example.com/data", timeout=(3.05, 27))

Reusing connections with a Session

session = requests.Session()
session.headers.update({"Authorization": f"Bearer {token}"})

response = session.get("https://api.example.com/profile")
response = session.get("https://api.example.com/settings")

Each plain requests.get() call opens a new TCP connection and, for HTTPS, redoes the TLS handshake — expensive when you're making several calls to the same host. A Session reuses the underlying connection via HTTP keep-alive, and lets you set headers (like an auth token) once instead of repeating them on every call.

Retrying failed requests automatically

A Session is also where automatic retries belong. requests doesn't retry anything on its own — a connection reset, a 503, a 429 rate-limit response all just fail once and stop, and it's genuinely surprising the first time you realize retry logic isn't built in at all. The real, standard fix is mounting a retry policy from urllib3 (the library requests is built on) onto an adapter:

from requests.adapters import HTTPAdapter
from urllib3.util import Retry

retry_strategy = Retry(
    total=3,
    backoff_factor=1,
    status_forcelist=[429, 500, 502, 503, 504],
    allowed_methods=["GET", "POST"],
)

session = requests.Session()
session.mount("https://", HTTPAdapter(max_retries=retry_strategy))

response = session.get("https://api.example.com/data", timeout=5)

backoff_factor=1 spaces retries out (roughly 1s, 2s, 4s) instead of hammering a struggling service immediately three times in a row — genuinely important, since an aggressive, unspaced retry loop from every failing client at once is exactly how a brief outage turns into a longer one. status_forcelist is deliberately narrow here: retrying a 404 or 400 wastes real time, since the request wasn't the problem — the request itself was invalid, and running it again produces the identical, correct-but-unwanted result.

Handling failures without a stack trace dump

try:
    response = session.get("https://api.example.com/profile", timeout=5)
    response.raise_for_status()
except requests.exceptions.Timeout:
    print("Request timed out — the service may be overloaded.")
except requests.exceptions.ConnectionError:
    print("Couldn't reach the service — check the URL or network.")
except requests.exceptions.HTTPError as err:
    print(f"Service returned an error: {err}")

requests.exceptions.RequestException is the base class every one of these inherits from, so except requests.exceptions.RequestException: catches all of them if you don't need to distinguish the cause. Catching the specific ones — the way exception handling in Python generally recommends — means a timeout and a 404 don't get reported to the user with the same generic message when they call for different fixes.

What a production-ready request looks like

import requests
from requests.adapters import HTTPAdapter
from urllib3.util import Retry

def build_session() -> requests.Session:
    session = requests.Session()
    retry_strategy = Retry(
        total=3, backoff_factor=1,
        status_forcelist=[429, 500, 502, 503, 504],
    )
    session.mount("https://", HTTPAdapter(max_retries=retry_strategy))
    return session

def fetch_profile(session: requests.Session, user_id: str) -> dict | None:
    try:
        response = session.get(
            f"https://api.example.com/users/{user_id}",
            timeout=5,
        )
        response.raise_for_status()
        return response.json()
    except requests.exceptions.Timeout:
        print(f"Timed out fetching profile for {user_id}")
        return None
    except requests.exceptions.HTTPError as err:
        print(f"Profile fetch failed for {user_id}: {err} — body: {response.text[:300]}")
        return None

A session for connection reuse, a mounted retry policy for the failures that are genuinely worth retrying, an explicit timeout so a stuck dependency can't hang the caller, raise_for_status() so a 500 doesn't get parsed as if it were valid data, and exception handling specific enough to tell "the network is down" apart from "this user doesn't exist." None of it is complicated — it's just the difference between code that works in a demo and code that survives a bad network day in production. This exact combination — timeout, retry, and translating a raw API response into something clean — is the same pattern this site's AI agent tutorials reach for the moment an agent needs to call a real, external API reliably.