You need a list of users from another service, so you open a browser and copy the data by hand. That works exactly once. It falls apart the moment you need it every hour, for 50 cities, saved to a file. The Python requests library is how you stop: send an HTTP call from code and get back clean JSON. This post covers GET through DELETE, API keys and bearer tokens, pagination, and surviving timeouts and flaky servers.

“An API is just a website for machines.”

Common saying among backend developers

Last Updated: July 2026 | Tested on: Python 3.14.6, requests 2.34.2 | Difficulty: Intermediate | Reading Time: 20 minutes

Almost every app you use talks to an API behind the scenes. Your weather app calls a weather service. Your CI pipeline calls GitHub. Your office Slack bot calls Slack. The shape of it never changes: send an HTTP request with the right headers, get back a response (usually JSON), read it. The requests library makes that whole round trip feel like calling a normal Python function.

An API, short for Application Programming Interface, is a contract between your code and a remote server. You say “give me user number 1” by sending GET /users/1, and the server hands back a JSON object describing that user. Think of it like ordering at a restaurant. You do not walk into the kitchen and cook. You read the menu (the API docs), tell the waiter exactly what you want (the request), and the kitchen sends out a plated dish (the response). You never touch the stove.

That is the big difference between an API and web scraping. Scraping a website is like reading someone’s messy handwriting off a napkin. Calling an API is like reading a clean spreadsheet they prepared just for you. APIs are built for programs to use, so they come structured, documented, and predictable.

Tip: click the diagram to open it full screen, then use the zoom and pan controls for a closer look.

Here is the whole API cycle in one picture. Your Python client attaches its headers and login (auth), sends an HTTP request like GET or POST to the server, and the server sends back a JSON response with a status code. If the data is split across pages, you loop: ask for page 2, then page 3, until there is nothing left. That loop is the backbone of every real API integration, and as you will see, requests turns each arrow into a single line of code.

Prerequisites

You should be comfortable with Dictionaries, CSV and JSON, and Exception Handling, since every API response comes back as JSON that you read like a dict, and real network code fails often enough that you will lean on try/except. Knowing a little about HTTP (URLs, status codes) helps, but it is not required. We cover the parts you need as we go.

Install & Verify

The requests library is not part of the standard library, so you install it from PyPI (the Python Package Index). Do this inside a virtual environment so it does not clutter your system Python.

📄 Terminal: install requests

pip install requests

Now check that it imported and see which version you got. If this prints a version number instead of an error, you are ready.

📄 Terminal: verify the install

python -c "import requests; print(requests.__version__)"

▶ Output

2.34.2

What happened here: your exact version may differ, and that is fine. Anything in the 2.3x range behaves the same for everything in this post. If instead you saw ModuleNotFoundError: No module named 'requests', the install landed in a different Python than the one running your script. That almost always means your virtual environment is not active, so activate it and run the install again.

The Quick Win: Your First API Call

Let us get a win on the board in under a minute. A Python requests call is like sending a text message to a very reliable friend: you ask one clear question, wait a moment, and get one clear answer back. Here the friend is JSONPlaceholder, a free fake API that needs no key, no signup, nothing. It is the perfect sandbox to practice against.

📄 first_api_call.py: fetch one user from a public API

import requests

# JSONPlaceholder is a free fake API, no auth required
response = requests.get("https://jsonplaceholder.typicode.com/users/1", timeout=15)
response.raise_for_status()  # raise an exception for any 4xx/5xx status

user = response.json()  # parse the JSON body into a Python dict
print(f"Name: {user['name']}")
print(f"Email: {user['email']}")
print(f"City: {user['address']['city']}")
print(f"Status: {response.status_code}")

▶ Output

Name: Leanne Graham
Email: Sincere@april.biz
City: Gwenborough
Status: 200

What happened here: requests.get() sent an HTTP GET request to that URL and waited for the reply. The server answered with a JSON object, and .json() turned that text into an ordinary Python dictionary, so user['address']['city'] is just normal dict access. The raise_for_status() line is your safety net. If the server had returned a 404 or a 500, it would raise an exception right there instead of letting you carry on with bad data. And notice timeout=15. We will come back to why that one keyword is what saves your script from hanging forever.

HTTP Methods: GET, POST, PUT, PATCH, DELETE

HTTP has a small set of verbs, and each one maps to a CRUD action (Create, Read, Update, Delete). The names are blunt on purpose. GET reads, POST creates, PUT and PATCH update, DELETE removes, and Python requests mirrors each one with a method of the same name. Think of a shared whiteboard: GET is reading what is on it, POST is adding a new note, PUT is wiping a note and rewriting it from scratch, PATCH is editing one word on an existing note, and DELETE is erasing it. In the code below, imagine a developer named Viraj publishing his first blog post through an API: we create it, read it, update it two ways, then delete it.

📄 crud_operations.py: GET, POST, PUT, PATCH, DELETE

import requests

BASE = "https://jsonplaceholder.typicode.com"

# CREATE: POST sends a JSON body to make a new resource
new_post = {
    "title": "Viraj's API Guide",
    "body": "REST APIs are just HTTP with conventions.",
    "userId": 1,
}
resp = requests.post(f"{BASE}/posts", json=new_post, timeout=15)
print(f"Created: {resp.status_code}, ID: {resp.json()['id']}")

# READ: GET fetches an existing resource
resp = requests.get(f"{BASE}/posts/1", timeout=15)
print(f"Read: {resp.json()['title'][:40]}...")

# UPDATE (full): PUT replaces the entire resource
updated = {"title": "Updated Title", "body": "New body", "userId": 1}
resp = requests.put(f"{BASE}/posts/1", json=updated, timeout=15)
print(f"Updated: {resp.json()['title']}")

# UPDATE (partial): PATCH only sends the fields that changed
resp = requests.patch(f"{BASE}/posts/1", json={"title": "Patched Title"}, timeout=15)
print(f"Patched: {resp.json()['title']}")

# DELETE: remove the resource
resp = requests.delete(f"{BASE}/posts/1", timeout=15)
print(f"Deleted: {resp.status_code}")

▶ Output

Created: 201, ID: 101
Read: sunt aut facere repellat provident occae...
Updated: Updated Title
Patched: Patched Title
Deleted: 200

What happened here: the one parameter doing the heavy lifting is json=. When you pass json=new_post, requests serializes your dict to a JSON string and sets the Content-Type: application/json header for you. POST returned 201 Created (the standard “new thing made” status) with a fresh id of 101. The difference between PUT and PATCH is worth burning into memory: PUT replaces the whole record, so you must send every field, while PATCH touches only the keys you hand it. One quick note on this sandbox: JSONPlaceholder fakes the writes, so it reports success without truly saving anything. Against a real API, that same code would create and change actual rows.

Query Parameters and the Real URL

You rarely want every record. You want “the comments on post 1” or “users sorted by name”. Those filters ride along in the URL as query parameters, the part after the ?. Think of it like a coffee order: the drink is the endpoint, and the query parameters are “oat milk, no sugar, extra hot”. You could glue that string together by hand, but escaping spaces and special characters yourself is a classic way to introduce bugs. Hand requests a dict through params= instead, and it builds a correct, encoded URL for you.

📄 query_params.py: let requests build the URL

import requests

# params is a dict; requests turns it into ?postId=1 for you
resp = requests.get(
    "https://jsonplaceholder.typicode.com/comments",
    params={"postId": 1},
    timeout=15,
)
print(f"Final URL: {resp.url}")
print(f"Status: {resp.status_code}")
print(f"Comments on post 1: {len(resp.json())}")
print(f"First commenter email: {resp.json()[0]['email']}")

▶ Output

Final URL: https://jsonplaceholder.typicode.com/comments?postId=1
Status: 200
Comments on post 1: 5
First commenter email: Eliseo@gardner.biz

What happened here: printing resp.url shows exactly what requests assembled, which is the first thing to check when an API returns nothing you expected. The dict {"postId": 1} became ?postId=1, properly encoded. Add more keys and they get joined with & automatically. When you read API docs and see a list of accepted query parameters, this is where they go.

Authentication: API Keys, Bearer Tokens, OAuth

Most real APIs need to know who you are before they hand over data. Authentication is just proving your identity on each Python requests call you send. There are a handful of common ways to do it, and a token is basically a hotel key card: the front desk issues it after you check in, and from then on the card alone opens your door. The card does not know your name, it just proves you are allowed in.

The code below shows the five patterns you will meet in practice, including basic auth, where an admin user named Aditi signs in with a plain username and password. It uses placeholder URLs and fake keys, so treat it as a reference to copy from rather than something to run.

📄 api_auth.py: five ways to authenticate (reference, fill in real values)

import requests

# Method 1: API key in a query parameter
resp = requests.get(
    "https://api.openweathermap.org/data/2.5/weather",
    params={"q": "Mumbai", "appid": "YOUR_API_KEY", "units": "metric"},
    timeout=15,
)

# Method 2: API key in a header
headers = {"X-API-Key": "YOUR_API_KEY"}
resp = requests.get("https://api.example.com/data", headers=headers, timeout=15)

# Method 3: Bearer token (OAuth2 / JWT), the most common today
token = "eyJhbGciOiJIUzI1NiIsInR5..."
headers = {"Authorization": f"Bearer {token}"}
resp = requests.get("https://api.example.com/me", headers=headers, timeout=15)

# Method 4: Basic auth (username and password)
resp = requests.get(
    "https://api.example.com/admin",
    auth=("aditi", "secure_password_123"),
    timeout=15,
)

# Method 5: a Session reuses auth and the connection across calls
session = requests.Session()
session.headers.update({"Authorization": f"Bearer {token}"})
resp1 = session.get("https://api.example.com/users", timeout=15)
resp2 = session.get("https://api.example.com/orders", timeout=15)  # same auth, no repeat

What happened here: the bearer token (Method 3) is the one you will use most. The API gives you a token after you log in, and you send it in the Authorization header on every call. The Session in Method 5 is the upgrade you reach for once you make more than one call: set the auth header once and every request from that session carries it, plus the session keeps the TCP (Transmission Control Protocol) connection open so back-to-back calls are faster. One rule with no exceptions: never paste a real key or token straight into your code. Read it from an environment variable or a .env file, because anything hardcoded will eventually get committed to git and leaked.

Pagination: Getting All the Data

An API will not dump 10,000 records on you in one response. That would be slow, and it could topple the server. Instead it serves data in pages, like the search results on a shopping site: page 1, page 2, and a “next” button at the bottom. Your job in Python requests code is to keep clicking that next button until there is nothing left. Here we grab every post written by one user, asking for small pages on purpose so the loop actually has to run a few times.

📄 pagination.py: loop through every page

import requests

BASE = "https://jsonplaceholder.typicode.com"

def fetch_all_posts(user_id):
    """Fetch every post by one user, one page at a time."""
    all_posts = []
    page = 1
    per_page = 4

    while True:
        resp = requests.get(
            f"{BASE}/posts",
            params={"userId": user_id, "_page": page, "_limit": per_page},
            timeout=15,
        )
        resp.raise_for_status()
        posts = resp.json()

        if not posts:                 # an empty page means we are done
            break

        all_posts.extend(posts)
        print(f"Page {page}: fetched {len(posts)} posts")

        if len(posts) < per_page:     # a short page is always the last one
            break
        page += 1

    return all_posts

posts = fetch_all_posts(user_id=1)
print(f"\nTotal posts fetched: {len(posts)}")
for p in posts[:3]:
    print(f"  #{p['id']}: {p['title'][:40]}")

▶ Output

Page 1: fetched 4 posts
Page 2: fetched 4 posts
Page 3: fetched 2 posts

Total posts fetched: 10
  #1: sunt aut facere repellat provident occae
  #2: qui est esse
  #3: ea molestias quasi exercitationem repell

What happened here: the while True loop keeps asking for the next page and only stops on one of two signals. Either the page comes back empty (if not posts), or the page is shorter than the size we asked for (len(posts) < per_page), which can only happen on the final page. User 1 has 10 posts, so with 4 per page we got pages of 4, 4, and 2, then stopped. Different APIs signal “last page” differently: some give you a total_pages number, some put a next URL in the response, some use a Link header. Always read the docs, but the loop shape stays the same: fetch, collect, check for more, repeat.

Error Handling: Timeouts, Retries, Status Codes

The network is the one part of your program you do not control. Servers go down, connections drop, and a request that took 50 milliseconds yesterday hangs for 30 seconds today. It is a bit like calling a friend: sometimes they pick up at once, sometimes it rings forever, sometimes the line is busy. Hopeful Python requests code that assumes every call just works will eventually freeze or crash in production. So we wrap it in armor. We add a timeout so it never waits forever, automatic retries for the errors that are usually temporary, and named exception handlers so each failure tells you what actually went wrong.

📄 robust_api_client.py: timeouts, retries, and clear error handling

import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry

def create_robust_session():
    """A session that retries flaky errors and never hangs forever."""
    session = requests.Session()

    retry_strategy = Retry(
        total=3,
        backoff_factor=1,     # wait 1s, then 2s, then 4s between tries
        status_forcelist=[429, 500, 502, 503, 504],
    )
    adapter = HTTPAdapter(max_retries=retry_strategy)
    session.mount("https://", adapter)
    session.mount("http://", adapter)

    return session

session = create_robust_session()

try:
    resp = session.get(
        "https://jsonplaceholder.typicode.com/posts/1",
        timeout=10,  # give up waiting after 10 seconds
    )
    resp.raise_for_status()
    print(f"Success: {resp.json()['title'][:40]}...")

except requests.exceptions.Timeout:
    print("Request timed out after 10 seconds")
except requests.exceptions.ConnectionError:
    print("Could not connect to the server")
except requests.exceptions.HTTPError as e:
    print(f"HTTP error: {e.response.status_code}")

▶ Output

Success: sunt aut facere repellat provident occae...

What happened here: the happy path printed success, but the real value is in the parts that did not fire today. The Retry strategy tells requests to automatically try again, up to three times, whenever it sees a 429 (too many requests) or a 5xx server error. The backoff_factor spaces those retries out (1s, then 2s, then 4s) instead of hammering a struggling server. The timeout=10 caps how long a single attempt can wait. And each except block names a specific failure, so when something breaks at 2am your logs say “timed out” or “could not connect” instead of a generic stack trace. This is roughly the shape of every production HTTP client.

It is also worth seeing what an unhandled error looks like, so you recognize it. Ask for a post that does not exist and call raise_for_status():

📄 error_404.py: what a failed request actually raises

import requests

resp = requests.get("https://jsonplaceholder.typicode.com/posts/99999", timeout=15)
print(f"Status code: {resp.status_code}")
resp.raise_for_status()   # this line raises because 404 is an error status

▶ Output

Status code: 404
Traceback (most recent call last):
  File "error_404.py", line 5, in <module>
    resp.raise_for_status()   # this line raises because 404 is an error status
    ~~~~~~~~~~~~~~~~~~~~~^^
  File ".../requests/models.py", line 1167, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://jsonplaceholder.typicode.com/posts/99999

What happened here: notice the request itself did not fail. The server happily returned a response with status code 404, and requests considered that a perfectly valid HTTP exchange. It is raise_for_status() that looks at the 404 and decides to raise an HTTPError. That is exactly why you call it: without it, your code would sail past the 404 and try to .json() an error page, then blow up somewhere far away with a confusing message. The traceback shown here is from running the file as error_404.py; your file path will differ.

Ecosystem: requests vs httpx vs aiohttp

Picking an HTTP library is like picking a vehicle: a scooter for quick errands, a car for daily commutes, a cargo truck when you move hundreds of boxes at once. requests is the scooter, and most days that is all you need. It is the default choice for synchronous HTTP: simple, battle tested, with the biggest ecosystem of any Python HTTP library. For scripts, CLI (command-line interface) tools, cron jobs, and any code that does one thing at a time, it is exactly what you want, which is why this whole post uses it.

httpx is the car: the modern sibling of requests. The API looks almost identical, so most code ports over by changing the import. What you gain is async/await support and HTTP/2, plus better type hints. Reach for httpx on a new project, especially anything built on asyncio. To prove how close they are, here is the quick-win call again, swapping only the import.

📄 httpx_demo.py: same call, httpx instead of requests

import httpx

resp = httpx.get("https://jsonplaceholder.typicode.com/users/1", timeout=15)
resp.raise_for_status()
user = resp.json()
print(f"httpx {httpx.__version__}")
print(f"Name: {user['name']}")
print(f"HTTP version: {resp.http_version}")

▶ Output

httpx 0.28.1
Name: Leanne Graham
HTTP version: HTTP/1.1

What happened here: the only change from the very first example was requests.get becoming httpx.get. Everything else, including raise_for_status() and .json(), works the same. httpx also exposes extras like resp.http_version. The third option, aiohttp, is the cargo truck: a pure async client built on asyncio. It shines when you need to fire off hundreds of requests at once (think scraping 500 URLs in parallel), where its lower overhead pays off. It takes more boilerplate than httpx, so most people only reach for it at real scale.

Common Mistakes

Mistake 1: Not checking the status code

🚫 Wrong

resp = requests.get("https://api.example.com/data", timeout=10)
data = resp.json()  # crashes later if the server sent a 500 with an HTML error page

✅ Correct

resp = requests.get("https://api.example.com/data", timeout=10)
resp.raise_for_status()  # raises HTTPError for any 4xx/5xx
data = resp.json()

Why: a 200 status is the only promise that the body is the data you wanted. On an error, many servers return an HTML page, and calling .json() on HTML throws a confusing parse error far from the real cause. raise_for_status() fails loudly at the right line.

Mistake 2: No timeout, so the script hangs forever

🚫 Wrong

resp = requests.get("https://slow-api.example.com/data")  # can wait forever

✅ Correct

resp = requests.get("https://slow-api.example.com/data", timeout=10)  # gives up after 10s

Why: by default requests has no timeout at all. If the server accepts your connection but never replies, your program just sits there. Put a timeout on every single request. It is one keyword and it is the difference between a slow API and a frozen program.

Mistake 3: Sending JSON with data= instead of json=

🚫 Wrong

import json
payload = {"title": "Anvay's note", "userId": 1}
resp = requests.post(url, data=json.dumps(payload), timeout=10)  # wrong content type

✅ Correct

payload = {"title": "Anvay's note", "userId": 1}
resp = requests.post(url, json=payload, timeout=10)  # sets Content-Type: application/json

Why: here a user named Anvay is saving a note through your API, and the data= parameter sends it form-encoded without setting the JSON content type. Many APIs then reject the body or misread it. The json= parameter does both jobs: it serializes your dict and sets Content-Type: application/json. Let requests do the work.

Wrapping Up

You now have the complete Python requests toolkit: GET, POST, PUT, PATCH, and DELETE mapped to CRUD, query parameters built safely with params=, all five authentication patterns from API keys to bearer tokens, a pagination loop that keeps going until the data runs out, and a production-grade session with timeouts and retries baked in. The two habits worth carrying into every script you write from today: put a timeout on every request, and call raise_for_status() before you touch the body.

Next we cross to the other side of the counter: instead of calling someone else’s API, you will build your own web app with Flask, complete with routes, templates, and forms. And if you want to see where this post sits in the bigger journey, browse the full Python + AI/ML tutorial series home.

Frequently Asked Questions

Try It Yourself

Build a script that fetches the top 30 GitHub repositories for a given language using the GitHub Search API (https://api.github.com/search/repositories?q=language:python&sort=stars). Pull out each repo name, star count, description, and last updated date. The Search API returns 30 results per page by default, so handle pagination with the page parameter if you want more, and set a timeout on every call. Save the results to a JSON file. Bonus: the unauthenticated rate limit is low, so add a clear message when you hit a 403, and read the X-RateLimit-Remaining header to see how many calls you have left.

Interview Questions on Python Requests

If you can walk through these without peeking, you are ready for this topic in an interview.

Q: Why should a script that makes many calls to the same API use requests.Session instead of plain requests.get()?

Every bare requests.get() opens a fresh TCP connection, does the TLS handshake, sends one request, and throws the connection away. A Session keeps a pool of open connections, so back-to-back calls to the same host skip the handshake entirely, which can cut per-request latency dramatically on HTTPS. It also carries shared state: set an Authorization header or cookies once on the session and every request sends them automatically. For anything beyond a one-off call, a Session is the correct default.

Q: An API call returns 200 OK, but resp.json() raises a JSONDecodeError. What do you check first?

Print resp.text and resp.headers.get("Content-Type") to see what the body actually is. A 200 with unparseable JSON usually means the server sent HTML: a login page after your session expired, a WAF or captcha challenge page, or a redirect landing page, and resp.url will show where you really ended up after redirects. It can also be an empty body (a 204-style response served as 200), which JSON parsing rejects. The status code only promises the request succeeded at the HTTP level, not that the body is JSON.

Q: What is idempotency, and why does it matter when you configure automatic retries?

An idempotent request produces the same result no matter how many times you repeat it: GET, PUT, and DELETE are idempotent by design, POST is not. If a POST times out after the server already processed it, an automatic retry creates the resource twice, which is how duplicate orders and double payments happen. That is why urllib3’s Retry only retries idempotent methods by default and you must opt in via allowed_methods to retry POST. The safer fix for POST is an idempotency key header, which many payment APIs support so a repeated request is deduplicated server-side.

Q: Your nightly cron job hits a partner API and fails two or three times a week with 502 or 503, but the same call succeeds when you rerun it in the morning. How do you make the job resilient without hammering their server?

Those are transient server-side errors, so the fix is automatic retries with exponential backoff, not tighter error handling. Mount an HTTPAdapter with a Retry(total=3, backoff_factor=1, status_forcelist=[429, 500, 502, 503, 504]) onto a Session, which waits roughly 1s, 2s, then 4s between attempts instead of retrying instantly. Honor the Retry-After header when the server sends one (Retry respects it by default for 429 and 503). Keep a per-request timeout too, because retries multiply the worst-case runtime, and log the final failure so a genuinely down partner still pages you.

Q: What is the difference between timeout=10 and timeout=(3, 10) in requests, and what does the read timeout actually measure?

A single number applies the same limit to both phases; the tuple form sets a connect timeout (time to establish the TCP connection) and a read timeout separately, so timeout=(3, 10) gives up after 3 seconds if it cannot connect and 10 seconds while reading. The subtle part: the read timeout is the maximum gap between bytes arriving, not a cap on the whole response. A server that trickles one byte every 9 seconds never trips timeout=(3, 10), so a slow download can still run far longer than 10 seconds.

If you need a hard wall-clock limit on the entire call, you have to enforce it yourself, for example with concurrent.futures or by switching to an async client with a total timeout.

Q: A teammate named Anvi silences an SSL certificate error by adding verify=False to every request. What do you tell her?

That flag turns off certificate verification entirely, so the script will happily talk to any machine that intercepts the traffic, which makes bearer tokens and API keys readable to a man-in-the-middle. The right fix is to repair trust: upgrade certifi so the CA bundle is current, or if the API uses an internal corporate CA, point requests at it with verify="/path/to/ca.pem" or the REQUESTS_CA_BUNDLE environment variable. verify=False is acceptable only as a temporary debugging step against a local test server, never in committed code.

Go deeper: when you outgrow this post, Requests documentation is the next stop.

Related Posts

Previous: Python: SQLAlchemy Object-Relational Mapping (ORM), Models, Sessions, Queries

Next: Python: Web Scraping with BeautifulSoup

Series Home: Python + AI/ML Tutorial Series