A chat tab is a demo; a product needs code. The moment you call an LLM (Large Language Model) from Python, three problems show up fast: streaming the reply, getting JSON you can trust, and letting the model trigger real tools. Function calling in Python solves that last one, and this guide covers all four building blocks with working OpenAI and Anthropic (Claude) code.

“An API call to an LLM is just an HTTP request. The magic is in what you do with the response.”

Simon Willison, Datasette creator

Last Updated: July 2026 | Tested on: Python 3.14.6, openai 2.44.0, anthropic 0.116.0, pydantic 2.13.4 | Difficulty: Advanced | Reading Time: 25 minutes

An LLM API (Application Programming Interface) is just an HTTP endpoint. You send it a prompt, it sends back the model’s reply plus some metadata (which model ran, how many tokens you used). The SDK (Software Development Kit) wraps that HTTP call in a tidy Python method so you never touch the raw request yourself. Function calling is the part that surprises people: the model does not run any code, it just looks at the user’s question, decides “I need the weather tool with city=Pune”, and hands you that decision as typed JSON. Your code runs the actual function and passes the result back. The model is the brain, your code is the hands.

Here is a quick way to picture it. Think of the LLM as a smart coworker on a phone call. They can reason brilliantly, but they cannot reach into your database or check today’s weather. So they say “can you look up X for me?”, you go do it, you read the answer back, and they fold it into their reply. Function calling is exactly that phone call, except the request comes back as clean JSON instead of a sentence you have to parse.

By the end of this guide you will have working code for both OpenAI and Anthropic, and you will see how chatbots, code assistants, data-extraction pipelines, and AI agents are all built from these same four pieces: a basic call, streaming, structured output, and function calling.

OpenAI API: The Quick Win

The diagram above shows the full function-calling loop we will build up to later in this post. For now, start at the simplest possible call. The OpenAI Python SDK wraps the REST API in a clean, typed interface. You store your key in the OPENAI_API_KEY environment variable, and the SDK reads it automatically, so it never sits in your source code. Every call is the same three steps: create a client, call chat.completions.create(), and read the response. Here is the quick win, one call from start to finish.

📄 openai_basics.py: your first OpenAI API call

from openai import OpenAI

# The SDK reads OPENAI_API_KEY from the environment, so no key in your code
client = OpenAI()  # Or, if you must: OpenAI(api_key="sk-...")

# Viraj makes his first API call
response = client.chat.completions.create(
    model="gpt-5.4-mini",            # cheap, fast model
    messages=[
        {"role": "system", "content": "You are a Python tutor. Be concise."},
        {"role": "user", "content": "What is a generator in Python? Two sentences max."},
    ],
    temperature=0.3,                 # low value = more predictable answers
    max_completion_tokens=100,                  # cap the response length
)

# Read the reply, the model name, and the token usage
message = response.choices[0].message
print(f"Response: {message.content}")
print(f"Model:    {response.model}")
print(f"Tokens:   {response.usage.prompt_tokens} prompt + "
      f"{response.usage.completion_tokens} completion = "
      f"{response.usage.total_tokens} total")

# Rough cost. The rates below are placeholders, NOT real prices.
# Always copy current per-token rates from platform.openai.com/pricing.
INPUT_RATE = 0.15   # dollars per 1M input tokens (placeholder)
OUTPUT_RATE = 0.60  # dollars per 1M output tokens (placeholder)
input_cost = response.usage.prompt_tokens * INPUT_RATE / 1_000_000
output_cost = response.usage.completion_tokens * OUTPUT_RATE / 1_000_000
print(f"Cost:     ${input_cost + output_cost:.6f} (rough, check live pricing)")

▶ Output (illustrative: a real call needs your API key and bills your account)

Response: A generator is a function that uses `yield` instead of `return` to produce a sequence of values lazily, one at a time, without storing the entire sequence in memory. This makes generators ideal for processing large datasets or infinite sequences.
Model:    gpt-5.4-mini
Tokens:   32 prompt + 47 completion = 79 total
Cost:     $0.000033 (rough, check live pricing)

What happened here: One API call, 79 tokens, a fraction of a cent. The response object hands you three things you will use constantly: the reply text, the token usage, and the model name. The messages list is the chat format you already know from the browser: a system message sets the rules, then user messages ask the questions, and you can append assistant messages to replay an earlier conversation.

Token usage matters because that is what you pay for, so we build a small cost tracker near the end of this post. One honest note: the output above is illustrative. Running this for real needs your own API key and charges your account, so the exact wording and token counts will vary a little each time.

Anthropic API: Claude in Python

Switching from OpenAI to Claude is like driving a friend’s car: same road, same rules, but the indicator stalk sits on the other side, so you flick the wrong thing once before your hands adjust. Anthropic’s Claude API works the same way, with two small differences that trip people up when they switch over. First, the system prompt is its own system= parameter, not a message inside the list. Second, the reply comes back as a list of content blocks, so you read response.content[0].text instead of a single string.

Claude is strong on long-context work (it reads up to a million tokens), careful analysis, and following detailed instructions. The model IDs below were current at the time of writing; check docs.claude.com before you ship, because new models land often.

📄 anthropic_basics.py: calling Claude from Python

import anthropic

# The SDK reads ANTHROPIC_API_KEY from the environment
client = anthropic.Anthropic()

# Rahul asks Claude for a quick code review
response = client.messages.create(
    model="claude-sonnet-4-6",       # current at the time of writing; check the docs
    max_tokens=300,
    system="You are a senior Python developer. Review code concisely.",
    messages=[
        {"role": "user", "content": (
            "Review this one-liner:\n"
            "result = [x**2 for x in range(1000000) if x % 2 == 0]"
        )},
    ],
)

# Read Claude's reply. Note content is a LIST of blocks, so index [0].text
print(f"Response: {response.content[0].text}")
print(f"Model:    {response.model}")
print(f"Tokens:   {response.usage.input_tokens} input + "
      f"{response.usage.output_tokens} output")
print(f"Stop:     {response.stop_reason}")

▶ Output (illustrative: a real call needs your API key and bills your account)

Response: This creates a list of 500,000 squared even numbers in memory all at once. Two issues:

1. **Memory**: Materializes the full list (~4MB). Use a generator expression `(x**2 for x in ...)` if you're iterating once.
2. **Efficiency**: `range(0, 1000000, 2)` skips odd numbers entirely instead of generating and filtering them.

Better: `result = [x**2 for x in range(0, 1000000, 2)]` or a generator if you don't need random access.
Model:    claude-sonnet-4-6
Tokens:   48 input + 112 output
Stop:     end_turn

What happened here: Claude caught both problems in the one-liner: the memory cost of building the whole list, and the wasted work of filtering odd numbers instead of stepping over them. Spot the three API differences from OpenAI. Anthropic calls the method messages.create(), not chat.completions.create(). The system prompt is the separate system= argument, not a message. And the reply is a list of content blocks, so you read response.content[0].text. None of these are hard, they just bite you the first time you copy OpenAI code and expect it to run against Claude.

Streaming Responses: Token by Token

Staring at a blank screen for five seconds while the model thinks feels broken, even when nothing is wrong. Streaming fixes that. Instead of waiting for the whole answer, you get each token the moment the model produces it, so words appear on screen like someone typing. It is the same trick ChatGPT and Claude.ai use in the browser. Both SDKs support it, and the code is almost identical between them.

📄 streaming.py: real-time token streaming

from openai import OpenAI
import anthropic
import time

# OpenAI: pass stream=True and loop over the chunks
openai_client = OpenAI()

print("--- OpenAI Streaming ---")
start = time.time()
stream = openai_client.chat.completions.create(
    model="gpt-5.4-mini",
    messages=[{"role": "user", "content": "Write a 4-line Python haiku about recursion."}],
    stream=True,
)

full_response = ""
for chunk in stream:
    if chunk.choices[0].delta.content:
        token = chunk.choices[0].delta.content
        print(token, end="", flush=True)   # print as it arrives, no newline
        full_response += token
print(f"\n  Total time: {time.time() - start:.1f}s")

# Anthropic: use the messages.stream() context manager
anthropic_client = anthropic.Anthropic()

print("\n--- Anthropic Streaming ---")
start = time.time()
with anthropic_client.messages.stream(
    model="claude-sonnet-4-6",
    max_tokens=200,
    messages=[{"role": "user", "content": "Write a 4-line Python haiku about decorators."}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)
print(f"\n  Total time: {time.time() - start:.1f}s")

▶ Output (illustrative: needs your API key; the poem and timing vary each run)

--- OpenAI Streaming ---
A function calls itself,
Stack frames grow like autumn leaves,
Base case sets them free.
  Total time: 1.2s

--- Anthropic Streaming ---
A wrapper descends,
@ above the function's name,
Behavior transformed.
  Total time: 1.4s

What happened here: OpenAI uses stream=True and Anthropic uses .messages.stream(), but both give you the same thing: an iterator you loop over, where each step hands you the next chunk of text the moment the model produces it. The first words usually show up in well under a second, long before the full answer is done, so the user feels an instant reply even when the whole response takes a couple of seconds. If you have used Python generators, this will feel familiar (see the generators tutorial): it is lazy evaluation, one piece at a time, instead of waiting for the whole batch.

Structured Output with Pydantic

Think of the difference between a handwritten note and a form where every box is labelled. A person reads the note just fine, but a machine wants the form. A wall of text is fine for a chatbot. For a pipeline that feeds the next step of your program, you need data with a shape: JSON whose fields your code can read without guessing. The trick is to define that shape as a Pydantic model (see the Pydantic tutorial), ask the model to return JSON, then validate the reply against your model. If the model leaves out a field or sends the wrong type, Pydantic raises a clear error instead of your code crashing three functions later.

📄 structured_output.py: type-safe LLM responses with Pydantic

from openai import OpenAI
from pydantic import BaseModel, Field
import json

client = OpenAI()

# Aditi describes the shape she wants back, as a Pydantic model
class CodeReview(BaseModel):
    file_name: str = Field(description="Name of the file being reviewed")
    issues: list[dict] = Field(description="List of issues found")
    overall_quality: str = Field(description="One of: excellent, good, needs_work, poor")
    suggested_tests: list[str] = Field(description="Test cases that should exist")

# Ask for JSON back. response_format forces the reply to be valid JSON.
response = client.chat.completions.create(
    model="gpt-5.4-mini",
    messages=[
        {"role": "system", "content": "You are a code reviewer. Output JSON matching the schema."},
        {"role": "user", "content": (
            "Review this code:\n\n"
            "def divide(a, b):\n"
            "    return a / b\n\n"
            "def average(numbers):\n"
            "    return sum(numbers) / len(numbers)"
        )},
    ],
    response_format={"type": "json_object"},
    temperature=0,
)

# Parse the text into a dict, then validate it against the model
raw = json.loads(response.choices[0].message.content)
review = CodeReview(**raw)   # raises if a field is missing or the wrong type

print(f"File: {review.file_name}")
print(f"Quality: {review.overall_quality}")
print(f"\nIssues ({len(review.issues)}):")
for issue in review.issues:
    print(f"  [{issue.get('severity', 'INFO')}] {issue.get('description', '')}")
print(f"\nSuggested Tests:")
for test in review.suggested_tests:
    print(f"  - {test}")

▶ Output (illustrative: needs your API key; the model picks the file name and wording)

File: math_utils.py
Quality: needs_work

Issues (3):
  [CRITICAL] divide() has no zero-division handling, will raise ZeroDivisionError
  [CRITICAL] average() crashes on an empty list (ZeroDivisionError from len([]) = 0)
  [WARNING] No type hints on either function

Suggested Tests:
  - test_divide_normal: assert divide(10, 2) == 5.0
  - test_divide_by_zero: assert raises ZeroDivisionError
  - test_average_normal: assert average([1, 2, 3]) == 2.0
  - test_average_empty: assert raises on empty list
  - test_average_single: assert average([5]) == 5.0

What happened here: We described the output we wanted as a Pydantic model, asked the model for JSON, then ran CodeReview(**raw) to check it. That one line is the whole point. If the model forgets a field or returns a number where you expected a string, Pydantic fails loudly right there, instead of letting bad data slip into the rest of your program. The pattern is always the same four steps: define the shape, call the LLM, validate the reply, then use the typed object.

No more fishing values out of free text with fragile regex. One tip for later: the newest SDKs also offer a helper that parses straight into your Pydantic model, but the validate-it-yourself version above shows exactly what is happening under the hood.

Function Calling: LLMs That Use Tools

This is the feature that turns a chatbot into something that can actually do work. Remember the phone-call analogy from the start: the model is the smart coworker who cannot reach your systems. Function calling is them saying “call get_weather with city=Pune for me.” The model never runs your code. It looks at the question, picks a tool, fills in the arguments, and hands you that decision as JSON. Your code runs the real function, sends the result back, and the model writes the final answer using it. That loop is how LLMs reach live data, query databases, and call other services.

📄 function_calling.py: letting the model call real Python functions

from openai import OpenAI
import json

client = OpenAI()

# Pravin writes two normal Python functions the model is allowed to call
def get_weather(city: str) -> dict:
    """Simulate weather API call."""
    weather_data = {
        "Pune": {"temp": 32, "condition": "Sunny", "humidity": 45},
        "Mumbai": {"temp": 29, "condition": "Cloudy", "humidity": 78},
        "Bangalore": {"temp": 26, "condition": "Rain", "humidity": 82},
    }
    return weather_data.get(city, {"temp": 0, "condition": "Unknown", "humidity": 0})

def calculate_bmi(weight_kg: float, height_m: float) -> dict:
    """Calculate Body Mass Index."""
    bmi = weight_kg / (height_m ** 2)
    category = (
        "Underweight" if bmi < 18.5 else
        "Normal" if bmi < 25 else
        "Overweight" if bmi < 30 else
        "Obese"
    )
    return {"bmi": round(bmi, 1), "category": category}

# Define tool schemas for the LLM
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather for a city in India",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {"type": "string", "description": "City name (e.g., Pune, Mumbai)"},
                },
                "required": ["city"],
            },
        },
    },
    {
        "type": "function",
        "function": {
            "name": "calculate_bmi",
            "description": "Calculate BMI from weight and height",
            "parameters": {
                "type": "object",
                "properties": {
                    "weight_kg": {"type": "number", "description": "Weight in kilograms"},
                    "height_m": {"type": "number", "description": "Height in meters"},
                },
                "required": ["weight_kg", "height_m"],
            },
        },
    },
]

# Map function names to actual functions
available_functions = {
    "get_weather": get_weather,
    "calculate_bmi": calculate_bmi,
}

# One message that needs TWO different tools
messages = [
    {"role": "user", "content": "What's the weather like in Pune? Also, my friend Anvi weighs 72kg and is 1.75m tall, what's her BMI?"},
]

response = client.chat.completions.create(
    model="gpt-5.4-mini",
    messages=messages,
    tools=tools,
    tool_choice="auto",  # let the model decide which tools to call
)

# The model did not run anything; it just told us what to call
tool_calls = response.choices[0].message.tool_calls
print(f"Model wants to call {len(tool_calls)} functions:\n")

messages.append(response.choices[0].message)  # keep the model's tool-call message

for tool_call in tool_calls:
    func_name = tool_call.function.name
    func_args = json.loads(tool_call.function.arguments)
    print(f"  Calling: {func_name}({func_args})")

    # YOUR code runs the real function here
    result = available_functions[func_name](**func_args)
    print(f"  Result:  {result}")

    # Hand the result back to the model, tagged with the call id
    messages.append({
        "role": "tool",
        "tool_call_id": tool_call.id,
        "content": json.dumps(result),
    })

# Ask again; now the model writes a sentence using both results
final = client.chat.completions.create(
    model="gpt-5.4-mini",
    messages=messages,
    tools=tools,
)

print(f"\nFinal response:")
print(final.choices[0].message.content)

▶ Output (illustrative: needs your API key; the model's final wording will vary)

Model wants to call 2 functions:

  Calling: get_weather({'city': 'Pune'})
  Result:  {'temp': 32, 'condition': 'Sunny', 'humidity': 45}

  Calling: calculate_bmi({'weight_kg': 72, 'height_m': 1.75})
  Result:  {'bmi': 23.5, 'category': 'Normal'}

Final response:
Here's what I found:

**Pune Weather:** It's currently 32°C and Sunny with 45% humidity, a warm and dry day.

**Anvi's BMI:** With a weight of 72kg and height of 1.75m, her BMI is 23.5, which falls in the Normal range. She's in a healthy weight category.

What happened here: From one sentence, the model figured out it needed two different tools, pulled city='Pune' and weight_kg=72, height_m=1.75 straight out of the plain English, and asked for both at once. Our code ran the real get_weather and calculate_bmi functions and passed the answers back. Only then did the model write a friendly reply that used both results. That round trip, model decides, your code acts, model answers, is the seed of every AI agent (see the AI agents tutorial): give it tools, and it strings them together to finish a task.

One safety reminder: the model can ask to call anything you expose, so validate the arguments before you run them. We come back to that in the FAQ.

The same loop in the Responses API. Here is the part that keeps this post current. The loop you just built (model decides, your code runs the function, model answers) is exactly the same in OpenAI’s newer Responses API, which OpenAI recommends for new projects at the time of writing (mid-2026). Only the field names move around. The tool schema is flatter (no nested "function" wrapper), you pass input= instead of messages=, tool calls come back as function_call items in response.output, and you hand results back as function_call_output items keyed by call_id. Learn the pattern once and swapping the API is a mechanical translation, not a rewrite.

📄 function_calling_responses.py: the same tool loop, Responses API style

from openai import OpenAI
import json

client = OpenAI()

def get_weather(city: str) -> dict:
    """Simulate a weather API call."""
    data = {"Pune": {"temp": 32, "condition": "Sunny", "humidity": 45}}
    return data.get(city, {"temp": 0, "condition": "Unknown", "humidity": 0})

# Responses API: the tool schema is flat (no nested "function" key)
tools = [{
    "type": "function",
    "name": "get_weather",
    "description": "Get current weather for a city in India",
    "parameters": {
        "type": "object",
        "properties": {"city": {"type": "string", "description": "City name"}},
        "required": ["city"],
        "additionalProperties": False,
    },
}]

# 'input' replaces 'messages'; it accepts a string or a list of items
context = [{"role": "user", "content": "What's the weather in Pune?"}]
response = client.responses.create(
    model="gpt-5.4-mini",            # current at the time of writing; check the docs
    input=context,
    tools=tools,
)

# Carry the model's function_call items into the next turn, then run each tool
context += response.output
for item in response.output:
    if item.type == "function_call":
        args = json.loads(item.arguments)
        result = get_weather(**args)
        print(f"Calling: {item.name}({args}) -> {result}")
        # Feed the result back as a function_call_output, keyed by call_id
        context.append({
            "type": "function_call_output",
            "call_id": item.call_id,
            "output": json.dumps(result),
        })

# Ask again; now the model writes the final answer from the tool result
final = client.responses.create(model="gpt-5.4-mini", input=context, tools=tools)
print("Final:", final.output_text)   # output_text joins all text output for you

▶ Example output (needs your API key; the model’s wording will vary)

Calling: get_weather({'city': 'Pune'}) -> {'temp': 32, 'condition': 'Sunny', 'humidity': 45}
Final: It is currently 32C and sunny in Pune, with about 45% humidity. A warm, dry day.

What happened here: Same three-beat loop, different plumbing. Notice the wins the Responses API gives you: final.output_text collapses the output items into a plain string so you do not index into choices[0].message.content, and passing context += response.output keeps the full turn intact for you. If you are maintaining an older Chat Completions codebase, there is no rush to move (it stays supported), but reach for the Responses API when you start something new, especially if you want its built-in tools or MCP support. The one API you should not build on today is the Assistants API, which retires on August 26, 2026; the Responses API is its official replacement. As always, confirm the current guidance at platform.openai.com/docs.

Production Error Handling & Rate Limiting

A demo talks to the API once and quits. A real product calls it thousands of times a day, and the network will not always cooperate: you will hit rate limits, timeouts, and the odd server hiccup. Think of a busy customer-care line. When it is engaged, you do not redial the instant it drops; you wait a little longer each time before trying again, so you are not adding to the pile-up. That is exactly the retry-with-backoff pattern below, plus a small cost tracker so a runaway loop cannot quietly run up your bill.

📄 error_handling.py: resilient LLM API calls

from openai import OpenAI, APIStatusError, RateLimitError, APITimeoutError
import time
from functools import wraps

client = OpenAI()

# Prathamesh wraps API calls in a retry decorator
def retry_with_backoff(max_retries=3, base_delay=1.0):
    """Retry on rate limits and transient server errors, with exponential backoff."""
    def decorator(func):
        @wraps(func)
        def wrapper(*args, **kwargs):
            for attempt in range(max_retries + 1):
                try:
                    return func(*args, **kwargs)
                except RateLimitError:
                    if attempt == max_retries:
                        raise
                    delay = base_delay * (2 ** attempt)   # 1s, 2s, 4s, ...
                    print(f"  Rate limited. Retrying in {delay}s (attempt {attempt + 1}/{max_retries})")
                    time.sleep(delay)
                except APITimeoutError:
                    if attempt == max_retries:
                        raise
                    print(f"  Timeout. Retrying (attempt {attempt + 1}/{max_retries})")
                except APIStatusError as e:
                    # APIStatusError covers any non-2xx HTTP response and carries
                    # .status_code (the base APIError does NOT). RateLimitError is a
                    # subclass, but we already handled it above.
                    if e.status_code >= 500:
                        if attempt == max_retries:
                            raise
                        print(f"  Server error {e.status_code}. Retrying...")
                        time.sleep(base_delay)
                    else:
                        raise  # 4xx (bad key, bad request) will not fix on retry
        return wrapper
    return decorator

@retry_with_backoff(max_retries=3)
def safe_completion(prompt, model="gpt-5.4-mini", **kwargs):
    """Make one API call, with automatic retries on transient failures."""
    return client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt}],
        timeout=30.0,
        **kwargs,
    )

# Cost tracking
class CostTracker:
    """Add up what you are spending across calls.
    The prices below are PLACEHOLDERS, not real rates. Copy current numbers
    from platform.openai.com/pricing and docs.claude.com before you rely on this."""
    PRICING = {  # dollars per 1M tokens (placeholders, update before production)
        "gpt-5.4-mini": {"input": 0.15, "output": 0.60},
        "gpt-5.5": {"input": 2.50, "output": 10.00},
        "claude-sonnet-4-6": {"input": 3.00, "output": 15.00},
    }

    def __init__(self):
        self.total_cost = 0.0
        self.calls = 0

    def track(self, model, input_tokens, output_tokens):
        pricing = self.PRICING.get(model, {"input": 5.0, "output": 15.0})
        cost = (input_tokens * pricing["input"] + output_tokens * pricing["output"]) / 1_000_000
        self.total_cost += cost
        self.calls += 1
        return cost

tracker = CostTracker()

response = safe_completion("What is 2+2?", max_completion_tokens=10)
cost = tracker.track(
    response.model,
    response.usage.prompt_tokens,
    response.usage.completion_tokens,
)
print(f"Call cost: ${cost:.6f}")
print(f"Running total: ${tracker.total_cost:.6f} across {tracker.calls} calls")

▶ Output (illustrative: needs your API key; the cost depends on live pricing)

Call cost: $0.000008
Running total: $0.000008 across 1 calls

What happened here: Two things every real LLM app needs. The retry decorator handles the three failures you will actually hit: rate limits (HTTP 429), timeouts, and server errors (5xx). It backs off a bit longer each time (1s, then 2s, then 4s) so you stop hammering a service that is already struggling. Notice the order of the except blocks: most specific first. We catch RateLimitError and APITimeoutError on their own, then APIStatusError for everything else with a real HTTP status code.

That last class is the one that carries .status_code; the plain APIError base class does not, so catching the base and reading .status_code would blow up on a connection error. We retry 5xx (the server’s problem) and immediately re-raise 4xx (your problem, like a bad key) because retrying a bad key just wastes time. The cost tracker is the seatbelt: a loop with one off-by-one bug can fire ten times the requests you expected and quietly run up a real bill.

Common Mistakes

Interview Corner

Q: What is function calling and why does it matter for AI agents?

Function calling lets the model pick which of your functions to call, and with what arguments, based on the user’s request. The model never runs the function itself. It returns a structured JSON payload naming the function and its arguments, your code runs the function, and you pass the result back. That loop is the foundation of AI agents (see the AI agents tutorial): the agent reasons about which tool to use, calls it, looks at the result, and repeats until the task is done. A good follow-up answer in an interview is the safety point: since the model decides what to call, you must validate the arguments before running anything with side effects.

Practice Exercises

1. Exercise 1: Make one basic call to either provider that summarizes a paragraph in three bullet points, then print the token usage.
2. Exercise 2: Define a Pydantic model with fields like name, email, and signup_date, then ask the model to extract those fields from a messy block of text and validate the JSON it returns.
3. Exercise 3: Add a third tool, convert_currency(amount, from_code, to_code), to the function-calling example and ask a question that needs the weather tool and the currency tool in the same message.

More in this series:

• How to Choose an LLM: Open vs Closed, Size, Cost, Licenses
• Python Chatbot Project: Multi-Provider Chat App with Costs
• LLM Cost Optimization: Tokens, Caching, and Model Tiers

Frequently Asked Questions

Interview Questions on LLM APIs and Function Calling

How interviewers actually probe this topic: real scenarios, with answers you can say out loud.

Q: Your function-calling loop works when the user asks one thing, but when a message needs two tools the model calls only the first one and then makes up the second answer. What do you check first?

First confirm you sent every tool result back before asking again: you must append the assistant’s tool-call message, then one tool-role message per tool_call_id, and only then make the second call. A missing result forces the model to guess, which is exactly the hallucination you are seeing. Also check that tool_choice is "auto" and not locked to a single tool, and that both tool descriptions are distinct enough for the model to tell them apart. Finally, verify the model supports parallel tool calls; if not, loop and re-call until the response no longer contains tool calls.

Q: A batch job starts throwing 429 errors and latency spikes halfway through. What do you check first?

A 429 means rate limited: you are sending requests or tokens faster than your tier allows. Check your requests-per-minute and tokens-per-minute limits in the provider dashboard, then lower concurrency or add the exponential-backoff retry from this post so callers wait 1s, 2s, 4s instead of hammering a service that is already struggling. Trim max_completion_tokens and prompt size to cut token throughput, and spread the batch out rather than firing it all at once. If it still stalls, request a higher rate tier from the provider.

Q: Both structured output (response_format JSON) and function calling return JSON. When do you use each?

Use structured output when you just want the model’s answer as typed data you can validate with Pydantic, like pulling name and email out of messy text. Use function calling when the model should decide whether to call one of your real functions, with what arguments, so your code can act on live systems. A simple rule: structured output shapes the answer, function calling triggers an action. They also compose, since a tool’s arguments are themselves a JSON schema the model must fill in.

Q: Why do you read response.content[0].text for Claude but response.choices[0].message.content for OpenAI?

The two SDKs model a reply differently. OpenAI returns a list of choices, each with a single message whose content is a string. Anthropic returns a list of content blocks (text, tool_use, and so on), so you index into content and read .text on the block you want. Same idea, different shape, and it is the number-one thing that breaks when you copy OpenAI code onto Claude.

Q: A long streaming chat session slowly eats more and more memory. What is the likely cause?

Usually the conversation history, not the stream itself. If you keep appending every user and assistant message to one growing list and resend it each turn, both memory and token cost climb with every message. Trim or summarize old turns, cap the history length, and make sure you are not also holding the full concatenated text of every past reply. Watch the context window too: past its limit the API will start rejecting the call outright.

Q: How do you stop an LLM feature from running up a surprise bill in production?

Read response.usage on every call and add up spend with a small tracker like the CostTracker in this post, then alert or hard-stop at a budget. Cap max_completion_tokens, keep prompts lean, and route cheap or simple jobs to a small model while saving the expensive one for hard tasks. Loops are the biggest risk: an off-by-one bug can fire ten times the calls you expected, so put the budget check in the loop, not just at the end.

What’s Next?

You can now call LLMs from Python, stream the reply, get typed output with Pydantic, and let the model call your functions. Those four pieces are the backbone of almost every AI feature you will build. Next, the fine-tuning tutorial shows how to bend these models to your own domain with LoRA (Low-Rank Adaptation) and QLoRA (Quantized LoRA), techniques that let you fine-tune billion-parameter models on a single Graphics Processing Unit (GPU).

To recap: you learned how to make a basic call to both OpenAI and Claude, stream the reply token by token, force typed JSON out with Pydantic, and let the model call your own Python functions, all wrapped in production-grade retries and cost tracking. Master these four pieces and you can build almost any AI feature. For the full learning path from the basics to advanced AI/ML, head back to the Python + AI/ML tutorial series home and pick your next stop.

Related Posts

Previous: GenAI: Prompt Engineering, Techniques and Best Practices

Next: LLM Tool Calling: Build a Raw Agent Loop From Scratch

Series Home: Python + AI/ML Tutorial Series