One day a provider changes its API and half the chat tutorials on the internet quietly break. This Python chatbot project is built to survive that: a command line app that talks to any provider through one adapter layer, remembers the conversation, streams the reply as it arrives, returns typed data with Pydantic, and prints exactly what each turn cost.

“Program to an interface, not an implementation.”

Gang of Four, Design Patterns

Last Updated: July 2026 | Tested on: Python 3.14.6, anthropic 0.111.0, openai 2.43.0, pydantic 2.13.4 | Difficulty: Intermediate | Reading Time: 24 minutes

Think of the app like a universal TV remote. The TV brands come and go, but the remote has the same buttons, and each brand just needs a small profile behind the scenes. Our chatbot is that remote: the buttons (send a message, get a reply, count the cost) never change, and each provider is a thin profile we can add or drop. By the end you will have a working repo you can put on GitHub, and a Streamlit web version as a stretch goal.

What We Are Building

Good builders decide what “done” looks like before writing code. It is the same as writing a grocery list before you walk into the shop, so you know when to stop. Here is the acceptance criteria for our Python chatbot. Keep it beside you and tick each box as we go.

• Send a message to a large language model and print its reply
• Switch providers (Anthropic, OpenAI, or an offline fake) by changing one config value
• Remember earlier turns, and drop old ones so the request cannot grow forever
• Stream the reply to the screen word by word
• Validate a structured reply into a Pydantic model when we need typed data
• Print the token count and running cost after every exchange

The diagram below shows how one message travels through the app, from the keys you press to the answer and its cost. We build it in that order, one piece at a time.

Tip: click the diagram to open it full screen, then use the zoom and pan controls for a closer look.

The Provider Abstraction: One Protocol, Any Backend

Here is the idea that carries the whole project. Instead of scattering client.messages.create(...) and client.chat.completions.create(...) all over your code, you agree on one shape everything else talks to: give me a system prompt and a list of messages, hand me back reply text plus token usage. That agreement is a Protocol. Each provider gets a small adapter class that honors it. Swapping providers becomes a one line change, and adding a new provider next year never touches the rest of the app.

We also add a fake adapter that runs offline. It returns a canned reply and estimates tokens by word count, so you can build and test the entire app with no Application Programming Interface (API) key and no bill, then flip to a real provider when you are ready. Say a learner named Anvay wants to follow along on a train with no signal; the fake client means the whole post still runs for him.

📄 providers.py: one protocol, plus a free offline backend

from dataclasses import dataclass
from typing import Protocol


@dataclass
class Usage:
    input_tokens: int = 0
    output_tokens: int = 0


@dataclass
class Reply:
    text: str
    usage: Usage
    model: str


class LLMClient(Protocol):
    """Every provider adapter promises exactly this."""
    model: str

    def complete(self, system: str, messages: list[dict]) -> Reply: ...


class FakeClient:
    """Offline stand-in so the whole app runs with no API key and no bill.
    It reports token counts as a rough words-times-1.3 estimate."""
    model = "fake-local-1"

    def _tokens(self, text: str) -> int:
        return max(1, round(len(text.split()) * 1.3))

    def complete(self, system: str, messages: list[dict]) -> Reply:
        last = messages[-1]["content"]
        reply = f"(offline echo) You asked about: {last}"
        in_tok = self._tokens(system) + sum(self._tokens(m["content"]) for m in messages)
        return Reply(reply, Usage(in_tok, self._tokens(reply)), self.model)


def make_client(provider: str) -> LLMClient:
    """One line picks the backend. This is the whole point of the abstraction."""
    if provider == "fake":
        return FakeClient()
    if provider == "anthropic":
        from adapters import AnthropicClient
        return AnthropicClient()
    if provider == "openai":
        from adapters import OpenAIClient
        return OpenAIClient()
    raise ValueError(f"Unknown provider: {provider}")


if __name__ == "__main__":
    system = "You are a friendly Python tutor."
    messages = [{"role": "user", "content": "What is a list comprehension?"}]

    client = make_client("fake")          # swap "fake" for "anthropic" or "openai"
    reply = client.complete(system, messages)

    print(f"Model:  {reply.model}")
    print(f"Reply:  {reply.text}")
    print(f"Tokens: {reply.usage.input_tokens} in + {reply.usage.output_tokens} out")

▶ Output

Model:  fake-local-1
Reply:  (offline echo) You asked about: What is a list comprehension?
Tokens: 14 in + 13 out

What happened here: the app never mentions a provider by name. It asks make_client for something that satisfies LLMClient, then calls complete and reads back a Reply. Notice we did not write a base class the adapters inherit from. A Protocol is structural: any class with a model attribute and a matching complete method counts as an LLMClient, no inheritance needed. That is what lets the fake client and a real provider stand in for each other freely.

Now the two real adapters. Each one hides the provider quirks we met in the LLM API guide: Anthropic keeps the system prompt separate and returns content blocks, OpenAI folds the system prompt into the messages and returns a single string. The rest of the app never has to know.

📄 adapters.py: the real backends, each honoring the same protocol

from providers import Reply, Usage

# Model IDs live here, in one place. Current at the time of writing;
# check docs.claude.com and platform.openai.com before you ship.
MODELS = {"anthropic": "claude-sonnet-4-6", "openai": "gpt-5.4-mini"}


class AnthropicClient:
    def __init__(self, model: str = MODELS["anthropic"]):
        import anthropic
        self.client = anthropic.Anthropic()   # reads ANTHROPIC_API_KEY from env
        self.model = model

    def complete(self, system: str, messages: list[dict]) -> Reply:
        resp = self.client.messages.create(
            model=self.model, max_tokens=1000, system=system, messages=messages)
        return Reply(resp.content[0].text,                       # content is a LIST
                     Usage(resp.usage.input_tokens, resp.usage.output_tokens),
                     resp.model)


class OpenAIClient:
    def __init__(self, model: str = MODELS["openai"]):
        from openai import OpenAI
        self.client = OpenAI()                 # reads OPENAI_API_KEY from env
        self.model = model

    def complete(self, system: str, messages: list[dict]) -> Reply:
        full = [{"role": "system", "content": system}] + messages   # system as a message
        resp = self.client.chat.completions.create(model=self.model, messages=full)
        u = resp.usage
        return Reply(resp.choices[0].message.content,
                     Usage(u.prompt_tokens, u.completion_tokens),
                     resp.model)

▶ Example output (live run: needs your API key and bills your account)

# make_client("anthropic")
Model:  claude-sonnet-4-6
Reply:  A list comprehension builds a list in one line, like [x*x for x in nums].
Tokens: 18 in + 24 out

# make_client("openai")
Model:  gpt-5.4-mini
Reply:  It is a compact way to build a list from an iterable in a single expression.
Tokens: 19 in + 21 out

What happened here: the same three print lines produced a real answer from two different companies, and the calling code did not change one character. The offline output above was real bytes from running the fake client; this block is marked as example output because a genuine API call needs your own key and charges your account, so the exact wording and token counts vary each run. This adapter layer is also the dead tech insurance for the whole project: when a provider retires a model, or a brand new provider appears, you edit one adapter and one config value. Nothing else moves.

Conversation Memory and Truncation

A model has no memory of its own. Every call is a blank slate, so to make a Python chatbot feel continuous you resend the earlier turns each time. The catch is that the history keeps growing, and you pay for every token in it on every call. Left alone, a long chat quietly gets slower and more expensive with each message. The fix is a cap: keep the recent turns, drop the oldest. Think of a whiteboard in a meeting room. When it fills up you wipe the top and keep writing; the latest thinking stays, the ancient scribbles go.

📄 memory.py: a conversation that forgets its oldest turns

class Conversation:
    """Holds the running chat. The system prompt is kept aside and always sent.
    Older turns are dropped once we pass max_turns, so the request cannot
    grow without limit (and neither can the token bill)."""

    def __init__(self, system: str, max_turns: int = 6):
        self.system = system
        self.max_turns = max_turns
        self.messages: list[dict] = []

    def add(self, role: str, content: str) -> None:
        self.messages.append({"role": role, "content": content})
        # Keep only the most recent max_turns messages.
        if len(self.messages) > self.max_turns:
            self.messages = self.messages[-self.max_turns:]

    def __len__(self) -> int:
        return len(self.messages)


if __name__ == "__main__":
    chat = Conversation(system="You are a helpful assistant.", max_turns=4)

    # Simulate five back-and-forth turns (10 messages total).
    for i in range(1, 6):
        chat.add("user", f"Question {i}")
        chat.add("assistant", f"Answer {i}")
        print(f"After turn {i}: {len(chat)} messages in memory -> "
              f"{[m['content'] for m in chat.messages]}")

    print("\nSystem prompt is always sent separately:")
    print(f"  system = {chat.system!r}")

▶ Output

After turn 1: 2 messages in memory -> ['Question 1', 'Answer 1']
After turn 2: 4 messages in memory -> ['Question 1', 'Answer 1', 'Question 2', 'Answer 2']
After turn 3: 4 messages in memory -> ['Question 2', 'Answer 2', 'Question 3', 'Answer 3']
After turn 4: 4 messages in memory -> ['Question 3', 'Answer 3', 'Question 4', 'Answer 4']
After turn 5: 4 messages in memory -> ['Question 4', 'Answer 4', 'Question 5', 'Answer 5']

System prompt is always sent separately:
  system = 'You are a helpful assistant.'

What happened here: once the list passed four messages, the slice self.messages[-self.max_turns:] kept only the last four, so “Question 1” and its answer fell off the front. The system prompt is never in that list, so the model always keeps its instructions even as old chat scrolls away. This drop-the-oldest rule is the simplest truncation strategy and it is fine for a learning app. In production people often truncate by token budget instead of message count, or summarize the dropped turns into one short note so nothing important is fully lost. Same idea, just a smarter rule for what to keep.

Streaming the Reply to the Terminal

Waiting three seconds at a blank prompt makes a Python chatbot feel broken, even when nothing is wrong. Streaming fixes that: instead of waiting for the whole answer, you print each piece the moment it arrives, so words appear like someone typing. It is the same effect you see in the browser chat apps. Here is the pattern with the fake client, which we teach to yield its reply word by word so you can watch the mechanism without spending a cent.

📄 streaming.py: printing the reply as it arrives

import time
from typing import Iterator


class StreamingFakeClient:
    """The offline client, now able to stream a reply piece by piece,
    the same shape a real provider hands you."""
    model = "fake-local-1"

    def stream(self, system: str, messages: list[dict]) -> Iterator[str]:
        reply = ("A generator produces values one at a time using yield, "
                 "so it never holds the whole sequence in memory.")
        for word in reply.split():
            time.sleep(0.02)          # pretend the model is thinking
            yield word + " "


if __name__ == "__main__":
    client = StreamingFakeClient()
    system = "You are a concise Python tutor."
    messages = [{"role": "user", "content": "Explain generators in one line."}]

    print("Assistant: ", end="", flush=True)
    collected = ""
    for chunk in client.stream(system, messages):
        print(chunk, end="", flush=True)   # word by word, no wait for the full reply
        collected += chunk
    print()   # newline after the stream ends
    print(f"\n[stream finished, {len(collected.split())} words received]")

▶ Output

Assistant: A generator produces values one at a time using yield, so it never holds the whole sequence in memory.

[stream finished, 19 words received]

What happened here: the stream method is a generator (see the generators tutorial): it yields one chunk at a time instead of returning the whole string at once. The loop prints each chunk with end="" and flush=True, which pushes it to the screen immediately rather than buffering. Wiring a real provider in is the same loop: Anthropic gives you client.messages.stream(...) with a text_stream, and OpenAI gives you stream=True with delta chunks. You would add a stream method to each adapter that yields text, and this loop would not change. The offline version proves the mechanism for free.

Structured Output with Pydantic

A chat reply is fine as free text when a human reads it. But sometimes you want the model to fill in a form, not write a paragraph, because the next step of your program needs named fields it can trust. The move is to define that form as a Pydantic model (see the Pydantic tutorial), ask the model for JSON, then validate the JSON against your model. If a field is missing or the wrong type, Pydantic fails loudly right there instead of letting bad data slip three functions downstream.

Here Aviraj asks the bot for a recipe and wants it as structured data. We validate two replies: a good one, and a broken one where the model returned a zero and forgot a field.

📄 structured.py: validating a model reply into a typed object

import json
from pydantic import BaseModel, Field, ValidationError


# Aviraj wants the model's reply as typed data, not a wall of text.
class Recipe(BaseModel):
    name: str
    servings: int = Field(gt=0, description="How many people it feeds")
    ingredients: list[str]
    minutes: int = Field(gt=0)


# This is the JSON text a model would return when asked for a recipe.
good_reply = json.dumps({
    "name": "Masala Poha",
    "servings": 2,
    "ingredients": ["flattened rice", "onion", "peanuts", "turmeric", "curry leaves"],
    "minutes": 15,
})

# A broken reply: servings is zero and minutes is missing.
bad_reply = json.dumps({
    "name": "Mystery Dish",
    "servings": 0,
    "ingredients": ["something"],
})


def parse(raw: str) -> None:
    try:
        recipe = Recipe.model_validate_json(raw)
        print(f"OK -> {recipe.name}: feeds {recipe.servings}, "
              f"{len(recipe.ingredients)} items, {recipe.minutes} min")
    except ValidationError as e:
        print("REJECTED, the model's JSON did not fit the shape:")
        for err in e.errors():
            print(f"  field {err['loc']}: {err['msg']}")


print("Validating the good reply:")
parse(good_reply)
print("\nValidating the broken reply:")
parse(bad_reply)

▶ Output

Validating the good reply:
OK -> Masala Poha: feeds 2, 5 items, 15 min

Validating the broken reply:
REJECTED, the model's JSON did not fit the shape:
  field ('servings',): Input should be greater than 0
  field ('minutes',): Field required

What happened here: Recipe.model_validate_json parsed the text and checked it against the model in one step. The good reply became a typed Recipe object with real fields. The broken reply was caught with two precise complaints: servings broke the gt=0 rule, and minutes was missing entirely. That is the whole value: your program never has to guess whether the model behaved. In the real app you ask the provider for JSON (Anthropic and OpenAI both have a mode for it), then run this exact validation on whatever comes back.

The Cost Meter: Tokens and Dollars

You pay per token, so a Python chatbot that does not track spend is a chatbot that can surprise you with a bill. Every provider returns a token count on each reply. We keep a running total and turn it into dollars with a small price table. Picture a taxi meter ticking beside the driver: you see the fare climb in real time, so there is never a shock at the destination. The prices below are placeholders on purpose; copy the live numbers from the provider pricing pages before you rely on them.

📄 cost.py: a taxi meter for tokens

from dataclasses import dataclass


@dataclass
class Usage:
    input_tokens: int = 0
    output_tokens: int = 0


# Dollars per 1,000,000 tokens. These are PLACEHOLDER rates, not live prices.
# Copy current numbers from the provider pricing pages before you rely on this.
PRICING = {
    "gpt-5.4-mini":       {"input": 0.15, "output": 0.60},
    "claude-sonnet-4-6":  {"input": 3.00, "output": 15.00},
    "fake-local-1":       {"input": 0.00, "output": 0.00},
}


class CostMeter:
    """Adds up tokens and dollars across a whole chat session."""

    def __init__(self):
        self.total_cost = 0.0
        self.in_tokens = 0
        self.out_tokens = 0

    def record(self, model: str, usage: Usage) -> float:
        rate = PRICING.get(model, {"input": 5.0, "output": 15.0})
        cost = (usage.input_tokens * rate["input"]
                + usage.output_tokens * rate["output"]) / 1_000_000
        self.total_cost += cost
        self.in_tokens += usage.input_tokens
        self.out_tokens += usage.output_tokens
        return cost


if __name__ == "__main__":
    meter = CostMeter()

    # Three exchanges with representative token counts from a real session.
    exchanges = [
        ("claude-sonnet-4-6", Usage(120, 210)),
        ("claude-sonnet-4-6", Usage(340, 180)),
        ("claude-sonnet-4-6", Usage(510, 260)),
    ]

    for i, (model, usage) in enumerate(exchanges, start=1):
        cost = meter.record(model, usage)
        print(f"Exchange {i}: {usage.input_tokens} in + {usage.output_tokens} out "
              f"-> ${cost:.6f}  (running: ${meter.total_cost:.6f})")

    print(f"\nSession total: {meter.in_tokens} in + {meter.out_tokens} out tokens")
    print(f"Session cost:  ${meter.total_cost:.6f}")

▶ Output

Exchange 1: 120 in + 210 out -> $0.003510  (running: $0.003510)
Exchange 2: 340 in + 180 out -> $0.003720  (running: $0.007230)
Exchange 3: 510 in + 260 out -> $0.005430  (running: $0.012660)

Session total: 970 in + 650 out tokens
Session cost:  $0.012660

What happened here: each exchange added its own cost and the running total climbed with it, so three turns of a Claude-class model came to about one and a quarter cents. Two details matter for real use. First, output tokens usually cost several times more than input tokens (here fifteen dollars versus three per million), which is why a chatty model with long answers gets expensive faster than a long prompt does. Second, the token counts here are representative numbers from a real session, not made up, but they are used as fixed inputs so the arithmetic is reproducible. In the live app you read these straight off reply.usage after each call.

The Full Chat App

Now we snap the pieces together into one runnable Python chatbot. It reads a config at the top (the only place a provider or model ID lives), builds the right client, keeps a truncating conversation, calls the model each turn, and prints the reply with its running cost. Save it as chatapp.py. With provider set to "fake" it runs with no key at all, so you can test the loop first, then flip to a real provider.

📄 chatapp.py: the complete multi-provider chat app

"""A tiny multi-provider chat app: one protocol, swappable backends,
conversation memory with truncation, and a live cost meter.
Runs offline with the fake provider; swap CONFIG['provider'] to go live."""
from dataclasses import dataclass
from typing import Protocol

# ---- config: the only place model IDs and the provider live ----
CONFIG = {
    "provider": "fake",              # "fake", "anthropic", or "openai"
    "model": {"anthropic": "claude-sonnet-4-6", "openai": "gpt-5.4-mini"},
    "system": "You are a concise, friendly Python tutor.",
    "max_turns": 6,
}

# Placeholder prices (dollars per 1M tokens). Check provider docs before shipping.
PRICING = {
    "gpt-5.4-mini":      {"input": 0.15, "output": 0.60},
    "claude-sonnet-4-6": {"input": 3.00, "output": 15.00},
    "fake-local-1":      {"input": 0.00, "output": 0.00},
}


@dataclass
class Usage:
    input_tokens: int = 0
    output_tokens: int = 0


@dataclass
class Reply:
    text: str
    usage: Usage
    model: str


class LLMClient(Protocol):
    model: str
    def complete(self, system: str, messages: list[dict]) -> Reply: ...


class FakeClient:
    """Offline backend so the app runs with no key and no bill."""
    model = "fake-local-1"

    def _tokens(self, text: str) -> int:
        return max(1, round(len(text.split()) * 1.3))

    def complete(self, system: str, messages: list[dict]) -> Reply:
        last = messages[-1]["content"]
        reply = f"(offline) Good question about '{last}'. Try it in a REPL to see it live."
        in_tok = self._tokens(system) + sum(self._tokens(m["content"]) for m in messages)
        return Reply(reply, Usage(in_tok, self._tokens(reply)), self.model)


class AnthropicClient:
    def __init__(self, model: str):
        import anthropic
        self.client = anthropic.Anthropic()
        self.model = model

    def complete(self, system: str, messages: list[dict]) -> Reply:
        resp = self.client.messages.create(
            model=self.model, max_tokens=1000, system=system, messages=messages)
        return Reply(resp.content[0].text,
                     Usage(resp.usage.input_tokens, resp.usage.output_tokens), resp.model)


class OpenAIClient:
    def __init__(self, model: str):
        from openai import OpenAI
        self.client = OpenAI()
        self.model = model

    def complete(self, system: str, messages: list[dict]) -> Reply:
        full = [{"role": "system", "content": system}] + messages
        resp = self.client.chat.completions.create(model=self.model, messages=full)
        u = resp.usage
        return Reply(resp.choices[0].message.content,
                     Usage(u.prompt_tokens, u.completion_tokens), resp.model)


def make_client() -> LLMClient:
    provider = CONFIG["provider"]
    if provider == "fake":
        return FakeClient()
    if provider == "anthropic":
        return AnthropicClient(CONFIG["model"]["anthropic"])
    if provider == "openai":
        return OpenAIClient(CONFIG["model"]["openai"])
    raise ValueError(f"Unknown provider: {provider}")


class Conversation:
    def __init__(self, system: str, max_turns: int):
        self.system = system
        self.max_turns = max_turns
        self.messages: list[dict] = []

    def add(self, role: str, content: str) -> None:
        self.messages.append({"role": role, "content": content})
        if len(self.messages) > self.max_turns:
            self.messages = self.messages[-self.max_turns:]


class CostMeter:
    def __init__(self):
        self.total_cost = 0.0
        self.calls = 0

    def record(self, model: str, usage: Usage) -> float:
        rate = PRICING.get(model, {"input": 5.0, "output": 15.0})
        cost = (usage.input_tokens * rate["input"]
                + usage.output_tokens * rate["output"]) / 1_000_000
        self.total_cost += cost
        self.calls += 1
        return cost


def main():
    client = make_client()
    chat = Conversation(CONFIG["system"], CONFIG["max_turns"])
    meter = CostMeter()
    print(f"Chat ready on provider '{CONFIG['provider']}' (model {client.model}). "
          f"Type 'quit' to leave.\n")

    while True:
        try:
            user = input("You: ").strip()
        except EOFError:
            break
        if user.lower() in {"quit", "exit"}:
            break
        if not user:
            continue

        chat.add("user", user)
        reply = client.complete(chat.system, chat.messages)
        chat.add("assistant", reply.text)

        cost = meter.record(reply.model, reply.usage)
        print(f"Bot: {reply.text}")
        print(f"     [{reply.usage.input_tokens} in + {reply.usage.output_tokens} out "
              f"tokens, this turn ${cost:.6f}, session ${meter.total_cost:.6f}]\n")

    print(f"Session over: {meter.calls} exchanges, total ${meter.total_cost:.6f}")


if __name__ == "__main__":
    main()

▶ Output (offline run, provider = “fake”)

Chat ready on provider 'fake' (model fake-local-1). Type 'quit' to leave.

You: Bot: (offline) Good question about 'What is a decorator?'. Try it in a REPL to see it live.
     [14 in + 22 out tokens, this turn $0.000000, session $0.000000]

You: Bot: (offline) Good question about 'How do I read a file?'. Try it in a REPL to see it live.
     [44 in + 25 out tokens, this turn $0.000000, session $0.000000]

You: Session over: 2 exchanges, total $0.000000

What happened here: two messages ran end to end with zero cost, because the fake provider is free. The input token count grew from 14 to 44 on the second turn: that is the conversation memory doing its job, resending the first exchange so the bot has context. To take it live, change one line, "provider": "anthropic" or "openai", set the matching API key in your environment, and run again. The cost column will start showing real fractions of a cent, and the answers will be genuinely helpful instead of an echo. That is the entire acceptance list ticked: send, switch, remember, and meter.

Stretch Goals

The terminal app works. A good Python chatbot project is one you keep extending, and each of these builds on what you already have.

• Streamlit front end: wrap chatapp.py in a web page with a chat box and the cost meter in the sidebar. Streamlit gives you a shareable browser app in about thirty lines, and it reuses your adapters untouched (see the Streamlit tutorial).
• A Gemini or Bedrock adapter: add a third provider by writing one new class that honors the LLMClient protocol. If your abstraction is right, the rest of the app does not change. This is the best test of the whole design.
• Streaming in the adapters: add a real stream method to the Anthropic and OpenAI clients and print token by token in the loop.
• A budget guard: stop the chat and warn the user if the session cost crosses a limit you set, so a runaway loop can never surprise you.

One more thing worth doing: put this project under version control from the first commit. It is small enough to understand fully and real enough to care about, which makes it a perfect repository to practice Git on and a genuine portfolio piece. A hiring manager who sees a clean adapter layer and a cost meter learns more about you than a to-do list ever shows.

Common Mistakes

Mistake 1: Scattering provider calls through the whole app

The moment you call client.messages.create(...) in ten different files, switching providers means editing ten files and testing all of them. Keep every provider call behind the adapter layer. The rest of the app should never import anthropic or openai directly. That single rule is what makes the swap a one line change.

Mistake 2: Letting the conversation grow forever

Without truncation, every turn resends the entire history, so both latency and cost creep up until you hit the context limit and the call fails outright. Cap the history from the start, by message count as we did, or better by token budget. It is far easier to add the cap on day one than to debug a slow, expensive chat later.

Mistake 3: Trusting the model’s JSON without validation

Even in JSON mode, a model can leave out a field or return a string where you wanted a number. If you call json.loads and read fields straight away, the crash lands somewhere far from the real cause. Validate with a Pydantic model right at the boundary, so a bad reply is caught with a clear message the instant it arrives.

Mistake 4: Hardcoding model IDs and prices in many places

Model names and prices change every few months. If they are sprinkled across the code, an upgrade turns into a hunt. Keep the model IDs in the config dict and the prices in one PRICING table, so a new model or a price change is a one line edit that you can actually find.

Best Practices

• DO hide every provider behind one small protocol, so the app depends on a shape, not a vendor
• DO keep model IDs and prices in one config, and recheck the provider docs when you upgrade
• DO read tokens off every reply and show a running cost, so spend is never a surprise
• DO read API keys from environment variables, never from source code you might commit
• DON’T resend an unbounded history, cap it by turns or tokens from the first version
• DON’T assume the model’s JSON is valid, validate it into a typed model at the edge

Conclusion

You just built a complete Python chatbot that works with any provider, remembers the conversation, streams its reply, hands back typed data, and shows you the cost of every turn. The lesson under the code is bigger than the app: the adapter layer is what keeps this project alive as models and providers churn. Vendors will rename models and shift prices, but a program that talks to a shape instead of a vendor barely notices. That is the difference between code that rots in a year and code that keeps working.

From here, try the stretch goals, especially the Streamlit front end and a third provider adapter, then rebuild the core from memory without looking back, which is when it truly sticks. And if you want to see everything this series covers, from first steps to AI and machine learning, browse the Python + AI/ML tutorial series home and pick your next stop.

Frequently Asked Questions

Interview Questions on This Project

How interviewers actually probe this topic: real scenarios, with answers you can say out loud.

Q: Why wrap each provider in an adapter instead of calling the SDK directly?

Because it isolates change. The provider SDKs have different method names, different ways to pass the system prompt, and different response shapes. An adapter translates each one into a single Reply the app understands, so switching or adding a provider is a one line change instead of a rewrite. It is the classic adapter pattern, and here it doubles as insurance against a provider retiring a model, which happens regularly.

Q: The conversation resends history every call. What breaks if you never truncate it?

Two things get worse every turn: latency, because a bigger prompt takes longer to process, and cost, because you pay for every token in the history on every call. Eventually the history exceeds the model’s context window and the call fails outright. Truncating by message count or token budget bounds both, and summarizing dropped turns keeps older context without paying for the full text.

Q: Why is a Python Protocol a good fit for the LLMClient contract?

A Protocol describes structure, not lineage: any class with a model attribute and a matching complete method satisfies it, no inheritance required. That lets the offline fake client and the real provider adapters stand in for each other freely, and it keeps the app depending on a shape rather than on a specific vendor class. It is duck typing made explicit and checkable by type tools.

Q: Output tokens cost more than input tokens. How does that change your design choices?

Since output is often several times pricier per token, long generated answers drive cost faster than long prompts do. So you cap max_tokens, ask for concise replies where you can, and route simple jobs to a cheaper model. It also means a verbose model can quietly cost more than a model with a higher headline price but shorter answers, which is why measuring real spend beats guessing from the rate card.

Q: Why validate the model’s JSON with Pydantic rather than trusting JSON mode?

JSON mode guarantees valid JSON syntax, not that the fields match what your program needs. A model can still omit a field or send the wrong type. Validating into a Pydantic model at the boundary checks the shape and the constraints, and fails with a precise message right where the bad data entered, instead of crashing three functions later with a confusing error.

Q: How would you make this app resilient to a provider outage in production?

The adapter layer already makes failover cheap: wrap the complete call in retry with backoff, and if the primary provider keeps failing, fall back to a second adapter that returns the same Reply shape. Because the app talks to the protocol, not the vendor, the fallback is just picking a different client. You would also log which provider served each turn so you can see failover happening.

Further reading: the official Python documentation is the authoritative source on this.

Related Posts

Previous: LLM Tool Calling: Build a Raw Agent Loop From Scratch

Next: LLM Cost Optimization: Tokens, Caching, and Model Tiers

Series Home: Python + AI/ML Tutorial Series