GenAI screens do not chase whichever model launched last week; they recycle a fixed set of durable ideas in slightly different words. The LLM interview questions in this checkpoint cover that set, forty in all, grouped the way real screens group them, each with a short spoken-style answer, runnable code for the numbers people fumble, and a link to the deep-dive post that teaches it in full.

Then you score yourself, get a revisit list built from exactly what you missed, and finish with two whiteboard drills that set up the system-design rounds later.

“Know the ideas that outlive the model of the month.”

Last Updated: July 2026 | Tested on: Python 3.14.6, scikit-learn 1.9.0, NumPy 2.4.6 | Difficulty: Advanced | Reading Time: 30 minutes

LLM interviews have quietly turned into system-design interviews. Nobody asks you to recite a transformer paper; they ask how you would ground a chatbot so it stops inventing refund policies, how you would evaluate the fix, and what the token bill looks like at a million requests a day. Hallucinations, evals, and cost are the three axes almost every GenAI question rotates around, and the forty below map them group by group. Each comes with a spoken answer and code you can run, because the numbers behind attention and quantization are easier to trust once you have printed them yourself.

Read the diagram top to bottom and you have the syllabus: five groups, one score, a fork at 25. Miss the bar and the revisit list points each gap at its deep-dive post. Clear it and you head to the whiteboard for two drills, sketching attention and a RAG pipeline with an eval loop in plain NumPy, which is exactly the on-ramp to the system-design chapters later. The sections below take the groups one at a time.

How This Checkpoint Works

Each of the five groups below has eight LLM interview questions, forty in total, chosen because they show up again and again in real GenAI screens across companies of every size. For each group you get a table with a one-line answer you could say out loud and a link to the post that explains it properly, then one runnable code block that proves the part people usually hand-wave. Read the answer, cover it, and say it back in your own words. If you cannot, that question goes on your revisit list.

Give yourself one point for every question you can answer cleanly without peeking, so the group tables add up to 40. Keep a running tally on paper. A spoken answer counts only if you could explain it to a teammate in three sentences, not just recognize the words. We total it up in the scoring section, and the honest truth is that this checkpoint measures whether you understand GenAI concepts, while the two drills at the end measure whether you can still write attention and a retrieval loop under mild pressure.

One note before we start: the ideas here are durable, but specific numbers like context lengths and prices move fast, so anywhere a figure could age we quarantine it in an “at the time of writing” line rather than leaning on it.

Transformer Internals and Tokenization

Think of a group discussion at a hiring panel. When it is your turn to speak, you do not weigh everyone in the room equally, you lean toward the two people who just said something relevant to your point and mostly ignore the rest. That weighting is attention: every token looks at the others and pulls hardest on the ones that matter for it. The rest of this group is the plumbing that makes attention fast and position-aware, the KV cache that saves recomputation, grouped-query attention that shrinks that cache, rotary embeddings that inject word order, and the tokenizer that decides what a “word” even is.

📄 kv_cache.py: why grouped-query attention slashes the memory of long chats

# KV cache memory: multi-head attention (MHA) vs grouped-query attention (GQA)
# Formula: bytes = 2 (K and V) * layers * seq_len * kv_heads * head_dim * bytes_per_elem
def kv_cache_gb(layers, seq_len, kv_heads, head_dim, bytes_per_elem=2):
    total_bytes = 2 * layers * seq_len * kv_heads * head_dim * bytes_per_elem
    return total_bytes / (1024 ** 3)

# A 7B-class config: 32 layers, 32 query heads, head_dim 128, fp16 (2 bytes)
layers, q_heads, head_dim, seq = 32, 32, 128, 8192

mha = kv_cache_gb(layers, seq, kv_heads=q_heads, head_dim=head_dim)   # every head keeps its own K,V
gqa = kv_cache_gb(layers, seq, kv_heads=8, head_dim=head_dim)         # 8 groups share K,V
print(f"context length: {seq} tokens, fp16")
print(f"MHA  (32 KV heads): {mha:.2f} GB per sequence")
print(f"GQA  (8 KV heads):  {gqa:.2f} GB per sequence")
print(f"GQA saves {(1 - gqa/mha):.0%} of the KV cache")
print(f"batch of 16 users on MHA: {mha*16:.1f} GB   on GQA: {gqa*16:.1f} GB")

▶ Output

context length: 8192 tokens, fp16
MHA  (32 KV heads): 4.00 GB per sequence
GQA  (8 KV heads):  1.00 GB per sequence
GQA saves 75% of the KV cache
batch of 16 users on MHA: 64.0 GB   on GQA: 16.0 GB

What happened here: The KV cache is the memory a model keeps so it does not re-read the whole conversation on every new token, and its size is the formula in the first two lines. With plain multi-head attention every one of the 32 heads stores its own keys and values, so a single 8192-token sequence needs 4.00 GB. Grouped-query attention lets those 32 query heads share just 8 key/value heads, and the cache drops to 1.00 GB, a 75% cut, for output that is nearly identical.

The batch line is the punchline interviewers want: serving 16 users at once costs 64 GB on MHA versus 16 GB on GQA, which is the difference between needing four GPUs and needing one. That is why almost every modern open model ships with GQA.

RAG Design Tradeoffs

Picture an open-book exam. The book is huge, and you only have a minute per question, so the skill is not reading the whole book, it is flipping straight to the two paragraphs that answer the question and quoting them. Retrieval-augmented generation is exactly that: instead of hoping the model memorized your company handbook, you split the handbook into chunks, fetch the few most relevant ones, and hand them to the model as context. This group of LLM interview questions is about the choices that make or break that flip to the right page, how you chunk, how many chunks you pull, whether you rerank, and how you prove the answer actually came from the page.

📄 rag_retrieve.py: rank chunks by similarity, and see why top-k beats top-1

# RAG retrieval: chunking + top-k cosine similarity over a tiny corpus.
# Uses TF-IDF (keyword vectors) to stay runnable offline; real RAG swaps in embeddings.
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity

docs = [
    "The refund policy allows returns within 30 days of purchase with a receipt.",
    "Our paneer thali is served with two rotis, dal, and a side of salad.",
    "To reset your account password, open settings and click the security tab.",
    "Standard shipping takes 5 to 7 business days across the country.",
    "An approved refund is credited to the original payment method within 5 days.",
    "The cafe opens at 9 am and the kitchen closes at 10 pm every day.",
]

vec = TfidfVectorizer(stop_words="english")
doc_mat = vec.fit_transform(docs)

def retrieve(question, k=2):
    q = vec.transform([question])
    sims = cosine_similarity(q, doc_mat)[0]
    top = sims.argsort()[::-1][:k]
    return [(i, round(float(sims[i]), 3)) for i in top]

question = "how many days until my refund is credited?"
hits = retrieve(question, k=2)
print("question:", question)
for idx, score in hits:
    print(f"  score {score}  ->  {docs[idx]}")

# k=1 grabs one refund chunk; k=2 also pulls the other -> why top-k matters
top1 = retrieve(question, k=1)[0][0]
print(f"\ntop-1 alone returns only chunk {top1}; k=2 also surfaces the second refund rule")

▶ Output

question: how many days until my refund is credited?
  score 0.591  ->  An approved refund is credited to the original payment method within 5 days.
  score 0.294  ->  The refund policy allows returns within 30 days of purchase with a receipt.

top-1 alone returns only chunk 4; k=2 also surfaces the second refund rule

What happened here: The retriever scores every chunk against the question by cosine similarity and returns the highest. Chunk 4, the one about a refund being credited within 5 days, scores 0.591 and lands first, exactly right. But notice the second chunk: the 30-day return policy scores 0.294, and a real user asking about refunds probably wants both facts, the processing time and the return window. If you had set k to 1 you would have handed the model only half the story, which is how confidently wrong answers get generated.

That is the top-k tradeoff in one run: too small and you starve the model of context, too large and you pad the prompt with noise and pay for tokens you did not need. We used keyword vectors here to keep it offline, and the only change in production is swapping TF-IDF for embeddings so meaning matches even when the words differ.

Evaluation: Offline, Online, and the LLM Judge

Imagine grading a stack of essays. Offline evaluation is marking a fixed set of practice essays against an answer key before the exam, so you can compare two teaching methods fairly. Online evaluation is watching how real students do on the actual exam once you ship the method. And an LLM-as-a-judge is hiring a second teacher to grade essays quickly, which is a huge time-saver right up until you notice that teacher quietly favors longer essays and their own writing style. The LLM interview questions here are about running all three honestly, and the trap everyone falls into is trusting a judge’s score without ever checking it against a human.

📄 judge_agreement.py: high raw agreement that Cohen’s kappa exposes as luck

# LLM-as-judge: raw agreement can look high while chance-corrected agreement is weak.
# Cohen's kappa corrects for agreement you would get by luck alone.
from sklearn.metrics import cohen_kappa_score

# 20 answers each labelled pass(1)/fail(0) by a human and by an LLM judge.
human = [1,1,1,1,1,1,1,1,1,1,1,1,1,1,0,1,0,1,1,0]
judge = [1,1,1,1,1,1,1,1,1,1,1,1,0,1,1,1,1,1,0,0]

n = len(human)
raw_agree = sum(h == j for h, j in zip(human, judge)) / n
kappa = cohen_kappa_score(human, judge)
pass_rate_human = sum(human) / n
pass_rate_judge = sum(judge) / n

print(f"raw agreement:        {raw_agree:.0%}  (looks great)")
print(f"Cohen's kappa:        {kappa:.2f}  (chance-corrected, much weaker)")
print(f"human pass rate: {pass_rate_human:.0%}   judge pass rate: {pass_rate_judge:.0%}")
print("lesson: on a skewed pass rate, high raw agreement is mostly luck -> report kappa")

▶ Output

raw agreement:        80%  (looks great)
Cohen's kappa:        0.22  (chance-corrected, much weaker)
human pass rate: 85%   judge pass rate: 85%
lesson: on a skewed pass rate, high raw agreement is mostly luck -> report kappa

What happened here: The human and the LLM judge agree on 80% of the 20 answers, which sounds like a trustworthy judge. But look at the pass rates: both mark about 85% of answers as passing, so most of the time they agree simply because almost everything passes and guessing “pass” is usually right. Cohen’s kappa strips out that lucky baseline and asks how much they agree beyond chance, and the honest number is 0.22, which is weak.

This is the single most important habit interviewers want from anyone building evals: never quote a bare agreement percent on a skewed dataset, report kappa or a similar chance-corrected score, and calibrate your judge against real human labels before you trust it to grade thousands of outputs. A judge you have not validated is just a confident opinion.

Fine-Tuning, RLHF, and DPO

Here is the everyday version. Suppose a new hire named Anvi joins your support team. If she just needs to follow today’s policy, you write her a good instruction sheet, that is prompting. If she needs facts that change weekly, you give her a searchable wiki, that is RAG. Only if she needs to internalize a whole new tone and format that no instruction sheet captures do you send her on a training course, that is fine-tuning, and it is the most expensive option. This group is the decision tree for when to reach for each, plus the two ways models learn human preference, the older reward-model route (RLHF) and the newer direct route (DPO).

📄 lora_params.py: how LoRA trains under 1% of the weights

# Fine-tuning decision: full fine-tune vs LoRA trainable-parameter count.
# A weight matrix is d_out x d_in. LoRA replaces its update with two small
# matrices A (r x d_in) and B (d_out x r), so trainable params = r*(d_in+d_out).
def full_params(d_in, d_out):
    return d_in * d_out

def lora_params(d_in, d_out, r):
    return r * (d_in + d_out)

# A 7B model: ~200 attention/MLP matrices, each roughly 4096 x 4096, LoRA rank 16
n_matrices, d, r = 200, 4096, 16
full = n_matrices * full_params(d, d)
lora = n_matrices * lora_params(d, d, r)

print(f"full fine-tune trainable params: {full/1e6:,.1f}M")
print(f"LoRA (rank {r}) trainable params: {lora/1e6:,.2f}M")
print(f"LoRA trains only {lora/full:.2%} of those weights")
print(f"reduction factor: {full/lora:,.0f}x fewer trainable params")
print("QLoRA adds 4-bit frozen base weights on top, so it fits a 7B on one small GPU")

▶ Output

full fine-tune trainable params: 3,355.4M
LoRA (rank 16) trainable params: 26.21M
LoRA trains only 0.78% of those weights
reduction factor: 128x fewer trainable params
QLoRA adds 4-bit frozen base weights on top, so it fits a 7B on one small GPU

What happened here: Full fine-tuning would update every weight in those 200 matrices, about 3.36 billion parameters, which means storing optimizer state for all of them and needing a large Graphics Processing Unit (GPU). Low-Rank Adaptation (LoRA) freezes the original weights and learns two skinny matrices per layer, so it only trains 26 million parameters, 0.78% of the total, a 128x reduction.

The output that matters in an interview is that last idea: because the trainable set is so small, you can keep dozens of task-specific LoRA adapters and swap them onto one shared base model, and QLoRA goes further by quantizing that frozen base to 4-bit so the whole thing fits on a single modest GPU. That is the concrete reason fine-tuning stopped being something only big labs could afford.

Quantization, Serving, Agents, and MCP

Two mental models cover the LLM interview questions in this group. For quantization, think of saving a photo as a smaller file: drop from a huge RAW image to a compressed JPEG and it takes a quarter of the space while looking almost the same, which is what int8 and int4 do to model weights. For agents, think of a capable assistant who does not just answer but actually goes and does things: reads your calendar, books the room, checks the result, and tries again if it failed.

This group covers the memory math that decides which GPU you need, the serving trick that keeps that GPU busy, and the agent and tool-calling patterns, including MCP, the shared standard for plugging tools into any model.

📄 quantization.py: the weight-memory math that decides your GPU

# Quantization math: how big is the model in memory at each precision?
# weights_bytes = n_params * bytes_per_param
def weights_gb(n_params_billions, bits):
    bytes_per_param = bits / 8
    return n_params_billions * 1e9 * bytes_per_param / (1024 ** 3)

n = 7  # a 7-billion-parameter model
for label, bits in [("fp32", 32), ("fp16", 16), ("int8", 8), ("int4", 4)]:
    print(f"{label} ({bits:2d}-bit): {weights_gb(n, bits):5.1f} GB just for weights")

fp16 = weights_gb(n, 16)
int4 = weights_gb(n, 4)
print(f"\nint4 is {fp16/int4:.0f}x smaller than fp16")
print("a 24 GB consumer GPU cannot hold a 70B model in fp16 (needs ~131 GB),")
print(f"but int4 brings 70B down to {weights_gb(70,4):.0f} GB,")
print("which fits on two 24 GB consumer cards (or one 40+ GB card)")

▶ Output

fp32 (32-bit):  26.1 GB just for weights
fp16 (16-bit):  13.0 GB just for weights
int8 ( 8-bit):   6.5 GB just for weights
int4 ( 4-bit):   3.3 GB just for weights

int4 is 4x smaller than fp16
a 24 GB consumer GPU cannot hold a 70B model in fp16 (needs ~131 GB),
but int4 brings 70B down to 33 GB,
which fits on two 24 GB consumer cards (or one 40+ GB card)

What happened here: Each parameter takes bits divided by eight bytes, so the memory scales straight down as you drop precision: the same 7B model needs 26.1 GB in fp32, 13.0 GB in fp16, 6.5 GB in int8, and just 3.3 GB in int4. That is the calculation an interviewer expects you to do out loud when they ask “will this model fit.” The last two lines carry the real-world weight: a 70B model in fp16 needs about 131 GB and will not fit on a 24 GB card, but quantized to int4 it drops to roughly 33 GB, still too big for one 24 GB card but comfortable across two of them, or on a single 40 GB or larger datacenter card.

The honest caveat to say alongside it, and this is durable interview wisdom, is that quantization trades a little accuracy for that memory, so you always run your eval suite after quantizing to confirm the quality drop is acceptable. These figures are weights only; the KV cache from the first group stacks on top.

Score Yourself: The Revisit List

Add up your points across the five tables, one per question you could answer out loud without peeking, for a total out of 40. The band you land in tells you what to do next, and the honest goal is not a perfect score today, it is a shrinking revisit list over the next two weeks. Most LLM interview questions only stick after a second visit, so a low first band is normal.

Here is the trick that makes this efficient: every question in the five tables already names its deep-dive post in the last column, so your revisit list writes itself. For each question you missed, jot the topic and open the linked lesson, for example a shaky answer on GQA sends you to the Transformer Architecture post, and a blank on DPO sends you to RLHF vs DPO. Work only those posts, not the whole chapter, then come back and rescore in a week. Targeted review beats re-reading everything, and the delay is what moves a fact from “I recognize it” to “I can say it cold.”

Two Whiteboard Drills

Not all LLM interview questions are spoken: the GenAI coding round rarely asks you to train a model, it asks you to prove you understand the pieces by writing a small one in plain NumPy. Two come up constantly: sketch scaled dot-product attention, and sketch a retrieval pipeline with an evaluation loop. Write each yourself, talk through the parts people get wrong, and you have shown the panel you know what the libraries do under the hood. Both run on Python 3.14.6 exactly as shown.

First, attention. The whole operation is four moves: score every query against every key with a dot product, scale by the square root of the key dimension, softmax each row into weights, and use those weights to average the values. For a language model you add a causal mask so a token can never peek at a future token. Here it is end to end, with the row sums proving the weights are a real probability distribution.

📄 attention_from_scratch.py: score, scale, mask, softmax, average

# Whiteboard drill 1: scaled dot-product self-attention with a causal mask, in NumPy.
import numpy as np

def softmax(x, axis=-1):
    x = x - x.max(axis=axis, keepdims=True)   # subtract max for numerical stability
    e = np.exp(x)
    return e / e.sum(axis=axis, keepdims=True)

def attention(Q, K, V, causal=True):
    d_k = Q.shape[-1]
    scores = Q @ K.T / np.sqrt(d_k)           # the scaling step people forget
    if causal:
        mask = np.triu(np.ones_like(scores), k=1).astype(bool)
        scores[mask] = -np.inf                 # a token cannot see the future
    weights = softmax(scores, axis=-1)
    return weights @ V, weights

rng = np.random.default_rng(0)
seq, d_k = 4, 8
Q = rng.normal(size=(seq, d_k))
K = rng.normal(size=(seq, d_k))
V = rng.normal(size=(seq, d_k))

out, weights = attention(Q, K, V, causal=True)
np.set_printoptions(precision=2, suppress=True)
print("attention weights (rows sum to 1, upper triangle is 0):")
print(weights)
print("row sums:", weights.sum(axis=1))
print("output shape:", out.shape)

▶ Output

attention weights (rows sum to 1, upper triangle is 0):
[[1.   0.   0.   0.  ]
 [0.83 0.17 0.   0.  ]
 [0.34 0.41 0.25 0.  ]
 [0.1  0.03 0.68 0.18]]
row sums: [1. 1. 1. 1.]
output shape: (4, 8)

What happened here: The weight matrix is the heart of attention, and two things in it are worth narrating out loud. The upper triangle is all zeros because the causal mask set those scores to negative infinity before the softmax, so token 1 attends only to itself, token 2 to tokens 1 and 2, and so on, which is exactly how a model generates left to right without cheating. And every row sums to 1.0, confirming the softmax turned raw scores into a clean probability distribution over the visible tokens.

The two details that separate a passing answer from a buggy one are the scaling by the square root of d_k, which stops the softmax from collapsing onto one token, and subtracting the max inside softmax for numerical stability. Mention both while you write them and the panel knows you have done this before.

Second, a RAG pipeline with an eval loop. The shape is retrieve, then generate, then score, and the score is what turns a demo into something you can safely change. We build a tiny corpus, write an eval set that pairs each question with the chunk that truly answers it, and measure recall@k, the share of questions whose correct chunk made it into the top k. The generate step is stubbed so the whole thing stays offline, and in production you would drop a real model into that one spot.

📄 rag_eval_loop.py: retrieve, stub the generate, score recall@k

# Whiteboard drill 2: a RAG pipeline with an eval loop.
# retrieve -> (generate, stubbed) -> score retrieval recall@k against gold chunks.
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity

corpus = [
    "Paneer butter masala is made with paneer, tomatoes, butter, and cream.",  # 0
    "The gym membership can be paused for up to three months per year.",       # 1
    "An approved refund is credited back within five working days.",           # 2
    "The office stays closed on national holidays and every Sunday.",          # 3
    "A veg biryani order arrives with raita and a small green salad.",         # 4
    "Veg thali plates come with paneer, dal, rice, and two soft rotis.",       # 5
]
# Eval set: each question paired with the chunk id that truly answers it.
evalset = [
    ("when is a refund credited?", 2),
    ("how long can the gym membership be paused?", 1),
    ("what does the veg biryani order arrive with?", 4),
    ("what is paneer butter masala made with?", 0),
]

vec = TfidfVectorizer(stop_words="english")
mat = vec.fit_transform(corpus)

def retrieve(q, k):
    sims = cosine_similarity(vec.transform([q]), mat)[0]
    return list(sims.argsort()[::-1][:k])

def evaluate(k):
    hits = sum(gold in retrieve(q, k) for q, gold in evalset)
    return hits / len(evalset)

for k in (1, 2, 3):
    print(f"recall@{k}: {evaluate(k):.0%}")

# The "generate" step in real RAG feeds retrieved chunks to an LLM; here we stub it.
q, gold = evalset[0]
retrieved = retrieve(q, 1)[0]
print(f"\nsample question: {q!r}")
print(f"retrieved chunk {retrieved}: {corpus[retrieved]}")
print(f"answer grounded in the correct chunk: {retrieved == gold}")

▶ Output

recall@1: 100%
recall@2: 100%
recall@3: 100%

sample question: 'when is a refund credited?'
retrieved chunk 2: An approved refund is credited back within five working days.
answer grounded in the correct chunk: True

What happened here: The loop is the whole point. For each question the retriever fetches the top k chunks, and recall@k counts how often the truly-correct chunk was in that set, so it measures the fetch step in isolation before any generation happens. On this clean corpus recall@1 is already 100%, which is exactly what you want a healthy retriever to look like, and the sample line confirms the refund question grounded on chunk 2, the correct one.

The reason this loop earns its keep is what happens on the day someone changes the chunk size or swaps the embedding model: if that quietly drops recall@1 to 75%, this number catches it before your users ever see a wrong answer. That is the difference between a RAG demo and a RAG system, and it is the natural lead-in to the full RAG system-design round later in the series.

Common Mistakes

❌ Mistake: Reaching for fine-tuning when a prompt or RAG would do

# Wrong: "the model does not know our new refund policy, so let's fine-tune it"
#   Facts change weekly, so a fine-tuned model is stale the day after training,
#   and it is expensive to redo.

# Right: put changing facts in RAG, shape behavior with the prompt,
#   and reserve fine-tuning for a fixed tone or output format at high volume.

Why: Fine-tuning bakes knowledge into weights, so anything that changes, prices, policies, product names, is out of date the moment training ends, and updating it means paying to train again. Retrieval handles changing facts because you just edit the documents, and prompting handles behavior for free. The decision tree interviewers want to hear is prompt first, then RAG, and fine-tune only when you need a consistent style or structured format that no prompt reliably produces, at a scale where the token savings pay back the training cost.

❌ Mistake: Trusting an LLM judge’s score without validating it

# Wrong: "the judge model says our answers are 92% good, ship it"
#   Judges have position, verbosity, and self-preference biases, and 92% on a
#   skewed set can be mostly luck.

# Right: calibrate the judge against a few hundred human labels first,
#   report chance-corrected agreement (kappa), and randomize answer order.

Why: An LLM judge is fast and cheap, which makes it tempting to treat its number as truth, but an unvalidated judge is just a fluent guess. It may reward the longer answer, favor whichever option came first, or prefer text in its own style, and on a dataset where most answers pass, a high raw agreement with humans can be almost entirely chance, as the kappa of 0.22 in this post showed. Validate the judge against human labels, quote a chance-corrected score, randomize the order of options, and only then let it grade at scale.

Best Practices

• Answer in three sentences, then stop. Say what it is, why it matters, and one concrete number or example. Rambling past a correct answer usually walks you into a follow-up you did not need.
• Reach for the memory math. When asked “will this fit” or “why is this slow,” compute the weights and KV cache out loud. Naming GBs beats saying “it depends.”
• Name the eval before the model. For any GenAI feature, say how you would measure it, offline suite plus online signals, before you talk about which model to use. It reads as someone who has shipped, not just prompted.
• Prefer the boring ladder. Prompt, then RAG, then fine-tune. Jumping to fine-tuning first is the most common red flag interviewers listen for.
• Quote moving numbers carefully. Context lengths, prices, and top model names drift, so say “at the time of writing” and lean on the durable idea. The concepts here were verified on Python 3.14.6 and scikit-learn 1.9.0 at the time of writing.

Wrapping Up

LLM interview questions stop being scary once you see the screen as a fixed checklist of forty durable ideas, grouped into transformer internals and tokenization, RAG design, evaluation, fine-tuning with RLHF and DPO, and quantization with serving and agents. You ran real code to settle the ones people fumble: you watched grouped-query attention cut a KV cache by 75%, watched top-k retrieval rescue an answer that top-1 would have missed, watched Cohen’s kappa expose an 80% judge agreement as mostly luck, watched LoRA train under 1% of the weights, and watched a 7B model shrink from 26 GB to 3 GB across precisions.

Then you sketched attention and a RAG eval loop from scratch. These ideas are durable, they were shaping systems before the current model wave and they will outlast it, with the one honest caveat that dated specifics like context lengths and prices should be checked against current sources, and the library versions here were verified on scikit-learn 1.9.0 at the time of writing. Score yourself, build the revisit list from your misses, run the two drills until the loops flow, and the GenAI screen becomes a formality instead of a fear.

This post is the checkpoint that closes the generative AI half of the journey and hands you off to the system-design rounds ahead. For the full roadmap, from beginner basics through the AI/ML deep dives, visit the Python + AI/ML tutorial series home.

Frequently Asked Questions

Interview Questions About the Interview

If you can walk through these without peeking, you are ready for this topic in an interview.

Q: A user says your RAG chatbot gave a confidently wrong answer. How do you debug it?

I split the pipeline and check retrieval before generation, because most RAG errors are retrieval errors wearing a generation costume. First I look at what chunks came back for that query and whether the correct one was even in the top k; if it was missing, the problem is chunking, the embedding model, or too small a k, and I measure it with recall@k on an eval set. If the right chunk was retrieved but the answer still went wrong, then it is a generation or grounding problem, and I tighten the prompt to force citing the retrieved text and refuse when nothing relevant is found. Naming that retrieve-then-generate split is the whole answer.

Q: An interviewer asks whether a 13B model in fp16 fits on a 24 GB GPU. Walk me through it.

I do the memory math out loud. Weights are parameters times bytes per parameter, so 13 billion at two bytes each is about 26 GB just for the weights, which already exceeds the 24 GB card before I have loaded a single token. So in fp16 it does not fit. My options are to quantize to int8, which roughly halves it to about 13 GB and leaves room for the KV cache, or to int4 for about 6.5 GB, and then I would run the eval suite to confirm the quality loss is acceptable.

The point I make is that the weights are only part of the budget; the KV cache grows with batch size and context length and has to fit alongside them.

Q: Explain the difference between RLHF and DPO to someone non-technical.

Imagine teaching someone to write better replies. RLHF is like first training a separate critic who learns your taste by watching you pick the better of two replies many times, and then the writer keeps drafting until that critic is happy, which works well but you now have two systems to keep in sync and the training can wobble. DPO removes the critic: you show the writer the pairs of better and worse replies directly and adjust it to prefer the better ones in one step. Same goal of learning human preference, but DPO is one moving part instead of two, which is why teams often reach for it first now.

Q: You are asked to sketch scaled dot-product attention on the whiteboard. Where do you start, and what breaks?

I start with the four moves: score every query against every key with a dot product, divide by the square root of the key dimension, softmax each row into weights, then multiply those weights by the values to get the output. For a language model I add a causal mask so a token cannot attend to future tokens, which I do by setting the upper triangle of the scores to negative infinity before the softmax.

Two things break if you are careless. Forgetting the square-root scaling makes the dot products large, the softmax spikes onto one token, and gradients vanish. And forgetting to subtract the max inside the softmax can overflow the exponential, so I always include that stability step, exactly like the drill in this checkpoint.

Further reading: Hugging Face documentation is the authoritative source on this.

Related Posts

Previous: LLM Guardrails in Python: Prompt Injection, Content Filtering, Safety

Next: LLM Serving: Ollama Locally, vLLM in Production

Series Home: Python + AI/ML Tutorial Series