In the LangChain State of AI Agents survey of 1,340 practitioners, quality beat cost and safety as the number one obstacle to shipping agents. AI agent evaluation is how you find out whether the agent you built in posts 225 and 234 actually finishes tasks or just looks busy calling tools. And agent evals are not chat evals: you grade a whole trajectory of tool calls, not one message.
Two agents can land on the same reply while one took two clean steps and the other flailed through five, looped on a bad argument, and burned triple the tokens. Good AI agent evaluation measures that difference, and this post shows you how to gate deploys on it.
“Testing leads to failure, and failure leads to understanding.”
Burt Rutan
Last Updated: July 2026 | Tested on: Python 3.14.6, stdlib only | Difficulty: Advanced | Reading Time: 22 minutes
- AI agents tutorial (the agent this post evaluates)
- LLM as a judge (the judge we reuse for the subjective calls)
- LLM observability with Langfuse (where trajectories become traces you can read)
Think of it like grading a driving test instead of a photograph of the finish line. You do not just check that the learner reached the parking spot, you watch how they got there: mirrors checked, no wrong turns, no stalling at every light. The agent version of that recording is the trajectory, and the whole discipline of agent evaluation is turning it into numbers you can trust, then wiring those numbers to a gate that stops a bad build before your users meet it.
Table of Contents
Why Agent Evals Are Not Chat Evals
A chat eval grades one turn: here is the question, here is the reply, score it. That is the world of the Large Language Model (LLM)-as-a-judge post, and it works because there is exactly one thing to look at. An agent breaks that assumption. Give a food-ordering agent the task “how much are two plates of paneer tikka?” and it might search the menu, look up a price, and answer.
Or it might look up the price with a wrong argument name, get an error, retry the same broken call three times, then give up. Same task, wildly different behavior, and if you only read the final message you cannot tell the two apart until the second one starts costing you real money and real latency at scale.
So AI agent evaluation grades the path, not just the destination. That means recording every step the agent takes and asking a different set of questions: did it pick the right tools, in a sensible order, with valid arguments, without going in circles, and did the final answer actually solve the task? Those questions turn into five metrics.
What to Measure: Five Agent Metrics
Five numbers cover most of what you care about. Task success rate is the headline: what fraction of tasks ended in a correct answer. Trajectory quality asks whether the agent used the right tools in the right order. Tool-error rate counts how many tool calls came back as failures. Loops and steps flag agents that repeat themselves or wander. Cost and latency per task put a price tag on the whole run. The diagram below shows two trajectories for the very same task so you can see, at a glance, what these metrics are separating.
The healthy run on the left reaches the answer in two clean steps with zero errors. The pathological run on the right reaches no answer at all: it passes a wrong argument name to get_price, retries the identical failing call three times, wanders into check_stock, and gives up. Both would look similar in a naive chat log. The code below turns each trajectory into those five numbers so the difference stops being a hunch and becomes data.
📄 trajectory_metrics.py: five metrics from one recorded run
# A trajectory is the ordered list of steps an agent took to finish one task.
# Each step: the tool it called, the args, and whether the tool returned cleanly.
# These two were captured from the food-ordering agent under test.
good = {
"task": "How much are two plates of paneer tikka?",
"steps": [
("search_menu", {"q": "paneer tikka"}, "ok"),
("get_price", {"item": "paneer_tikka"}, "ok"),
],
"final": "Two plates of paneer tikka cost 360 rupees.",
"expected_facts": ["360"],
}
pathological = {
"task": "How much are two plates of paneer tikka?",
"steps": [
("search_menu", {"q": "paneer tikka"}, "ok"),
("get_price", {"dish": "paneer_tikka"}, "error: bad arg 'dish'"),
("get_price", {"dish": "paneer_tikka"}, "error: bad arg 'dish'"),
("get_price", {"dish": "paneer_tikka"}, "error: bad arg 'dish'"),
("check_stock", {"item": "paneer_tikka"}, "ok"),
],
"final": "I could not find the price, please check the menu board.",
"expected_facts": ["360"],
}
def metrics(traj, step_cost=0.002, step_latency_ms=550):
steps = traj["steps"]
n_steps = len(steps)
tool_errors = sum(1 for _, _, status in steps if status.startswith("error"))
# a loop = the same (tool, args) pair repeated back to back
repeats = sum(1 for a, b in zip(steps, steps[1:])
if (a[0], str(a[1])) == (b[0], str(b[1])))
success = all(fact in traj["final"] for fact in traj["expected_facts"])
return {
"success": success,
"steps": n_steps,
"tool_errors": tool_errors,
"looped": repeats > 0,
"cost_usd": round(n_steps * step_cost, 4),
"latency_ms": n_steps * step_latency_ms,
}
for label, traj in [("GOOD", good), ("PATHOLOGICAL", pathological)]:
m = metrics(traj)
print(f"[{label}] {traj['task']}")
print(f" success={m['success']} steps={m['steps']} "
f"tool_errors={m['tool_errors']} looped={m['looped']} "
f"cost=${m['cost_usd']} latency={m['latency_ms']}ms")
▶ Output
[GOOD] How much are two plates of paneer tikka? success=True steps=2 tool_errors=0 looped=False cost=$0.004 latency=1100ms [PATHOLOGICAL] How much are two plates of paneer tikka? success=False steps=5 tool_errors=3 looped=True cost=$0.01 latency=2750ms
What happened here: The same task produced two very different report cards. The good run: success, two steps, no errors, under half a cent. The pathological run: failure, five steps, three tool errors, a detected loop, and two and a half times the cost and latency. Notice the loop detector is dead simple, just checking whether the same tool and arguments repeat back to back, but that one check is what separates “took a couple of extra steps” from “stuck spinning”. These are the units the rest of the post aggregates across a real eval set.
A 20-Task Eval Set
One trajectory tells you about one run. To judge an agent you need a set of tasks that looks like real usage, with the correct outcome pinned for each so a check can run without a human. Twenty tasks is a sensible starting point for a small agent: enough to catch patterns, small enough to run on every change. Where the answer is a hard fact (a price, opening hours, an order total) an automated check verifies it with no model in the loop. The harder, subjective calls get an LLM judge later. Here is the suite running against the agent, with the aggregate metrics it produces.
📄 eval_set.py: 20 tasks, automated success checks, aggregate metrics
# 20 tasks for the vegetarian food-ordering agent from posts 225 and 234.
# Each task ships with the trajectory the agent produced, the final text, and
# expected_facts an automated check can verify. Status "ok" or "error: ..."
# lets us count tool failures without a model. (Full list trimmed for print;
# 10 clean tasks, then 5 designed failures, then 5 more clean tasks.)
TASKS = [
("Price of paneer tikka?",
[("get_price", {"item": "paneer_tikka"}, "ok")],
"Paneer tikka is 180 rupees.", ["180"]),
("Order 3 chole bhature",
[("check_stock", {"item": "chole_bhature"}, "ok"),
("place_order", {"item": "chole_bhature", "qty": 3}, "ok")],
"Order placed: 3 chole bhature, total 450 rupees.", ["450"]),
# ... 8 more clean tasks ...
("Price of dal makhani?", # wrong tool: searched, never priced
[("search_menu", {"q": "dal makhani"}, "ok")],
"Dal makhani is on the menu.", ["220"]),
("Order 2 samosa", # bad args: qty as a string, repeated
[("place_order", {"item": "samosa", "qty": "two"}, "error: qty must be int"),
("place_order", {"item": "samosa", "qty": "two"}, "error: qty must be int")],
"Sorry, I could not place that order.", ["120"]),
("Price of two plates paneer tikka?", # runaway loop on a bad arg
[("get_price", {"dish": "paneer_tikka"}, "error: bad arg 'dish'")] * 4,
"I hit an error looking that up.", ["360"]),
("Is rasgulla available?", # gave up without calling any tool
[],
"I am not sure, please ask staff.", ["available"]),
("Price of veg pulao?", # wrong tool then stopped
[("check_stock", {"item": "veg_pulao"}, "ok")],
"Veg pulao is in stock.", ["150"]),
# ... 3 more clean tasks ...
]
STEP_COST = 0.002 # rough dollars per model step (tool call round trip)
STEP_LATENCY_MS = 550 # rough wall-clock per step
def eval_task(steps, final, expected):
n = len(steps)
errs = sum(1 for _, _, s in steps if s.startswith("error"))
repeats = sum(1 for a, b in zip(steps, steps[1:])
if (a[0], str(a[1])) == (b[0], str(b[1])))
success = n > 0 and all(f in final for f in expected)
return success, n, errs, repeats > 0
def run_suite(tasks):
passed = steps_sum = err_sum = looped = 0
for _task, steps, final, expected in tasks:
ok, n, errs, loop = eval_task(steps, final, expected)
passed += ok; steps_sum += n; err_sum += errs; looped += loop
return {"total": len(tasks), "passed": passed, "steps_sum": steps_sum,
"err_sum": err_sum, "looped": looped,
"success_rate": passed / len(tasks),
"avg_steps": steps_sum / len(tasks),
"tool_error_rate": err_sum / max(steps_sum, 1)}
r = run_suite(TASKS)
print(f"tasks evaluated : {r['total']}")
print(f"task success rate : {r['passed']}/{r['total']} = {r['success_rate']:.0%}")
print(f"avg steps per task : {r['avg_steps']:.2f}")
print(f"tool-error rate : {r['err_sum']}/{r['steps_sum']} steps = {r['tool_error_rate']:.0%}")
print(f"tasks that looped : {r['looped']}")
print(f"avg cost per task : ${r['avg_steps'] * STEP_COST:.4f}")
print(f"avg latency per task : {r['avg_steps'] * STEP_LATENCY_MS:.0f} ms")
print(f"gate decision (>=80%) : {'PASS' if r['success_rate'] >= 0.80 else 'FAIL'}")
▶ Output
tasks evaluated : 20 task success rate : 15/20 = 75% avg steps per task : 1.30 tool-error rate : 6/26 steps = 23% tasks that looped : 2 avg cost per task : $0.0026 avg latency per task : 715 ms gate decision (>=80%) : FAIL
What happened here: Fifteen of twenty tasks succeeded, so the agent scores 75 percent, under our 80 percent bar. The aggregate numbers point straight at the problem: a 23 percent tool-error rate and two looping tasks are dragging the success rate down. A single number (“75 percent”) tells you the build is not good enough, but the supporting metrics tell you why, which is the whole reason we record trajectories instead of just final answers. Now we need to know what kind of failures these five are.
A Failure Taxonomy for Agents
A success rate tells you how many tasks failed. A failure taxonomy tells you how they failed, which is what you actually fix. Agent failures fall into a small number of repeating shapes. Wrong tool: the agent picked a tool that cannot answer the question, like checking stock when it needed a price. Bad args: the right tool with malformed arguments. Gave up: it stopped without seriously trying. Runaway loop: it repeated the same failing call over and over. The classifier below sorts every failed run into exactly one bucket, checking the loudest symptom first.
📄 failure_taxonomy.py: sort each failed run into one bucket
from eval_set import TASKS, eval_task
# Rules that sort a failed run into ONE bucket. Order matters:
# check the loudest, most specific symptom first.
def classify(steps, final, expected):
n = len(steps)
errs = [s for _, _, s in steps if s.startswith("error")]
repeats = sum(1 for a, b in zip(steps, steps[1:])
if (a[0], str(a[1])) == (b[0], str(b[1])))
called = {t for t, _, _ in steps}
if n == 0:
return "gave_up" # never even tried a tool
if repeats >= 2 and errs:
return "runaway_loop" # same failing call, again and again
if errs:
return "bad_args" # tool exists, arguments were wrong
if "get_price" not in called and any(f.isdigit() for f in expected):
return "wrong_tool" # needed a price, never priced
return "other"
buckets = {}
print("Failed runs and their failure type:\n")
for task, steps, final, expected in TASKS:
ok, *_ = eval_task(steps, final, expected)
if ok:
continue
kind = classify(steps, final, expected)
buckets[kind] = buckets.get(kind, 0) + 1
print(f" {kind:13s} <- {task}")
print("\nFailure taxonomy (counts):")
for kind, count in sorted(buckets.items(), key=lambda kv: -kv[1]):
print(f" {kind:13s} : {count}")
▶ Output
Failed runs and their failure type: wrong_tool <- Price of dal makhani? bad_args <- Order 2 samosa runaway_loop <- Price of two plates paneer tikka? gave_up <- Is rasgulla available? wrong_tool <- Price of veg pulao? Failure taxonomy (counts): wrong_tool : 2 bad_args : 1 runaway_loop : 1 gave_up : 1
What happened here: The five failures are not random, they cluster. Two are wrong-tool errors, where the agent answered “in stock” or “on the menu” when the user asked for a price, so the fix is a sharper tool description telling the model when to reach for get_price. The bad-args and runaway-loop cases both trace back to the same wrong argument name, which points at a tool schema the model keeps misreading. The taxonomy just turned a vague “improve the agent” into three concrete, prioritized fixes. That is the difference between a score and a diagnosis.
Judging Trajectory Quality with an LLM
Automated checks handle every task whose answer is a hard fact. But some tasks have no keyword to grep for. “Was the tool order sensible?” and “did the agent recover gracefully from an error?” are judgment calls, and that is where you reuse the LLM-as-a-judge pattern, pointed at the trajectory instead of a single answer. The code below is the real modern Anthropic SDK shape. We did not run it here because a live judge call needs an Application Programming Interface (API) key and spends tokens, but this is exactly what you would run, and the rest of the post stays fully reproducible with the stdlib scripts above.
📄 trajectory_judge.py: grade the path, not just the answer
import os, json
from anthropic import Anthropic
client = Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])
MODEL = "claude-opus-4-8" # current at the time of writing; models change, check provider docs
RUBRIC = """You grade an AI agent's TRAJECTORY, not just its final answer.
Score 1 (bad) to 5 (ideal) on whether it used the right tools, in a sensible
order, with valid arguments and no wasted or repeated calls. Recovering from an
error counts as good; repeating the same failing call counts as bad.
Reply as: score|one-line reason.
TASK: {task}
STEPS: {steps}
FINAL ANSWER: {final}"""
def judge_trajectory(task, steps, final):
prompt = RUBRIC.format(task=task, steps=json.dumps(steps), final=final)
msg = client.messages.create(model=MODEL, max_tokens=80,
messages=[{"role": "user", "content": prompt}])
return msg.content[0].text.strip()
print(judge_trajectory(
task="Price of two plates paneer tikka?",
steps=[["get_price", {"dish": "paneer_tikka"}, "error: bad arg"]] * 3,
final="I hit an error looking that up."))
▶ Example output (needs an API key, see note)
1|Right tool choice, but it repeated the identical failing call three times, never fixed the bad argument, and gave up, so the task failed.
client.messages.create) at the time of writing, and the grade shown is the kind of response the rubric asks for. Your exact wording will vary because the model picks the phrasing. Every other output block in this post is real, captured from stdlib-only Python you can run yourself.What happened here: The judge read the whole trajectory and gave it a 1 with a reason you can act on: right tool, but three identical failing calls and no recovery. Reading trajectories one at a time by hand does not scale past a handful, which is why teams pipe them into a tracing tool. In Langfuse (current at the time of writing, with Arize Phoenix a common alternative), each agent run becomes a trace of nested spans, one per tool call, and you attach this judge score to the trace. Then reviewing failures is filtering to score <= 2 and reading the spans, instead of grepping raw logs.
A Regression Gate in CI
Metrics you look at once are trivia. AI agent evaluation pays off when the metrics are wired to a gate that acts as a safety net. The rule is the same one from the eval post: the eval suite runs, and if the agent’s success rate falls below a threshold, or drops under the last shipped baseline, the build fails with a non-zero exit code and the deploy is blocked. The script below is the real gate logic, comparing the current agent against both an absolute bar and the previous baseline.
📄 regression_gate.py: block the deploy when the agent regresses
import sys, json, pathlib
from eval_set import TASKS, run_suite
THRESHOLD = 0.80 # candidate must clear this success rate to deploy
BASELINE_FILE = pathlib.Path("baseline.json") # last shipped score
def load_baseline():
if BASELINE_FILE.exists():
return json.loads(BASELINE_FILE.read_text())["success_rate"]
return None
r = run_suite(TASKS)
rate = r["success_rate"]
baseline = load_baseline()
print(f"candidate success rate : {rate:.0%}")
print(f"absolute threshold : {THRESHOLD:.0%}")
print(f"last shipped baseline : {baseline:.0%}" if baseline is not None
else "last shipped baseline : (none, first run)")
clears_bar = rate >= THRESHOLD
no_regression = baseline is None or rate >= baseline - 0.02 # allow 2pt noise
gate_open = clears_bar and no_regression
reasons = []
if not clears_bar:
reasons.append(f"below absolute bar ({rate:.0%} < {THRESHOLD:.0%})")
if not no_regression:
reasons.append(f"regressed vs baseline ({rate:.0%} < {baseline:.0%})")
print(f"gate : {'OPEN, deploy allowed' if gate_open else 'BLOCKED'}")
if reasons:
print(f"reason : {'; '.join(reasons)}")
sys.exit(0 if gate_open else 1) # non-zero exit fails the CI job
▶ Output (process exit code was 1)
candidate success rate : 75% absolute threshold : 80% last shipped baseline : (none, first run) gate : BLOCKED reason : below absolute bar (75% < 80%)
What happened here: The agent scored 75 percent, the bar is 80, so the gate blocked and the script exited with code 1. In CI a non-zero exit is a red build and the deploy never happens, so a flaky agent gets stopped by a machine before it reaches a user. The baseline check is the second guard: even an agent above the absolute bar gets blocked if a change quietly dropped its score below what you last shipped, which is how you catch silent regressions from a prompt tweak or a new model version. Wire this into a workflow that runs on every pull request.
📄 .github/workflows/agent-eval.yml
# .github/workflows/agent-eval.yml
name: agent-eval-gate
on: [pull_request]
jobs:
eval:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with: { python-version: "3.14" }
- run: python regression_gate.py # non-zero exit blocks the merge
Common Mistakes
- Grading only the final answer. An agent that reaches the right answer through a lucky, wasteful, five-retry path is a regression waiting to happen. Score the trajectory, not just the destination, or you will miss the loops and tool errors until they show up on your bill.
- Inventing tasks at your desk. Like canary sets, the strongest eval tasks are harvested from real usage and real failures. Twenty tasks that mirror actual user requests catch more than a hundred you imagined.
- A gate with no baseline. An absolute threshold catches a build that is bad, but not one that quietly slid from 92 to 84 percent. Compare against the last shipped score too, so silent regressions get caught.
- Running evals against live tools. An eval that calls production tools will send real email and place real orders every run. Mock every tool with a side effect.
- No loop or step ceiling. Without a cap, a stuck agent can spin until it exhausts your token budget. Track steps per task and set a hard limit that ends the run.
Conclusion
You went from a single trajectory to a working AI agent evaluation system. You turned one recorded run into five metrics, built a 20-task suite with automated success checks, sorted the failures into a taxonomy that told you exactly what to fix, reused an LLM judge for the subjective trajectory calls, and wired the whole thing to a CI gate that blocked a 75 percent build with a red exit code. The theme underneath it all: for agents, the path is the product. Grade how the agent works, not just whether it happened to land the answer, and you catch the failures that a final-answer check sails right past.
Next, the series moves from evaluating agents to shipping them safely under load. Want the bigger picture? Browse the complete Python + AI/ML tutorial series home to see where agent evaluation fits among every other lesson, from your first script to production AI systems.
Frequently Asked Questions
What is AI agent evaluation?
AI agent evaluation is measuring whether an autonomous agent actually completes tasks, judged on the whole trajectory of tool calls it made, not just its final message. It tracks metrics like task success rate, trajectory quality, tool-error rate, loops and steps, and cost and latency per task. It differs from chat evaluation because an agent produces a sequence of actions, so two agents can give the same answer while one took a clean path and the other looped through errors.
Which metrics matter most for an AI agent?
Start with task success rate, the fraction of tasks that ended correctly. Then add trajectory quality (right tools in the right order), tool-error rate (how many tool calls failed), loops and steps (whether it repeats itself or wanders), and cost and latency per task. Success rate is the headline number, but the others tell you why a build is failing, which is what you actually fix.
What is an agent trajectory?
A trajectory is the ordered record of everything an agent did to finish one task: each tool it called, the arguments it passed, whether the call succeeded, and the final answer. Evaluating the trajectory instead of just the answer is what lets you catch wasted steps, wrong tool choices, bad arguments, and runaway loops that a final-answer check would miss entirely.
When should I use an LLM judge for agent evals?
Use automated checks wherever the correct answer is a hard fact you can verify with a keyword or exact match, because they are free and deterministic. Reach for an LLM judge only for the subjective calls a keyword cannot make, like whether the tool order was sensible or the agent recovered gracefully from an error. Audit the judge against human labels first, exactly as you would for any LLM-as-a-judge setup.
How do I stop a bad agent from being deployed?
Run the eval suite in CI and make a failing result exit non-zero, so the job goes red and the deploy is blocked. Gate on two things: an absolute success threshold and the last shipped baseline, so you catch both an outright bad build and a silent regression that slid a few points. Always run the eval against mock or sandboxed tools so grading the agent has no side effects in production.
Which tools do agent evaluations use?
The metrics are tool-agnostic, so you can compute them with plain Python as shown here. For scale, teams use tracing and eval platforms: Langfuse and DeepEval are common at the time of writing, with Arize Phoenix offering trajectory evals as an alternative. Whatever you pick, the durable ideas stay the same: record the trajectory, score success and path quality, taxonomize failures, and gate deploys on the result.
Interview Questions on AI Agent Evaluation
Try each one aloud first. The phrasing you produce under mild pressure is what interviews measure.
Q: Why is evaluating an AI agent harder than evaluating a chatbot?
A chatbot produces a single turn you can grade in isolation, but an agent produces a trajectory: a sequence of tool calls, arguments, and retries on the way to an answer. Two agents can return the same final message while one took a clean two-step path and the other looped through errors and burned triple the tokens. So agent evaluation has to grade the whole path, which means recording every step and measuring success, tool errors, loops, and cost, not just checking whether the last message looks right.
Q: What does trajectory quality mean, and how do you measure it?
Trajectory quality is whether the agent used the right tools, in a sensible order, with valid arguments and no wasted or repeated calls. Some of it is measurable with plain rules: count tool errors, detect a loop when the same tool and arguments repeat back to back, and count total steps. The subjective part, like whether the ordering made sense or the agent recovered gracefully from an error, is where you point an LLM judge at the recorded trajectory and have it score the path against a rubric.
Q: Walk me through building an eval set for an agent.
Collect a set of tasks that mirror real usage, ideally harvested from actual user requests and past failures rather than invented. For each task, pin the correct outcome so a check can verify it without a human, using automated keyword or exact-match checks wherever the answer is a hard fact, and an LLM judge only for the subjective cases. Run the agent against mock tools so there are no side effects, record each trajectory, and aggregate the metrics: success rate, tool-error rate, loops, steps, cost, and latency.
Q: What is a failure taxonomy and why is it useful?
A failure taxonomy sorts each failed run into a category such as wrong tool, bad arguments, gave up, or runaway loop. A success rate alone tells you how many tasks failed; the taxonomy tells you how they failed, which converts a vague “make the agent better” into concrete, prioritized fixes. If most failures are wrong-tool, you sharpen the tool descriptions; if they are bad-args, you fix the tool schema the model keeps misreading.
Q: How would you gate deploys on agent quality in CI?
Run the eval suite in the pipeline and make a failing result exit non-zero so the job goes red and the deploy is blocked. Gate on both an absolute success threshold and the last shipped baseline, because the absolute bar catches an outright bad build while the baseline check catches a silent regression that slid a few points from a prompt tweak or a new model version. Store the shipped score, compare against it every run, and allow a small noise margin so normal variance does not block healthy builds.
Q: Scenario: your agent’s success rate is unchanged but its average steps per task doubled after a change. What do you do?
Success rate held, so the agent still finishes tasks, but doubling the steps means it is finishing them more expensively, which shows up as higher cost and latency and often a rising tool-error or loop count. I would pull the trajectories that grew longest, classify their failures and near-failures, and look for a pattern: a tool description change that makes the model second-guess, a schema tweak causing retries, or a prompt edit that removed a stopping cue. The step-count metric is exactly why you gate on trajectory quality and not just success, since a change can keep answers correct while quietly making every run wasteful.
Q: How do you keep an agent eval from causing real-world side effects?
Run every task against mock or sandboxed versions of the tools, never the production ones. If the agent can place orders, send email, or move money, an eval wired to live tools will do all of that on every CI run. The mock returns canned data with the same shape as the real tool, so the agent behaves identically for grading purposes while its actions stay contained. This is the same isolation discipline you apply to any code that touches external systems in a test.
Series: Python + AI/ML Cookbook, Part 6: GenAI & LLMs
Go deeper: the official Python documentation covers every edge case of this topic.
Related Posts
Previous: Python: LangChain vs LlamaIndex vs CrewAI, AI Agent Frameworks Compared
Next: AI Agent Project: From Raw Loop to LangGraph, With Evals
Series Home: Python + AI/ML Tutorial Series
No comment