Why AI agents need observability too

01.Why ordinary monitoring misses agent failures

Traditional monitoring looks for symptoms that are easy to detect: errors, slow responses, crashed processes. A web service either returns the right page or it does not. Failure is usually visible.

An agent is different. Its output is language and action, and both can be wrong while looking fine. It might summarise a document and omit the clause that matters. It might call a tool with the wrong customer identifier. It might loop between two steps, each valid, until it has spent a day's budget. Every request returned a successful response. Nothing is broken, by the measures you are watching.

Change adds to the problem. A provider updates a model, someone edits a prompt, a tool changes its output format. Behaviour shifts quietly, and no test written last month covers it.

02.What to track

• Every step and tool call in a run. Record the prompt, the retrieved context, the tool name and arguments, the result and the final output. When something goes wrong, an engineer should be able to replay the run.
• Quality scores from automated and human review. Score a sample of runs against clear criteria. Use rules where you can, a grader model where you must, and human review to check both.
• Cost and latency per task. Not per request. A task may involve a dozen calls. Track the total, and watch for outliers.
• Hand-off rate to humans. A rising rate may mean the agent is struggling. A falling rate may mean it is overconfident. Either way it tells you something.
• Changes after model or prompt updates. Tag every run with the prompt version and model, so you can compare before and after.

Without this, you learn about failures from customers.

03.An example

Consider an agent that drafts replies to supplier queries. It begins to quote the wrong delivery terms for one product line. The cause is a change in how a source document is formatted, which breaks retrieval for that line. No error is raised. Replies look polished.

With tracing, an engineer can filter runs by product line and see the retrieved context was empty. With quality scoring, the drop in factual accuracy for that line shows within a day. With neither, the issue lasts until a supplier complains.

04.A practical way to begin

You do not need a large platform to start. A sensible order is:

• Add a run identifier and log every model call and tool call with it.
• Store traces somewhere searchable, with sensitive fields masked.
• Collect 30 to 50 real cases with agreed good outcomes, and write them as an evaluation set.
• Run the set before every prompt or model change.
• Sample live runs for scoring and review.
• Add alerts only when you know your baseline.

05.Anatomy of a useful trace

A trace is more than a log. A log line says something happened. A trace shows how a whole task unfolded, in order, with timing and relationships.

A useful trace for an agent includes:

• The user request or triggering event.
• Each model call, with prompt, settings, output and token use.
• Each tool call, with name, arguments, result and duration.
• The data retrieved, such as documents or records, and where they came from.
• Decisions the agent made, with its stated reason.
• Final output and whether a human took over.
• Version tags for the prompt, model and tools.

When an engineer opens a trace, they should be able to answer three questions quickly. What did the agent see? What did it decide? What happened next? If the trace cannot answer those, it needs more detail.

06.Measuring quality without a perfect answer key

Many agent tasks have no single correct answer, which makes people give up on scoring. That is a mistake. You can still score the parts that matter.

• Rules: Did the answer include the required fields? Did it avoid forbidden content? Is the format valid?
• Reference checks: Does the answer match the facts in the source record?
• Grader models: Does the answer meet a written rubric for completeness and tone?
• Human review: Does an expert agree, on a sample?

Use the cheap checks on everything, and the costly ones on samples. Compare your automated scores with human ratings regularly, so that you know how far to trust them.

07.The cost side

Agents can spend money without any error appearing. A prompt that grows with each step, a retry loop or a tool that returns very long text can double the cost of a task. Track cost per task and per agent, and set a limit. A hard cap per run, with a clean failure and a hand-off, protects you from the worst case.

08.Who should look at the data

Traces are only useful if the right people look at them. Engineers need them during incidents. Product owners need quality trends before they approve a change. Operations staff need hand-off rates and queue sizes. Give each group a view built around its questions, and do not hand everyone the same raw table. A weekly fifteen-minute review of the worst runs, with all three groups present, often teaches more than any dashboard. People see real failures, agree on what counts as wrong, and leave with a concrete fix to make.

09.Designing for safe failure

Observability tells you what went wrong. Safe failure design limits the harm while you fix it. Three habits help.

• Fallbacks. If the agent cannot complete a task with confidence, return a safe default, such as a message that a person will follow up.
• Rate limits. Cap the number of actions an agent can take in a period, especially actions that are hard to reverse.
• Human hand-off. Give the agent a clear way to pass a case to a person, with a summary, and make sure someone is watching that queue.

10.Common mistakes

• Logging everything with no structure. A pile of text is not a trace. Link steps under a single run.
• Trusting an automated grader without checking it. Compare it with human ratings. Graders can favour fluent wrong answers.
• Alerting on every wobble. Noisy alerts get ignored. Use windows and minimum sample sizes.
• Ignoring privacy. Traces can contain personal or confidential data. Decide what to store and for how long.
• Treating prompt changes as small. A one-line edit can change behaviour widely. Run the evaluation set every time.

11.What to do next

Pick your most important agent. Ask: if it started giving wrong answers today, how long before we found out, and how? If the honest answer is "when a customer tells us", that is where to begin.

If you do only one thing after reading this, record every run under a single identifier. It costs little and it makes every later step easier.

Brandsmashers builds tracing, evaluation and alerting for production agents, and designs the fallbacks and hand-offs that keep failures small. If you want help with a first pass, we are happy to talk it through.