Escalation rate is one of the most obvious AI support metrics. If an agent can resolve more issues without human transfer, escalation should fall. Buyers often expect that.

Reality is messier.

A lower escalation rate can mean better automation. It can also mean the agent is holding onto conversations too long, avoiding handoff, or giving customers answers that appear complete but create repeat contact later. A higher escalation rate can mean weak automation. It can also mean the system is correctly identifying risky, high-judgment, or emotionally sensitive cases.

Escalation is a signal. Interpretation requires context.

Useful escalation tracking should break down by scenario, customer type, language, policy path, agent version, and resolution outcome. A 10 percent escalation rate in password resets may be poor. A 10 percent escalation rate in regulated financial disputes may be excellent. The number means little without the operating context.

Support intelligence should show where escalation happens, why it happens, and whether it was the right choice.

Track repeat contact

Repeat contact is one of the cleanest ways to detect fake resolution.

A customer may end a conversation politely. The ticket may close. The agent may mark the issue resolved. Then the customer comes back twelve hours later with the same problem. From the first conversation’s point of view, everything looked fine. From the customer’s point of view, nothing worked.

Repeat contact catches this.

For AI agents, repeated contact is especially important because automation can create polished failure. A fluent answer can reduce immediate friction while failing to complete the job. The customer may leave the interaction thinking the issue is handled, then discover the refund did not happen, the order did not update, the account change did not persist, or the promised follow-up never arrived.

A good AI support scorecard should track repeat contact by intent, agent version, channel, language, and tool path. If repeat contact rises after an automation change, something important happened. The agent may be overconfident. The policy may be incomplete. The backend action may be failing. The customer may need a confirmation step that the workflow skipped.

Repeat contact is where hidden failure reappears.

First-call resolution still matters

First-call resolution is old, but still useful. The core idea remains strong: did the customer get the issue handled in the first interaction?

AI agents change how this metric should be understood. A first-call resolution may involve speech recognition, translation, tool use, policy lookup, browser execution, ticket updates, and a summary. The “call” becomes a surface over a larger system. Measuring first-call resolution without inspecting the underlying path may hide important defects.

A multilingual voice agent, for example, might resolve English calls well and Spanish calls poorly. An agent might resolve simple cases on first contact but fail when a browser action is required. Another version might improve first-call resolution but increase latency enough that customers abandon.

First-call resolution remains a valuable outcome metric.

Support intelligence makes it diagnosable.

Latency is a KPI, not a technical footnote

Latency often gets treated as an engineering metric. In voice AI, it is a customer experience metric. In agentic support, it is also an operational constraint.

Every live conversation has a time budget. Speech recognition takes time. Translation takes time. Retrieval takes time. Reasoning takes time. Tool use takes time. Speech generation takes time. Human patience also has a limit.

A slow agent feels broken even when it is technically correct. A fast agent can still fail if it sacrifices reasoning or skips necessary checks. The goal is not always the lowest possible latency. The goal is the right tradeoff between responsiveness, reasoning quality, and task completion.

Latency should therefore be measured by step:

·speech recognition latency

·translation latency

·model response latency

·tool-call latency

·browser-action latency

·text-to-speech latency

·total turn latency

For voice support, subcomponents matter because customers experience the system as one rhythm. A delay anywhere in the chain becomes conversational friction.

Latency is the shape of the agent’s thinking time.

Tool success rate matters

An AI support agent that cannot use tools reliably is just a fluent explainer. Support work often requires action: retrieve an order, update an address, issue a credit, check eligibility, create a case, send a message, schedule a follow-up, or trigger an outbound workflow. In Giga’s product world, the relationship between conversation and action also extends into surfaces like Browser Agent, where the agent can operate inside browser-based systems.

Tool success rate measures whether the agent’s actions actually worked.

This should include:

·tool-call attempts

·successful tool calls

·failed tool calls

·retries

·timeout rate

·invalid arguments

·unauthorized actions

·human fallback after tool failure

Anthropic’s agent engineering guidance emphasizes that tool design strongly affects agent performance. Clear tool names, good descriptions, constrained inputs, and useful responses all shape whether the agent can act effectively. That observation matters for support KPIs because a failed support conversation may be a tool-design failure rather than a model-quality failure.

A serious AI support scorecard should not only grade the final answer.

It should grade the machinery underneath.

Policy adherence deserves its own measurement layer

Support agents operate inside policies. Refund policies. Safety policies. Escalation policies. Compliance policies. Brand policies. Regional policies. Language policies. Tool-use policies.

A response can sound helpful while violating policy. A response can follow policy while failing the customer. Both cases matter.

Policy adherence should be measured directly, especially in regulated or high-volume environments. This can include rubric-based grading, deterministic checks, contradiction detection, unauthorized tool reference detection, and review of high-risk conversations.

NIST’s AI monitoring work separates monitoring categories such as functionality, operational behavior, human factors, security, and compliance. That distinction is useful because an AI agent can perform well on one dimension and poorly on another.

Support teams should avoid the temptation to collapse every KPI into one “quality score.” A composite score can be useful, but only if the underlying dimensions remain visible.

Policy adherence is not a vibe.

It is a system property.

Customer effort is harder, but important

Customers do not always express effort directly. They show it through repeated explanations, long pauses, clarification loops, recontacts, transfers, abandonment, and frustration. AI support teams should measure those signals.

Useful customer effort proxies include:

·number of turns before resolution

·number of repeated details

·clarification loops

·transfer count

·time to action

·abandonment

·repeat contact

·negative sentiment signals

·explicit dissatisfaction

Customer effort is especially important because AI systems can accidentally make support feel efficient for the company and exhausting for the customer. A bot that asks too many questions may reduce human workload while increasing customer burden. A voice agent that sounds natural but cannot act may feel polite and useless at the same time.

The best support automation reduces effort on both sides of the conversation.

Anything else is just workload displacement.

Segment every KPI

Average metrics hide product truth.

An AI support agent may perform well overall and poorly for one language, region, scenario, channel, or customer tier. Averages can bury the exact segments where support quality matters most.

Every serious AI support KPI should be segmented by:

·support scenario

·intent and subintent

·language

·region

·customer tier

·channel

·agent version

·policy path

·tool path

·escalation reason

·resolution outcome

Segmentation matters because AI systems do not fail evenly. Translation may work well in one language pair and struggle in another. Browser actions may work well for one workflow and fail in another. A policy may be clear for standard customers and ambiguous for enterprise exceptions.

Smart Insights should make those differences visible. When a metric moves, the next question should always be: where?

Measure improvement over time

A support AI program should improve. If it does not, something is wrong with the loop.

Improvement metrics should track whether changes to instructions, policies, tools, knowledge, or workflow design actually create better outcomes. This is where KPI tracking becomes more than reporting. It becomes release discipline.

A good improvement system should connect:

Without this connection, teams may ship changes without knowing whether they helped. With this connection, support operations starts to look more like product development.

Google’s Conversational Insights materials describe the value of detecting and visualizing patterns in contact center data, while NIST emphasizes the importance of monitoring deployed AI systems in real-world settings. Together, those ideas point toward a practical conclusion: AI support teams need both pattern detection and ongoing measurement.

A static dashboard tells you what the score is.

A continuous improvement loop tells you whether the system is learning.

A practical KPI stack for AI support agents

For teams starting from scratch, the KPI stack should stay simple enough to use and complete enough to diagnose failure.

Start with these:

·Outcome layer: DWR / resolution rate, first-call resolution, repeat contact, escalation rate, CSAT or customer satisfaction proxy.

·Runtime layer: total turn latency, tool success rate, transfer rate, clarification loops, abandonment.

·Quality layer: policy adherence, hallucination flags, QA score, unsafe action attempts, human override rate.

·Segment layer: language, intent, region, channel, agent version, customer tier.

·Improvement layer: improvement item completion, experiment lift, KPI delta after change, regression rate, recurring issue reduction.

That stack gives leaders a clean view of outcomes, while giving operators enough depth to fix the system.

The point of measurement

Metrics should not exist to decorate a dashboard. They should tell the team what to do next.

An AI support agent is a production system that talks to customers. It should be measured like one. That means outcomes, runtime behavior, policy adherence, segment performance, and improvement over time all matter.

The wrong scorecard will make the agent look better than it is.

A useful scorecard will make the agent easier to improve.

Support teams should therefore ask one final question of every KPI: does this metric help us make the next version of the agent better? If the answer is no, the metric may still be interesting. It is not yet operational.

AI support does not become mature when it answers more conversations. It becomes mature when the organization can prove which conversations are resolved, which ones fail, why they fail, and what changed after the team tried to fix them. For a broader view of how this fits the contact-center system, see Giga’s guide to contact center automation.