Policy score
86%
Measures whether the answer stays inside the written support policy.
Back to Studio
Studio / Consulting
Customer Support Agent Scorecard
A scorecard for support agents covering escalation quality, policy adherence, multilingual handling, hallucination risk, and customer outcome.
Live Studio demo
Support scorecard console
Turn a support policy into scored conversations: policy boundary, tone, evidence citation, queue choice, and safe escalation behavior.
Refund decisionsMediumpublic
Refund policy boundary case
Respond to an angry customer whose refund request sits just outside the stated policy window while preserving tone and escalation boundaries.
Escalation
94%
Checks whether the agent routes only the unresolved part to a human.
Tone and language
91%
Rewards calm customer language and faithful multilingual summaries.
Handoff risk
32%
Lower is better; this rises when evidence gates are relaxed.
Model support ranking
Which agent can answer safely?
Frontier reasoning model
88/100
Accepted
Firm policy answer, empathetic tone, and correct escalation path.
Fast mid-tier model
84/100
Accepted
Strong answer with one minor wording edit.
Open-weight local model
58/100
Partial
Tone was acceptable but policy boundary was vague.
Small routing model
42/100
Rejected
Classified refund intent but could not safely resolve.
Policy evidence
policy date windowcustomer order dateapproved escalation path
An empathetic support response that does not invent a refund exception, cites the policy boundary, and offers the approved escalation path.
Best response excerpt
I understand why this is frustrating. The request is outside the refund window, but I can escalate the approved exception-review path.
retrieve refund policycheck order datedraft support response
Rollout readout
Safe for supervised rollout after policy owner signs off on escalation evidence.
Failure mode to watch: Models either deny too harshly or invent a refund exception not present in policy.
How it works
Each Studio surface is designed as a practical operating loop: capture the buyer problem, run measured evidence, and return a decision artifact that can be acted on.
Current demo state
Live support scorecard is connected to the Support Agent Policy Suite; client-specific policies can be converted into private benchmark packs.
1
Convert real support policies into scenario packets with gold outcomes and escalation boundaries.
2
Run agents through refunds, complaints, policy lookups, angry customers, and ambiguous exceptions.
3
Score resolution, tone, hallucination risk, policy citation, and human handoff precision.
4
Deliver an implementation memo that separates safe automation from workflows needing review.
Buyer questions
These are the questions the product needs to answer before someone deploys, buys, or scales the system.
Will the agent respect our refund and exception policy?
Can it handle bilingual or region-specific context?
When should it escalate instead of improvising?
Which support queues should be automated first?
Deliverables
What a buyer gets
Support scenario pack
Policy adherence matrix
Escalation audit
Rollout recommendation
Connected evidence
Read the benchmark trail
Studio packet
Buyer-ready demo packet.
This generated packet gives Sanjay and Saujas a consistent follow-up artifact for demos, consulting calls, and product conversations.
Next build step
Turn this Studio surface from a populated product brief into a live demo by wiring real run data, screenshots, and client-approved examples into the same page.