Elbuz scores every resolved ticket itself — politeness, whether the issue was resolved, reply accuracy and customer satisfaction judged from the conversation — and notices when tickets on one topic suddenly pile up.
When you need this:
- Customers don't fill in surveys — satisfaction shows in the customer's own words and tone.
- You can't reread every conversation — the worst tickets are collected in a table with a hint on what to improve.
- A carrier failed deliveries and identical questions poured in — the manager gets a signal, and every customer gets one consistent reply.
Where to start: “Autonomous support” → “Channels and proactive” → turn on AutoQA.
What quality control does
- AutoQA — AI scores every resolved conversation on three criteria from 1 to 5 and judges customer satisfaction from the text.
- 30-day summary — average scores, how many customers are satisfied, and a list of the worst conversations.
- Single-topic spike — a signal to the manager when tickets with one reason are several times above normal.
- AI instruction and bulk draft — a consistent reply for the duration of the spike; staff send it.
Neither feature sends anything to customers, so both work regardless of the main autonomous support switch. Reviewing the AI's own decisions is a separate “Review” queue (how to handle it).
How AI scores closed conversations
In the “AutoQA — conversation scoring” card, set the mode to “Test” or “Live” and click “Apply”. Both modes work the same way: scoring sends nothing and doesn't change the ticket.
Every 30 minutes, Elbuz picks unscored tickets with the “Solved” or “Closed” status, solved or closed in the last 7 days — up to 20 at a time. AI reads the last 30 messages from the customer, the store and AI across emails, chats, calls and imported correspondence. Internal notes are not included in the score.
| Metric | What it means | Scale |
|---|---|---|
| “Politeness” | politeness and tone of the store's replies | 1–5 |
| “Resolved” | whether the customer's issue was resolved | 1–5 |
| “Accuracy” | accuracy and completeness of the replies | 1–5 |
| “Satisfaction” | how satisfied the customer is, judging by their words and tone | “Satisfied”, “Neutral”, “Dissatisfied” |
| “What to improve” | one sentence: the main thing worth changing | text |
The “Per day, at most” field limits how many conversations AI scores per day: 50 by default, up to 500. Tickets over the limit are scored the next day if they were closed no more than 7 days ago.
Where to see the summary
The 30-day summary sits in the same card below the settings: “Scored in 30 days”, average “Politeness”, “Resolved”, “Accuracy” and the counts of “Satisfied”, “Neutral”, “Dissatisfied”. Below is a table of up to 10 worst conversations: the customer is dissatisfied or at least one score is 2 or lower. Each row shows the ticket number, satisfaction and “What to improve”.
1
Mode and daily limit2
Average scores3
Satisfaction4
What to improve table- Mode and daily limit — how AutoQA runs and how many conversations it scores per day.
- Average scores — politeness, resolution and accuracy over the last 30 days.
- Satisfaction — counts of satisfied, neutral and dissatisfied conversations.
- What to improve — the worst conversations with the AI's note on what went wrong.
AutoQA satisfaction is AI's judgment from the text, not the customer's answer. Ratings that customers give themselves are counted separately in the “Tickets” reports.
How to spot a spike in one topic
A spike is when tickets with one AI-detected reason over the last few hours are several times above normal. The reason is the same one shown in the ticket's “AI · Support” section: “Delivery”, “Order status”, “Return” and so on.
| Field | What it does | Default |
|---|---|---|
| “Mode” | “Test” — the spike is only added to the list; “Live” — also a signal and an AI instruction | “Off” |
| “Window, h” | how many recent hours new tickets are counted over | 3 (1 to 48) |
| “At least tickets” | fewer tickets than this never count as a spike, however much they grow | 5 (3 to 500) |
| “Times above usual” | usual is the average for the same window over the previous 7 days | 3 (2 to 20) |
| “AI instruction lasts, h” | how many hours the spike stays active | 24 (up to 72) |
| “Notify” | the employee who receives the notification | “Manager (the “Manager” target of questions to colleagues, otherwise the head of the “Customer Service Department”)” |
Elbuz checks reasons every 30 minutes. Example with factory defaults: in a normal week there's about one delivery question per 3 hours; today there are 7 in 3 hours. That's at least 5 and more than three times usual — a spike. One topic gives at most one spike per window.
In “Live” mode, the recipient gets the “Ticket spike: Delivery” notification (the title shows the reason name) with the text “N tickets in 3 h (usually …). Check the cause and prepare a reply.” The button in the notification opens the “Channels and proactive” tab.
Spikes are listed below the card settings: topic, number of tickets, over how many hours and the usual level. “Ongoing” means the spike is active, “Over” means the instruction time has run out, “Test” means it was recorded in test mode.
1
Thresholds2
Who gets the signal3
Ongoing spike4
Instruction for AI5
Mass reply draft- Thresholds — rating, window in hours, minimum tickets and how many times above usual make a spike.
- Who gets the signal — recipients notified when similar tickets pile up.
- A spike row with the Ongoing pill shows a spike that is still active.
- Instruction for AI — a hint the AI follows in these tickets while the spike lasts.
- Mass reply draft — becomes the AI draft in the open tickets of the spike.
How to give all spike customers the same reply
Each spike's panel has two fields. Fill in the one you need and click “Save” below the panel.
- “Instruction for AI during the spike”, up to 1,000 characters — your words about the cause, for example “Nova Poshta delayed by bad weather, delivery up to 3 days, don't promise compensation”. While the spike is active, the AI responder adds the instruction to its rules and answers everyone consistently, promising no more than it says.
- “Draft bulk reply”, up to 4,000 characters — ready text for customers. After “Save”, it becomes the AI draft in the spike's open tickets — statuses “New”, “Open”, “On hold”; a toast shows how many.
AI doesn't send the bulk reply. A staff member opens the ticket and clicks “AI draft” — the text from the spike panel appears in the field — edits it if needed and sends it (how the draft works). Tickets that arrived after the spike was recorded don't get the draft: AI answers them, taking the instruction into account.
The instruction applies only in “Live” mode and only while the spike is “Ongoing”. You can also save a draft for a spike in “Test” mode.
Scenarios
Weekly quality review
AutoQA is set to “Live”, limit 50. On Monday, open the AutoQA card: average scores, the share of dissatisfied customers and the 10 worst conversations. Open the tickets from the table, read “What to improve” and go over the specific replies with your team.
A carrier delays every parcel
Spikes are set to “Live”, “Notify” — the support manager. Over the morning, 12 “where is my parcel?” questions came in — the manager gets a signal and writes an AI instruction and a bulk draft. Staff close the open tickets with one reply, and new ones get the same account of events from AI.
Identical questions about a new product
After a product launch, the “Product and price” topic spiked. Put the answer in the bulk draft, and turn questions AI couldn't answer into articles from the “Knowledge base gaps” queue (how).
What to do when there are no scores or signals
| What you see | Why | What to do |
|---|---|---|
| “No scores for the last 30 days yet” | AutoQA mode is “Off”, there are no solved or closed tickets in 7 days, or “AI · Support” is off or has used up its budget | Turn on the mode and the AI employee; scores appear within half an hour (AutoQA) |
| Fewer tickets scored than closed | The “Per day, at most” limit, or tickets were closed more than 7 days ago | Raise the limit; AutoQA doesn't pick up older tickets |
| The spike is in the list, but nobody got a notification | “Test” mode: the spike is only added to the list | Switch to “Live” |
| Lots of tickets, but no spike | The topic's usual level is high too, there are fewer tickets than the threshold, or they have no AI-detected reason | Lower “Times above usual” or “At least tickets” (how it's calculated) |
| After “Save”, the toast says “Drafts in tickets: 0” | The spike's tickets are already solved or closed | Nothing: the bulk draft goes only into open tickets |
FAQ
Does the customer see the AutoQA score?
No. The score is stored for the store, nothing is sent to the customer, and the ticket doesn't change.
Does AutoQA score AI replies or staff replies?
The whole conversation: staff and AI replies together. AI decisions are scored separately by people in the “Review” queue (how).
Will AI send the bulk reply by itself?
No. The bulk draft goes into the reply field of open tickets, and a staff member sends it.
Can I score a ticket closed a month ago?
No. AutoQA only picks up tickets solved or closed in the last 7 days.
What happens to the AI instruction when the spike is over?
After “AI instruction lasts, h”, the spike becomes “Over” and AI stops taking the instruction into account. The text stays in the spike panel.
Where to find it and who has access
- Settings
- “Tickets” → gear → “Autonomous support” → “Channels and proactive” tab
- Cards
- “AutoQA — conversation scoring” and “Single-topic spikes”
- Other entry points
- the “Ticket spike: …” notification → “Ticket spikes”
- Access right
- the “Autonomous support” window — by default only “Management”
- Prepare
- the “AI · Support” AI employee turned on: AutoQA runs on its model and budget
Related topics
- For the big picture — modes, reviewing AI decisions, the breaker — see Autonomous support.
- For ratings that customers give themselves — “Tickets” reports.
- If the spike is about questions the knowledge base has no answer to — knowledge base gaps.
