The Conversation Quality Index (CQI) is an AI-powered metric designed to help team managers and admins evaluate human agent performance once a conversation has been assigned to them. CQI automatically analyzes conversation dynamics to measure support efficiency and agent behavior without requiring manual audit sampling.
How CQI Works
Behind the scenes, our Large Language Model (LLM) evaluates assigned support conversations based on agent behavior rather than fact-checking the specific information shared.
Trigger Conditions for Analysis
The AI evaluates a conversation in two scenarios:
Conversation Closed: A final analysis is triggered as soon as the ticket/conversation is closed.
72-Hour Inactivity: If a conversation remains open for more than 72 hours with no new activity, the AI takes a temporary snapshot evaluation so management gets timely performance metrics without waiting indefinitely.
Evaluation Categories & Logic
The AI evaluates conversations and assigns them to one of three statuses, which feed into either a Positive or Negative score:
Status | Outcome | Criteria & Agent Behavior |
Resolved | Positive | The agent handled the customer request properly and fulfilled their query before closing the conversation. |
Abandoned | Positive | The agent replied or requested more information, but the customer went silent. Note: Agents are never penalized if the customer stops responding after the agent asks for details. |
Unresolved | Negative | The agent failed to handle the request properly due to specific flagged behaviors. |
Flagged Behaviors (Negative Impact)
The AI automatically flags the following negative behaviors and marks the conversation as Unresolved:
Silent Closure / Neglect: An agent was assigned the conversation but closed it without replying or allowed 72 hours to pass without responding.
Generic Response Drop: The customer asked a specific question, the agent provided a generic standardized reply (e.g., "How can I help you?"), and the customer subsequently went silent.
Unfulfilled Follow-Up Promises: The agent promised a follow-up (e.g., "I will check and get back to you"), but closed the conversation or let 72 hours elapse without providing the promised update.
Partial Resolution: The agent addressed only part of the customer’s multi-part inquiry, leaving other questions unanswered.
Dashboard Features & Admin Overrides
Administrators can review and manage CQI scores directly from the Analytics Dashboard:
Detailed Reporting Panel: View overall performance percentage scores, historical trend charts, and daily conversation breakdowns.
Status Badges: Conversations are labeled with clear badges indicating their resolution state (Resolved, Abandoned, or Unresolved), along with whether the evaluation score is Final (closed conversation) or Temporary (open >72 hours).
Manual Overrides: If an admin believes the AI misjudged a conversation, they can manually change the rating score directly from the dashboard or from within the highlighted conversation view.
How to locate the CQI insights?
To view the Conversations Quality Index insights, follow these steps:
Click on Reports, from the main BusinessChat dashboard
Choose Conversations Reports
Locate the CQI as the top chart
Use the time filter to filter for the desired period you want to check, choose which tem or user you need to see their conversations and apply filters.
This report has a drill-in feature, meaning if you click any bar, a side window will open with the corresponding conversations. Click any conversation to open the inbox view and check the entire conversation.
How to change a conversation rating?
Admins can manually change a conversation rating if they think it's incorrect. There are two way to do so:
Either from the report's page
Or from the conversation inbox view
Note: Whenever an admin manually adjusts a rating on the dashboard, feedback is automatically submitted to our Product Team to continuously train and improve the AI model.
CQI Activation & Status
Beta Feature: CQI is currently in Beta.
Independent of AI Bot: CQI functions independently of our AI Bot, meaning your business can enable CQI performance tracking even if you do not use the AI Bot.
Need Help Enabling CQI? If you would like to enable the Conversation Quality Index for your organization's workspace, please contact your account manager or reach out to our support team.