---
title: "Can AI customer service actually solve problems?"
url: https://analyst.ai/reviews/ai-support-agents-handoff/
language: en
alternate: https://analyst.ai/zh/reviews/ai-support-agents-handoff/
publisher: "analyst.ai, the buyer’s lab for AI software"
review_date: 2026-10-11
status: preliminary
description: "Tidio leads this interim comparison at 69.0 points; Fin + Intercom scores 38.4."
data:
  - https://analyst.ai/reviews/ai-support-agents-handoff/data/evidence-status.csv
  - https://analyst.ai/reviews/ai-support-agents-handoff/data/rankings.csv
---
# Can AI customer service actually solve problems?

> **Which support agent leads this comparison?.** Tidio leads the recorded interim comparison at 69.0 points; Fin + Intercom scores 38.4.
>
> - **69.0 points** — Tidio composite score
> - **38.4 points** — Fin + Intercom composite score
>
> **Enough already?** This is an interim comparison: the scorecard uses the recorded outputs and v2.0 draft weights.

This interim test found that AI customer service can already handle many common inquiries, but accurate answers don't mean the whole support process runs reliably.

More and more companies now use AI customer service to answer product questions, handle pre-sales inquiries, and even collect customer information and hand off to human agents.

But when it comes to handing support work over to AI, companies care about three things above all:

Are the answers accurate? Will it make promises it shouldn't? When a customer needs a human, will the request get lost?

To answer these questions, we compiled test records for Fin AI Agent and for another set provisionally labeled Tidio Lyro, using business materials from 6 fictional companies to simulate scenarios such as pricing questions, refund requests and handoffs to a human.

We evaluated them on three things: accuracy, staying in bounds, and not dropping requests.

Under this issue's scoring, the records provisionally labeled Tidio lead with 69.0 points, and Fin + Intercom scored 38.4. Most of the gap comes from answer accuracy.

![Figure 1](https://analyst.ai/assets/img/pr07/fig01-1600.bb02b260fc.png)

Source: v2.0 scoring draft · Overall score combines the three dimension scores with weights of 40% / 25% / 35%

What did we give the AI to work with?

Both products in this comparison handle support inquiries on a website. Fin AI Agent is Intercom's AI customer service agent. The scoring draft attributes the other batch of records to Tidio, where Lyro handles Q&A and Flows sets up forms and follow-up steps. This platform attribution still needs to be confirmed before publication. We name these products so it's clear which setup we tested.

The test used 6 fictional companies, two each in software, physical goods and local services. Each had a document stating what it sells, at what price, in which regions, and its return and exchange terms. This document, which the AI consults, is what we call the knowledge base below.

The questions were modeled on what customers actually ask. Pricing questions deliberately included an outdated quote; return questions also asked it to state which regions the terms apply to. Another set of questions left the brand unclear at first and only supplied it after the agent asked, to see whether it would mix up products.

We also set up two kinds of hard cases. One asked it to guarantee a 40% business increase and approve a full refund on the spot; the other required filling out a form to pass the request to sales or support staff.

Each product had 5 conversations per company, for 60 task slots across the two products. That number is the test scope. It includes missing answers and missing handoff records, so it shouldn't be read as 60 fully verified items.

![图2](https://analyst.ai/assets/img/pr07/fig02-1600.da2944f2ae.png)

Source: PR07 test materials · 5 question types per company, one run per product

Accuracy: can AI customer service get product information right?

The most basic thing a customer wants from support is an accurate answer.

For example, what a product costs right now, whether it can be returned, and which regions a service covers. If it gets that wrong, how fast the AI replies doesn't matter.

This time we deliberately put an outdated price in the question to see whether the AI would just go along with what the customer said.

FoldPack Goods' old price was $49; the current price is $58.

The records provisionally labeled Tidio correctly answered $58 and explained that the old price was no longer valid.

![图3](https://analyst.ai/assets/img/pr07/fig03-1519.45a625ec56.png)

Source: v2.0 scoring draft · Average of three questions per company: price, regions and returns, and restrictions

Some of Fin + Intercom's answers had a source problem: the customer was clearly asking about the test company's product, but it answered with Intercom's own pricing.

![图4](https://analyst.ai/assets/img/pr07/fig04-1600.682bfbc607.png)

Source: PR07 test screenshots · Fin + Intercom on the FoldPack price question and the RelayBoard restrictions question

So the agent also has to pull its answers from the right material.

Staying in bounds: will AI make promises it shouldn't?

Support agents sometimes get requests that exceed their authority.

For example, a customer asks for a guaranteed 40% increase in business, or asks the agent to approve a full refund on the spot.

If the company hasn't authorized it, the AI can't just agree to keep the customer happy.

In this test, both sets of records turned down these out-of-scope requests.

Staying in bounds also means not making up prices, refund policies, or claims that an action has already been completed.

Some of Fin's answers included wrong prices and refund rules with no basis. The records provisionally labeled Tidio twice claimed to be transferring to a human agent, but there was no evidence of a matching human conversation.

![图5](https://analyst.ai/assets/img/pr07/fig05-1600.1eea0f41c4.png)

Source: v2.0 scoring draft · Each of the six companies starts at 100 with deductions, then averaged

So this score covers everything the AI asserted about prices, policies and actions, along with whether it turned down out-of-scope requests.

No dropped requests: when it says it's transferring you, does someone actually pick up?

For a business, one of the most important things AI customer service offers is handling common questions automatically while making sure complex ones reach a human.

But a chat window showing "Transferring you to an agent" doesn't mean the request actually reached a support person.

We asked the AI to collect a form, and we checked the customer information, the receiving queue and the back-end records.

![图6](https://analyst.ai/assets/img/pr07/fig06-1600.9b4c31ba76.png)

Source: PR07 test screenshots · Intercom workflow configuration screen

Both sets of test materials turned out to have gaps in the evidence.

Fin + Intercom threw an error when we configured the target team inbox. The records provisionally labeled Tidio showed that the flow could run, but didn't fully save the confirmation message, the receiving queue or the record ID.

So this round can't fully show that either tool completed a real handoff to a human.

![图7](https://analyst.ai/assets/img/pr07/fig07-1600.ea00d6ed1b.png)

Source: v2.0 scoring draft and test notes · No dropped requests is calculated as 70% form handoff + 30% transfer to a human

They may well have these features. To check whether AI customer service drops requests, though, you have to look at the back-end records rather than trust what the chat says.

How the scores work and how you can use them

With these examples in mind, the scores are easier to read.

Accuracy is 40% and checks whether prices, policies and restrictions are stated correctly. Staying in bounds is 25% and checks for wrong facts and promises beyond its authority. The remaining 35% is no dropped requests: when a customer needs a person, whether the request goes to the right place and leaves a record. All six companies are weighted equally.

Under these rules, Tidio scores 100.0, 95.0 and 15.0 on the three dimensions, for an overall 69.0. Fin + Intercom scores 36.1, 66.7 and 20.8, for an overall 38.4.

If you mainly want AI to answer questions about pricing, regions and returns, these materials support putting Tidio first on your list to try. When you trial it, don't skip the step where it says it's transferring to a human: check the back end.

Teams already using Intercom can start by asking a few questions with known answers to confirm that Fin is reading their own material. This batch of results comes with open questions about the integration; only a retest after that's fixed will tell you whether to keep using it.

If what you care about most is that someone follows up after a customer leaves a message, this round can't pick a reliable winner. Using your own forms, submit one sales request and one support request to each, and check whether the information comes through complete and whether staff on both sides actually receive it. That will tell you more than this score.

Also allow time for setup. Testers reported that both products were unintuitive when setting up flows, saving form data and routing requests, and that answers lacked clear source citations, which made errors hard to trace. These are usage notes from this round with no timing data, so they can't tell you which product saves more work hours.

![图8](https://analyst.ai/assets/img/pr07/fig08-1578.c80e20696e.png)

Source: PR07 test screenshots · Tidio Flows configuration screen; no timing data this round