Imagine enjoying a customer service experience

The weirdest thing happened to me recently. I contacted a customer service department and enjoyed it. I sent an email, heard back promptly, and got a refund. What was most notable about the positive problem-solving experience was the fact that I couldn’t tell if there was a human other than me involved.

It dawned on me, however briefly, that the prophecies were finally coming true. AI was finally making it easier for me to complain to companies and get results. At least that’s what I wanted to believe.

Customer service is supposed to be one of those things that AI can just do. Indeed, that one good experience was powered by an AI-first company called Intercom. They have an AI agent called Fin that handles most of its clients’ queries. Why not all of them?

“I’m confident that a lot of current customer support that happens over a phone or computer, those people will lose their jobs, and that’ll be better done by an AI,” OpenAI CEO Sam Altman told Tucker Carlson, of all people, in September. 

Altman is hardly the only Silicon Valley executive pushing to automate customer service. Last year, Salesforce cut 4,000 customer service jobs in favor of AI tools, and Verizon launched a chatbot powered by Google Gemini as its front door for customer service. Then there’s Klarna, whose CEO bragged about replacing humans with AI before backtracking last May and launching a recruiting drive to hire more human customer service agents.

There’s the rub. It turns out that AI, and especially generative AI, is really good at doing some things…until it isn’t. That’s why you still have to fact-check everything ChatGPT tells you and why, even though they’re good at diagnosing certain medical conditions, chatbots can’t replace human doctors. When it comes to customer service, AI can be good at simple tasks, like issuing refunds, but terrible at handling more complicated cases, especially when customers are upset and could benefit from some human empathy. To quote Anchorman, “Sixty percent of the time, it works every time.”

Still, human customer service agents are losing their jobs to AI in large numbers, and have been for the last few years, both in the United States and abroad. Whether to cut costs or look cool, a lot of companies rolled out AI-powered chatbots as the first point of contact for customers, only to realize that customers actually hate this concept. Now, these organizations are pulling back from those plans, according to Brad Fager, chief of research for customer service and support leaders at Gartner.

“The idea that you could replace your workforce is really just not viable, and it’s not even preferable,” Fager told me, noting that executives might think replacing human agents with AI is a good way to cut costs. “The reality is it’s just not working.”

There’s also evidence that customers just don’t like interacting with AI. One 2024 Gartner survey found that 61 percent of customers would prefer companies didn’t use AI at all for customer service, and 53 percent of them would consider switching to a competitor if they did. As Fager explained to me, Gartner has broadly taken the stance that AI and automation will transform the future of customer service, but that humans will play a big role in that transformation. And to many customers’ delight, a lot of the AI integration will happen on the back end, helping human agents do their jobs better rather than leading interactions. The customers themselves may never know that AI was involved.

This approach reminded me of a study I read a couple of years ago from researchers at MIT and Stanford who looked into how generative AI improved productivity in call center workers. It did, mostly for the less experienced agents. With access to an AI tool that offered real-time suggestions on how to handle calls, the workers were able to resolve 14 percent more cases per hour. The tool had been trained on data from more experienced agents and could even help novice workers be more empathetic to customers. 

Contrast this with what you’ve probably experienced with chatbots: the AI version of a phone tree. This is where you ask a customer service bot for help and are met with a menu of options prompting you to narrow down your request in order to get you to the correct, probably AI-powered agent. It’s a slightly updated version of the infuriating phone tree that asks you to say or press one for billing, two for technical support, and so forth. 

These front-end solutions to identify customers and their needs are essentially AI tools bolted onto old customer service systems, and they’re awful. Werner Kunz, a professor of marketing at the University of Massachusetts Boston, argues that a lot of companies are doing this just to do something with AI. 

“It doesn’t work very well,” he told me. “The failure rate is way too high in comparison to the older systems, and if this is what companies are using AI at the moment for, I think it destroys customer relationships.” Kunz added that using AI in the backend would provide better results in a safer environment, and also, “Who cares about if you use AI or not?”

Which brings me back to my recent, surprisingly positive customer service experience. I reached out to Intercom, the company that built the software, and confirmed that it was an AI agent that solved my problem. There was no phone tree analog and, in a sense, no fight with a chatbot to reach a human agent. Fin, the AI agent, registered my complaint, offered me a solution in a human-sounding email — there were even emojis used in the correct context — and closed the case before I even considered getting annoyed. 

It wouldn’t quite be correct to say that customer service, thanks to AI, is finally starting to get good. As Kunz and Fager explained, lots of companies are getting it wrong by using AI for the wrong things or tacking it onto legacy systems. However, Intercom co-founder and chief strategy officer Des Traynor says that going all in on AI is the best way to give customers what they want: instant outcomes. 

“You don’t want to wait,” Traynor said. “It’s the same reason why people Google before they pick up the phone: People just want instant resolution to problems and that’s what AI offers.” He added, “It’s just categorically better for users — when it works.”

Traynor admitted that AI ushered in an era of software that left people wondering if it worked, and that problem guided the development of Fin. He said his company “put a phenomenal amount of time into building an AI evaluation engine” and “torture-tests every release” to make sure Fin doesn’t hallucinate or get things wrong. As a result, Fin resolves a million customer queries a week with a 67 percent resolution rate, which is not 100 percent, but Traynor said that number is going up 1 percent every month. He conceded that some interactions needed human intervention, but in most cases, the AI can get the job done better. In my case, that was true.

The big problem here, if you’re a consumer, is that you don’t necessarily get to choose how any given company is handling its customer service. There’s also a sort of income equality gap between the haves and the have-nots, whereby bigger companies, like Amazon, can invest more and offer better customer service and small companies, like local utility boards, just do the best they can.

What’s clear, however, is that a transformation is happening. There are signals that complaining to companies is getting easier to do but also strong evidence that many companies will continue to make it difficult, even though they want to make it easier. AI is here to help make things work better, but only if it can stop making them worse first.

A version of this story was also published in the User Friendly newsletter. Sign up here so you don’t miss the next one!