When AI Becomes The Mystery Shopper

Mystery shoppers have been helping retailers test the customer experience for years. They pose as ordinary customers, ask questions, attempt purchases and report back on what worked and what didn’t.

Now, some organisations are beginning to use AI to do the same thing; AI agents can browse websites, ask questions through customer service chatbots and even attempt to complete purchases, testing different journeys and spotting where things go wrong. They can repeat the process far more times than a human mystery shopper could, making it possible to test thousands of different interactions.

Which leads to a slightly strange situation: AI is pretending to be a customer, to test the AI that’s serving the customer.

There is a practical reason for doing it. As organisations introduce more AI into customer journeys, they need ways to check that those systems actually work when faced with different questions, behaviours and scenarios. Using AI to run those tests means they can do far more of them, while people review the results and decide what needs fixing.

But I think it’s the loop that’s interesting.

We’re starting to see more examples of AI not simply helping people complete a task, but interacting with other AI systems as part of the process. Here, one plays the customer, another plays the retailer and the human increasingly sits outside the interaction, looking at what happened. It’s a small shift, but an interesting one. As AI agents become more common, the customer journey may increasingly involve interactions that aren’t between a business and a person at all.

And that raises another question for retailers: if your customers start using AI too, who exactly are you designing the experience for?