Why You Should Never Chat with 'Synthetic Users': The Ethical Case Against AI-Generated User Research

The Allure of the Instant Focus Group

Let us be entirely honest: user research is exhausting. If you are a product manager, developer, or founder, you know the drill. You have to source participants, coordinate schedules across three time zones, offer Amazon gift cards as bribes, and sit through forty-minute Zoom calls where half the time is spent helping someone find the "share screen" button.

So when modern AI tools promised the ability to create "synthetic users"—fully-formed LLM agents programmed to behave like "a 35-year-old mid-level sysadmin who hates Slack and loves open-source tools"—the collective sigh of relief in Silicon Valley was deafening.

Suddenly, you do not need to schedule anything. You can just prompt Gemini or Claude with your design mockups and ask: "What do you think of this onboarding flow?"

It is fast. It is incredibly cheap. It is also a complete ethical and practical disaster.

Replacing real human feedback with AI-generated avatars does not just lead to boring, generic products; it actively erases real human lived experiences, reinforces dangerous biases, and cuts you off from the messy, unpredictable truths of what makes your users tick.

The Echo Chamber of the Average

To understand why synthetic users are so dangerous, we have to look at how LLMs actually work. At their core, these models are predictive engines designed to output the most statistically probable next token based on their training data.

When you ask an LLM to pretend to be a specific persona, it does not conjure up a real human mind with complex quirks. It generates a caricature. It gives you the average of every blog post, forum comment, and stock description of that demographic ever written.

If you ask a synthetic "working-class single mother" what she thinks of your budgeting app, the AI will spit back tired clichés about saving money on groceries and feeling stressed. It will not tell you about the broken screen on her three-year-old budget Android phone that makes your fancy SVG graphics unclickable. It will not tell you about the spotty bus Wi-Fi she uses to access her bank account.

By relying on synthetic users, you are designing for a stereotype. You are building in a closed loop of confirmation bias, where your AI validates your assumptions, and you use that validation to build products that ignore real human friction.

The Ethical Hazard of Erasing Real Voices

Beyond the practical issues of bad data, there is a profound ethical concern here: the systemic erasure of marginalised and non-standard users.

When a design team decides to bypass human interviews in favour of synthetic users, they are choosing convenience over inclusion. The voices that get squeezed out of training data—people with accessibility needs, non-native English speakers, those from lower socio-economic backgrounds, or people living outside major tech hubs—are the exact same voices that get completely erased by synthetic personas.

If your training data doesn't have deep, nuanced representation of a blind user navigating a website with a screen reader, your synthetic "blind user" persona will be a shallow hallucination. Designing a product based on that hallucination is not just bad UX; it is a form of exclusion disguised as innovation.

We cannot build accessible, equitable technology if we refuse to look at, speak to, and listen to the actual people we claim to serve.

Where AI Actually Belongs in User Research

This does not mean AI has no place in your research toolkit. The key is to use LLMs as an analytical tool for real data, rather than a generator of fake data.

Instead of generating synthetic people, use AI to scale your understanding of real ones. Here is how to do that ethically:

1. Synthesise, Don't Fabricate: Conduct ten real, messy interviews with actual human beings. Record them, transcribe them, and feed those transcriptions into a high-context model to extract themes. The AI is brilliant at finding patterns in real human speech that you might have missed. If you need help structuring these analysis prompts, check out our prompt library for research synthesis.
2. De-biasing Prompt Engineering: When using LLMs to analyse feedback, explicitly instruct the model to look for outliers and dissenting opinions, rather than just the majority sentiment.
3. Accessibility Audits: While AI cannot replace testing with real disabled users, you can use automated tools and LLMs to flag technical accessibility failures in your code before you ever show it to a human. For technical troubleshooting on building these automated pipelines, look at the Gemini Support Site.

Here is an example of an ethical prompt you can use to analyse real customer interview transcripts without fabricating data:

`markdown You are a UX research assistant. I am going to provide you with the transcripts of 5 real user interviews.

Your task is to: 1. Identify the top 3 friction points mentioned by the users. 2. Specifically highlight any "outlier" comments—feedback that contradicts the rest of the group or points out unique accessibility or environmental challenges. 3. Do not assume or extrapolate beyond the words in the transcripts. If a user did not mention a pain point, do not invent one. `

Talk to Humans

Product design is not an academic exercise in statistical probability. It is an act of empathy. It requires you to sit with the discomfort of hearing that your beautiful, prized interface is confusing, frustrating, or completely useless to someone who is tired and just trying to get through their workday.

When you replace real people with synthetic ones, you are hiding from that discomfort. You are choosing the easy path of the echo chamber over the hard, rewarding path of real human connection.

Put down the API key, close the chat interface, and go talk to a real person. Your product—and your conscience—will thank you for it.