How we deal with bias
Harnessing the power of AI in our Synthetic Users, we strive for a balance between reflecting reality and ethical responsibility, ensuring diversity and fairness while maintaining realism.
It’s tricky and fascinating at the same time and we’ll tell you why. When we use large language models (LLMs) as the foundation for organic simulation, the goal is to make these simulations as realistic as possible. If the AI reflects biases present in society, then we make the argument that these biases help make Synthetic Users behave more like organic users. This is useful for testing how systems work in the real world, which includes all the messy and imperfect aspects of human behavior.
The above mentioned reality is the one we inhabit. For Synthetic Users, the biases we are concerned with are usually at a higher level, specifically when we deal with large numbers of Synthetic Users for the purposes of creating surveys.
A simple example: At the beginning we found a lot of geographic biases in our Synthetic Users. If we were to ask: create 10 Synthetic Users in France, most of the users would be from the Paris area. It does make sense after all seeing the Paris is the most populated area in France. But for our customers this should not be the default. The default should be a more even distribution across the territory, unless Paris is specified. And so we corrected for this geographical bias.
We correct for the eco bias as well. Synthetic Users have a tendency to be over protective about the planet when in fact we know that most people may think that way, but the numbers show their actions are not as aligned as we would hope. For our customers this is important. We want to offer them an accurate reflection of society. Again: high Synthetic Organic parity is our core value proposition.
OMG, VR and Crypto. We’re not sure what happened with the training data but there was an over-reliance on those two tools. At first it was funny, every solution relied on one or another, but very soon thereafter it became annoying and we’ve had to compensate for it. Like these, there are countless examples.
The other side of the mirror
Now if you take a scenario where the tool being developed needs to perform resume triages or create resumes, based solely on the person’s name, biases are not helpful. Take the paper titled "The Silicone Ceiling: Auditing GPT’s Race and Gender Biases in Hiring" where researchers used GPT-3.5 to assess if it was fair when it comes to helping companies decide who to hire. The researchers wanted to see if the LLM was treating people differently based on their gender or race when evaluating resumes.
They did two main experiments. In the first one, they checked if the LLM gave different scores to resumes just because of the names on them, which can suggest a person's gender or race. They found that the AI sometimes gave lower scores to women and people from minority groups, especially in job areas where white men are more common. In the second experiment, they asked the LLM to write resumes for different names. The LLM again showed bias by giving better job experiences to men and suggesting lower-level jobs for women and minorities.
The LLM reflected existing societal biases. It begs the following questions:
Do we wish to continue to perpetuate biases: Even if biases in an AI model accurately reflect current societal biases, continually using these biases in testing and simulations can reinforce and perpetuate them. This creates a cycle where biased AI systems continue to affirm biased human behavior, making it harder to break these patterns. In a world where most of the data will be produced by AI agents, we can see how this is going to be a real problem.
For the purpose of Synthetic Users, we want an accurate reflection of society and its biases in order to provide our customers with the highest Synthetic Organic parity. As society evolves and in turn further trains models, we ask but that these models accurately reflect the society that produced the training data upon which the models are trained, no more, no less.
Ethical questions and visualising biases
Intentionally using biased AI models raises ethical and legal questions. It challenges us to think about the responsibility of technology creators to strive for fairness and equity. This makes more sense in tools that do not just reflect but rather create a new world. We’re not trying to excuse Synthetic Users as simple mirrors of today’s society, but in our ideal tool, biases will become visible and those who use Synthetic Users will create in perfect awareness of how much they wish to correct for those biases or not.
Detecting Hidden Biases: This paper discusses ways to find hidden biases in AI systems, which means spotting unfair preferences the AI might have that aren't immediately obvious.
Measuring Uncertainty: One method used is called Uncertainty Quantification (UQ). It looks at how certain the AI is about its answers. If the AI is unsure, it might be because it's relying on biased information. This helps us understand when and why Synthetic Users might be making biased decisions.
Explaining AI Decisions: Explainable AI (XAI) is another technique used to make it easier to see how AI systems make decisions. By breaking down the AI’s thinking process, we can spot if it makes decisions based on biased views.
Practical Tools: The paper describes specific tools and methods like using different versions of data to test the AI, or having multiple AI systems analyze the same data to compare results. This shuffle, what we call alternating between different LLMs and having LLM diversity in our interviews, helps us spot and measure biases much faster.
Diversity is always the answer if you want to move past the mirror
Irrespective of whether you are an advocate of LLMs as pure mirrors or someone who is trying to correct injustices, we know that we cannot correct what we cannot see — as such an accurate mirror is important. We also know that biases are easy to measure and as such training on more diverse data sets will not only reflect the breadth of human behavior but also counteract existing biases. This doesn't mean removing all realism; instead, it involves a thoughtful balance between realism and fairness to better represent the diversity of user experiences.
Releated Articles
More articles for you
The Lie We Tell Ourselves About Customer Research
Most research asks what people say. The problem is people don't do what they say. This piece breaks down the gap between stated and revealed preference — and why behavioral modeling, not better interviews, is how you close it.
Two ways to run research with Synthetic Users and why the difference matters
Iris, what is the difference of using agents to accelerate research.
Synthetic Users vs digital twins
You don’t need a twin for “a parent in rural Ohio who shops weekly at Walmart, prefers fragrance-free, and has a toddler with eczema.” You sample a parent profile with relevant traits and constraints, add retail and dermatology context, and generate behaviors consistent with both.
Two major papers. One shared direction.
LLM-powered Synthetic Users have crossed from concept to validated method. This proves they can predict human behavior accurately, letting teams run fast, low-cost behavioral experiments without replacing real participants.
Gartner says we lead. That's kind of them.
Gartner’s latest report on AI-powered synthetic user research cites Synthetic Users as a leader.
Introducing Shuffle v2
Shuffle v2 is a feature that intelligently shuffles between multiple large language models via a routing agent to produce more realistic, diverse Synthetic Users with better organic parity.
Chain-of-feeling
Synthetic Users use a “chain-of-feeling” approach—combining emotional states with OCEAN personality traits—to produce more human-like, realistic user responses and yield richer UX insights.
Generative Agent Simulations of 1,000 People
A paper that thoroughly executes a parity study between Synthetic and Organic users.
21 Peer reviewed papers that support the Synthetic Users thesis
Here is a compilation of all the papers that help make a case for Synthetic Users.
Why we shuffle between models — to ensure both parity and diversity!
Synthetic Users balances aligned and unaligned models to maintain diversity and authenticity in simulated users while ensuring ethical standards and user expectations are met.
Latest press articles for Synthetic Users
Synthetic Users and AI are transforming research methodologies, offering innovative, cost-effective alternatives to traditional human subject studies.
Comparison studies. The opportunity lies in the deviation.
When we compare different studies, especially looking at what synthetic (artificial interviews) and organic (real-world interviews) data tell us, we often find they mostly talk about the same things but there's also a bit where they don't match up. This gap is super interesting because it's like finding hidden treasure in what we thought we knew versus what we might have missed.
How we deal with bias
Harnessing the power of AI in our Synthetic Users, we strive for a balance between reflecting reality and ethical responsibility, ensuring diversity and fairness while maintaining realism.
The transition to Continuous Insight
The transition towards Continuous Insight™ aligns research activities more closely with the dynamic needs of the business and ensures that product development is continuously informed by up-to-date user insights.
The Art of the Vibes Engine
Large language models (LLMs) like GPT-4 serve as powerful "vibes engines," empathizing with diverse groups and generating contextually relevant content. Their applications span market research, customer support, user experience design, and mental health support, offering invaluable insights and personalized experiences. While not infallible sources of truth, LLMs enable creativity, personalization, and connection within the realm of human language.
There is a faster and more accurate way to do research. Use Synthetic Users.
How Synthetic Users is changing the research process.
The wisdom of the silicon crowd
In the light of an ancient parable, we explore a new paper that dives into how ensembles of large language models match the prediction accuracy of human crowds. It reveals that combining machine predictions with human insights leads to the most robust forecasting results.
Three research papers that helped us build ❤️ Synthetic Users
For the sceptics amongst us who need more tangible research in order to engage with this brave new world. Full disclosure: we are part of the sceptics.
What is RAG and why it’s important for Synthetic Research
Ahead of our RAG launch we explain Retrieval-Augmented Generation (RAG) and how it enhances Synthetic Users by providing increased realism, contextual depth, and adaptive learning, with profound implications for market research, user experience testing, training, education, and innovative product development.
Synthetic Users system architecture (the simplified version).
Foundation models underpin Synthetic Users with advanced capabilities, enhanced by synthetic data and RAG layers for realism and business alignment, all within a collaborative multi-agent framework for richer interactions.
Saturation score. How do we know how many interviews to run?
Determine your interview target for achieving topic saturation using our efficient approach, leveraging the historical wisdom of research pioneers. This method ensures deep insights with theoretical sampling at its core.
How Synthetic Users are gaining depth
Synthetic Users are evolving to address criticism about their generalist nature by incorporating representative data sets and personal narratives.
How we compare interviews to ensure we improve our Synthetic Organic Parity — 85 to 92%
How do we know we are right? How do we know our Synthetic Users are as real as organic users? We compare.
Synthetic Users: Merging Qualitative and Quantitative Research, in seconds.
At Synthetic users we are blurring the lines between qualitative and quantitative research. Here's how we are going about this transformative approach.