One of the questions I get asked most often is…

How should I build my audience?

And my answer is always another question.

What are you trying to learn, and what do you have to work with?

This usually produces a pause. People expect me to launch into a feature demo or explain our calibration engine. Instead, I'm asking about their constraints.

Because here's the thing. The biggest mistake I see isn't building audiences wrong, it's starting at the wrong level of sophistication for what you actually need.

The Paralysis Problem

You're staring at the audience generation interface. You have four options for generating personas, and a nagging voice in your head asking "but how do I know these will be accurate?"

So you close the tab and schedule another meeting to "discuss requirements."

Or worse. You assume you need the most sophisticated solution from day one, without first having created internal appetite to explore synthetics. Both roads lead to the same place. Stuck.

I get it. The trust barrier is real. When someone is told they can simulate human responses with AI, a common objection is to call bullshit. The funny thing is, this is also true of AI personas. Here’s what they said when we showed them our landing pages.

 

This is why we spent the last 8 months running experiments to understand what these systems can and cannot do.

What AI Personas Actually Give You

Before we go further, let's be clear about what we're building here.

AI personas don't predict individual behavior. They can't tell you whether Customer #4847 will click "buy now" on Tuesday. As I wrote previously, what they can do is help you understand how people talk about decisions, the reasoning, objections, and narratives that shape behavior in the conversational spaces where purchases actually get influenced.

They help you:

• Stress-test messaging before you spend the budget

• Explore positioning options before you commit

• Understand why an idea that seems obvious to you might land completely differently with your audience

• Ask follow-up questions to research that's already complete

The catch? You have to stop thinking about prediction and start thinking about exploration.

If you need the full philosophical framework for why this matters, read "What Focus Groups Are Really Telling You". For this guide, I'm assuming you're past the "should I use synthetic research?" question and you're ready for the "how do I actually build this?" answer.

The Right Audience Depends on Your Stage

Here's what most people missed. The four levels of AI audience sophistication aren't quality tiers. They're fit-for-purpose tiers.

A Level 1 audience generated in 30 seconds isn't a "worse" version of a Level 3 calibrated panel. It's a different tool for a different job. Like using a screwdriver versus a power drill. Sometimes you actually want the screwdriver.

The question isn't "what's the best audience I can build?"

The question is "what's the right audience for what I'm trying to achieve right now?"

Let me show you what I mean.

The Four Levels of AI Audience Maturity

Think of these as different tools in your research toolkit, not as a ladder you're obligated to climb.

Level 1: AI-Generated | Uncalibrated

Time to build: Seconds to minutes

What it is: You describe your target audience in a text box. The AI generates personas based purely on that description and its training data.

Best for: Rapid prototyping, initial concept exploration, brainstorming sessions

The tradeoff: Fast and cheap, but you're getting generic AI outputs with all the biases baked in (human and AI). As we've documented, uncalibrated LLMs skew liberal, overrepresent environmental preferences, and miss real-world cost sensitivity. You're getting responses, but they're filtered through the AI's default worldview.

Cost: $ (lowest)

 

Level 2: Clone or Memory-Enriched | Uncalibrated

Time to build: Minutes to hours (depending on data prep)

What it is: You ground your personas in real data, survey responses, interview transcripts, social media posts, customer reviews.

Two approaches:

• Memory enrichment: Add real human responses as "memories" to existing personas
• Cloning: Generate new personas directly from your source material (AI infers persona traits from the data)
• Or use both approaches together

Best for: Survey boosting, leveraging existing research, building stakeholder confidence with compelling demo and convincing outputs.

The tradeoff: Still fast and relatively cheap, now anchored to a dataset your team already trusts. The personas will reflect patterns in your data. However, their performance is still "black box", you don't know exactly how accurately they'll mirror known truths in the source material.

Cost: $$ (moderate)

Operators Edge: If you’re sharing a real research artefact, consider using our Google Sheet  template with some pre-prepared audience segments based on your real-human research.
Link this prepared simulator worksheet in your presentation so teams can:

• Explore the segments in more detail

• Ask follow-up questions

• Reduce survey fatigue (without blowing the budget)

Level 3: Virtual Panel | Calibrated

Time to build: Days to weeks

What it is: Personas trained through our calibration engine (custom service) that runs thousands of experiments comparing AI responses to real human responses. We build tailored eval systems, measure accuracy, identify systematic errors. Then we optimize persona-of thought prompts (unique per persona) until response quality beats expert AI judges.

Best for: Strategic decisions, board presentations, anything where you need accuracy and transparency, improved response diversity, or to tackle a discovered case of AI bias. Great for media diet testing.

The tradeoff: Higher cost and longer timeline, but you get transparent performance. You know how accurate your personas are because we'll have tested them against real humans responses. The responses are more realistic, diverse, and grounded in human lived (spoken) experience.

Cost: $$$ (bigger investment - contract floor $10k)

Operators Edge: Optimizing to remove bias from AI personas can remove other types of bias to.

Want access to a virtual panel? We're building a virtual panel of real US consumers covering all major consumption categories. Our source data is verified (human video interviews) and can speed up the time to using calibrated audiences. Request a demo.

Level 4: Virtual Panel with Infinite Memory | Calibrated

Time to build: Weeks to months

What it is: Calibrated personas with long-running memory systems and branched timeline management. They expand the limits of our in-session memory, utilizing Mem0 to power a dynamic approach to base_memories. This means they can remember every conversation, can reason about their past decisions, and dare I say, develop emergent identity… with options for you to turn back time and branch off different universes. (e.g. ‘how would this voter have reacted had they known about x before 'y’)

Best for: Long-term strategic intelligence infrastructure, scenario planning over advanced timeframe management, advanced media diet simulations.

The tradeoff: This is the closest thing to "uploaded intelligence" if you're a Pantheon fan. But it requires more serious infrastructure investment.

Cost: $$$$$ (enterprise-level commitment)

Why This Framework Matters

Most clients to date don't start at Level 3 or 4 because they typically either don’t have the data to calibrate against, or need to be made more aware of the difference between calibrated vs uncalibrated. They start at Level 1 or 2, build internal confidence, identify specific use cases, and then invest in calibration. 

This common adoption path might have been due to us not having had an on-the-ready calibrated audience (something that will soon change). But even so. I expect there to always be a time and place for being quick, scrappy and cheap with some audiences. Perhaps to have more budget for custom developement building proof-of-concept automations that illustrate what an end-to-end synthetic persona capability could look like. 

This isn't settling for "good enough." It's recognizing that you need to answer different questions at different stages:

Early stage: "What could this approach look like for us?"
→ Level 1 answers this in an afternoon

Mid stage: "Do these personas sound like [enter audience of interest]"
→ Level 2 grounds them in data your team trusts

Advanced stage: "Are these personas exploring realistic conversational terrain"
→ Level 3 removes systematic AI bias for higher-quality exploration with transparent benchmark + improvement scores

Strategic infrastructure: "Can we build a dynamic research capability with the safety guard rails?"
→ Level 4 creates long-term capability

The audience level framework exists because different questions jobs to be done require different levels of investment. Let's look at how to choose your starting point.

Finding Your Starting Point

The right level depends on three factors. What you're trying to learn, what data you have available, and what kind of organizational buy-in you need.

Let's walk through the most common scenarios I see, and which level makes sense for each.

Scenario A: "I Need to Test Messaging Concepts by Friday"

Your situation: You're an agency creative team with three positioning angles for a product launch. The client meeting is Friday. You need to stress-test these concepts against different audience segments to identify obvious problems before you present.

Starts at: Level 1

How to do it:

1. Open AskRally and use the description box to generate 3-4 audience segments (e.g., "budget-conscious millennials," "tech-forward enterprise buyers," "skeptical traditionalists"). 

2. Ask each segment to react to your messaging concepts

3. Look for patterns: Which objections surface repeatedly? Which value propositions fall flat? What language resonates versus confuses?

4. Generate 20-30 personas per segment

Tip: Do you have more accurate descriptions than "budget-conscious millennials"? Great. Paste it in the description box. Or try clicking ‘enrich description’ and then edit (description enrichment works best with OpenAI model).

What you'll learn: You'll quickly identify messaging that triggers resistance, confusion, or enthusiasm. You won't know if these reactions perfectly mirror your real audience, but you'll get ideas and spot obvious problems to refine your concepts before the client meeting.

What you won't learn: Whether these responses represent genuine diversity or just AI's tendency toward certain bias.

Operators' edge: You can try to mitigate diversity and audience trait bias with Personification which just means correcting for persona distribution across known facts. Ie, if 70% of your base is female with 25% chrome usage vs 30/30 for male? Mirror that when making the personas. This adds audience prep time/complexity and isn't a sure fire defense against AI bias. Another option is you could create your personas outside of AskRally and import them in with our CSV template. Just make sure to leave the ‘data column’ free or AI will infer the persona based on the content you provide in the 'data column' (cloning).

 

Scenario B: "I Have Survey Data But Limited Budget for Follow-Ups"

Your situation: You just completed a 500-person survey. The report is done. But now stakeholders have follow-up questions that weren't in the original questionnaire. Commissioning new research would cost $30k and take six weeks. Teams are making decisions without the answers.

Starts at: Level 2

How to do it:

1. Export your survey data and organize it by segment

2. Format it into AskRally's CSV upload where each unique respondent’s entire questionnaire is included in the ‘data’ column as well as in the ‘base_memories’ column. This will clone personas from your survey AND give them memories of their survey responses. If you don't want to clone, then fill in the demograhphic data in the CSV, but skip adding anything to the 'data' column. Instead just add your survey in the base memories column if you're doing memory enrichment. 

3. Configure our Google Sheet template, add your API key and make map audience IDs to segment descriptions (see example below)

4. Share the template with stakeholders so they can ask their follow-up questions

What you'll learn: You'll get responses more likely to reflect the patterns and preferences already evident in your survey data. When you ask follow-up questions, the personas are more likely to reason from the same values and constraints your real respondents expressed.

What you won't learn: Whether the personas are faithfully representing patterns in your data or introducing AI's default biases when reasoning beyond what's explicitly in the survey. The performance is "black box"—grounded in your data, but you can't see where your data ends and AI inference begins.

Real example: This is exactly what we did for the vehicle manufacturer. They had completed research across three markets with segments at different purchase funnel stages. We took their survey data, cloned personas with memories of the actual responses, and translated them into native languages for each market. The client's team got a Google Sheet where they could ask 300 follow-up questions per day to these survey-grounded personas (using only one API key on smart plan $100/m). The initial time investment was 1-2 days of data prep and audience setup. The result? Teams who weren't involved in the original research brief could explore the segments, test new messaging concepts, and get answers to their specific questions, all without survey fatigue or additional fieldwork costs. This became what I call a "paid demo" that built so much internal enthusiasm that they commissioned Level 3 calibrated personas for their next phase. Now we’re building their virtual panel in collaboration with their qual research partner. 

Scenario C: "I Need to Present Findings to the Board"

Your situation: You're making a strategic recommendation that will shape the company's direction for the next 18 months. The board wants evidence, not assumptions. Your CMO understands that marketing is about navigating conversational realities, but your CEO needs to see proof that these AI personas actually reflect how customers think and talk, not just how AI thinks customers think.

You need calibrated personas.

Starts at: Level 3

How to do it:

1. Engage us for custom calibration work aka, “Virtual Panels” Request a demo.

2. We build tailored evaluation systems tailored to your project goals that compare AI persona responses to real human responses

3. Our engine runs hundreds (sometimes thousands) of experiments, identifying and removing systematic AI biases, optimizing unique persona-level prompts until expert AI judges can no longer distinguish between individual real human and AI persona responses

4. You receive bias-corrected personas in your AskRally account—ready to explore the conversational reality your product will actually enter. 24/7. Millions of conversations per year ;)

What you'll learn: Which conversational realities you're actually navigating versus which ones AI's biases would have shown you. The responses reflect authentic diversity, not AI's tendency toward consensus or its learned defaults. You're stress-testing ideas against realistic discourse patterns, the actual objections and narratives that will shape your market, not the sanitized version AI generates by default.

What you won't learn: How to explore branching timelines or accumulated context beyond session context window limits. These personas give you authentic discourse patterns for the present moment, but to place them in context of present moments, you’ll need to build a media diet testing workflow like this one. However, Level 4 turns this idea into a living research asset where personas remember every ad they've seen that moved them, every conversation they've had, and lets you branch timelines and explore "what if they’d never seen that first campaign?"

Scenario D: "We're Building Long-Term Strategic Intelligence Infrastructure"

Your situation: You want a permanent research asset that evolves with your market understanding. Personas that remember past research, accumulate insights over time, and lets you explore different scenarios across branched universes.

Starts at: Level 4

The reality: Just like with virtual panels (ie, calibrated personas), we’re not quite ready to serve this on a self-serve level (yet). We're building toward it, but for now, this is the a pretty big custom project for us. If your project required it, we could accelerate development quickly and could build in a modular way tailored to your project needs. If you want it, DM me on X and mention 'advanced media diets', 'universe branching', or 'deep memory'. 

What it would enable: Imagine asking your personas "How has your perception of our brand changed over the past six months?" or spinning up a new universe, one for where each of your competitors got in your prospects' ear first with their message, and finding out how it influences perspectives as their message collides with yours in the prospect's simulated mind.

This is uploaded intelligence territory, personas that function as long-term research participants rather than one-off simulation tools.

The Decision Tree

Still not sure where to start? Here's the quick version:

Start at Level 1 if:

• You need answers in hours/days, not weeks

• You're exploring whether synthetic research makes sense for your organization

• You have no existing research data to ground personas in

• Budget is under $500

Start at Level 2 if:

• You have survey data, interview transcripts, or other research you can clone from

• You need to build stakeholder confidence with convincing demos

• You want to reduce survey fatigue by enabling follow-up questions

• You want to overindex on building POC automations rather than the audience

• Budget is $500-$5k

Start at Level 3 if:

• You need to remove systematic AI bias and explore authentic conversational reality

• You're making strategic decisions where generic AI outputs could lead you astray

• You need transparency on how closely personas mirror real customer discourse

• You've already proven value with Level 1 or 2 and want bias-corrected exploration

• Budget is $10k+

Start at Level 4 if:

• You need personas that accumulate memory across months of interactions

• You want to explore branching timelines and counterfactual scenarios

• You're building long-term strategic intelligence infrastructure, not one-off research

• You're willing to work with us on development of next-gen capabilities

• Get in touch to discuss

Start Somewhere

The paralysis at the beginning of this guide? It comes from thinking you need to make the "right" choice.

But here's the truth: the right choice is the one that gets you moving.

If you have no data and need to explore fast → Level 1. Spin up personas in minutes and see what surfaces.

If you have research gathering dust → Level 2. Turn it into a living resource your team can actually use.

If you're making strategic bets → Level 3. Request a demo and let's talk about virtual panels (calibrated personas).

The worst outcome isn't starting at the "wrong" level. It's staying stuck in requirements meetings while your competitors are already exploring.

Remember. AI personas don't predict what people will do. They help you explore what people might say, and that's where decisions actually get shaped.

Stop optimizing the decision. Start exploring the conversational reality your product is about to enter.