A subtle but crucial detail in this study is the use of synonym-grouping when scoring model responses—meaning, instead of just checking if the AI picks a single “correct” word in response to a prompt, the researchers aggregate probabilities across a set of synonyms that all reasonably answer the same question (see Figure 2A and methods, pages 3–6). This addresses what’s known as “surface form competition,” where language models might spread probability mass across similar words, underestimating the true likelihood that the model “knows” the right answer.

Why it matters: This approach is important because it makes the AI’s simulated responses robust to how survey questions are worded and to the natural variation in human language. Without synonym grouping, you’d miss a lot of true signal—models might look inaccurate simply because they’re picking “required” instead of “necessary.” By capturing the full range of plausible answers, you get a much truer measure of whether your AI audience is aligned with real people—and this is a key reason the method outperforms simpler or naive scoring approaches.

Example of use: Suppose a brand manager is testing synthetic customer reactions to a new product description. Instead of measuring only whether the AI personas label it “exciting,” she aggregates across related terms like “innovative,” “interesting,” or “intriguing.” This allows her to see the total positive response, regardless of exact wording, giving a more stable and realistic read on likely customer sentiment—especially useful if the market uses varied language.

Example of misapplication: If the manager only scores for a single word—say, “exciting”—any AI persona that picks “innovative” or “game-changing” would be counted as a miss, leading to artificially low or misleading results. In practice, this could cause her to reject good messaging or underestimate product appeal just because the simulated audience didn’t use the exact terms she expected, not because they actually responded negatively.

• Ground your simulated audience in recent, real media exposure: Market research simulations are much more accurate when virtual personas are built around the actual news, TV, and radio content recently consumed by your target audience, rather than generic or outdated information.

• Use advanced language models fine-tuned on audience-specific media: Off-the-shelf AI models don’t capture real group differences well; the study found that fine-tuning models (like BERT) on real media diets for each group increased prediction accuracy from r=0.19 to r=0.46 (see Figure 2A, page 5).

• Account for how closely people pay attention to media: The accuracy of simulated audience responses improves when you factor in how much attention the real-world group pays to the news—adding this variable to the model made predictions even more closely match survey results.

• Write survey and study prompts using synonyms and natural wording: The way questions are phrased affects AI accuracy, but using synonym grouping and multiple phrasings makes simulated results more robust and less sensitive to wording quirks (see Figure 2C, page 5).

• Don’t rely on demographic targeting alone: Demographic variables like age, gender, or education were much less predictive than recent media exposure and attention, so focus on understanding what your audience is actually seeing and reading.

• Test the approach first on issues where opinions are known to shift with the news: The method works best for questions about national trends, public health, or the future—areas where media coverage strongly shapes perceptions—rather than purely personal or retrospective topics.

• Consider supplementing traditional research with fast, low-cost simulations: Because this approach can track opinion shifts daily or weekly at much lower cost than repeated human surveys, it’s a practical way to spot early trends or test messaging before launching expensive studies.

Prompt Explanation: The AI model was probed using fill-in-the-blank prompts derived from real survey questions, with target words representing survey response options, to estimate the likelihood of each answer as a proxy for subpopulation opinions.

Requiring most businesses other than grocery stores and pharmacies to close is [BLANK] in order to address the coronavirus outbreak

1. Define customer segments: Create three AI persona profiles. For each, describe age range, financial goals, digital habits, and typical concerns.
 • Young adults new to credit: 23, first job, uses mobile banking, wants to build credit and avoid debt.
 • Middle-aged homeowner: 45, married, owns a home, manages family expenses, uses online banking but prefers simple tools.
 • Retiree: 68, on fixed income, cautious with new technology, values stability and avoiding scams.

2. Prepare prompt template for persona simulation: Use this structure for each simulated interview:

You are a [persona description].
Here is the new product being tested: ""A digital budgeting tool that links to your bank accounts, categorizes spending, offers monthly savings tips, and provides credit score updates, all in a simple mobile app.""
You are answering an open-ended survey for your bank.
Respond as yourself in 3–5 sentences.
First question: What is your first reaction to this budgeting tool? What would make you want to try it or not?

3. Run the prompt for each persona: For each segment, generate 10–15 AI responses using the prompt above. Slightly vary the question phrasing for diversity (e.g., ""What would you find most helpful or confusing about this tool?"" or ""How does this compare to how you currently manage your budget?"").

4. Add follow-up prompts: For each initial response, ask a relevant follow-up, such as ""What concerns would you have about security or privacy?"" or ""Would you use this regularly, or only for occasional check-ins? Why?""

5. Tag and summarize key themes: After collecting responses, label them with qualitative tags like ""mentions privacy,"" ""positive about savings tips,"" ""skeptical of technology,"" or ""asks about family budgeting."" Group responses by segment to see common themes.

6. Compare segments and messaging: Summarize which features and messages were most appealing or raised concerns in each group. Identify patterns (e.g., retirees care about simplicity and security; young adults want help with credit-building; homeowners focus on family or bill management).