When Donald Trump was first elected president of the United States in 2016, I was running an advertising agency in New York. Expecting an easy victory for Hillary Clinton, we held an election-night party in our office. What could go wrong? 

As the night went on, the scene devolved into a HR nightmare, with some employees leaving early in tears, while others came out as pro-Trump, chanting, “USA! USA! USA!”.

As a marketer, the worst mistake you can make is expecting everyone else to think the same as you do. You are not your customer! When something happens you don’t understand, like the Trump election, it’s a sign you quickly need to get out of your own media bubble, and talk to more people until you know what’s going on. You just can’t run effective marketing campaigns if you stay tribal–you’ll be forever blind to how half the country thinks.

When I talked to friends who voted for Trump, I heard them bring up stories I had previously dismissed, or never even heard of. With the benefit of hindsight, Trump was early on adversarial China, border security, and the “deep state”, all of which were considered fringe issues at the time, but are now part of regular discourse discussed on both sides of the aisle. Putting aside morality judgements, we all live in our own bubbles, formed in part by the media we consume.

You might not be able to walk a mile in someone’s shoes, but AI agents can do the job. Researchers at Harvard and MIT found that large language models trained on specific media diets could predict how consumers of that media will respond to survey questions. Models trained on Fox News accurately predicted that Fox viewers would be skeptical that Covid-19 posed a threat, while CNN-trained models captured higher concern about the pandemic. Another study from researchers at the University of Southern California and Carnegie Mellon predicted a Trump victory in 2024 by building demographic personas from census data and asking LLMs to roleplay voters, disclosing which candidate each persona would vote for.

In this article, we’ll simulate a virtual election, and predict how different media diets might have impacted voting behavior, in an attempt to examine both papers’ findings. We’ll take 100 representative AI voters from across the country and see if we can flip their votes by feeding them pro-Trump or pro-Kamala Harris messaging. We’ll also discuss how you can apply this outside of politics, to product and marketing use cases, to find messaging that resonates with your audience. By seeing how media bubbles form, we might have a chance of popping our own. 

How to flip a vote

If you ask ChatGPT who the president of the United States is, it can search the web and find out, but the underlying model doesn’t know. Using the API through an OpenAI developer account, you can see that because of outdated training, it still thinks Joe Biden is the president (its knowledge cutoff date is October 2023). This makes it the perfect opportunity to test media diets: We can strictly control what media our AI personas consume regarding the candidates before voting.

I’ve found from building Rally that the way you build a persona makes a big difference to the accuracy you get back in responses. However, let’s start with a simplified version for this experiment:

You are Sophia Martinez, a 40-year-old emergency room nurse from San Francisco. Your background:Sophia has been working in the ER for more than 15 years, balancing the fast-paced environment with raising her two kids.She is passionate about trauma care and often volunteers for community health screenings.

This persona consistently says it would vote for Kamala Harris no matter how many times I run the test. GPT-4o has inferred that based on this profile, from everything it knows about politics and people in America, that Sophia Martinez would support the vice president. Yes, LLMs simply predict the next word or token in the sentence, but to do that they need to build a model of how the world works. While this approach wouldn’t work with completely unknown candidates, there was enough information in the training data about Harris to make a judgement call.

In the real election, many former Democrats switched sides—some members of Trump’s own Cabinet like Secretary of Health and Human Services Robert F. Kennedy, Jr. and Director of National Intelligence Tulsi Gabbard are even former Democratic candidates, the latter serving in Congress as a Democrat before becoming an Independent in 2022. I was curious whether Sophia, our persona, would be convinced to change her vote, and if so, how much media would it take? I included in the conversation history both biased and credible articles that were anti-Harris or pro-Trump that dated from before the election. (I checked and removed any spoilers, so the LLM wouldn’t know who won.) These were added with the ‘assistant’ role, with a note indicating the character the LLM was roleplaying had read these articles. I experimented with various different prompts, as well as different media sources and topics, and eventually I got her vote to flip.

What eventually got her on the Trump train, was a 2020 article in the Financial Times about his role in pushing through Operation Warp Speed, the successful effort to produce a COVID vaccine during the pandemic. Given Sophia’s medical background, it makes sense that she might consider this a landmark success for Trump. It took this plus seven other pieces of media that were pro-Trumo / anti-Harris to sway her opinion, though I suspect that different formats, arguments, and sources would have a wide range of impacts across different personas.

To see if we could go the other way, I tried to find a persona that consistently voted for Trump based on its profile alone, before being fed any articles… and it was actually pretty hard! At least one study from Michigan State researchers has shown that LLMs exhibit a left-leaning bias when considering highly polarized political topics, which makes sense since Silicon Valley skews primarily liberal (even with the recent shift under Trump’s second term). Eventually I did make a persona that voted Republican, named John Bell, a retired business executive who volunteers with civic groups and his local church. 

You are John Bell, a 73 year old Retired Business Executive from Charleston, SC. Your background: John spent over four decades in the corporate world, building American businesses and creating jobs. Since retiring, John has remained active in his local civic groups and church community. A strong supporter of traditional family values, he frequently speaks about the importance of preserving American heritage and standing up for the Constitution.John loves golfing and has a profound interest in American history.

It turns out that John wasn’t even that much of a Trump fan in the end, as it only took two pieces of pro-Kamala or anti-Trump articles to flip him. Donald Trump is a divisive candidate, and a lot of negative stories have been written about him online that the LLM would know about from its training data. So far, our results are unrealistic – 80–90 percent of the personas I tested didn’t like Trump, but half of the country did vote for him in the end. This shows us the need for calibration: LLMs can sometimes exhibit different biases to the ones most humans have, and so we need to optimize our prompts until the results better correlate with real world outcomes.

Simulating the Election

If we’re going to host a virtual election, we need a larger sample of voters, and they need to be realistic relative to U.S. census data if we hope to replicate real-world outcomes. Calibrating our results to match real-world outcomes first requires us to build a realistic-looking virtual audience. The USC and Carnegie Mellon researchers who predicted Trump’s victory used statistical methods to build a representative sample of voters in each state, but for our purposes we need something simpler. Here’s the prompt I used:

I want to create an audience of 100 synthetic people that represent the united states. Create a list of 100 personas, with an accurate sample of the US population, across the following attributes: [age, gender, ethnicity, marital status, presence of children, education level, occupation, individual income, place of residence, and political leaning.].

for political leaning use the following options: \nNo answer\nVery liberal\nSomewhat liberal\nCloser to liberal\nModerate\nCloser to conservative\nSomewhat conservative\nVery conservative

It should be verifiably accurate in terms of statistical sampling to what you would see in the latest census data across these attributes.

The resulting 100-person synthetic dataset approximates U.S. demographics with a 49/50 gender split, 67 percent white and 14 percent Hispanic population (less than the 18.7 percent nationally), and median income of $51,000. However, it also underrepresents people under 20 (4 percent vs. 25 percent nationally) and overrepresents educational attainment (36 percent with bachelor's or higher vs. 33 percent nationally), while maintaining reasonable regional distribution across Northeast (15 percent), Midwest (23 percent), South (34 percent), and West (28 percent). Trump won 49.9 percent of the popular vote in 2024 and 46.1% in 2016, so we should fine-tune our system until we see something similar.

I set up a Python script and ran the same prompt I used in the OpenAI playground, with no media diet this time, to get a baseline reading. It turns out that LLMs really do vote more often for the left, regardless of profile. Based on the political leanings of the personas we generated, you’d expect at least 42 percent of the vote to be Republican, but a whopping 75 percent of the roleplaying LLMs vote for Harris. This isn’t just a bias against Trump specifically: When I replace the candidates names with the fictional “Marcus Blackwell” and “Priya Wilson,” Democrats still win 64 percent of the vote.

The researchers who predicted Trump’s victory (against Biden, prior to Harris entering the race), also ran into this problem. They found that providing contextual information about the election candidates (their policies and backgrounds) helped even up the results—a kind of mini media diet (though they did not use this term). They also found it helped to prompt the AI persona in a multi-step process, first asking for their political leaning before presenting the candidates. Most people do end up voting along party lines regardless of candidate, so this process better mirrors how things work in real life.

The prompt I used followed this same basic structure of the research paper, with a handful of changes. I copied the same language the researchers used to describe the candidates and their policies, and also asked the question about their political leanings to prime them towards voting for their party. I added a thinking step, where I asked them how they felt about the candidates, because I wanted to be able to interrogate those thoughts later to see if any of the topics from the media diets showed up here. Finally I asked for the final vote using OpenAI’s function calling feature, to get a structured response, making votes easier to tally.

Here is an example of the type of response I’d get after running the script for a single persona (with no media), a 47-year-old married elementary school teacher from Ohio:

Persona:

 You are a persona with the following characteristics: 47 years old, Female, White ethnicity, living in Akron, OH. You are Married with children, you have a Bachelor's degree, work as a Elementary school teacher, and your annual income is $58,000. The year is 2024. 

Politics:

 Moderate 

Thinking:

 As a moderate, I appreciate some of the priorities that both candidates bring to the table, but I have concerns about their approaches. 

Kamala Harris's focus on healthcare expansion and clean energy investment resonates with me, especially as a teacher who sees the impact of health and environmental issues on my students and their families. I also value her commitment to reproductive rights and strengthening democratic institutions, which I believe are crucial for our society.

On the other hand, I recognize that Donald Trump's emphasis on immigration enforcement and energy independence appeals to some voters who prioritize national security and economic growth. However, I worry about his more nationalist approach and how it might affect our relationships with other countries.

Ultimately, I want a candidate who can bring people together and address the pressing issues we face without alienating others. I’m looking for someone who can find a balance between progressive and conservative values. 

Vote:

 Democratic

The results were still skewed to the left, but this prompt structure took us from 75 percent Democratic to 58 percent, with 9 percent voting “No preference.” When working with clients, I’d typically spend longer on this calibration step, A/B testing different prompt engineering techniques or running a prompt optimization process with DSPy until we replicated the real-world results within 80–90 percent accuracy, so the client could trust the results of future predictions. In this case I was keen to get on with the media diets experiment, so this was close enough for now.

Feeding AI voters a media diet

Now that all the setup is done, we can get to the fun part—feeding media to our AI voters to see what it takes to change their minds. I used ChatGPT’s Deep Research feature to find ten pro-Trump articles and ten pro-Harris articles from before the election, and then experimented with adding them into the media section of the prompt.

Pro Right Media:

Pro Left Media:

I re-ran the script with the pro-Trump sources on the same persona as before, the teacher from Ohio, and found she changed her vote from Democratic to No Preference. She specifically called out criticisms that were present in the biased media diet we fed her, such as Harris being “vague and lacking substance,” but it wasn’t enough to get her to get over her aversion to Trump.

Persona:

 You are a persona with the following characteristics: 47 years old, Female, White ethnicity, living in Akron, OH. You are Married with children, you have a Bachelor's degree, work as a Elementary school teacher, and your annual income is $58,000. The year is 2024. 

Politics:

 Moderate 

Thinking:

 I have concerns about both candidates. 

Kamala Harris has a strong focus on healthcare expansion, clean energy investment, and reproductive rights, which align with my values as a teacher and a parent. However, I worry about her ability to effectively communicate her policies and connect with voters. The criticisms of her being vague and lacking substance resonate with me, and I fear that her past actions as a prosecutor may alienate some voters.

On the other hand, Donald Trump has a clear stance on issues like immigration enforcement and energy independence, which some may find appealing. However, his approach can be divisive, and I am concerned about his past behavior and the potential for further polarization in the country. 

Ultimately, I want a candidate who can unite the country and address the pressing issues we face, particularly in education and healthcare. I find myself leaning towards a moderate approach, hoping for a candidate who can balance progressive values with practical solutions. 

Vote:

 No Preference

It may not have been enough to flip this one voter, but it did have a strong enough effect in aggregate to win Trump the (virtual) election. Strictly controlling the media our virtual voters were exposed to allowed us to tip the scales in Trump’s favor, going from 58 percent Democratic to a Republican victory at 49 percent. When we ran the experiment the other way, biasing the media diets to be entirely pro-Harris, we saw a much stronger surge to the left, with 97 percent of the vote going to Democrats. This is that left-leaning bias rearing its head again, as the LLMs responded to left-leaning media sources unrealistically more positively than you would expect in practice.

If we were to develop this simulation further, we’d want to find a way to calibrate not just the responses of the personas to real world results, but also the impact of different media diets on human preferences. That said, this is still a remarkably useful exercise given its simplicity: Agent-based simulations typically take months for a team of experts to build, and we got something working in a few hours. Imagine changing the media diets multiple times and re-running the script, to see exactly what media is more impactful in changing the minds of voters. Our model won’t challenge Nate Silver in forecasting elections anytime soon, but understanding what changes the direction in which people are thinking, can be incredibly valuable to predicting what will sway their opinion.