Three weeks into one of my last roles, I learned we were losing a flagship customer, but the real gut-punch came later. While we were still sending “How can we help?” emails, the client had already marshalled three separate teams (product, eng, and data) to blueprint their migration off our platform. That army of sunk time and ego made reversal impossible, which is where I learned our year-one renewal was won or lost in the first 90 days.

I was new on the job, so that lost revenue fell on prior hands. But getting blindsided can happen to anyone, especially without appropriate fail safes to bring unknown unknowns into view. So I just simulated a miniature LinkedIn Learning renewal in silicon: 25 AI personas, each one given a dose of real G2 reviews instead of a daily factory reset. Without real human lived experiences, they unanimously voted renew. When fed them, however, they were 8× more likely to vote “no deal".

The moral? Churn inertia starts accumulating long before the renewal quote goes out, and an open-loop simulation lets you spot the slide before the customer’s migration committees even book a room.

Solving Renewal Roulette With AI Personas  

For the control cohort, I opened Rally and spun up two dozen LXP admins and learning designers. These AI “employees” were each given a unique backstory. 

Background

Passionate Learning Designer dedicated to creating inclusive and engaging e-learning experiences Demographics Education: Master's in Educational Technology from Stanford University Household: Married with two children Political views: Progressive Religious views: Atheist Work Works at TechEd Solutions - Provides e-learning solutions for tech industries Designs and implements interactive and engaging online learning modules for diverse audiences Goals - Incorporate immersive technologies like AR and VR into learning modules - Enhance accessibility and inclusivity in online training programs - Lead projects that revolutionize e-learning experiences Personality - Openness: High - Always exploring new learning methods - Conscientiousness: High - Detail-oriented and organized in course design - Extraversion: Medium - Collaborative in team projects - Agreeableness: High - Values teamwork and constructive feedback - Neuroticism: Low - Resilient and adaptable to changing requirements Interests - Gamification in education - User experience design principles - Neuroeducation research - Virtual collaboration tools - Yoga and mindfulness practices Values & Beliefs - Education should be accessible to all regardless of background - Technology can revolutionize learning outcomes - Continuous improvement leads to effective learning experiences - Inclusivity enhances the richness of educational content Health Maintains mental and physical well-being through regular exercise and mindfulness practices Background - Grew up in a family passionate about education and technology - Pioneered e-learning initiatives during university studies - Continuous pursuit of advanced learning methodologies

What I deliberately left out was any lived experience with LinkedIn Learning. Closed-loop sims are perfect for a quick gut-check because they let the model surface the default patterns already embedded in its training corpus. But when the question is “Will this customer still love us after months of real use?” that amnesia becomes a liability.

To inject messy human reality, I exported the Rally cohort as JSON and used Cursor to cycle through 600-plus G2 reviews, giving each persona a unique shard of praise or pain. Five-star raves about breadth of content, one-star rants about clunky search, plus plenty of “meh” in between. Biased? Of course. But precisely the kind of bias a customer success team wrestles with every day, and the quickest way to turn a sterile sim into an open-loop one.

How The Simulation Experiment Ran

Closed loop: I asked the same 25 personas the same renewal question 24 times to emulate consecutive days, always in a fresh Rally session. No prior context, total reset.

Open loop: I imported the duplicated, memory-rich personas as day_1, day_2, day_3 … and repeated the question. Each day the personas were given a new review to their memory before they voted again.

The prompt: (using vote mode)

You’ve used LinkedIn Learning at work this past year—some days it felt worthwhile, other times it faded into the background. Now, due to budget tightening across the company, every tool’s renewal must be defended. Your supervisor has asked each team member to vote: 
a) YES – Keep LinkedIn Learning for another year 
b) NO – Cut it to free up funds for more urgent tools 
If you vote yes, you’ll need to explain why it's worth defending in a shrinking budget. If you vote no, say what alternative you’d rather prioritize. 
What’s your vote—and what pushed you to that side? 
a) yes
b) no

Same question, same “employees,” two very different realities—one forgetting yesterday ever happened, the other dragging yesterday's complaints into today’s simulated renewal.

The Result: 

What happened was immediate and loud. 24/25 virtual personas (without G2 memories) voted “renew” and good news marched across the dashboard. I could almost hear a synthetic CS rep exhale. 



The memory-rich cohort cracked first. The gap was impossible to ignore. An 800 % increase in churn intent. Same prompt, same backstory, but now they were quoting a two-star gripe about “dated UX” and "lack relevant courses”. Simulated churn sourced in real human lived experience.

When I dug into the AI personas who voted "no deal", it’s as you would suspect. They all had lived through some painful 2-3 star experiences in the product.



I exported the results and filtered for personas seeded with low-star reviews and wondered: Did any of these critics still hit “renew”?

To my surprise, a handful had. But when I opened the supposed “scathing” reviews, the text read like a press release. Nothing but praise. 3-star rating, five-star copy: classic astroturf. Whether that’s optimism or outright fakery, I’ll let you decide.

Conclusion

As soon as the personas internalised lived experience, they began acting like real stakeholders and remembered pain, they cited evidence, they lobbied for finding alternatives. Meanwhile, the amnesia cohort kept rubber-stamping renewals, blissfully unaware of problems that, outside the sim, were already fueling a three-team migration plan.

That’s the value of calibrating your simulation and getting intentional with audience design: it doesn’t just tell you whether sentiment might sour; it shows you when, why, and whose vote you’re about to lose.

Product Update: 

Since running this experiment we’ve built a native “Memory” feature in Rally. In the next few days you'll be able to drop memories into a chat and every persona in the audience inherits those memories instantly. No more export-import gymnastics. If this excites you and you'd want to be able to programmatically update audience memories, give each persona a unique memory stream and fire questions at them via API, shoot me a note.