Backstory: Why the CCO Threw Out Her CS Playbook and Started Measuring AI Maturity

Natalie Wolf had built the playbook over a decade. She knew how to map the customer journey end to end, how to sequence onboarding sprints, how to set health scores and run QBRs. She had done it at Deloitte, at Accenture, at Anaplan, at Salonis. It worked.

Then, roughly four to five months before this conversation, it stopped working.

"Everything I thought I knew and felt like I had a playbook for really got thrown out the window," she said. "In fast moving categories like AI, optimization doesn't just come fast enough — it's compounding."

This wasn't a product problem or a market problem. It was a structural problem with how CS itself was designed. The assumption baked into every journey map — that you can identify a moment, optimize it, and move on — collapsed when the underlying category started moving faster than the iteration cycle. By the time you'd fixed the onboarding touchpoint, the customer's definition of value had shifted. Natalie wasn't behind on execution. She was behind on the unit of work.

In a recent episode of BUILDERS, Natalie shared how she rebuilt her CS motion from scratch at Backstory, where she leads customer success, services, support, and the full revenue and customer operations function.

Replace the Journey Map with the Experiment

The first thing Natalie changed was the format. Instead of onboarding sprints built around milestones, she started running AI hackathons — same-day value experimentation sessions designed to show customers what was possible immediately. The logic is direct: if the market is moving faster than you can build a journey, stop trying to build journeys and start compressing the time to insight.

"What I'm doing with my customers is hosting AI hackathons, value experimentation sessions, iteration on what could be possible in one day," she said. "The market and AI are just moving faster than you can build a journey map or build a solution — and therefore you're solving the wrong problem."

The shift isn't just tactical. It reflects a different theory of customer value. Traditional CS assumes value is delivered through structured progression. Natalie's model assumes value has to be demonstrated before the customer fully understands what they're trying to achieve. The hackathon isn't an onboarding alternative — it's a mechanism for collapsing the time between contract and conviction.

The Change Agent Model: Internal First, Customer-Facing Second

Natalie didn't hire to solve this. She redeployed.

She identified the most curious people already on her CS and services teams and redesignated them as internal AI change agents. Their job: deploy new AI workflows inside Backstory first, prove they work, then bring those workflows directly to customers.

"I've basically taken our most curious people in my services team and my CSM team and said, you're going to be now our change agents internally for deploying AI in everything we do and you're also going to bring that expertise to customers," she said.

The structural insight is the sequencing. Internal deployment isn't separate from customer work — it generates the proof points that make customer conversations credible. A CSM who has actually built a working AI workflow inside their own company is not pitching a capability. They're demonstrating one.

One concrete output of this model: success plans. Natalie's team feeds every customer email, Slack thread, and Confluence page into an AI tool and generates a working success plan in minutes — one the CSM, AE, and SE can iterate on together. "You could take all of the data that you have and insights around the customers, every email, every conversation, and then pair that with your confluence page of knowledge, everything that's happening in Slack and input that into Claude or your AI tool of choice and make a success plan in mere moments," she said. The time saved matters less than what it enables: cross-functional iteration on account strategy that rarely happened before because the friction was too high.

The New Retention Metric: AI Maturity Progression

The change that signals how seriously Natalie has rethought CS fundamentals is what she now measures.

Her team is no longer measured primarily on health scores. They're measured on how many healthy customers they can move up an AI maturity curve — from experimentation to scaling to transformation.

"My team has measured on how many customers in green health can you move from experimentation to scaling to transformation? And that's our new measure," she said.

The problem this solves is concrete. Health scores are a lagging indicator of relationship stability, not a leading indicator of customer growth. A customer can be green — no churn risk, regular engagement — while being completely stuck at the experimentation stage. Natalie visits large enterprise customers and finds them nowhere near scaling. The health score says they're fine. The maturity curve says they need help.

Structural Alignment: Take the Silos Out of Forecasting

Natalie's other structural change was to forecasting rhythms. At Backstory, no renewal forecast call happens without shared ownership between the CSM, AE, and CRO. She shows up for new growth calls for the same reason.

"We don't do any of our forecasting rhythms in silos. So think every renewal forecast call has shared ownership between the CSM and the AE and the CRO," she said.

The insight isn't that collaboration matters. The insight is that shared presence in revenue rhythms changes accountability in ways that async communication never does. When the CSM and AE are looking at the same renewal number in the same room, ownership is unambiguous.

The Activation Debt Threshold

For founders building their CS motion, Natalie's most specific warning is about timing. Activation and onboarding debt compounds quietly until it becomes structural — and the threshold is earlier than most founders expect.

"A lot of startups hit like 30 to 40 million and then decide to fix this and then by then it's so hard to do," she said. The downstream consequences are specific: innovation slows because teams get stuck on fundamentals, field teams spend cycles on problems the product should solve, and the market moves on.

The fix isn't a CS hire or a new onboarding sequence. It's product architecture — which is why it has to happen before scale makes product changes expensive.

The principle connecting all of Natalie's changes is the same: stop optimizing the system you built for a market that no longer exists. In a category moving as fast as AI, that's not a one-time reset. It's the permanent operating condition.