The Hidden Dangers of Early Customer Success: Sleuth's Journey to True Product-Market Fit

When Dylan Etkin joined Atlassian as one of Jira's first three developers in 2005, he couldn't have predicted that fifteen years later, he'd be tackling one of the most pressing challenges in software development: engineering efficiency. But sometimes the most compelling startup ideas come from years of firsthand experience with a problem.

In a recent episode of Category Visionaries, Dylan shared how his experience running BitBucket at Atlassian led to the creation of Sleuth, a platform that helps engineering teams measure and improve their efficiency. The journey reveals important lessons about product-market fit, the dangers of early validation, and the evolution of enterprise software categories.

The Origin Story That Wouldn't Die

The seed for Sleuth was planted during Dylan's time leading BitBucket at Atlassian. "When I started running that team, I think there was like seven of us. By the time I finished, there was something like 40," he recalls. As the team grew, tracking deployment effectiveness became increasingly challenging.

"It kind of struck me like a lightning bolt that deploy, which is this thing that at the time, we treated almost like as a second class citizen, was the most important thing," Dylan explains. "That was when all of this hard work that a developer had done and even, like, design had done, product had done to bring into life hit customers, which is the moment when it actually matters."

This insight led Dylan to start building a tool, initially called Deploy Hub, around 2015. Even after returning to Atlassian through an acquisition, the idea persisted. "I gave the idea and the opportunity, a lot of opportunity, to just die on the vine and it refused. Every time I would talk to somebody about it, I'd get super animated and interested."

The False Signal of Friendly Customers

When Sleuth launched in 2019, they took an open development approach that yielded early success - perhaps too early. "We're lucky enough to have a lot of folks in the industry that we had known who were bought into the idea and the vision," Dylan shares. "And so they started paying us reasonably early."

But this early traction proved deceptive. "We started with this thing that we're calling deployment tracking, and it's still at the core of what we do. But it was not the value that organizations are looking for," Dylan explains. "We had some people that were paying us and giving us a little bit of false sense of having found our product market fit, when in reality, I think we're a little bit off the mark."

Evolution in a Nascent Market

The journey to true product-market fit required constant evolution, particularly in how Sleuth communicated its value. "Being in a market that is nascent and currently evolving, our messaging and what resounds and what customers are looking for is evolving daily," Dylan notes.

This evolution led Sleuth to focus on DORA metrics - deployment frequency, lead time, failure rate, and mean time to recovery. "The Dora metrics have really allowed us to take the real information and glean some sense of usefulness out of them," Dylan explains. "And so a market has grown up around that."

The Vision: Salesforce for Engineering

Looking ahead, Dylan envisions Sleuth becoming "Salesforce for engineering." He draws a compelling parallel: "The best sales teams back in the day before Salesforce were doing a lot of the things that Salesforce allows other teams to do... But what Salesforce did was sort of say to everybody, this is how you do it. You don't have to be a top tier team in order to use these tools."

For engineering teams today, Dylan sees a similar opportunity. "Engineering has been ripe for that transformation for quite some time," he notes, pointing to the need for standardized ways to measure and improve engineering effectiveness.

The lesson for founders? Sometimes the most valuable startups don't come from inventing entirely new categories, but from standardizing and democratizing best practices that top performers already use. The challenge lies in identifying the right moment when the market is ready for that transformation - and being willing to evolve your approach until you find it.