The following interview is a conversation we had with Jason Corso, Co-Founder and Chief Science Officer of Voxel 51, on our podcast Category Visionaries. You can view the full episode here: $15.5 Million Raised to Build the Future of Developer Tools for Machine Learning

Jason Corso
Brett, it’s a pleasure to be here. Thanks for the invitation. 

Brett
Not a problem. I’m super excited. 

Jason Corso
I’d like to begin with a quick. 

Brett
Summary of who you are and a bit more about your background. 

Jason Corso
Certainly. So I’m, as you mentioned, the Chief Scientist at Voxel 51. I was the CEO for the first five years of the company’s creation, but that was the beginning, I guess. So. I’m also on the faculty at the University of Michigan, go Blue, where I research problems around computer vision, robotics, and the relationship between, like, vision, language and the physical world. And in some sense to give rise to the creation of the company, which is fun. I’ve always had 1ft in the sort of like industrial door as a consultant or some sort of a builder. But when you’re making the leap from the academic world to the actual startup world, it’s a lot of learning and you really have to dive in all the way. So. Yep. 

Brett
Was that a master plan? As you were working through your academic career, did you think someday you’re going to go out and build a tech company, or where did that come from? 

Jason Corso
Yeah, good question. So I don’t think I would say it’s a master plan. I did dabble in some sort of startup ideas from the beginning. Right. Like when I was an undergrad, I made this website called Grep Cs.com, which was kind of like a yahoo. Like directory search service, but only for computer science angles. That was fun. As opposed to I did something where it was like spatial search on maps. I love being an academic. I love the problem space, right? What is it now and what does the future look like? And you have almost full control to really set to choose what problems you want to look into. 

Jason Corso
And then at a great school, Michigan, where I’m at wonderful colleagues, wonderful students, and sitting in a room and really tackling a new problem with some colleagues is probably one of the most fun things for me. But over time, as an academic, our economy is papers, essentially, right? Like, we have to do other things, like write grants and teach courses and do some service both in and outside of the university. But, like, really, we’re measured by and, you know, really want to emphasize. I think most people get into academia because they want to set the future of the research domain they’re involved in. And I guess over time, this was counting on, like, maybe almost 15 years. I was like, should I write another paper on computer vision and video understanding, or should I try to make a bigger impact? 

Jason Corso
Cause as your research papers go out, they get read by folks, probably. Now, some of my early work is incorporated into various products that I know nothing about. Right. Because you don’t really know the long tail of what your impact is. But as a startup Founder, I really thought I could have a chance to make a more near term and bigger impact. So I wouldn’t say it was pretty fine, but it’s certainly been a fun journey taking the lead into it, for sure. 

Brett
Let’s dive right into that journey. So take me back to December 2016, or in 2016 when you were founding the company. What was the aha that you had that made you say, yep, this is it. I want to go build this into a company. 

Jason Corso
Yeah. So Brian Moore, my Co-Founder. So I met Brian when he was one of those students who sits in the front of your class and, like, really makes it hard and interesting. Like, hard and a good way to be a faculty member. Right. Always asking good questions and. Anyway, so he took my computer vision course. We stayed in touch for a few years after that, and I think we kind of reconnected over this auto grader that he had written for the EECS department at Michigan. And I wanted to use it in my computer vision course as the numbers went from like 30 to 50 to 80 or 100 or more. And I think were just chatting one afternoon, thinking, like, there’s just so much video out there. 

Jason Corso
I think in my office at the time, there were something like six video cameras, not all on looking us at the same time, but just this realization that, wow, I’ve studied computer vision for over a decade. He’s doing a dissertation audit with a colleague of mine. What can we do now to make an impact for going forward? Right. And we actually initially got started trying to become. Well, initially we got started as consultants through a grant that we secured from NIST. And our thought was like, well, we know how to understand video. Let’s just build a system in which our users, our ultimate customers down the line, could upload their video to our SaaS offering and we’ll find them the insights in the video, kind of like we see today in some sense, with foundation models that can do video understanding. 

Jason Corso
This was, I don’t know, seven, eight years ago. But were lucky to have the grant first because that didn’t work. The two of us got started. We actually hired a couple of the people and had some pilot projects with various vendors. A notable one was the Baltimore City Police Department. Some others that were in the automotive space and the insurance space kind of built these initial successful deployments of the system. When it came time to go and sign a long term production, like, let’s get some business intelligence or production value out of this video understanding system, the users just weren’t ready. So we really had to pull back and take learnings from that effort to then go and ultimately pivot into what 51 is. And Voxel 51 is today, which was really much earlier in the lifecycle of the workflow. Right? 

Jason Corso
Like, the biggest learning we had was that way. One, it was a little too early to understand and spend money on video as a value prop and the ultimate lifecycle of businesses. And at the same time, were becoming a consulting company. For every vertical went into, we had to spend months getting data, labeling the data, like training models on it, and then were able to get insights. It really didn’t generalize that well. Like, we just didn’t have the capacity that current models do. And I’m a little doubtful that current models still have the capacity that people are saying they do. And we ultimately pivoted way earlier in the lifecycle to kind of meet the users where they were at and release the developer tools that we had been building for that consulting work to go. And that is essentially what 51 is today. 

Jason Corso
It’s like our set of internal tools that have been further matured, but really are at the early stages of the lifecycle, like help you curate data, help you figure out where your models are failing, ultimately leading to other downstream etls and various model deployment lifecycle stuff. Yep. 

Brett
Let’s imagine that we’re at a Thanksgiving table and cousin or aunt, says Jason. So what do you guys do there? How would you explain it? In very simple terms? 

Jason Corso
Cool. So we’re ultimately a visual AI company. So have you heard the news about the new autonomous driving companies that are coming out right one key problem that those companies need to do is avoid pedestrians like kids on bicycles, or even avoid other obstacles, like deer in the middle of the road. And oftentimes these vehicles have cameras that are looking forward, looking backward, and so on. So 51. Our software helps the engineers of those systems, and it’s a very horizontal tool, right? So it works in autonomous driving for various use cases, like I gave you, or it will work in like retail scenarios, right? Like if a company wants to understand the traffic flow through their stores and they use video to do that. Well, to build the models to do that, they need something like. 

Jason Corso
So we are the underlying developer infrastructure to support the work that data engineers, machine learning scientists do when they’re building these cutting edge systems on what we call unstructured data, right? So data that doesn’t fit on, like your checking book or like a bank account or a customer database, data that has no schema videos or natural language or audio. We help find the, like kind of tighten the loop in the development process, right? Like in that field, you do things with data, you get the data, you annotate it, you index and so on. Then what you’re ultimately trying to do, though, is use that data to train a model. Just kind of like a computer code or computer algorithm to take new data and predict the output you want, right? 

Jason Corso
So I actually just released a blog earlier today where I gave an example of dog breed classification, right? The data in that case is the image, for example, and the label or the prediction is the actual breed of the dog, like coconut retriever or laBrettoodle or whatever. And our software really kind of makes the work. You do it with the data and the models more effective, more rapid. Really helps you understand, not like the typical cases, like getting the typical distributions of the data, you know, like the mean, the typical scenario, right? Like highway driving, or, I don’t know what. Dogs are the most popular breed in America. Let’s just say golden retriever is right. 

Jason Corso
But instead of those, we want to help you find the tails, like the parts of the data distribution that your models are going to struggle on most, like crowded intersections in snowy conditions, or akita dog breed or something else that’s not as popular. And we really make that more effective through like a python library and a visual app software. 

Brett
And when it comes to the go to market motion, what does that look like? How are you acquiring customers? 

Jason Corso
Right, so when we initially got started, were in the full SaaS mindset. We’ll have some cloud based services. You send us your data and we will provide the insights and send it back to you. But post pivot, despite keeping some of that baggage along with us, we tried to be as flexible as possible, and we ultimately decided to go with an open source, go to market strategy. And we did this for a few reasons. One reason was we really believe in having a kind of like an educational impact on the best practices in the unstructured AI space. When I was a grad student some 20 years ago, one of my datasets was going into the cafeteria early in the morning. I went, I was a student at Hopkins and taking snapshots of this really new and colorful, like, lots of pictures and stuff, cafeteria. 

Jason Corso
And that comprised a dataset that was a significant portion of my thesis. There were like 80 images in that data set. It’s kind of laughable now, today. Since then, data sets went from dozens to hundreds to thousands to tens of thousands to millions. And when I was working with that 80 sample data set, I knew those images in and out. I built an intuition about each of those images, and the practices were really trying to understand, like, okay, for each image or each model variation, what is the model doing? Like, what are the corner cases? What are the challenges? What are the failure modes? Like, really being almost like, really a scientist about the work. But it’s really difficult to be a scientist about the work if you have millions of samples in your data set. 

Jason Corso
Which became popular as it became more available, it was more open and shared on things like Flickr and YouTube, and computing was bigger and so on. In the era of deep learning, many datasets are in the millions, so it no longer became typical to look at every sample. What did people do? They just began looking at precision or recall, or just various aggregate measures of the model performance on the given dataset. And that’s just not a good practice. It’s kind of like one wishes you can start with a problem definition, then, step one, go get some data. Step two, train your model. Step three, deploy. It never works like that. It’s always like, step one, get some data. Step two, train some models. Step three, analyze your problem step. Or is it deployable? Step one, again, like you. Step two, again. 

Jason Corso
And if you don’t actually study and look at your data, find out failure modes. Kind of like walking in a dark hallway. You’re just adding more data to it, trying more model variations. You don’t know what to do. So we really believed in this idea that our software, which is called 51, was going to help scientists espouse best practices. So our go to market actually is give it away for free. We give away the full machine learning stack, the software, as long as it is one user, one machine, local data. In fact, there’s a really cool bridge in the open source software package, which, between the python session you’re working in and the visual apps, anything you do in Python is immediately mimicked in the visual app and vice versa. And that only works if it’s a stateful relationship and one user’s experience. 

Jason Corso
But again, there’s no limitation from a compute stance or machine learning stance. If it can run on the computer, you’re using it on all good commercial or academic or research. But we don’t make money doing that. In fact, about a year. So we released open source, I think it was August of 2020, and there were no tools like it at the time, so our traction was really slow for a while. We can get back to that in a little bit. But it was about a year later when we inbounded a question from someone in the community. We set up a slack community, and it’s on GitHub and stuff like that. So were always interacting with customers or users the whole time. 

Jason Corso
But it wasn’t for another year where a customer inbounded and was like, wait, I got to use this with my teammates at company x. And they kind of gave us the how to establish our go to market. I feel very lucky that conversation happened. And now our go to market begins with the open source. Typically we have some outbound sales and so on, but generally what we sell is a wrapper around the open source software that adds the multi user capabilities, the enterprise security support, the system versioning like dataset versioning all the things that you may want to do in a company, you don’t really necessarily need to do as an individual who’s trying it out, or even an individual in a company just kind of hacking around to see what your data set is all about. 

Jason Corso
And we have a fully, since we are like a community based, community driven company, we’re very active in that community where we’re lucky to have a lot of community members now that answer each other’s questions in our slack space. But we are aware of who’s active in that space, and we do leverage that as a window into how to actuate our account execs when they’re going outbound to see who to talk to. 

Brett
What would you say you’ve learned about. 

Jason Corso
Marketing since launching the company? I would say that you should ride as many coattails as you possibly can in a marketing sense. I’m not a trained marketer. I’m lucky to have some team members who are indeed trained marketers. But what worked best for us was authenticity. And providing a good piece of software, I think is fundamental. Right. We challenge some barriers. We’re not SaaS. In order to use 51, you do need to install locally, or if you buy our software, it’s generally like installed in your private cloud, basically. Mostly because, like, we listened early on and learned from what were hearing, like, were not going to be trusted to house the data of Fortune 500 companies. And we have many of them who are our customers now. So maybe they would now. We’re much bigger now, but in the early days that wouldn’t happen. 

Jason Corso
The idea that it’s running on their space, we don’t have much windows to their properties and so on. But what really worked early on in terms of getting us attention was, like I said, riding the coattails of successful tools. Right. We were doing category creation in this notion of like, data and model analysis tools for unstructured AI. That was not a definition beforehand. Again, mostly because it was appreciated that you should look at your data and like, be able to ask key questions about it. Like, what are the key false positives in this scenario? Let me visualize them. Are they clustered together? Natural questions any data scientist wants to ask. But to do that, they were just writing their own python code, throwing it away a week later. So we kind of capitalized on that. 

Jason Corso
But it took a while to gain traction in the first months, and so we had like 20 weekly active users was like a big number for us. It really wasn’t until we began to cultivate integrations with leading datasets. Like, we are the preferred mechanism for accessing a dataset called Google open images. That’s an open source dataset from Google that we really began to see those numbers take up. So it was like a combination of writing somewhat aggressive blogs that showed the value of a tool like ours. That’s how Google ultimately found us. In fact, we found some errors in the data set through our software and ultimately led to a partnership or a relationship, and then also integration with other tools in the space that really worked as well. 

Jason Corso
I mean, nowadays if you look at our website, it’s a very technical website, many links to the documentation directly. There aren’t many like these. These are workflow x or workflow y landing page. So I think we have more to do. We’re still a young company when it comes to, like, understanding the right way to go to market for our space just about two and a half years, really. But I think the right thing we’ve learned is like, yeah, just bring as much attention as you can to the software and the authentic value the software breaks. We never talk about vaporware. We never promise things in our customer discussions. We were very careful to build that trust in that relationship over time. 

Brett
This show is brought to you by Front Lines Media podcast production studio that helps B2B founders launch, manage and grow their own podcast. Now, if you’re a Founder, you may be thinking, I don’t have time to host a podcast. I’ve got a company to build. Well, that’s exactly what we built our service to do. You show up and host and we handle literally everything else. To set up a call to discuss launching your own podcast, visit frontlines.io podcast. Now back today’s episode. And I know you hinted at this at the start of the conversation, but I want to talk about the transition that you made in summer of last year when you transitioned out of the Yale role and into the current role that you’re in. What was that transition like, and what did you learn from making that transition? 

Jason Corso
Yeah, it was an interesting one. Then Brian and I had been talking about it for a while and were kind of waiting for the right time to do it. So as startup founders, especially co-founders who are very similar, right, like Brian and I, are both technical, both come from like the machine learning world, one has to choose and like build some temporary walls between your responsibility. From the beginning, since were both learning about a whole new problem space in starting a company, and like, we really thrived in leadership by committee, we talked a lot. We really, I don’t want to use the word negotiation, but we kind of like worked through the data that was available for any decision we had in most cases. But I mean, that said, my primary roles at the company were very non technical. 

Jason Corso
I was in strategy, which it is a technical tool to some degree. But like, I focused on strategy. I did all the sales. Founder led sales was key for our 1st 15 customers or 20 customers, and help somewhat with operations. We’re lucky to have a great coo that we hired one of the first non Founder like leadership level hires. But really, we’re not capitalizing on some of my superpower. Whereas Brian, on the other hand, his title was CTO, really led the engineering team, really led that Founder, led product development as well. But he was also keenly interested in building and high level strategy. 

Jason Corso
And we thought that we had reached a critical point in the evolution of point, the company that we really needed to broaden our reach to some of the more like potential scientific user communities or research oriented user communities, and also even begin to establish, like, how can we keep 51 as the leading tool, not only from a like, oh wow, we build infrastructure. It’s the right infrastructure, you need it, and it’s very flexible, build on top of it. But also like, these are the right questions to ask about the work you do that 51 supports. That’s much more technical research than we had done beforehand. We were very much infrastructure only mindset, and we just decided it was kind of like the right thing to do. Were there hiccups in the transition? Absolutely. 

Jason Corso
It was only, I don’t know, six months before that we hired our first AE, basically. So I was still in this elongated partial chief scientist role for some months after the official transition. Likewise, we realized only a few months after the transition that we needed to think about engineering management, because Brian really, theres no room or time for that as a CEO, and he had been doing that for the two years earlier or more. But even through those hiccups, I think we are in a much better place now, both poised for growth. Thats coming in 24 and 25. So I think it was the right thing to do. 

Brett
Trey, something else you mentioned a couple of questions ago is category creation, which is of course a big passion of mine. When it comes to your market category, what would you call this or what would Gartner call it if they were trying to you into an existing category? 

Jason Corso
It is a great question and we think about it a lot. So one category that is easy to point to is annotation. It was like the first or second wave maybe, of AI enablers, if you will, or AI support tools. And what is the problem of annotation in this market? Basically, you collect raw data images of dogs, like were talking about earlier. And in order to train the model, you need to have a human look at some of those images and tell you what the dog is in the model, because it’s not given when you just take the photo, your camera doesn’t tell you the dog, right? So the human goes and provides the label, the ground truth, sad label. And then you use those pairs like for every image as a label to train your models. 

Jason Corso
Adaptation is often the first thing people think of when they see what we do. Right. We don’t have a demo to show in the podcast here, but if you go over to 51 AI, it’ll take you to like the first thing that shows up as a video, just kind of walk through what it is. You see an interface that shows images or videos or even point clouds, and it shows labels and bounding boxes and classification things on top of that. So we actually often confused as annotation company, but in fact, when we decided to go to market and build this like early stage developer tool, we explicitly said we are not doing any annotation. There were dozens of companies in the space. We do not support it. We rather would integrate with it, and that actually worked out well for us. 

Jason Corso
But what’s the next thing that’s a little bit more popular now? Maybe data quality, because in order to train good models, you need high quality data. I mean, that could mean everywhere from like, you can’t have errors in your data or you can have errors in your annotation, right? So it has to be like high quality, but it could also mean like the sample space of images and video or language, phrases, whatever. There are no holes, no missing data. And if there’s missing, you got to go find it and fill those holes. That is a also pretty well defined and easy answer, but I don’t think that’s one workflow for 51, right? 51 is a much broader set of enabling workflows that allow you to squeeze as much juice out of your data to train the best models. 

Jason Corso
So we tend to think of ourselves as a data set management and workflow management system. It is not a well defined category yet, but we have enough of a foothold in the open source user base that it really hasn’t been a problem that there aren’t many unstructured data set management systems that are out there. It’s not like a sexy thing like data set quality or whatever, but we think of it as critical, right? Like roadways in american cities are not sexy either, but you need them to get around it, right? So it is that infrastructure, and we think it’s the right way to think about the category that we’re a part of because it involves data, which is critical to the work you do in a space. It involves models and model evaluation. 

Jason Corso
And in our view, it’s not like a sequential process, like I was saying earlier, right? Like you are actually co developing the data and the models for a particular AI problem, and then you go deploy the result, which is a trained model. And it’s only if you have the appropriate infrastructure to manage the data, to manage your models, set up and train and execute the workflows to do that work. And it makes sense. So I would say dataset management systems and connect that to workflows is probably the right way to think about where we fit in the category. But I think the jury’s out on the wording for it because it’s still pretty evolved, rapidly evolving space. 

Brett
When it comes to fundraising, what are some of the lessons that you’ve learned? As I mentioned in the intro, it’s been over 15 and a half million or, sorry, 15 and a half millions been raised so far. What are some of those lessons? 

Jason Corso
Fundraising is a natural component of doing a startup. As you also mentioned, I came from academia, where we fundraise all the time. Professors and research universities have to fund their research labs and so on. Typically thats done in a less face to face Manner and more like write a grant, send it in, and it gets reviewed. So there is some relationship building there. But I think fundraising in startups is really interesting and unique. Some key lessons we learn, right? Get multiple term sheets or anytime you’re fundraising, a key value is learning how to shop who you are and what you’re building in a voice that is contextualized in the community and is confident, obviously, but also not again, we have both Brian and I tend to be rather willing to express what we don’t know. 

Jason Corso
I’m not sure how many founders share that, but it has played well for us. It is a learning process and you have to sort of be who you are in those meetings. But it has helped for us to talk to as many investors as possible prior to actually doing the fundraising and keep those conversations warm because the investors really care about the market, they care about the evolving market. And it may be different than how you view the market because generally you tend to be pretty laser focused as a Founder. And sometimes it’s are to come away from that laser focus and learn the right way to contextualize what you would bring to the market. So that’s definitely been a key lesson that we learned. 

Jason Corso
But I think maybe another lesson would be like, just because someone tells, you know, it doesn’t mean what you are doing is not the right direction, right? Like, you have to have a lot of confidence in what you’re doing. I mean, we’ve been at this for many years. Right before you mentioned, like we first started this, the LLC for that grant in December 2016. We didn’t decide to convert that to a C Corp until I think it was October 2018. So it was almost two years. And then we took our first seed funding then. And then we pivoted, kind of like after that, away from the initial angle, but the whole time, like were kind of stubborn. We listened obviously to our customers and the users and like really committed to this vision of what is the right support we can provide to people. 

Jason Corso
Building the future of unstructured AI software capabilities. And we knew infrastructure was the way to go. We knew open source was the way to go. I think being willing to stick to your guns and hear many knows was a value to us and it made us. It’s not easy to hear, but ultimately there’s some lesson to learn in each of those conversations. 

Brett
Let’s imagine that you were speaking with an early stage Founder who wants to build in a similar space. Based on everything that you’ve learned so far, what would be the number one. 

Jason Corso
Piece of advice that you give them? Number one piece of advice for a similar Founder in this or a similar space? I would say open source is whats so valuable to our growth and its so important to the open exchange of ideas. I think that if you can do open source as a go to market strategy in this, anything related to the AI space, I think it will pay its dividends, maybe not immediately, but in the long term. And I dont mean open source like youre expecting the community to go and build your main features or even supplementary features, one in 8000 or something like that situation. I just think it’s so important to understand how users of your technology use the technology, which may not be the way you think you expected them to use the technology. 

Jason Corso
It is so valuable and you’re going to get that feedback very rapidly if you engage the open source community. And at the same time it does also help the field. The whole level of the water rises. Right. I think if you do that as well. 

Brett
Final question, let’s zoom out into the future. So three to five years from today, what’s the big picture vision that you’re building here? 

Jason Corso
Yeah, so I think data is the heart of machine learning and AI, unstructured data. It’s the essence. It’s how we take a capability in the form of an algorithm or a model and we render it down or distill it down to functionality that actually works in a domain. And we believe that across the space of data, right. Like we focused on computer vision modalities first like images, video, then point clouds and soon 3d meshes and so on. But we are already expanding beyond those to other domains like audio and text and IoT and medical and so on. But it doesnt stop there, right? One cant just build the infrastructure and then hope for the best. 

Jason Corso
In my view, the world involves ever evolving problem spaces that need to have a feedback loop all the way, starting at the initial engineers, data scientists, data architects, and so on, that are rendering down this language based or natural language problem or problem description into a functioning system, and then figuring out how to make it functional, how to get the right data for it, what’s the right model for it, that code development I mentioned, and then figuring out how to go and deploy it and connect it to what we think of as data apps to end users that are non technical. They may be insurance claims adjusters, or they may be real estate agents, or they may be QA people on a factory line, technical in their own way, but not from an AI standpoint. 

Jason Corso
They do things with the data app as that world is being executed in real time. And then I think the key is providing the infrastructure to support those types of data apps and then actually closing the loop in the sense that what those non technical end users do with that app, that needs to feed all the way back to the engineers and the scientists. Because it’s not a static problem. You can’t train a machine learning model on a Monday and not come back to it on a Friday. It’s like a constantly evolving space, understanding of limitations, improvements you can make, and so on. So I think that’s the future of AI. 

Jason Corso
And the future of AI that Voxel will support is this world of data apps that are constantly evolving and thriving and connecting, like engineers who are sharing workflows and working on co development of models and data sets and so on with their end users through these apps and then getting the feedback over the long haul. Amazing. 

Brett
I love the vision and I really love this conversation. I’ve definitely learned a lot from you and the audience is going to learn a lot as well. 

Jason Corso
Before we wrap up here, if there’s. 

Brett
Any founders that are listening in, they feel inspired and they just want to follow along with your journey. Where should they go? 

Jason Corso
Right on. So the best place to go is 51 AI, where you would land on our docs page or if you go to my LinkedIn page. I am standard Jason Corso. You can even book a meeting with me. In fact, I’m happy to talk. I have open meetings available daily and I also hold recently began about a month ago holding open office hours every Monday at noon eastern time. And really, anyone can attend at my personal Zoom meeting. And I have had some of the most engaging and interesting conversations on those Monday moon hours in just the last four or five weeks. It’s been really exciting. So find me there amazing. 

Brett
Jason, thanks so much. Really appreciate it. It’s been blast. 

Jason Corso
Thanks, Brett. It’s been fun. 

Brett
This episode of Category Visionaries is brought to you by Front Lines Media, Silicon Valley’s leading podcast production studio. If you’re a B2B Founder looking for help launching and growing your own podcast, visit frontlines.io podcast. 

Jason Corso
And for the latest episode, search for. 

Brett
Category Visionaries on your podcast platform of choice. Thanks for listening, and we’ll catch you. 

Jason Corso
On the next episode.