Own or Be Owned: Why Every Company Needs Its Own AI Model (Yash Patil, Co-Founder & CEO of Applied Compute)

[00:00] There is truly some atrophying happening by relying too much on AI tools. The one thing I'm worried about people losing is this ability to actually deeply think about how things should be architected and designed. The best engineers right now are the ones who are like, learn how to code before AI. Because at the end of the day, I think humans will be the overlords over all of these AI tools, and you will still need to do critical thinking to know what you want to build. Palantir is one of the few companies that signs massive contracts without knowing if they can solve the problem.

[00:30] you know, $100 million deals before they've started working out the problem. Right? So, I think we've been into that category where there's open research questions and people who like to work on things that are uncertain, just naturally attracted to companies like AC. I know there's a lot of folks who are like, oh, 50% of jobs are going to go away in the next two years. Personally, I think that is not going to happen. I think just the diffusion of AI into the real world take time. [00:54] you [01:02] Demand for intelligence is outrunning the chips that supply it.

Meanwhile, tokens are being sold at subsidized prices that won't last, [01:11] Demand keeps climbing. [01:12] and companies are discovering that using a frontier model for every task is like cooking with a blowtorch. [01:18] Spectacular, and the wrong tool for most jobs. Yash Patil lives at the center of that squeeze. [01:25] The 23-year-old ex-OpenAI researcher runs Applied Compute, [01:29] a $1.3 billion company that trains custom models on a business's own data. [01:35] smaller, [01:36] cheaper, [01:37] and built for the work that [01:38] that company actually does. [01:40] One year in, Aniyash's startup is already serving a mix of tech startups and established corporations.

[01:46] helping them enter the AI era, all while owning their own intelligence. In this episode, we discuss why open-weight models are quietly taking over real workloads, [01:57] Why Yash believes AI's transformation of the economy will take [02:00] decades. [02:01] rather than years. [02:02] and what it was like inside OpenAI the weekend the board fired CEO Sam Altman. [02:08] I'm Mario. [02:09] And this is The Generalist. [02:11] The best founders aren't spending their time on expense reports. They're busy building. Brex is the agentic finance platform that makes that possible. High limit corporate cards, banking, and AI that handles the back office automatically so your team never has to.

[02:26] Expenses get captured, books get closed, and spend stays in policy without anyone chasing it down. [02:32] your team gets their time back to focus on what actually moves the company forward. [02:38] Vercel, OpenAI, Anthropic, Granola, and Deepgram. [02:42] already run on Brax. [02:44] One in three startups in the US does too. [02:46] It's time to get bricks. [02:48] Go to com/solutions/startups. [02:54] Every revolution in AI creates one question that never changes: Can you trust the output? [03:00] AI for work is incredible, but without trust, it's just leading to faster mistakes.

[03:06] The challenge isn't building an AI that can answer questions, it's making sure those answers are right. [03:12] That's where Guru comes in. [03:14] It's the AI source of truth that connects everything your company knows. [03:17] So every insight, every answer, every recommendation [03:21] is grounded in verified knowledge, not outdated information or hallucinations. [03:27] When your teams and your AIs share one trusted foundation, [03:30] everything moves faster. [03:32] with fewer redos, fewer blind spots, [03:34] and more confidence in every decision. [03:36] Because in the age of AI, truth isn't just power, it's protection.

[03:41] See what Guru is doing for thousands of companies like Spotify, DHL, and Stripe at com. [03:48] That's com. [03:51] Yash, lovely to have you here. [03:54] And we've been chatting already sort of before we we've sort of kicked this off. And I think it makes sense to to maybe start with some of the things we were chatting about. That's good to you. Of course. Yeah. Yesterday, Fable 5 came out. [04:06] uh big day in the world of ai and has implications [04:11] for all of us, but including on the sort of the business you're running.

I know you've got a few thoughts on it. What came to mind for you when you sort of saw it? I think, first of all, the model is amazing. Like, a ton of our team members were using it. It's like, [04:26] clearly feels like a step function. I think [04:30] There have been moments in time where we've seen better and better models, and this is definitely like a point in time that should be noted. [04:37] I think, though, the thing that you're referring to is kind of, [04:41] when we were looking at the system card, and this was kind of blowing up on Twitter and whatnot, there were some stipulations about how you could use the model, particularly some of the guardrails where the model might [04:54] not answer in a super intelligent manner, or it might sort of like, [04:59] answer around particular types of questions.

And I think in the past, [05:04] it's been focused on things that are very reasonable, like safety concerns and things like that, things that kind of are consensus view. [05:13] you know, model helping with this kind of stuff is not great. But yeah, in this case, it was, you know, queries related to like AI model development. [05:22] which really sparked this question of like [05:26] hey, how much control do the frontier model providers have and how you use these models? And if they do end up sort of, [05:34] holding out capabilities for particular types of work, what does that mean for the downstream users of these models?

So the sort of natural conclusion and what the core thesis of our company is, is in a sort of [05:47] own or be owned. [05:49] having ownership over your own model and your own intelligence is actually really [05:55] existential for a lot of people. And if you don't start investing in owning your own models, you might [06:02] kind of be in a tough position when, you know, these things do change underneath you. It's kind of like a model-less company is sitting on shifting sand. Yeah. And, you know, I think the consensus things that people...

[06:15] understand, you know, that Anthropoc might not want to help you with is like biochemical weapons and such. But, you know, folks that are relying on these models to do more AI research, discovering that that is sort of maybe in some places subtly being degraded is [06:30] Totally. You know, difficult to, to... [06:33] to adapt to. Totally. And like, you know, there are stated reasons for why they're not doing this, particularly around like, [06:41] you know, safety, [06:42] in terms of model development where [06:45] It's unclear whether... [06:47] having everyone have access to these tools is beneficial for humanity but the the big thing is who's going to make that call yeah and uh [06:55] The people who make the models are going to make that call.

And then this not-so-charitable view of this, right, is Anthropic is doing very well and making some extremely powerful models. And this kind of prevents other people from being in that race as well if they want to use Anthropic models to build up their model development stack. So it's a little anti-competitive. Yeah, my view would be that I think they really believe it. I think they sincerely worry about this stuff. [07:25] into the world is fairly consistent around that. I don't know. Do you find it to be more [07:31] of an anti-competitive behavior.

No, I don't. I actually don't think it's anti-competitive. I think it does come from a really good place, which is like, you know, I have friends at Anthropic and I really do respect the leadership. They handled the whole Project Glasswing and deployment of Mythos really well. Like, [07:49] They do get some flack for it, but I think if it [07:53] If you do have a powerful new technology, it makes a lot of sense for you to take things slowly and roll it out to preferred partners first. Also, with this new Fable model, there's some stuff around data retention, which makes sense.

[08:07] You cannot cover the full surface area of what these models can do just within the walls of Anthropic. So you actually need to see how people use it. [08:15] I think the important point is, and these things can be both true at the same time. I think, you know, Anthropica is acting in really good faith, but also at the same time, it does raise the concerns of like, hey, if you don't have control of your model, if you [08:29] Anthropic ever does want to hold out a capability for competitive reasons.

You know, what happens to the users of those models? Because, you know, I think I think Anthropic is an amazing business. They're like, I mean, they're obviously doing incredibly well. And they're moving into the application layer and things like that. So there are. [08:49] products that rely on Anthropic models that are going to be competitive with whatever Anthropic puts out into the application layer. So I guess the question is what happens when you're relying on those and they want to push their products higher up the stack. Yeah, there was always the phrase, I think it was around Apple, when they would release something that sort of immediately threatened something within their developer ecosystem is Sherlocked, right?

I saw that people would say that. It's like Anthropic, every time one of these models comes out, it's like [09:19] one is depositioned potentially in these different ways. To come back to the idea of owning one's intelligence, I think [09:27] Maybe we should talk about it in the context of your company, Applied Compute. What does that mean? Why is that so important? And why is that the challenge that you're spending your time on? [09:37] Yeah, so, so, [09:39] What we do at Applied Compute, maybe I'll give a little bit of an overview, is we help companies own their intelligence by training custom models on their data, being able to serve these in production at scale, and then continuing to improve them the more they're used.

In practice, what that looks like is we benefit from the open source community basically taking open-weight models that are extremely powerful, actually, I think. [10:02] You know, if you sort of track the development of open source models compared to closed source models, you know, they've kept up. And a lot could argue that they've sort of been closing the gap from when the first open weight models started to come out. But, yeah, we basically are able to harness these open weight models, combine it with customer data to create task specific or specialized models.

[10:32] Right? There are places where you want to use the right model for the right task and you want to optimize for cost, latency, performance, modality, deployment, flexibility, that kind of stuff. [10:45] So our vision is that AI is actually going to be very decentralized in the sense that many companies will own their own models rather than there being a sort of single model that does everything. Yes. And so to put it just in probably a two-dumb analogy for folks, it would be sort of a Goldman Sachs of the world has all of this useful data.

I think you've even used maybe Goldman as an example before. That's top of mind. [11:15] And, you know, just unleashing a general model into that. [11:19] loses a lot of what, you know, the value of this place is. And you also don't want to necessarily give that data away. Totally. Yeah. I mean, the way these companies operate on the inside is like, [11:30] a lot of what makes them special and differentiated. So, you know, I had Brendan, a good friend of mine, and CEO of Mercore, he was the one talking about the Goldman stuff.

And, [11:41] He was over at our office last week kind of talking about what they're seeing in enterprises. And, you know, I think the sort of manifestation of what differentiates a company when it comes to AI is kind of coming up in this talk around evals. Right. So, you know, companies have their own definition of what good looks like, how they do certain things. [12:02] tasks and procedures inside of their company, and they're sort of codifying it within evals. And, you know, there's no way they would ever sort of give those evals away to [12:12] to the model providers because that's a lot of their secret sauce.

You know, there's some things that they might, you know, want to share to sort of, [12:22] improve the capabilities of the models, but anything that's sort of proprietary and unique to them, that's what they're going to want to hold close. [12:31] So many of these topics are, I feel like, so... [12:34] live right now like the question of the value of routing and cost and open apocalypse yes yeah um so i want to come back to it but maybe to [12:44] to sort of set the stage a little bit, I'd love to hear about your story a bit.

I was seeing that when you announced Applied Compute for the first time, there was a tweet from someone you went to middle school with, which was very sweet. And it described you in such an interesting way that it made me very curious. It describes you as, [13:03] The middle schooler hanging out with high schoolers. [13:05] And someone who had a very unusual sort of unconventional mind. Yeah. Yeah. What were you like as a kid? Were you just building things from the very beginning? Yeah. So that's actually really funny. I know exactly what you're talking about.

It's a friend of mine. Her name is Sarah. We went to high school together. But she was a few years above me. So on a little tangent, there is this club called Science Olympiad. And we had like there was like this middle school where a lot of people were. [13:32] went to high school like this specific high school afterwards she was there in high school i was there in middle school and those those science olympiad teams often like hung out together so that's how i got to know her before before i was there but [13:45] Yeah, in terms of childhood, like, [13:49] I can tell you a little about my family.

So, you know, small family of four. Mom is like a pediatrician, runs her own sort of medical practice. My dad's like an electrical engineer, works on chips and things like that. Very relevant now. [14:04] And I have an older brother who is in tech and is a software engineer, now works at a really amazing company called Chai Discovery. Plug for Chai. Yeah, I love it. Chai Discovery. [14:18] Growing up, yeah, like, [14:20] My brother was actually the person who got me into coding. So I feel like older brothers always like, [14:27] get way less credit than they deserve.

Like, they really paved the, or, you know, older siblings in general paved the way for the younger kids. And, yeah, he was the first to sort of start fiddling around with, like, [14:38] building apps and games and things like that. And one of the things he did sort of when he joined high school is he built this grade checking app. So the grade checking system in our school district was terrible. Everyone hated it. So he just like built this app and then got a bunch of his friends to use it.

Eventually the whole school is using it. Eventually a bunch of schools in the [15:04] And I just thought it was so cool that [15:05] He, as a single person, was able to build this cool thing. And now everybody I know is using it. So that really got me into this mode of like, oh, wow, I want to do that too. Basically, classic copy of the older sibling. So my dad actually encouraged me a lot to build these kind of little fun, unique apps or platforms and stuff. But he doesn't know how to code.

So he would give me the ideas and then he'd say, go build it. [15:35] and he'd be like, oh, you should do this or that or something. So, like, you know, for example, so my dad grew up in India, like a village, and it was very communal type living. And the way it would work is each family would, like, cook a bunch of food of, like, the thing that they were really good at cooking. Yes. And then they would, like, go and, like, sort of trade food with, like, other families and things like that.

Oh, sweet. Yeah. So he was like, oh, you should just, you know, your mom makes really amazing burritos. [16:05] here so he was like you should go and like build something that helps us trade food or we're not [16:10] Built it. It was really fun. No one ever used it. But it was a lot of fun to just kind of like iterate on this thing and see something come to life. [16:21] And that project actually got me my first internship. Oh, wow. Yeah, my neighbor, he's in tech, and he was working at this small fintech startup that was like four people.

And... [16:34] My dad was like in classic dad fashion was like in the yard talking to him about, oh, you know, Yasha's working on this thing, yada, yada, yada. And he was like, oh, do you want to do you want to come and work at our startup for the summer? And I was I'd never done anything like that. So I was like, oh, that's super cool. This was like kind of freshman year of high school. [16:53] So, um, [16:55] Yeah, he took me to their office, which was just an apartment.

[16:59] building, like one unit there. And I thought it was super cool to see like, [17:04] All of these just normal... [17:07] people like [17:09] working like is such a small group of people working on this [17:13] This. [17:14] sort of like hackathon project. Yes. You know, it was very different than when I visited my dad at work and he's in the cubicles. Yes. And like, you know, like, [17:22] - Serious. - And things like that. Yeah, so I was like, cool, this is like a bunch of people who are doing exactly what I wanted to do in my free time with my dad.

- Yes, as a job. - But doing it as a job. And so I was like, this is super cool. So I interned there and a couple other places. That's how I kind of got into startups and stuff. [17:40] Yeah, it was kind of cool. Amazing. And then you go to Stanford, and I think I've seen you say somewhere that, you know, you're a very good student in middle school and high school. And then when you came to college, it was sort of like, you know, I've arrived, and I'm going to basically watch the lectures and just build stuff.

No, no, exactly. So, I mean, in middle school and high school, grades were super important. I think it's like a common thing in a lot of immigrant families as well. But yeah, when I went to college, I was like, oh, yeah, I was like, oh, yeah, I was like, oh, yeah, [18:09] It was kind of like, give me an inch and I'll take it. So I was like, oh, like, you know, classes are online. That's cool. I'll just like watch lectures online. I spend most of my time just like hacking on like basically like, [18:20] uh manifestations of what i was working on like [18:23] childhood like middle middle school and high school so but for like the university setting so nothing i want to be very clear none of this was impressive or like interesting or anything like that i was like working on like [18:34] every single generic college campus coding project you can think of like social calendar for like me and all my friends you have to do these things or like you get these reps exactly or like a dining hall voting app or something like that because you know some food is better than others and things like that but yeah just just through that like found other people who were kind of doing the same thing and i think stanford's a [18:57] really cool place because [18:59] there's these sort of natural communities that form and, um, you know, they're like, [19:04] little informal clubs of like, oh, let's get together and hack on a project or something like that.

And then, [19:10] sort of the main organization I was affiliated with at Stanford was Treehacks, which was like, [19:15] the school's hackathon, um, that we throw for, for like, [19:19] everyone across the nation and even some folks internationally so yeah spend most of my time just like kind of [19:25] working on projects and stuff rather than attending class. You'll have to correct me if I'm wrong, but you're a Teal fellow, but you also seem to have gotten a degree. How can this happen? So I did not get a degree, actually.

Yeah, yeah. So I dropped out of [19:41] Uh, [19:42] - I was at Stanford during my sophomore year. About halfway through my sophomore year. So yeah, was basically there. [19:49] for a year and a half, dropped out to join OpenAI, though, and then after OpenAI started, started to apply. And OpenAI, I think, you know, the way you've talked about it before is, like, you were just so mesmerized by this technology. Like, can you, yeah, tell me about that. I mean, I look super fondly upon OpenAI because... [20:10] Obviously the technology was amazing, but the people were like...

[20:14] even more amazing. So I actually didn't really take any AI classes when I was at [20:20] Stanford. [20:22] So joining OpenAI is how I learned a lot of this stuff. And there's [20:26] that were incredible people on the post-training side that kind of took them under their wing and taught me a lot of stuff. [20:33] you know, Luke Metz, Barrett Zoff, a bunch of folks on the post-training side were like, they were my managers. And I was just like, kind of looked up to them. They're legends in like the language modeling space.

They like made kind of made the first versions of ChatGPT with John Shulman. It was actually really funny. So on the sort of, [20:54] So Barrett, who used to be my skip, told me on his last day when he was leaving OpenAI, he was like, Yash, by the way, I don't know if you know. [21:03] But, um, [21:04] when you're joining, you were originally supposed to be on like chat GPT engineering team. And we just [21:10] We just really needed more people on post-training infrastructure, which is on the research side.

[21:15] So it was like last minute we pulled you in. And I didn't know because you don't know your team before you join. And it kind of, you know, that's... [21:21] basically the reason why I got into all this AI and research stuff is completely by luck. Yeah, it set a few things in motion. Yeah, it set a ton of things in motion. I feel super grateful for the people that were there and then the fact that like, [21:35] they were willing to teach me a bunch of stuff and take me under their wing.

And yeah, I mean, like this guy, Ian Osmond, who's now at DeepMind through my 21st birthday party. It was a lot of fun. [21:47] I was lucky to be surrounded by some really, really awesome people. Am I right in recalling that you got the OpenAI job by cold emailing Sam? Yeah. So the backstory there is rewind to freshman year. [22:02] We had been hacking on some projects and stuff, and my friend and I both had, like, internships lined up for the summer. But we were like, oh, let's just, like, keep working on cool things and, like, [22:12] you know, keep keep hacking on stuff.

So we, [22:17] Stanford is a very unique place because I think there are a lot of people who encourage folks to do this, like to go and just build stuff. [22:25] So we were, you know, there were a few venture firms that were like, hey, like here, take a angel check or something like that and go and spend the summer building something. And then if you know, if you if you end up not wanting to continue to find whatever. But growing up, like I knew that nothing comes comes free. So we prior to this.

[22:45] I had been invited to Sam's place for a dinner. He puts on some dinner sometimes for people who are building stuff on campus, whatnot. There were, I think, a couple dozen folks who went. And Sam is like, [23:00] someone I deeply, deeply admire. I think he's amazing. And so I had watched all his YC videos and read all his blogs and stuff like that. [23:07] Couldn't work up courage to actually talk to him at that dinner. But a couple weeks later, when we were like, oh, should we do our internships or not?

[23:16] shot him an email and was like, hey, like there's some people who are offering to sponsor us for the summer. We don't really know who they are. We're not even sure if we feel comfortable. Like, what do you think we should do? Do you know these people? And he was like, you should totally do it. If you're [23:31] This is back when Sam was kind of talking about like, oh, people should spend more time building stuff and take time off of school to like try and explore things. He was like, if you want, I can basically give you a small stipend to just like pay for rent and food to go and work on it.

And I'll sponsor you for the summer, which is like deeply generous of him. And we were like, wow, this is cool. [24:01] sort of his family office at the time who became a friend and then ended up [24:07] shutting that down though it was fun it was like an e-commerce company uh but ended up shutting it down and coming back to school and like returning the the money and things like that but it's [24:17] Yeah, that's how I got to know him. And then when ChatGPT came out, I was like, [24:22] wow, this is the coolest thing I've ever seen.

I have to work on it. So then I was like, oh, hey, Sam, I don't know if you remember me, but like, [24:31] You know, I think Chachubisi is super cool. I'd love to work on it. Like, are there any ways I can apply for a job? [24:36] And then he was like, [24:38] "Oh yeah, we have this residency program. You should apply for it." So he introduced me to the residency folks. [24:44] Talk to them. They were like, oh, you have to drop out. And I was like, [24:47] I'm fine with that.

I can take time off. And they were like, no, you really have to drop out. You have to commit to never coming back. Burn the boats. Yeah. And I was like, no. [24:57] I don't know if I could... [24:58] fully convinced my parents of that. And then so I was like, Sam, like, you know, should I should I do? And he's like, [25:04] Let me chat with them. The residency is like a six month thing. So you'll have the option if you want. Then I interviewed, was very lucky to get the residency position and then joined like basically like a week after.

They needed you to drop out like almost as a proof of conviction somehow. I think at the time it was just it was actually just much smaller company. Right. So like I think they're there. It's not the. [25:28] you know, massive company that it is today. This was actually something that I don't think they had done before. Like they hadn't had people do the residency that had like dropped out of college just yet. But after after I did, I think there were a couple other folks who became more mainstream. [25:43] But yeah, I guess I think they just wanted like, [25:46] commitment and stuff.

But I already kind of knew at the time I was like, I would be so lucky to be able to work here. You know, I totally would. If it was up to me, it was not up to me. My parents had to have to have a say, right? So clearly the right choice. Yeah. [25:59] you ended up living through like [26:02] one of the most interesting periods of technological history ever, and also one of the most dramatic, like, weeks of technological history ever. Yeah. I'm sure there's lots of that you don't want to talk about.

Yeah. But you were there when Sam... [26:15] It was ousted, came back, [26:17] What was that experience like for you as someone having what was, I think, their first proper job? Like, that's a very dramatic thing to go through. [26:25] It really sucked. So I remember that week pretty well. And I don't want to be overdramatic or anything. Like, you know, things... [26:34] things happen and stuff. But like it was, it was kind of this feeling of like, [26:38] Oh my gosh, things are going so well. We're making so much progress.

Everything is [26:43] You know, everything's happening. You know, why are we... [26:46] halting the the progress like let's just go like things are going so well and so i just remember there were like a bunch of basically that whole week was actually like you know we couldn't go into the office all our laptops were shut down for security reasons and whatnot so it was like every day there'd be someone who'd be who's like oh i'm like hosting at my place just come hang out um because we know nothing so um so yeah i just remember like everyone being [27:12] down on the fact that it wasn't even about like the situation.

[27:16] you know, everyone wanted Sam to come back and not no one more than me, because I think he's really awesome. But everyone is kind of sad about, [27:24] the mission just being on pause. They were like, this is... [27:27] this is weird, like why we were doing so well. So, and then it was a rollercoaster too, because, you know, Sam was, [27:34] I. [27:35] talking with the board and things like that and [27:38] you know, we all thought he was coming back and stuff, and then [27:41] there is an announcement that, you know, he wasn't and there's someone else stepping in.

But eventually, right, I think we were super excited that [27:50] Sam get back at CEO and is actually funny. I remember. [27:53] So after that was announced, [27:56] everyone was going to meet back up at the office. And I lived, so I lived across the street from the office in this, like the building, the Madeline. And so I like came her to the office and Sam was actually coming down the stairs. So, um, [28:08] He gave me a big hug, and I'll definitely remember that for a while. That was probably one of the coolest things.

[28:14] close days was just like, [28:16] It's back on track. Yeah. Let's go. Right. So it sounds like, you know, uh, there's obviously been, I'm thinking of all the stories that have been out over this year at the books. Sounds like you still are basically, yeah, big supporter of Sam and think he's a good executive in that respect. Oh yeah. I think, I think, [28:33] Yeah, a little side tangent on Sam. I think he... [28:36] He's an amazing CEO, and his superpower is he's able to make [28:41] impossible things. [28:43] sound extremely possible.

[28:45] If I were to tell you, hey, we're going to spend hundreds of billions of dollars to build the biggest supercomputer in the world and train a model that can do like, [28:57] 80% of all knowledge work [29:00] You'd think I was crazy. But somehow he's able to go up in front of [29:04] hundreds of the smartest people in the field and be like, guys, this is possible. We can do it. And I think that extends to the whole leadership team as well. Like, Elio is really, really good at this too.

He had his, you know, the feel the AGI stuff. Like, yeah, I really admire Sam's ability to make him [29:23] impossible sounding things sound possible. [29:26] And that's really how you get the best people is you work on really hard problems because all the best people want to work on hard problems. No one wants to know one who's really, really smart. [29:35] wants to work on things that are easy. I think that's a, yeah, I mean, that is a skill. I won't make you answer for Sam's journey. In terms of your time at Opening Eye, you mentioned that you ended up...

[29:48] spending it on post-training and this sort of had you know such an impact on on what you're building today uh for folks that i'm sure most of the audience or a lot of the audience will [29:58] will be really intimately acquainted with this, but for folks that maybe aren't, can you explain a little bit about why post-training matters and how it led to this work you're doing with Applied at the moment? Totally. Yeah. So kind of a high-level overview of training models in general is there's a couple of phases of training.

A lot of people talk about the big, expensive, high-capex phase, which is pre-training, which is [30:24] Let's go and take... [30:25] internet scale data, use this architecture called the transformer to sort of learn language patterns. And out of that falls intelligence, right? So you get this model that can do next token prediction. And that's kind of like how we got the first sort of GPT models where they just got really good at predicting the next token. And it felt like these models could actually reason and answer questions and things like that.

[30:50] After you do pre-training, you basically have a unaligned model that will just predict the next token until you tell it to stop. [30:59] To make it useful, that's when you move to things in the post-training realm. So there's a couple of different types of post-training. When we were first training these chat models, where it's a user sends a message and an assistant sends a message, there's a variety of techniques of supervised fine-tuning as well as reinforcement learning with human feedback [31:29] that a human would like and whatnot. But you're sort of just basically doing behavior cloning on the supervised fine-tuning side.

So you're saying, here's a bunch of examples for good answer. If someone inputs this prompt, [31:42] These are examples of what good completions look like. And then you just do, you know, [31:46] You just train the model to reduce the loss on those target tokens. Now, what people have been talking about, [31:53] when they say post-training today is something quite different than supervised fine-tuning or reinforcement learning with human feedback. [32:01] It's this... [32:02] this idea of RLVR, so reinforcement learning with verifiable rewards. It turns out if you take a really small, like a relatively small amount of high quality data as compared to like when you were doing pre-training or SFT or something like that, [32:17] And you actually...

[32:18] have a model try to attempt the same problem a bunch of different times and you have a verifiable way of checking if the model gets the answer right [32:27] You can use this new... [32:30] reinforcement learning algorithm to sort of show the model, hey, these are examples of what good looks like. These are examples of what bad looks like. And sort of nudge the weights to do more of the good reasoning versus bad reasoning. This thing that people talk about when they talk about chain of thought, so all these tokens before the model outputs an answer, is actually entirely...

[32:51] an emergent property. [32:52] Turns out, like, the models... [32:55] learn to steer themselves, learn to think, and that's how they get more accurate. And [33:02] get, you know, basically like, [33:03] hill climb any sort of task that you give it. So what the first domain that this started with was math. [33:10] Part of it is because, first of all, it's got a lot of really good properties, which is easily verifiable. You don't need that many complex tools or things like that to be able to do math. [33:22] And the second thing is just like, [33:24] everyone on the research team at OpenAI was like competitive math people or things like that.

So it was like a domain that they deeply understood and also were very excited about. And there's also a lot of other philosophical reasons around like why math is like kind of the root of like a lot of other things. So basically the first experiments were like, [33:44] actually hill climbing on a variety of different math problems ranging from like basic arithmetic to sort of like harder, you know, proofs like, you know, algebra, calculus proofs, whatnot. And turns out that the model is able to learn on sort of easier problems and sort of internalize that and be able to unlock harder and harder problems.

So that's this like hill climbing that you've seen. [34:08] And there was kind of like a couple of scaling laws that came out. So obviously you have the pre-training scaling laws. Now you have this like post-training scaling law. And now you have this test time compute scaling law, which is like, if you actually allow the models to reason for longer when you're using them in production, they give you better answers. So that's this like, [34:27] When you pick a reasoning level in Chachipati or something like that, that's just reasoning for longer, expending more test time computed before.

[34:35] Give me a final answer. [34:36] This episode is brought to you by Persona, the B2B identity platform helping businesses verify users, fight fraud, and build trust. Fraudsters are already using AI to spoof faces, voices, and documents, so your defenses need to adapt just as fast. [34:51] Persona helps secure some of the Internet's largest and most trusted platforms with identity verification. If you're building a product where trust matters, identity should be a priority. You've probably already experienced Persona without realizing it. Verifying your LinkedIn profile, signing up for Etsy, [35:08] or renting a scooter with Lime.

Trusted by leading companies like Square, Brex, and Twilio, [35:14] Persona gives you the building blocks to create identity flows that adapt to your customers, risk tolerance, and locales you operate in, whether you're verifying age, onboarding businesses, or automating KYC. It's fully configurable, so you can launch in days, not quarters. [35:30] Want to see for yourself? Generalist listeners get a free year of the starter plan. Head to com slash generalist and check it out. [35:39] You're doing this work at OpenAI. What was the moment that you realized there's a need for the sort of service that applied compute offers where you are going into these companies and doing this more hands on work?

So I think Microsoft AI, when they put out their their thinking models recently, you know, they the title of their paper is like the hill climbing machine or something like that. I think the way of thinking about RL is it really is that is like if you define the hill to climb well, the algorithm works. [36:09] Thank you. [36:09] So the way we were training these models was actually just like on the post-training side was, hey, let's go and make a bunch of high-quality products. [36:18] data sets that target the capabilities that we want the models to get really good at.

Like, basically, we'd start with an eval and be like, OK, if this is what the eval looks like, let's just like go make training data that looks like the eval. Yes. And... [36:30] That was the way that we were proving these models. So the natural conclusion was kind of like, hey, [36:35] you know, everyone is going to have [36:38] the their own evals and the things that they want these models to be really good at and evals extend beyond just uh you know the intelligence of the model it's like oh it needs to [36:48] you know, be this intelligent in this cost and latency regime or this modality or something like that.

So [36:54] If everyone's going to have their own evals, [36:57] everyone should also be able to train their own models that do well on those evals, right? So our sort of natural conclusion was like, hey, maybe since we have this hill climbing machine, the best way to sort of manifest it in the world, obviously the frontier models are amazing. These will be like workhorse models for a lot of things. But if people want to be very specific about the task that they're trying to optimize for, [37:22] they should have the ability to go and push drain their own models on their own data.

Yeah. Evals are, you know, the scorecard for all of these models. And that's so general. It's the new PRD. Yeah. Yeah. And the reality is that every organization has a different scorecard for what they consider valuable. And that's, you know, sort of what you're saying is like, you know, if you have a different set of things that you're optimizing for, it starts to make sense that you have a machine that is made to [37:48] hit those optimizations. Exactly. And if we go back to the Goldman example, right? Like, if a lot of those evals are going to say private, people are going to want to [37:56] own their intelligence, own the post-training capability to go and create those models that are good at those evals, right?

Yeah. So I think that was the primary motivator. And then I think the main sort of driver for why people are thinking about this right now is cost. The token apocalypse has really shown, hey, you need to use the right model for the right task. And [38:16] even beyond just like optimizing for evals, right? [38:20] Yeah, that's I'm curious, like how you think about it today. Clearly, that's top of mind. I think in the past you've talked about how general models sort of set the floor and, you know, specialized models can sort of raise the ceiling in some sense.

Yeah. [38:34] In some ways, it feels like, you know, the distance between those two might be getting smaller. But the cost at which you are accessing these different offerings is like, has it become more of a cost story in your head? Cost has become like a. [38:49] the main driver i think right now but i think in principle [38:53] the argument still stands, which like take Kirkland and Alice, right? They caused waves in the Twitter community. I'm sure broadly as well about how they're going to spend like half a billion dollars to, I forgot what the exact quote was, but essentially the idea was to create AI tools that [39:11] their competitors can't also use.

Right. So I think the the idea is and this extends beyond just the model, to be clear. Right. The model, the model is. [39:20] part of the moat. [39:22] but also training that model inside of your harness with your context [39:26] that's actually how you're going to get differentiation. So I think a lot of people are thinking about, hey, if I'm using the same stuff as my competitors, how am I different? And that's also another driver for why they should be. [39:39] train their own models on their own data. [39:41] And I think it's been just almost a little over a year.

Yeah. Yeah. And you've gone from starting the company to, you know, a last valuation, 1.3 billion. Yeah. Been a good year. Been a good year. Yeah. Can you tell me a little bit of how you started to work with folks and what that process looks like with these companies? Totally. Yeah. So we work with like a variety of companies across AI natives, digital natives and large enterprises and hyperscalers. [40:11] right there's [40:12] some amount spent on training. And it's like a non-trivial amount, but most of the money in [40:18] most of the usage is on inference, right?

So when we were thinking about what the future of open models looks like or models that people are going to inference a lot for different things, [40:30] Our conclusion was kind of [40:32] if you are going to train or if you are going to use open weight models and specialize that model, you're going to do some sort of training on top of it. So the actual wedge into [40:44] Inference is. [40:46] getting really good at training. So what we do with our customers is we [40:50] sort of we have like a post-training platform where they can integrate all of their data.

And we have a team of applied researchers that actually deeply embed with the customer, work with their ML and research teams to train these custom models. And we really want to make sure that the economics are in their favor. So we try to have most of the stuff that they pay for come on the inference side. So it's like once we have a really good model and you guys want to use [41:20] Yes. The sort of [41:22] Forward deployed engineer model is something that I think people obviously associate with Palantir as a successful example of it.

With Palantir, I think the way they've made it work is that clearly they found a way to get good leverage on it, such that the 50th version of it, of them taking on a project, is very different than the 5th. What's the version of that for you that stops it from... [41:46] being sort of just pure consulting leverage. There is always some component of this that is going to be unique per company. That's kind of the whole thesis, right? It's like each company is going to have different data and want a different model, but really it's the infrastructure that is repeatable across all of these companies, right?

It's like, [42:06] the scaled infrastructure for training these models, the scaled infrastructure for serving them, and then the sort of techniques for capturing production usage and turning that into more data that we can go and continually train. So our equivalent of Foundry, which is Palantir's platform that they use to give to all their FDs to go and move faster, we have our own post-training platform where all of our applied researchers and researchers from the companies we work with [42:34] can both collaborate to train different models. And really, I think the way of thinking about it is, it's the infrastructure to make training, [42:44] performant and easy.

Mm hmm. Yeah. Does it end up becoming more of an ongoing engagement than, you know, people might realize from the outside? I could imagine that, you know. [42:54] one version of it would people would think oh you go in you're sort of doing an ai transformation and then you know you're out the door but i would [43:01] Imagine like the demand for, you know, [43:04] Improving this, deepening it, leads to quite long engagements. Totally, yeah. I think there's always a desire to continually improve these things, which is why when we deploy a model in production, a lot of our product is built around, hey, how do we take the usage of this model and turn it into more data that we can continue to train on or build context around the model so that it's learning.

[43:34] use it. So yeah, I think the idea is that the engagements don't end. Obviously, the way we generate revenue is by serving the models, and we need to make sure that those models are [43:45] always performant. And maybe even to get even more sort of [43:50] tangible, like, can you, um, [43:52] Maybe talk us through an example or two of what taking this more specialized contextual approach unlocks for a customer, for instance. Yeah. So I think one of the case studies we have on our website is this engagement we did with DoorDash, which is, you know, they have more than 100,000 merchants that come to their platform every year and they want to create a new DoorDash storefront.

[44:22] who they serve, things like that. And one of the things is a picture of their menu. So it turns out this was actually an example of where we were able to exceed the frontier in terms of quality, 'cause we trained a small specialized model [44:39] that was able to take pictures of menus and turn them into high-fidelity DoorDash storefronts. Basically, the process was create an eval for what good looks like, create the training data for us to go and, like, [44:53] do RL training on top of, train the model, and then deploy that and host that.

The trend feels like [44:59] You know, if applied compute continues to grow and this is part of sort of a broader movement. [45:05] is like sort of trending towards more of these small models where you do sort of say here's a set of [45:11] tasks that like are maybe a bit more [45:14] narrow, a bit more deterministic that like we can actually very inexpensively do this very effectively. Is that roughly how you see things going? Yeah. And so I think the motto is better, cheaper, faster. What people are starting to realize is that these open weight models.

[45:33] provide a ton of flexibility around ownership and they're actually good. Like you can actually go and use them to, to perform optimally on, on your task. Um, so I think you're going to see just a lot of people for, you know, reasons related to cost. I think cost is the big driver right now, but there's just so many other downstream benefits you get by, um, [45:54] owning your own model. When you sort of move towards the world of, you know, [45:58] A company having multiple of its own models for different tasks.

Does that create like some sort of maintenance debt for them that they have to sort of stay on top of? And yeah, I mean, there's probably still cases where that's very much worth it for them. But is that something you've noticed come up? [46:16] - Yeah, I mean, so this is part of the continuous improvement part is there's obviously like, [46:21] model drift and you know the distribution of tasks might be changing [46:26] slightly, but over a long period of time that really, really matters. Our view is just like, [46:31] Training and inference loops.

[46:33] They're like this right now. [46:34] but they're starting to move closer and closer together. [46:39] Right now, the form is people are just getting a lot faster at making new data and training the model again. But eventually, these things are going to be quite intertwined. I think there are some cool examples of this that are maybe the first beginnings of it. If you saw Cursors, Composers, real-time RL work, that was them doing online training by taking a bunch of user trajectories. [47:06] you know, getting extracting the implicit reward, taking a training step and then deploying a new new model in production.

I think like. [47:14] that is just kind of an indication of where things are going to go is like [47:18] you're actually going to be doing [47:20] more and more continuous training because as your model is being used it's kind of like an exploration technique right yes so and and and so you know what you're describing there is almost like a continuous learning model where it's like every time i happen to do this task it's actually training this model to be better the next time around totally and and the name of the game is learning from sparse rewards right so [47:43] If you think about the different phases of training, pre-training is like super data inefficient, right?

You need to like learn the whole... [47:50] where you need to train on the whole internet in order to learn the language. Then we had this SFT stuff, which was kind of behavior cloning. So you say, hey, here's- I'm not sure what you're talking about. Yeah, yeah. So SFT is described as behavior cloning because you're giving it examples- I see. Of what the correct output should look like. And so you give it a bunch of examples. And then for a new task or a new prompt, the model is sort of interpolating, like, oh, I should say these things.

Okay. [48:20] a lot more data efficient, which is I'm going to take one data point, one math problem, [48:24] generate a thousand trajectories. [48:28] all with different thinking tokens or whatnot, all with different rewards. And then I'm going to do backprop on those. And there the model will kind of learn what good reasoning looks like and bad reasoning looks like. [48:38] the ultimate [48:40] thing, right, is being able to do super, super sparse learning, which is like you go and roll out a trajectory. You have some indication of whether it was good or bad or something, and you're able to directly learn from that.

That is difficult, and that is the problem to solve in continual learning. On the open weights models, you were mentioning earlier that you think like maybe... [49:02] The gap between open weight and closed models is has sort of narrowed over the past year. In some ways, if I look at like the hardest benchmarks, I almost feel like it's widened. But actually, what we're learning is that it does just beyond a certain threshold. It almost doesn't matter. Yeah. Like, I don't know. Do you think that's wrong? Yeah. Yes. So I think what we were talking about before is.

[49:24] Frontier models are amazing and you should still use them for your hardest tasks. But if you have like $100 of spend, maybe 20 of those dollars go to the Frontier models and the $80 can be... [49:36] actually $20 that go to cheaper, better, cheaper, faster models that you train on this, this task. Right. So I think like, like the frontier models are obviously very, very, very good. But, you know, you don't need to use them for everything. There's all these memes on, on Twitter, right? Like, where like people are using some big sword to cut like a blowtorch.

Exactly. So, so I think like, you know, there, there's a lot of truth to that. And that's what, you know, you're talking about [50:06] innovations there. We're working on some of that as well. The other thing I think about with open wage models is like China has really dominated that so far. Yeah. You know, it feels like America is clearly leading with some of the frontier models. But yeah, it's like how do you think about the fact that an implication of this could be that a lot of American companies become like actually quite dependent on.

[50:31] Chinese models that the rules could get changed up on them. Those could go closed at some point. Is there a robust enough open weight system right now? So I think that... [50:42] you know you're right it it all boils down to incentives so i'm extremely excited that [50:49] companies like NVIDIA are investing a ton in open source models. And NVIDIA has a real reason to do it, right? Like the dream for NVIDIA is that every... [51:00] company is [51:02] has their own model and is running it on NVIDIA chips.

So I think that there are real, there are companies like NVIDIA, NVIDIA being the big one, that have true incentives to keep models open. And I think there is enough of, [51:18] investment and a lot of smart people working on open models that i think it's going to be a pretty [51:23] pretty safe and big bet over [51:25] a period of time. One of the things that, uh, [51:28] I think is really interesting about applied compute sort of [51:32] outside of the technology piece, [51:34] is that you seem to have managed to attract a huge amount of ex-founders, which is always a very bullish sign, I think, for an investor to see that happen.

How have you managed to do that? It's something like two-thirds, right? Yeah, so it's almost...