Building the most AI-pilled engineering team in the world | Fiona Fung (Anthropic)

[00:00] Anthropic engineers on average ship eight times as much code per quarter as they did compared to 2025. Coding is no longer the bottleneck. It's left at the ceiling of what anyone is able to do. Everything is now possible in theory. Now it's about how ambitious can you be? It's always something we ask ourselves, what's better than me doing it? I haven't thought to. The people that seem to be doing best are taking the most initiative, getting the most proactive, have the most agency. We say with high agency, it's also high accountability.

So it's all about making sure folks have that freedom to cook. [00:30] for it? What's a hypothesis of what you're trying to solve? I'm curious what is lost in this new world of software engineering. I could start being a lonely experience because we all started just working with our agents so much on the ClockCode team. Recently, we started a pairwise programming lunch. Something you think about is this gap forming between people that are leaning into AI, killing it, and then people that are not super frustrated, fighting, resisting. In terms of frustration, I think sometimes I also see a little bit of fear.

For anything that there's a fear, [01:00] What can I do about it? What is within my control? [01:04] Today my guest is Fiona Fung. Fiona leads the teams behind Claude Code and Cowork at Anthropic. She oversees both Boris Cherney and Kat Wu, both of whom who have been on the podcast and whose episodes are in the top 10 most listened to episodes of all time. [01:20] Before Anthropic, at Microsoft, Fiona ran the teams that built TypeScript and Visual Studio. After that, she went to Facebook, where she started the Facebook Marketplace team, which she took from idea to launch.

Today, Facebook Marketplace generates over $100 billion in GMV every year, [01:36] Also while at Meta, she oversaw work on Meta's first smart glasses product, [01:40] And then she helped build Orion, their first AR glasses product. [01:44] Then she went to Instagram, where she led infrastructure, growth, integrity, and safety teams. While at Instagram and at Meta, she oversaw an org of over 500 people. [01:53] Fiona has been an engineer for over 25 years and as a longtime engineering leader, especially now at Anthropic. [02:01] She has such a unique lens into where things are heading, what's worth paying attention to, and what teams should be thinking about right now as AI transforms the world of building.

A huge thank you to Kat Wu, Boris Cherny, and Mohamed Hegazi for suggesting topics and questions for this conversation. Before we get into it, don't forget to check out Lenny's Product Pass dot com for a free year of the hottest and most well-crafted AI products in the world, available exclusively to Lenny's newsletter subscribers. [02:29] With that, I bring you Fiona Fung. [02:35] Fiona, thank you so much for being here. Welcome to the podcast. [02:39] Thanks so much for having me, Lenny. [02:41] So I was at the Code with Clot event, I don't know, a month ago at this point, and I went to your talk.

[02:47] And I was just like, holy shit, I got to get Fiona on this podcast. She's thinking [02:51] so far ahead of where everybody else is going and where people are at with AI. So you've been an engineer for 25 years. I was browsing your LinkedIn. You started at IBM, of all places. Such a different place to be these days. [03:06] And it's just insane how much the job of an engineer has changed over the past just like two years. It's like a completely different job. [03:14] Like people may forget 100% of code was written by humans not long ago.

[03:19] and now it's getting to 100% of code written by AI. [03:23] As Boris famously said, coding is solved. [03:27] Along these lines, there's just this tweet that you guys put out yesterday where you showed [03:32] Here's the tweet. Anthropic engineers on average ship eight times as much code per quarter as they did compared to 21 to 2025. We'll show this chart on the screen. It's just like. [03:41] stable, stable, stable, stable, shooting off into the moon. So it's just insane how much this role has changed. [03:47] I'm curious. [03:48] about your [03:49] kind of path as an engineer going living through this having been an engineer for a long time [03:54] What have been kind of like the big moments along the way where it's shifted your [03:58] way of thinking and operating that have led you to what you do now and how you operate now.

[04:03] Oh, I love this kind of look back in time. Yeah, like IBM working on DB2, the operating system services team. Like back then I was thinking, oh, how can I how can I be like the like what's a hard area of the stack? And I really thought the lower level you go closer to the OS, then it's more hardcore and you learn more. So that was I was really fortunate to gotten into the IBM internship. [04:26] But the funny thing was, I think there's even a big shift from IBM to Microsoft.

So at IBM, it was, I think VIM. I didn't have an IDE that we used. I think there might have been [04:37] An Eclipse license, but for some reasons, most of us didn't use it. I remember it was mainly Vim and terminal debugging. [04:46] And then when I joined Microsoft, I mean, this is how naive I was. I didn't even know about IDEs and such. And back then, you didn't really get to pick teams. Like, this was early 2000s. [04:56] Actually, first off, I should say I was so grateful that I landed the internship and the role because to take us all the way back in time, the dot com bubble burst in 2000.

[05:05] And so for my graduating class, like a lot of the companies [05:08] weren't hiring or were having freezes. I was so fortunate when Microsoft sent me an offer, [05:13] They're like, "You're going to work on Visual Studio." I did not even know what Visual Studio was because I came from a Unix school. I remember asking, because I was thinking about the name Visual Studio. I'm like, "Oh, is this a better paint program?" I could tell the look of my manager's face like, "What is going on?" Then it ended up becoming the love of my life for the first 11 years of my career.

That was the first time I used an IDE, so joining the Visual Studio team. [05:40] seeing, "Oh, wow, here's an IDE with debuggers, and you can set breakpoints and do multi-threaded debugging." That was also always mind-blowing for me to think about the stepwise change. So yeah, that was the story going from Ibrahim to Visual Studio. Then what I really loved actually about Visual Studio is I was on the Visual Studio editor team. [06:01] So I use a VS editor to build the VS editor. And that's where my whole love of dog feeding comes from.

Like I remembered I wanted to first and foremost create a delightful experience, not only for myself, but for my teammates. [06:14] Because also, if we go back in that time, if you remember, [06:17] I mean, when did Twitter come out? [06:20] 2000 and [06:21] - Six or something. - Feels like it's been around my whole life. - But back then, like before social media, it was also harder for most engineers to hear fast customer feedback. Like for sure there would be user research sessions or we would have customers visit us, but you didn't get the rapid feedback that you do nowadays.

But back then I was so lucky because I was on VS, we ourselves gave each other so much rapid feedback because we were all heavy VS users on the team. [06:49] This episode is brought to you by our season's presenting sponsor, WorkOS. What do OpenAI, Anthropic, Cursor, Vercel, Replit, Sierra, Clay, and hundreds of other winning companies all have in common? They are all powered by WorkOS. If you're building a product for the enterprise, you've felt the pain of integrating single sign-on, SCIM, RBAC, audit logs, and other features required by large companies.

WorkOS turns those deal blockers into drop-in APIs with a modern developer platform built [07:19] specifically for B2B SaaS. Literally every startup that I'm an investor in that starts to expand upmarket ends up working with WorkOS. And that's because they are the best. Whether you are a seed stage startup trying to land your first enterprise customer or a unicorn expanding globally, WorkOS is the fastest path to becoming enterprise ready and unblocking growth. It's essentially Stripe for enterprise features. Visit com to get started or just hit up their [07:49] WorkOS allows you to build faster with delightful APIs, comprehensive docs, and a smooth developer experience.

Go to com to make your app enterprise-ready today. [07:59] People think about these milestones along the way of an engineer's journey and we forget there's also there's been like a lot of transformation over the years. Not quite what we're living through, but just like IDEs, Visual Studio. [08:09] So I love that you're reminding us of these moments in the history of software engineering that have changed the work in a big way. Yeah, when I worked on Visual Steel back then, we also shipped software on CDs. [08:20] I mean, and that's why there were really hard deadlines, because you had to make sure the software was ready for us to give to manufacturing to then, you know, put on the CDs for us to then put on the shelves.

And then so once that was another shift when we actually started to be able to, you know, ship software online. And I think that's the interesting thing. And I kind of mentioned this in my talk. It's before when you like. [08:42] Engineering time was like really precious resource. But you also have these really hard deadlines, for example, like printing software CDs. And so back then you would do a lot more planning because [08:53] you just wanted to make sure given the time you have you you make the best of it and that's the shift that you know like we're seeing with with uh clock code and co-work is [09:02] Coding is no longer the bottleneck.

[09:04] And so now like, you know, you showed up, you showed the tweet and that graphic. So, but now it's all about like, where has that shift happened? Like now, not only engineers, we also have designers, PMs, everybody on the Clark code team checks and code. So like when, [09:18] not only more people checking in code but like kind of different disciplines but also the throughput is so high how do we think about verification like that's kind of this [09:26] other shift that I'm seeing. Maybe just kind of set this theme that I want to have for this conversation.

[09:31] A lot of people are just wondering, what is what is software engineering, managing software engineers, managing software and product teams look like? [09:38] in the future. And you are living through that right now. [09:41] So you mentioned this point about a more focus on verification, making sure the quality of the code being... [09:49] 8x is actually high and something that will work. [09:54] So just like let me ask this broad question. Let's kind of see where this conversation goes. What is a what is an AI pilled software team look like in 2026?

[10:03] Because the roads are blowing, it's shifting more to this builder [10:06] Everybody starts being a builder, I would say. The other shift that I've recently done is I actually have a Cloud Code remote session that I enlist in all of our repos. [10:18] And so this way I have full visibility into the work that everybody's doing. [10:22] And this instance, it also has access to all our Slack channels and I'll have access to like how are the metrics of everything we track. And so every month, like when like I have been like, hey, you know, it's fun.

Like, let's let's take a look back. Like and so we'll actually do it together. I'll share my, you know, like code code, you know, I share my screen and we do it by code. [10:42] session. And it's just about, hey, what were the focus areas? Like, what were some of the products that got shipped? How did it do? Oh, what were the feedback channels? And so I, even though like before I would have just used these sessions to like generate PRs and bug fixes, I actually have these sessions to enable me to have conversations with folks that I support.

[11:00] Say more about that. So this is this is like a management technique, let's say, to help people [11:07] Not just ship, but actually ship. [11:10] better understand the [11:11] if they're shipping things that have impact. Is that kind of the idea here? Just like use Claw to keep on top of all the things people are shipping and then make that a conversation with them. [11:20] Exactly. So, yeah, outside of just the action of shipping is how did it do in market or, hey, did we have, you know, like, did we cause some bugs?

And it's okay to, like, I have the saying, make new mistakes. Like, it's okay to make mistakes, just make new ones so that we're always learning because if you aim to make zero mistakes, like that probably means you're not, you know, [11:38] moving fast enough or being a little bit too cautious. So yeah, by having Claude, so then they can also look at, for example, actually, yeah, I was just looking at, oh, given some of the incidents that we have, let's look across all of this. Can we generate a theme?

What's a good area investment for us next, especially we think about quality. Are we seeing any hotspots of where there could be a gap? I think that used to be just a much more manual, [12:04] process, I would say. If I look back a year ago, I don't think I would have been able to have some of these insights with Claude. Yeah, well, partly it's because there's that, and also people weren't shipping as much, so you could just make a little bullet list. Here's the things I shipped last quarter, this feature, that feature, this feature, that feature.

[12:21] So I think what I'm hearing here is this is like one of the only ways to stay on top of all the things a teammate is shipping. [12:27] So this is a really cool thread of just how you found ways to stay on top of this 8x increase in code. [12:33] What else has worked in helping you and your team stay on top of and [12:36] maintain quality of all the stuff that y'all are shipping because that's obviously a challenge a bigger challenge yeah and so definitely like the feedback channels are really important to us but then we also get a lot of feedback and like for example even i myself usually what my my morning ritual would be you know i get my morning cup of coffee and then i look at the feedback channels and then i try to pick up what are you know if i have some maker time what's something that maybe i would be able to help out or what i could like see some gaps like that just used to be something [13:06] I think maybe a month or two ago, we launched routines.

[13:09] And that's also completely changed. Like now I just have a routine that automates all this for me. And then there's also it's almost like before I would be able to kind of like [13:18] you know, generate some prompts. But now with routines, it's almost like I'm [13:22] having an agent help me generate the prompts and the PR. So for example, one is, "Hey, keep a look on this feedback channel. What are some of the themes?" And then when I wake up, then I have a really good summary of that and then even some PRs that I'll be able to take a look and review.

And the feedback channel, where's that feedback coming from? Is that like emails, Twitter, is that a combo of everything? [13:44] - Ah, our feedback channels are definitely, like, we have a lot from internal ads, but also, like, emails, channels. Actually, everybody, like, when we all get feedback on even, like, one friend's ping us or our LinkedIn or socials, we'll all actually, like, post all of that in Slack as well. And so, and we also have, like, of course, partnerships. And so we have different channels for all the various sources. But that's what I mean of I need Claude's help to help me stay on top because there is so much incoming feedback.

[14:14] Got it. Okay, so this is cool. So this is a, like a way of working that you've built to stay on top of all the stuff shipping, which is this kind of daily ritual slash routine where you, as a manager, look at what people are saying about the current state of business. [14:29] Cloud Code and Cowork. [14:31] and used to just like, okay, someone go fix this, go fix that. Now it's like, [14:36] Here's the PR that'll fix this thing. Check it out. We're ready to ship it if you want.

[14:39] Awesome. Okay. Like, obviously, a big challenge for people is also just code review. I imagine Clot is also doing a lot of its own code review. Is there anything there you've... [14:49] recently figured out that allows your teams to ship [14:51] Faster stuff that they are confident is great. Yeah, and honestly, it's crazy when you think we didn't even have cloud code reviews last year And so speak of bottlenecks that that was a really really big bottleneck of you know the human reviewers So we definitely for the important like like areas that need deep subject matter expertise We definitely want to make sure we have the proper, you know, like humans still reviewing I would say what helps us though is the more that we can automate to almost like [15:17] check in the framework for what good looks like.

[15:20] Claude is very good when you give it a framework to validate against those frameworks. So I had mentioned like, you know, like, you know, like recently we just updated like content design to have a skill in it. Like my like I and this is why, like, I think if you have specs or like or like like check those into the repo and then make sure the spec also keeps up to date with the code like frequently. But that's what I found like works really well of any time you have like a statement of what good looks like.

[15:47] get that into the repo and then code review can make sure it's still matching what you set up to do. Basically, it's like the evolution of test-driven development. [15:56] Yes, for TDD, because I remember that was a big thing, my gosh, maybe in the 2000s, right? The test first and then you can make sure the test fails, then you do actually the code. In principle, it's really good, but I think... [16:09] I know I remember it, I myself struggling a bit because it was almost like you have to eat the broccoli first.

I was like, oh, you have to write this test first. And I just get so much thrill out of shipping and building product. So it's funny, actually, the first bug I fixed on Cloud Code, I remember asking Cloud, hey, I want to do test driven development. Help me write the test first, make sure it fails, and then we'll actually do the fix. And then and then now that has passed. And the fact that that used to be, you know, like that test generation used to just be this tax that I remember having to pay.

[16:39] that that's not automated and you can even revisit [16:42] All these principles that have been around for a while, but now they actually might be even more efficient just because you have the models that can do more of the work for you. Yeah, like that's that's so unfair. It's just right to test for you first. So on this point of builders, one of my favorite slides from your talk that I want to ask you about is who you are hiring and what you look for in people. [17:05] And so I'll read what you said there, and I want to hear more here.

[17:08] So the two profiles that you now look for when you're hiring are [17:12] creative builders with Product Sense, [17:15] and deep systems experts for the hard parts? Yeah, the deep subject matter expertise. Like, for example, when I first joined Cloud Code, we had really great kind of like product generalists. And then I realized, oh, we were missing folks with systems background. And so that was definitely an area that we needed more folks with kind of like systems and distributed systems expertise. And so I would say whatever are the parts that it's all about trust, but verify the models are

[17:45] you know, the verification. And so wherever you need the deep [17:48] subject matter expertise, I would say that's, you know, an area to definitely still invest in. And the other one is kind of like the product, the product sense books, almost like the dreamers. Like these are folks that usually will be like, my gosh, you're really passionate about a product and they have an idea, they build it. And then it's, [18:05] always like looking at the feedback and then iterating and polishing and making sure that the product is a delightful experience like owning that product end to end uh that's [18:16] That's like another skill set that sort of does really well on clock head.

[18:19] This super resonates. There's this word ambition. I don't know if you used it, but that's what I thought of as you were talking. That's been coming up a bunch recently in the podcast and other work I've been doing. [18:30] There's this 10XC engineer I was talking to the other day, and he was just like... [18:34] I used to [18:35] Like I heard about a feature idea, like someone's like, hey, we should build this. And I was like, no, that's really hard and complicated. [18:42] And he's like, but now I'm like, no, no, that's so possible.

I just ask Claude Coe to do it. And it just does it. And it's this whole mindset shift. [18:49] And he's just realizing now it's about how ambitious can you be? [18:53] Like, [18:54] everything is now possible in theory. Now it's about how ambitious and how big can you think [18:59] versus just like, okay, it's all these stupid little features and things they have to unblock. Does that resonate? Is that something that you think about? Yes, actually, I was just catching up with an engineer yesterday and he actually is not a mobile engineer by trade, but we really needed to update this feature to also have a mobile footprint.

[19:18] And it was just amazing. He's like, "You know what? [19:21] Because it's common because you might think, wait, but I don't I'm not like an Android expert. [19:26] But now, thanks to Claude, actually, I can actually have a partner and actually also do this on the mobile surfaces. And so that definitely resonates. It's like it's left at the ceiling of what anyone is able to do. So let me follow that thread. [19:40] As the role has transformed in such a crazy way, [19:45] Some people are thriving. [19:46] Some people super frustrated, unhappy.

[19:50] fighting, resisting. [19:52] What do you see... [19:53] common across the people that are doing really well, the engineers that have adapted and are thriving versus the engineers and, you know, even outside engineering that are just like frustrated and having a bad time. I would say a growth mindset really, really helps. Like actually, even before AI tooling, I found that has been just so valuable. And I learned that a lot. Actually, it was the shift from Microsoft to Meta. That was where I first realized that first year, I'm like, [20:23] always be learning.

Also what served you to get you to this point may not [20:28] We serve you no longer, but it's really hard because of course everybody like we all you know, you [20:35] we've all gotten to the state by acting or, you know, like operating in a certain way. And so sometimes it is a little bit scary to think, wait, you're like, but I've been successful so far. You're asking me to change what has made me successful. And so I would say like the growth mindset is, is really, really has, has really served me well.

And I think it's also served others well, um, that, that I noticed, like always leaning in with curiosity and always being able [21:05] I think sometimes I also see a little bit of fear. [21:08] And so my advice there is at least this is how just in life I've [21:13] Because we all have, and fear is of course an evolutionary, like I didn't study anthropology, but I think it makes sense. It helps us make sure we were able to survive and not get eaten by larger predators. [21:27] But for anything that [21:29] you know, [21:30] that there is a fear, my advice is kind of lean in and ask, okay, is there some, what can I do about it?

What is within my control? Because sometimes the frustration comes from fear and feeling like, but everything is outside of my control. And so it's happening to me. And so if you think about, okay, what is in your control? Um, [21:48] instead of happening to you, is it happening for you? And then what's something that you might be able to kind of like do and change? [21:56] that's been helpful because if not, it is super frustrating to have the fear and then feeling like, [22:02] everything's outside your control. Like I actually remembered, um, [22:06] When I was in high school, I actually didn't go into computer science or engineering.

I really wanted to be a visual artist. [22:19] And this is how far back we go. Back then computers were really expensive. So I actually didn't even have access to a computer. [22:25] my first computer with access to computer was [22:28] grade nine, I think in high school. I don't even know if this happens in high school anymore. My [22:34] where you went to class just to learn how to type on the keyboard and learn to be really proficient. And that was my first time. And then, you know, the next class was maybe like some HTML programming.

So anyways, but the reason why I fell in love with it was why what I love about art is creating and being able. If you have an idea, you can go and create and tell a story. And then I realized, oh, computers and programming, you know, that that's that's enables me to do that. [23:04] for me to get into university. But I had this fear of, oh, but how will I afford to get into an engineering school? [23:10] And it was this big unknown. I grew up in Ontario, so I was very grateful.

I knew there was an Ontario, you know, school assistance program, OSAP, I think is called, which I'm very grateful for, but I didn't know. [23:22] how much of that would cover my tuition or like expenses and it was just this unknown and it would be like [23:27] a year out. And I remember thinking, "Okay, what can I do about it?" And then as luck would have it, the National Bank of Canada just posted this flyer in our high school saying, "Hey, we're hiring high school interns to be a bank teller."

[23:41] And I remember, oh, that like I and of course it was going to be minimum wage, but I thought that that could be a lifeline. But it was funny because the class I hated the most in high school was accounting. So I don't know if I'll be any good at it at all. But I signed up to be a bank teller and that ended up being such a great decision because I worked all summer, saved up. And then actually I was able to work as a bank teller on the weekends.

[24:04] So like, yeah, well, I would go to school Monday to Friday and then I'm tired of a bank teller. And that ended up being this lifeline that enabled me to. [24:12] you know, like pay for all my school expenses. And, you know, we talked about the year 2000 dot com crash. Then when folks weren't hiring as much interns, actually, I continued being a bank teller for [24:23] for two years. [24:26] But like that was the one action I thought I could take that was within my control to try to counter this fear I have of if I'm really going to go down this path, I don't even know if I could have afford to go to school.

So that's probably my other advice of like growth mindset and the source of frustration or anxiety. If it's coming from the like, see, is there one action that's within your control that you can take to, you know, so it like, because I think there's, you know, [24:56] What would you do if you're not afraid, actually? Those were my two favorite things before. Like, what would you do if you're not afraid and do something scary once in a while? Because that's also usually how we grow. I found that when you're really good and professional, what you do, you're kind of at peak [25:10] you know, like maximum efficiency.

But then how do you keep growing is you then do something scary that you might not have done before. And yeah, you will have a dip because you need to learn. But that's kind of how you keep pushing yourself to learn. The quote that I have probably used the most on this podcast of all quotes is, the cave you fear contains the treasure you seek. [25:30] Okay. [25:30] Oh, I love that. Yeah, and it's so true. [25:33] Just like, I forget who put this, but just the thing that is scariest is like...

[25:38] That's a compass towards that's what you should be doing. [25:40] I'm going to steal your quote. Please do. Like, you know, don't do all the scary things. Like, maybe don't drip off a clip. [25:48] But in career moves, it's probably a good... [25:51] A good choice. [25:53] Following this path, something that [25:56] I know that is important to you, something you think about that I also think about is this kind of gap. [26:01] that is [26:02] forming. [26:03] between people that are just like leaning into AI, killing it, doing super well, and then people that are [26:08] just like not.

And this is like [26:10] a scary time for people that may be left behind in this [26:14] a new world that is emerging. [26:16] I know you spend a lot of time with small businesses. That's a big passion of yours to help people. [26:20] learn how to use AI in their work. [26:22] Talk about just like how you think about that and what maybe we should be thinking about. [26:26] to help folks stay [26:28] not fall behind basically. [26:30] Oh, I love this. Yeah, it's one of my passion topics because I kind of mentioned, you know, like growing up in Canada.

And I moved there when I was a kid. I was born in Hong Kong. So I didn't speak any English. And my parents had to work all the time. So my grandma who [26:46] is the best grandmother anyone could have ever asked for. I know everybody thinks her grandma's the best, but I really was so lucky I had the best grandma. She moved with us just to take care of me while my parents were working. [26:57] But neither of us spoke English, but I was able to learn how to speak English by, you know, going to school and speaking with classmates.

And when I think about my grandma, it was very alienating for her to be, you know, in a new country where, and, you know, back then it wasn't as walkable. But one summer, I remember we just happened to find this little yarn shop that was owned by a lady that also spoke Cantonese. [27:20] And so that became every week we would go to this yarn shop this summer. My grandma found her knitting circle. And then I think I learned how to do like macrame, which I think is having a comeback, by the way.

It's always funny to see what's coming back. [27:33] Macrame, I think, yeah. Heard it here first on Lenny's podcast. I think macrame is coming down. But that was probably where my love came from. I'm like, oh, wow, this little small business created this wonderful sense of community. And I've been really fortunate to, you know, become friends with all the small businesses that I love. So that's kind of like where the passion comes from. And then [27:53] How it happened was I was using co-work for my own business expensing travel. [27:59] I don't know what it is.

I really don't like doing business expensing. And when I was using Cowork, it was magic. I'm like, oh, all these things that I don't like to do, like Cowork is doing for me. And I'm like, [28:10] Wait a minute, all my friends, and honestly, small business owners, they really bust their butt. They work incredibly hard and sometimes might be operating on really small margins. [28:20] And so I thought if Claude Cowork is so good at helping me do my little business travel expense, this would be huge for it. Because I see my friends sometimes sitting at the bar with stacks of bills and all they're doing is invoicing and expensing.

And I think nobody actually really likes to do it. So that's kind of where it came from. And then so, yeah, I remembered helping a couple of them onboard. And it was also very humbling to see them go through our onboarding flow. [28:50] good bugs. So it was really like a win-win. But it was also delightful for me because they also use coworking in ways that I wouldn't have thought about because I was all fixated on, look at how it is amazing with PDFs and invoice and then... [29:02] One of my friends who runs two restaurants, she's like, "Oh my gosh, this folder I have is like a junk drawer of like, it's basically our documents folder.

It just becomes this junk drawer of every, or downloads, like just, you know, like the kitchen junk drawer." And she's like, "I know I have a few menus in here and I can't find it." And I'm like, "Well, let's ask Co-Work." So we gave Co-Work access to the directory, found the menus. And then she's in a really unique way. She goes, "I want to make sure I keep my prices reasonable for locals and tourists." So she goes, "Hey, [29:32] cuisine in this area is a comparable [29:35] And it came back with really cool, almost like market analysis.

And she goes, "Hey, I actually just went to that restaurant in Seattle and that was pretty good." So I learned something every time. And yeah, they've been giving me great feedback as well. [29:48] So the question then is just like, how do we how do we spread this to everybody? Because as you see, you know, there's like, like a lot of people are just like, I don't have time for this or I hate it. It's just like, [29:58] I want to ignore it. Is it like talking? Is it just talking about this, sharing examples?

[30:02] What do you think? What's like a way to make a dent in this? [30:06] uh in this problem for all of us especially to your listeners are probably very ai pilled if there's anyone either in their community or their family that that has like i would start with what has something that really you felt um has [30:21] that you really have felt has made a meaningful life change for you with the AI tools. And then seeing if that is a conversation starter for, because for me, AI is a tool, again, it's the whole light in the dark.

I totally understand the frustrating part as well. But for me, I'm also about like knowledge is power, like have to learn how to use the tools because it could actually, you know, be the light part of that light and dark equation. So I think that that's something that I would love everybody's help with, whether it's a community member or even if there's a [30:51] really like anything, hey, have you ever, it's a little awkward to start. Like I remember when I first reached out my friends, I'm like, [30:58] because I don't talk about what I do with them a lot, but I'm like, hey, I kind of, you know, working AI.

It sounds it's even unnatural for me to do it because I'm, you know, like, hey, can I show you what, you know, co-work is possible? And then but it ended up we ended up having a lot of fun. So, yeah, I would love for like that conversation starter would be great of. Yeah, because I just want to make sure we keep sharing the knowledge and, you know, making the tool equitable, [31:28] frills larger and larger. Me too. [31:30] I find that sharing use cases, as you said, isn't so powerful. [31:34] just using Cowork the other day to fill out my son's camp forms.

[31:38] And just sharing that on Twitter, just like a lot of people are like, oh, wow, I didn't think about that. [31:43] It's just like these little things you don't think about that Kork and Kalko can do. [31:47] Kind of along those lines, speaking at cowork, [31:49] If you think about it, [31:51] Anthropic has been super early [31:54] on uncovering these really big opportunities [31:57] ahead of everyone else. For example, coding. [32:00] So far ahead, like realizing this is a huge market, whether it was intentional or not, it was just like, whoa, that's maybe the biggest new business opportunity in history.

[32:10] And then Cowork is a great example, just leaning into... [32:13] Knowledge work, let's just solve all the knowledge work. Why not? [32:17] so far ahead of everyone else. [32:19] Another element is it feels like a focus on the personality of the model, something you all were very early on, just how important that is, not just for the experience, but just also the intelligence and success of the model. [32:31] What is it? [32:32] that you think Anthropic and the teams do [32:35] differently that allows you [32:37] to uncover these opportunities and then just go big on them before other labs, let's say.

[32:43] Well, I haven't worked in any other labs, so I'm not sure how the labs operate, but I will share like, yeah, with, um, [32:50] And actually on the Cloud Code team as well and co-work, we also keep an eye on latent demand. Now we're very fortunate with coding as a use case because so many ants that we were our own first customers and we're able to do really rapid feedback. And so I think, but latent demand has been like, for example, a co-work we notice a lot of folks that were not necessarily coders were using Cloud Code.

Can we make that experience better? I think that's actually even outside of Anthropic that served me well in all the different [33:20] products I've worked on. But and actually, it's funny, you mentioned like, you know, the the cloud for, you know, my passion was small business after I, you know, had a few of this business, we ended up now launching cloud for small business. [33:33] which is really cool because I and it totally didn't come from me. So I'm not taking credit, but I noticed it too of like when I was working with it because they were asking me, oh, does it have this plugin or this plugin?

I'm like, I think it does. And then we have to go and search for it. So actually now cloud with small business bundles it all up. So inside cowork, you just have this little toggle and there were there was a wonderful team that's been kind of like, you know, doing coworker sessions with small business that probably found, hey, we could have like an efficiency gain here. [34:00] or like make the experience better. And so I would say like always not only for products that you're on, like making sure you're always listening to feedback and always iterating, trying to make a delightful, reliable, high quality experience, but then also keeping an eye out for, oh, what are these other use cases that that are popping up?

And can we also make that experience better? [34:22] It's interesting, like after, like, you know, [34:26] In software, I've learned that [34:27] Customers will use your product in ways that you did not intend for good or for bad. [34:33] And so the best way is really like it's all about the iteration, learn and keep close to the feedback. [34:40] Wait in demand. That's a that terms come up a bunch on this podcast. There's something there. Might be something there. So essentially, it's just watching closely for. [34:48] Behavior that you may not have expected or that is just kind of emerging.

[34:52] and then just... [34:53] Going big on that, essentially, exploring it, building something. [34:56] Awesome. Yeah. [34:57] and having a hypothesis for hey like because actually when you see people jumping through hoops to make something work [35:03] Can you actually make that an even like a smoother and better experience? Yeah. [35:08] Coming back to the way that your teams operate, [35:12] You guys are so... [35:13] at the edge of what's possible. [35:15] And the role of just engineering has changed so much. [35:18] I'm curious what you think is the next kind of frontier of how engineering in particular is going to change.

[35:25] Is it like fleets of agents? Is it [35:28] Something else? Just like what's like the next big shift to how engineers operate? [35:32] that either you're ready [35:33] working out or implementing or just like it's starting to happen? I would say we're shifting more towards async, like asynchronous. And so to your point, the fleets of agent like that, that's why routines are so interesting, because almost like I used to be like, you know, doing a prompt and synchronously. [35:48] And then I would maybe kick off different props asynchronously. But now I can actually have a routine that actually generates these prompts for me.

So it's almost like the level of abstraction keeps [36:00] pulling up a little bit. And so I would, yeah, I wish I have a crystal ball to see what this is going to look like. It'll be fun actually for you and me to revisit a year from now. No, I want to hear more about this. This is so help us understand what is what is routines and then what is what do you say synchronous? You're [36:16] writing a prompt and it just kind of goes off and is it writing it immediately?

Talk about what this actually looks like. [36:21] Yeah, so routines is like how you can, like, so if you remember, I was trying about my, what I used to do is always like, you know, wake up with my morning cup of coffee and hey, look at through this Slack channel for me. But then now, like I have a routine that I set to run every morning at a certain time. It's like a cron job. [36:38] That's right. But then it's able to actually go and kick off agents on your behalf.

And so that's because like that's what like, you know, even, you know, before it would be, oh, you could like, you know, do a cron job to automate. Like, but now it's, hey, look at these feedback. And then if you're seeing some of these bugs that what are some polish fixes that you might be able to knock out and then it would like go kick off. And then I wake up and I ended up having PRs that I could review versus before, which is still more of a kind of different agents.

And then it's still I'm still thinking about, OK, now what do I do with this information? [37:08] So it's like that higher level abstraction of okay now how can I actually write a routine that also basically does prompts for me for spawning different agents? So I think what will move more towards that kind of like asynchronous style of working so interesting so like in a sense as a manager you have these kind of things you do daily and [37:27] And what you're saying here is you can set up these prompts that kind of check in every day on the things you would be doing.

Like, how are the projects going? What's falling behind? What should I implore? [37:37] who's struggling and [37:39] How do I improve some polish? [37:43] And so the idea here, what you're describing here is kind of write these things that you almost do as a manager daily. [37:48] and have Claude essentially do them for you and then show you here's what I'm doing. Here's what I here's what there's to review. And then it's it's even like then giving even more autonomy at some point of it. You know what? Like go for it.

Like for example, a verification is really good. Yeah. Go for it. I see. So you give it like a little bit of freedom to do more of this. [38:05] Okay, that is so interesting. [38:07] It reminds me, speaking on this idea of go for it, I wasn't planning to talk about this, but I just watched Tyler Cohen have this awesome talk. [38:14] about just like what's happening. I don't know if you know Tyler Cohen, it's really smart. [38:18] The Economist as a podcast. And he just had... There's so much talk of...

[38:23] the people that are doing best in this world [38:26] His term is initiative. They have initiative. [38:30] Another term people use agency. [38:33] Is that something just like, you know, people hear this a lot, just like the people that seem to be doing best. [38:38] are taking the most initiative, getting the most proactive, have the most agency. [38:41] Does that kind of spark any thoughts to you? Just what people may need to, I don't know, think about or improve on? [38:47] Actually, that's agent. I love that word agency. That's what we really hold important on the Cloud Code and Co-Work team.

But it's interesting. I say along with high because like we really it's about like, hey, here's a problem. And then it's really everybody on the team has ideas for how to address the problem. So it's really high agency. And then we say with high agency is also high accountability. So it's all about like making sure folks have that freedom to cook. [39:17] to like what's a hypothesis of what you're you know like trying to solve so i think yeah the the balance like we're you know almost like two sides are the same kind of agency and then accountability i think has served our team really well such an important element of an agency okay cool everyone's off doing stuff but okay what have you actually what have you busted up what have you done so kind of along those lines [39:40] It feels like there's this vibe shift recently from token maxing, just go crazy, spend as much as possible, just see what's possible, to like, wait, what are we actually getting out of this?

How much does it cost? Dang, what's the ROI? [39:51] Interestingly, Boris was actually head of [39:54] productivity and meta that was like his job [39:57] is measuring edge productivity. [39:59] And [40:00] And then also just this tweet of lights per code, like code shift. There's this interesting discussion of just [40:07] How do you actually measure productivity increase and ROI of [40:12] AI tools in spend. [40:13] How do you like, you know, you guys have the unfair advantage of getting free free tokens working in Anthropic? [40:18] With that in mind, [40:20] What have you learned about just how to measure, say, ROI of engineers in today's world?

This end productivity is such a fascinating topic. Yeah, because I remember, you know, like even Boris mentioned, like, you know, we'll first start with like lines of code. And then that's Rupa. And then I remembered once then there was a debate of, well, lines of code. This engineer had this crazy lines of code, but they just took some library and then just was porting it. So they checked it in and then I was like, well, maybe it's significant lines of code. [40:50] we're updating our frameworks, and now we're generating less code, but the output's still the same.

So now I was like, okay, maybe it's time to land PR. But it's very interesting, if it's always been, whichever metric, if you're really focused on the output, [41:08] My advice here is [41:10] One is output, like, [41:12] is the output really going towards the outcome? Because yeah, the token maxing, it's almost like the lines of code that we used to have, but I'm really much more about what is it that we're trying to do? There was another saying, don't forsake motion for progress. [41:31] Because if you're measuring like, [41:33] you know, like tool user usage, then you're measuring the action, but is it really making whatever the end outcome of yours like important?

[41:41] And so I really try to zoom out and focus on what is the problem we're trying to solve? What's a good way to measure that? And then that's what we focus mostly on versus kind of like the productivity measurement. [42:11] focus on the senior engineers as well. Like hear from them on what's working, what's not, how can we actually make it better? Because they will also help you multiply and scale that out across a full engineering team. And sometimes it's those conversations where you might get spark an idea that comes up and also really good shared learning versus, you know, like metric dashboards.

[42:29] It's very interesting. I've learned some good lessons from metrics. I'm all about like, it's really great when you have [42:36] a metric that you can actually hill climb on, but always keep in mind of is, you know, kind of my whole growth mindset of, is it still serving you? Always keep in mind like, [42:45] Is that metric really still serving the outcome that you were aiming for? [42:49] A fun example I have of this is in the early [42:52] Facebook marketplace days, we were kind of like launching by region and we really want to make sure we're building a delightful product before we expand.

And I remember in the early days, one thing we would keep an eye on is kind of like number of sellers. And I remembered after launching to our first region, I'm like, huh, in this area, the number of sellers [43:10] is low, but actually people are finding items that they're looking for, which is what we're aiming for, like helping people find items that they need. And then I realized in that region, it wasn't a large number of sellers, but there were power sellers. [43:24] But our first gate before we expand would have just been like, you know, factoring heavily number of sellers.

And I remember that quick conversation of, hey, and this goes back to that whole people use things in ways you may not expect and shift, iterate, learn. And so so then we updated the metric to go, oh, you know, like it's not. [43:41] a number of solid because it didn't factor in power sellers and so that's the advice i have to like whatever metric whether for productivity or even for product always keep an eye and make sure that you're not just having blinders on that's blindly following a metric that used to make sense because sometimes the landscape can change so fast uh even the metrics themselves might need to be adjusted and this comes back to your uh process that you shared of having claude [44:11] not focusing on the metrics, but focusing on that leading to a conversation about [44:15] What impact does this have?

What was maybe some bugs that happened? [44:18] and that being a really powerful way of understanding what's going on with this engineer. [44:23] Awesome. [44:25] Let me come back to this question of speed and quality and just like... [44:29] impact, [44:30] Uh, [44:31] Is there anything else you've learned about just how to... [44:34] balance those things, just like this crazy velocity of code that's being generated and just staying on top of quality and impact. Is there anything else there that might be helpful to other teams that are trying to wrangle all this, all these PRs being shipped every day?

I would say, and this is what honestly we want to keep doing more of and being better at it, too, like the proactive quality. So especially for quality, making sure that what are the experiences that we're doing? [44:58] that are key and making sure you actually, you know, actually speaking of metrics, those are really good things that you make sure you can kind of like see trends over time. And so like on the quality front, we found like, [45:11] And this is like the more proactive we can be of like making sure we can get an earlier detection into quality.

And so like, [45:21] That's been one thing that we've been paying a lot of attention to. Like, so like, you know, I started this, hey, let's have a concept of. [45:28] uh what's bad versus what's sad and bad is like a very bad irrecoverable error and sad is something that's kind of like a pain point recoverable but it's interesting when you stack up sads it could you know generally go to bad but even having a like starting with a high level framework like that and because if not like i think sometimes with dashboards you can have you know like time to load or all these other but when you're dealing with a lot of different product services it's harder to go wait is that a good number or not a good number and so one thing that's helped us is [45:58] performance or, you know, like reliability numbers, also having some framework of what we think is, you know, like a bad experience and making sure we're focused on addressing those and then also keeping an eye on where we're seeing in terms of the sad.

I like that. Yeah, I like the bad and sad. So these are kind of like thresholds of this is bad. Okay, this is serious. And this is in terms of like performance or failures or what sort of things are you measuring here? [46:23] So for example, we allow each team, speaking of agency, so knowing that bad is a really bad, irrecoverable error, we enable each team a further surface areas or it could be services that they lead. So for example, on CLI could be crash rates, like a crash is pretty bad, you lost work.

And for example, a sub might be, hey, is it flickering? It might be recoverable. But we have each team, and that's why it's been interesting because surface areas are different, [46:53] zoom out and go, okay, what's the overall theme of the experience? So to your point, we give high agency to each team of what we think constitutes a bad, and what's a sad and then what's the goal that each team wants to take. [47:06] One of the main takeaway I'm hearing here is one of the best tools for staying on top of quality.

[47:12] is just monitoring and tests. [47:15] versus [47:16] spending more time reviewing. [47:18] which makes sense because the speed is just impossible to stay on top of. [47:22] And it almost speaks to this idea of closing the loop [47:24] for agents to be able to kind of figure things out themselves. They know what success looks like, cool, we'll fix it ourselves. [47:30] So that's a really interesting takeaway, is just invest more in the tests [47:33] evals I imagine is part of this and then just like monitoring failures and speed and things like that [47:38] That's right.

[47:39] Something I think this is public. I know that you guys have this dashboard that tracks just like F words, like how often people are like, because they're so pissed and frustrated. There's like a funny term for it, I think. I forget what it's called. [47:50] Yes. [47:51] Actually, I remember that was last September because we were all we were all seeing some frustrations and yeah, that was an engineer on the team of, hey, we should maybe track square words. I'm like, oh, that's a great idea. I remember it. I had just, you know, we were having that really fun conversation.

Yeah. So it's it's again like it's very interesting to kind of like look at. [48:09] And that's why like evals is hard too, because it's [48:13] Going back to that user experience and how we can make sure it's a delightful experience and less frustrating. But yeah, the swear word dashboard is a fun one. [48:21] This episode is brought to you by Mercury. Radically different banking loved by over 300,000 entrepreneurs and now with command. I've been a customer of Mercury's for over six years. I have never once thought about leaving. Mercury is basically what happens when banking is built by product people, not by bankers.

They make it so easy, dare I say fun, to send invoices, move money around, set up virtual cards for folks on my team. Does your bank have an API, [48:51] or an AI-ready MCP server? I don't think so. And just recently, they launched Command, a conversational interface built directly into Mercury, which acts as your financial operator. I've been using Command to transfer money around to figure out what categories I've been spending the most money in, analyze my cash flows. And just today, I used it to find out how much I've made from a specific sponsor over the past year.

I just asked, how much have I made from X over [49:21] cool. Visit com to learn more and apply online in minutes. Mercury is a fintech company, not an FDIC insured bank. Banking services provided through Choice Financial Group and Column A. members FDIC. [49:35] Kind of going back to the way you operate and ways that you've figured out to work in this crazy new world that we're in. I've heard about a couple of things that you've implemented that are pretty unique, I think, to how teams operate. And I think is something that has worked really well for you.

[49:47] One is... [49:48] making every manager start as an ic and then just every manager has to continue being an ic part-time kind of this player coach [49:56] approach. Talk about that, why that's so important in today's world. [50:00] I love it. It's funny, when I first joined... [50:04] because I have amazing recruiting partners and I noticed we were, you know, this whole theme of kind of growth mindset, it's just because the landscape is changing so fast. [50:13] what worked well before, like may not make sense. And even what makes sense today may need to change tomorrow, right?

Like that's what I have to keep reminding myself. So, yeah, when I first joined Recruiters, like, okay, yeah, we have these, um, [50:24] couple manager postings and I'm like, you know, because actually how it came from like a listening tour I did with like all the all the members on the team. And I heard a lot of these, hey, I really appreciate the agency, but how can I make sure prioritization? And so I kind of through that, I realized and then there was some really good feedback to making sure that [50:44] It's not too many layers of reviews.

There was good feedback. Some folks might have joined from other companies. Then I'm like, hmm. [50:53] I actually think as a leader, if you actually start as IC first, without the worry of supporting people, because that's a very heavy responsibility that, you know, I think like matches. But like that. But before you have to take on that full responsibility, give yourself that maker time to actually tap deep into the code and learn the code base. And I or the product, like whatever it is, it doesn't have to like honestly, the peers I do are like.

[51:23] because we're making so many changes to co-work and code. So even me doing PRs, it's less about what it is I'm fixing, it's more about me using the priority data just to keep that touch. 'Cause as amazing as metrics and everything are, and I do look at those dashboards, if you, [51:38] As a leader, if you're not living and breathing your product every day, you sometimes lose the touch and feel of the product. [51:47] But anyways, on the manager front, I think giving time for managers that join to be able to go deep and to do that before supporting people.

And then they actually like end up building really great rapport with the team. [52:00] Because if not, I think sometimes as managers, you might come and join a new team and you instantly think, "I have to manage. Let me dig into my manager toolbox and do manager-y things." But if you actually give yourself time to not have to worry about that first and actually learn what it's like to be an engineer and teammate on the team, that also goes really far to building rapport. [52:21] And in terms of using the product, I think that's important.

It's interesting. Every team I've joined, one of the first, actually across all the different products, it could be VR, it could be smart glasses, it was like Instagram. Usually when I first joined, one comment that comes back to me is, "Hey, you know what? I really love how you're actually using what we build." It's refreshing to see you giving user feedback. And so I think as leaders too, it's also a way for us to experience up the work personally that the team does. [52:50] Something that people may not realize, you were overseeing an org of like 500 people at Beta before you moved to Anthropic, right?

[52:57] And you move to IC, IC Engineer, basically, and Anthropic from that. [53:02] I started out like it was for a very short amount of time, but actually this was my journey between Microsoft and Meta. So at Meta, I interviewed as a manager, but I think at least for the first quarter, I was also an IC because I really wanted to learn what it's like to be a Meta engineer. Because before then I was at Microsoft, kind of like cut my teeth on engineering at Microsoft. So I knew all the code bases, the tools, the languages.