In today's episode, Michael and Ben dissect the process of building maintainable and impactful products, emphasizing the crucial balance between innovation and simplicity. They explore personal and group learning curves, the value of collaboration, and the indispensable role of peer review in creating robust solutions.

They'll also touch upon the nuanced perspectives of working at top tech companies like Google and Databricks, examining how timing and project involvement can shape a developer's skillset and career trajectory. From the importance of understanding one's career goals to the powerful impact of a company's culture on code quality, they aim to uncover the multifaceted aspects of professional growth in tech.

Join they as they delve into stories of overengineered solutions, the necessity of constructive feedback, and the collaborative efforts that define truly great products. Whether you're aspiring to join the elite 1% of developers, or simply looking to understand the dynamics of a high-functioning team, this episode is packed with insights and practical advice. So, tune in and let's explore the path to greatness together!

Michael Berk [00:00:05]:
Welcome back to another episode of Adventures in Machine Learning. I'm 1 of your hosts, Michael Burke, and I am joined by my beautiful, wonderful, amazing, and terrific cohost, Ben Wilson. Do you do anything or is your name just Ben Wilson?

Ben Wilson [00:00:17]:
Oh, we're doing that again this week. I always forget. Just kidding. I review design integrations at Databricks.

Michael Berk [00:00:28]:
Great. Today, we have a panelist episode. Per usual, Ben and I have been talking for the last hour and 7 minutes, and we sort of settled on this topic a while back but, honed the topic throughout the last hour. And hopefully, this will be really valuable to, the listeners yourselves because it's it's it helps you sort of inform how you go about learning, how you go about associating yourselves with different types of people in the organization, and then how you also interact with people sort of, quote, unquote, below you in the organizational hierarchy. So, Ben said that, 1 of the core Databricks hiring tenants is looking for people who are exposed to greatness. So worked with very top engineers or really smart people in general. And the rationale for this is what then?

Ben Wilson [00:01:20]:
So the foundation aspect of, like this goes back to looking at somebody's resume, right? Let's say we work for 1 of the or we're looking at somebody's resume who's coming from a household name company in the tech space. Like, hey. I work for, you know, this super famous company. I've worked there for the last year. And at face value, when you're looking at somebody that claims, like, on like, you look at their LinkedIn profile and you're like, 0II work at this big name company and people who don't work there or who have never worked for a company like that, look at that. They're like, oh, this person's legit. Like, they're they're an elite programmer or elite data scientist, and we don't care about that. It's it's the temporal effect of where were you at that company at what time.

Ben Wilson [00:02:27]:
So what what projects were you working on, What company was that, and what stage of their evolution were you there for?

Michael Berk [00:02:39]:
Yeah. So we were talking about Google as an example, and I remember hearing an anecdote from, someone who works at Google that the first Google search implementation was linked lists that, like, the founders actually built it with linked lists. And then from there, it got a lot more advanced. So if you join Google just a little bit. Yeah. The yeah. Just a tad. Anyway, if you joined at the very beginning of the organization, you would have been building Google search with link lists.

Michael Berk [00:03:10]:
If you joined at the very end or whatever now is, you'd be, like, working on a very, very tiny component of a very complex search stack. Ben, in your opinion, what is the optimal area that you should be or optimal time frame that you should have worked at Google to maximize your exposure to greatness?

Ben Wilson [00:03:34]:
So it's a really tough question and a really tough answer to provide. It depends on what you're looking to do. Right? So I've had many people reach out to me, over the years about, like, oh, how can I get a job at Databricks? Or I wanna do this thing at this other company that's really, you know, reputable. And I'll I'll I'll usually answer their question with another question. I'll ask, like, what are you looking to do? And sometimes the the response is, well, I'd like to, you know, work for you know, we're not making this podcast about Google. I've never worked there. I don't know too much about it. I work with a bunch of people who have worked there at all the different stages, of that company's evolution.

Ben Wilson [00:04:22]:
But, if we're looking to get in we'll just keep on using that as an example, but somebody's like, oh, I wanna go, you know, get a job at Google because I wanna build, you know, search. I'm like, sorry. That ship has sailed. They're iteratively improving it all the time, but you're not building it from scratch. It's never gonna happen because it already exists. And it's probably a truly monumentally complex, like, not just single code base. Like, there's a lot of services involved in that. Mhmm.

Ben Wilson [00:05:01]:
Most of them around making money, on ad revenue. But if somebody says something like that, and that's my response, well, I'll I'll say, what are you what are you trying to do? Are you trying to learn the process of how something like that is built from the ground up when a new idea comes out or a dramatic improvement on preexisting technology, then you'd need a time machine to get back and, like, try to get hired in that core team, you know, back in their first 6 months of of working there and go through the pain and the learnings over time of figuring this thing out. And if you were to do that, you would be imminently hireable anywhere. Like, anybody would hire 1 of those people because they had to go through figuring out how to invent new technology and make it work. And by the way, it made that company an astronomical amount of money. Really? I didn't know that. So to the point where they can afford to hire loads more people and build all sorts of new cool products, and then become a commodity and a household name. So the answers that I get are pretty varied from people, which is always interesting to me.

Ben Wilson [00:06:25]:
Some people are, will come back and say, well, I'm looking to, you know, get that street cred. It's like, cool. Go work for a startup. You know, play the lottery, find a startup that aligns to your desires and get ready to learn like that and get ready to get stressed out and have to figure stuff out and worry about profitability. And, like, how do we get more users and how do we build features quickly enough that people actually care about? It's a very unique experience, being in that high pressure environment, and it could also just go out of business. Right? Lots of factors that go into that, so you have to be willing to take on a whole lot of risk to get into that. And then some people come back and they're just like, I want the name. Like, I wanna be able to put that on my resume.

Ben Wilson [00:07:23]:
The and they think that companies are are kind of dumb, I guess. Like, oh, if I go work at at Meta right now,

Michael Berk [00:07:32]:
I think a lot are.

Ben Wilson [00:07:36]:
It depends on where you wanna go. So if you're going to a FAANG company right now and you put in, you know, 2 years working at 1 of these companies and then wanna go work at a startup that's founded by people who came from a company like that, who are looking for a core team to build a product, they're not gonna hire you. Like, they don't care, like, where you worked. They care about when you worked, where you worked. Mhmm. Because they're looking for that skill set, the scrappy, you know, entrepreneurial minded developers who were who were willing to build new stuff with no context and just get it out there and get it working and then refine it over years of work. So it that's that foundation thing. We're talking about the hiring process.

Ben Wilson [00:08:32]:
It depends on where the company is right now in their own evolution, where they wanna be in 2 years, 5 years, 10 years, and what their their DNA is at their organization. So if it's a an r and d focused org that is constantly looking to build whatever people want and, you know, be on the cutting edge for as long as they possibly can be and continue to do that. The desire for hiring is you need a good mix of people who have done that before to be able to be, like, the leads and guide the next generation of people who are building stuff and learning, you know, how to do that. And then you need people who are, like, recent college grads who have that sort of, like, the technical capabilities, the the training. They have the knowledge, and they're they're hungry to learn and break stuff with guidance and fix it. But what they're not looking for is somebody who spent, you know, 2 to 3 years at a bunch of established tech companies that you know that that person is basically just in, like, maintaining stuff and then hire them for a role where they're gonna have to be building stuff from scratch, they were never exposed to that greatness. And that's that foundation thing.

Michael Berk [00:10:00]:
So we're sort of talking about the top like, the 99th percentile, the very top. And I feel like if you have Google on your resume, that'll land you, like, an interview at probably 90% of organizations with a lot of caveats. But you're talking about sort of getting to the that, like, 1% of developers, that 1% of startups, the the really tippy top. Is that right?

Ben Wilson [00:10:24]:
That's where everybody wants to go. Like, for I mean, based on people that have reached out to me.

Michael Berk [00:10:29]:
Surprised, actually. But yes. I mean, yeah, there

Ben Wilson [00:10:32]:
are people that they've learned some skills earlier in their career. They want to just do a 9 to 5 job. Mhmm. And it's not a passion for them. They don't get excited about building new things. Some of the some people get terrified of having to build new things. They'd rather hire a consultant to do it for them, or they just don't care. Like, Hey, I have my, my quarterly goals, which is to maintain uptime and, you know, limit the number of trouble tickets that come in and, you know, make it so that any bugs that are that come up, we have a patch for them.

Ben Wilson [00:11:13]:
There's tons of maintenance engineers out there. They have job title of software engineer, but that's what they do, and that's what they like to do. I'm not knocking that. Like, it is it's a needed thing in any tech company. You can do an entire career of doing that. It sounds like pure torture to me, but I'm I'm more of the I like to build stuff, and I like to be around people that are building stuff.

Michael Berk [00:11:38]:
Right. Yeah. There's just different ways of living life. Like, I know, at least in American culture, it's pretty common at, like, high performing tech companies. But I spent a good amount of time in a few South American countries, and it's like it seems like just a very different set of priorities. It's more about family, friends having fun, for at least the people that I was around. And, yeah. So not knocking anybody who likes the 9 to 5, getting a big paycheck, and then going and living life.

Michael Berk [00:12:07]:
But we're sort of more talk or maybe yeah. What's that face?

Ben Wilson [00:12:12]:
You're not gonna get a truly big paycheck being, like Fair. Keeping stuff in maintenance. But, like, good enough. Your mileage may vary. Like, it it depends on if you're if you're working for some small company that's been around for 30 years, and you're on a team of 4 software engineers that are just maintaining things, that company is probably maybe paying you a 150 k a year total comp to just keep the like, keep the lights on. That's what you're doing. Maybe you get a new feature or, like, some, like, some small minor tweaks to the the product in a year, but it's not like you're focused on reinventing things or fundamentally building new product for that company. There might not even be an opportunity to do that.

Ben Wilson [00:13:04]:
Like, the business model for the company might be like, hey. We we sell education software. Like, that's what we do, and we have this full suite of things that all of our customers like and rely on. There's no other products that anybody's asking for. Nobody cares. They're just happy with what it is. You're now in maintenance mode, and that's what you're doing. Right.

Michael Berk [00:13:27]:
Yeah. Okay. So the like, just some caveats that there's different ways to approach professional development. And if you wanna be in that top percent, being around greatness is definitely helpful, but it's not a requirement if you wanna just hang out with friends a lot. Anyway, that's sort of an aside. So we've been talking about this this concept of greatness. And let's say in the we're using Google as a case study in its, whatever, 20 year history. If you entered at year 2 and then stayed till till year 14, that would be a pretty damn impressive, duration to or, like, time to spend at Google because you get to see how it was pre success and then helped it build success into being 1 of the most, like, impressive companies of all time.

Michael Berk [00:14:16]:
What types of greatness would you expect to be exposed to in that time frame?

Ben Wilson [00:14:22]:
Yeah. A lot of people have asked me this about Databricks actually, thinking that it's just code. Right? Like, oh, you've seen amazing implementations. Yeah. There is that. There are some very smart people that I work with, who are very good at implementing features, shockingly so. Very fast too. But just because you you code good, does not guarantee you a role in what you might wanna be doing.

Ben Wilson [00:14:59]:
There's a lot to developing products in a software company that has nothing to do with the code you're writing. In fact, most of the work that goes into building things has nothing to do with writing code. It does tangentially have to do with it. You're talking about how you're gonna potentially build something, but most of it's deciding not just what should we build, but what should we not build? What are the things that are irrelevant? You know, because if there's no guardrails, you can go and just build all the features. Like, hey. I think this would be a good idea, or somebody's gonna need this API someday. And if you allow a bunch of people to do that in a a fast moving tech company, All you're doing is creating an unmaintainable mess of spaghetti that's just gonna eventually break. You know, it's it's really easy when you're talking about, like, building something from scratch the first time, and you're like, I'm gonna put all the features in it, and I'm gonna it's gonna do all these cool things.

Ben Wilson [00:16:06]:
And if you're not thinking about, what does this look 5 years from now when I need to maintain this and add in 200 extra features to this over time, or I need to change how this behavior works completely because it doesn't scale or there's there's some fundamental shift in what the priority is of this product. You're now left with, hey. Those, you know, 200 APIs that we created, We collected data and only 15 of them are ever really used. The other one's like, yeah, people use them, but they feel like dev APIs that were made public and but we have to maintain them. And in order to make this change, we have to update all of them and all of the logical, you know, intersections that they all have with at the back end, and it becomes very expensive to make changes at some point if you don't restrict what it is that you're doing and knowing what to build and how to build how to design and build that in a community with your fellow engineers that are all gonna have to be maintaining it, being on call, making feature changes over time. If you're not in that environment where somebody is controlling that and seeing greatness in product About what are our decisions about how to build, get feedback, fix, improve and iterative development like that, if you've never been involved in that, even if you weren't the primary person doing that, just being in the room and being 1 of the people that are filing future PRs and getting review from the person who's got the the large context around that, If you've never been in that room, it's hard to be it's hard to sell yourself for a company that's in the process of doing that. Because they'll ask you questions that like, in the interview process, you'll be asked questions about, like, hey. Like, tell me about this feature that you worked on.

Ben Wilson [00:18:24]:
What was your role in it? What did you do? What did you decide to build and not and what not to build? And if you can't answer that question about, like, what not to build, now that person that interviewer has questions. Like, it does this person have an understanding of this? Like, do they know what that process is like? Have they ever had to maintain something, or did they just build something that could work somewhere for a year, built a feature, and then pop smoke and and move to another tech company? Like, we don't hire those people because they've never had to maintain what they built. So you've been calling greatness

Michael Berk [00:19:04]:
or, like, alluding to greatness being the ability to determine what should

Ben Wilson [00:19:09]:
and should not be built. What is greatness, though? Something that is successful in not just in the business sense, but in the ability to retain talent. So you can build the coolest thing ever. Right? You can go wild and build, like, this awesome new product that in the process of building it, everybody's super pumped and they're like, yeah. This is gonna make us tons of money, and we're gonna get famous off this, and it's gonna be awesome. And then 2, 3 years down the line without having a good product direction, it becomes so unmaintainable. And people from the business side are saying, we need more. Like, our customers are asking for these things.

Ben Wilson [00:19:59]:
We don't have evidence, but this is what they're saying. And you just end up just gluing more and more garbage onto the code base to the point where when you go on call, it becomes like a waking nightmare because you're getting pinged every 30 minutes for sub 0 incidents. Services are going down. Like, nothing works correctly. You're having to do patch fix after patch fix after patch fix, and all of your development time is spent just fixing the mess that people have created. Good people leave from companies like that. So, like, hey. Like, this is so broken, and management doesn't want us to fix it.

Ben Wilson [00:20:39]:
They're telling us, no. You can't, like, refactor this or get rid of all of these stupid features that nobody cares about. They just want us to perpetuate this monster that's been created. If you get no traction, you know, good people just leave. So greatness is building something that people love to build, is making money and people love to maintain, or at least the maintenance isn't painful. Okay.

Michael Berk [00:21:13]:
I think I'm falling. The 50 questions are popping up, though. The first 1 of which is how is greatness disseminated throughout organization? Does someone walk up to someone else and say, hello. This is greatness. Goodbye. Or is it implicit in how you go about your work? Is it a combination? Is it mentorship? What does it look like?

Ben Wilson [00:21:34]:
No one's ever gonna tell anybody else how to build something great because anybody who knows how that stuff works knows what to say because you never know. You don't you're always questioning that. Like, are we building the right thing? And there's processes around that to evaluate and to shift directions if it's not working out.

Michael Berk [00:21:58]:
So it's it's like 1 step above what to build. It's the process about deciding what to build. That's greatness?

Ben Wilson [00:22:04]:
It's having voices involved in that conversation and a an agreed upon way of doing things. So you have processes in place. You have people that are driving those processes. You have accountability to adhere to those processes, and everyone is sharing this sort of hive mind approach. Like, this is how we do business. This is how we're building. And you absorb that by doing that, not just by looking at it or studying it in a book. You have to be there.

Ben Wilson [00:22:41]:
You have to feel the pain. You have to be that person who created the incident that broke something, and then you went and fixed it and learned from that. Or you built something thinking it was gonna be used in this 1 way, and you find out later on, whoops. I didn't account for that. We're not we're not preventing our users from using this this way, and all of a sudden problems happen. You learn from that retroactively. You discuss that. You analyze that, and then you fix that so that the next time you build something like that, you remember that pain intimately.

Ben Wilson [00:23:21]:
And that's the wisdom that comes along with it. So learning greatness is suffering through your own personal idiocy or a group idiocy or group ignorance and learning from it. And eventually, you know, the organization and all the members of it are attracting other people from outside experiences who have done things and learned from them, and they've also experienced greatness.

Michael Berk [00:23:50]:
Do you need someone in the room to have experienced that idiocy, or can you just know how to do stuff correctly?

Ben Wilson [00:24:02]:
So, I mean, it depends on what you're trying to do. So some problems are kind of academic. You you know, like, the foundation for what it is that you're trying to do and you know your requirements. So you can academically test something. You know what to test because you can think through what could go wrong. And a lot of things are that. Right? We do acceptance testing. We do, you know, bug bashing.

Ben Wilson [00:24:30]:
We try to break what we just built, and inevitably, we do. Right? We break it and fix it. And there's some times where something so new, you just don't know. Should you instantly start building a product out of something that you don't understand? Probably not. Go and try it out first. Build a prototype. Break the prototype. Have other people play around with it, test it, and have other people's ideas be included in it.

Ben Wilson [00:25:03]:
You know, if somebody has a great idea of how to fix this issue where we're not directionally aligned to the product vision, don't tell them to shut up and and stop talking. Listen to them and hear their feedback. And whether that be a a customer that's using your product, or, you know, an end user who's struggling with what you built or another engineer in a different department or, you know, a different team who's bringing this stuff up. The greatness aspect of that of development and listening to. Thoughts that people have is being open and, like, respecting people's thoughts and opinions and and listening to them.

Michael Berk [00:25:49]:
Yeah. Let's take turns, giving some examples of a story in our life that informed us learning about how greatness should be defined. And I'll I'll kick it off really quickly with a prior, project in a prior company. Basically, I was building a AB testing pipeline that was, bespoke and sort of outside of our main AB testing stack, and it was used for a few special cases. And I remember it was my baby. Like, I I tried 7 different things. I wrote it in Spark, Pandas, Modin, which is like a a paralyzed version of Pandas. I did, what's that library that allows you Numba, which basically allows you to write CSK Python applications and Redshift SQL, which is the worst thing on planet Earth.

Michael Berk [00:26:40]:
And I remember I built it. It finally was working. And then 1 day, it was like someone was reviewing the dashboard, and they said, why do we have 200% revenue lift on this experiment? Like, is it actually that good? And turns out it was a bug, and I had to fix it. And there was, like, a little bit of embarrassment saying, oh, a product manager saw this, got excited, told someone, and then it actually was just a bug. And that really taught me the importance of keeping things simple. Because if I in that case, it was not overengineered, but there have been so many times where I overengineer solutions. And then, a, it's more error prone, and, b, it's so much harder to fix errors when they come up. And now that, like, implementations are just a solution to me, about a year ago, they weren't.

Michael Berk [00:27:30]:
It was like, oh, cool to, like, over engineer stuff. Now if a 1 liner does it versus a 10 liner, I will always go with the 1 liner. And, the sooner you can learn that, the better your code will be.

Ben Wilson [00:27:44]:
Yeah. And the the way you learn that can is 1 of 2 ways when we're talking about, like, the the process of directionally moving towards greatness. 1 way is the hard way, which what you just explained. Like, you you built an abomination, and now you gotta you gotta make you gotta sleep in the bed that you made, and fix it, and it's probably super painful. The fast track way of learning that is to have peer review where somebody who looks at the implementation and just ask you that question. Like, why not 1 API? Like, do we need all of these like, the same implementation in 7 different languages or 7 different implementations? Because if something goes wrong, you gotta fix this everywhere. And how do you test all of that in a an economical fashion?

Michael Berk [00:28:43]:
I don't think it hits as hard unless you've actually gone through the pain of re like, reworking a crappy overengineered solution. Like, yes, someone can say you should probably do it this way and you logically see that. But now I have, like, a visceral hate for overengineered things. Like, it likes it, like, is jarring to look at, and that's super powerful.

Ben Wilson [00:29:05]:
Yeah. It definitely. And you'll never forget that. But it there's also a way to do that in a team environment where, like, a tech lead can get you to realize the same thing by just a single sentence. Or, like, to ask you to make a change that might not be super relevant, but just like, hey. Could you just show me what it would look like to change this 1 thing? Or like, hey, what if we add this feature? How are you gonna add it? How long is it gonna take? And show me the tests. And if you get that request on a review and you're like you you'll have that realization before it gets the point where people are using it and finding, like, crippling bugs in it. Sometimes that question can open your mind, and you'll remember it.

Ben Wilson [00:29:58]:
When you have a mentor that's like, not trying to be a jerk here, but can you, like, do you think that what you built is maintainable? And sometimes that just opens your mind up into realizing maybe there was a simpler way to do this. Like Yeah. Am I doing something dumb here? And you kinda have that realization, like, oops. Yeah. Time to rethink what I'm doing.

Michael Berk [00:30:26]:
Yeah. And just to wrap that thought, greatness isn't making simple code, but it sort of is. Like, if you can I I would argue that sort of greatness is asymptotically finding finding the simplest solution that is a solution? Because simple is easy to maintain, it's typically cheap, and it's easy to build. And so I I mean, that might be going out on a limb. Would you call, like, the simplest solution to a problem greatness, Ben?

Ben Wilson [00:31:01]:
I'd say that the thing that solves a problem in the most maintainable way for me personally is what I go for. That's perfection, quote, unquote. Well, nothing's perfect. Yeah. Nothing in software's perfect. But the the weird thing about software development that I've learned over the years is that it's way easier to make shit code, like, way easier than it is to make simple code. And that might sound stupid, but the amount of thought that is required to go into a well crafted API that meets all of the requirements for extensibility and, most importantly, ease of use and making it You can create an API interface that people never have to look at the docs because it's just intuitive. It just makes sense.

Ben Wilson [00:32:01]:
There's tons of examples in pick a language, look at the core library APIs. And when you're trying to do something, like, man, how do I how do I cast this from a string to an integer? And you look and you're like, oh, there's a there's like a dot to int or a as int open parenthesis, close parenthesis. You know, like, that just intuitively makes sense. And if you do have to look at the docs, it's very easy to find. You understand the things that it does, but most importantly, the things that it doesn't do, And you realize, man, the core language developers are really good at what they did. But if you look at when they implemented that, it had dozens of comments and, you know, arguments about, like, should we do this? Should we do this other thing? Or what if what about this edge case? And you look at how consensus was arrived at and the simplicity that was that was generated from that. You know, like, that's really clever, but it took a lot of work to get to that. And then if you're really interested in how to properly craft, like, that sort of API, but make it as simple as possible for something that is actually complex underneath the covers.

Ben Wilson [00:33:20]:
And you go look at that implementation, and you see the same sort of pattern that was done there. Like, the first implementation was really complicated, and then people are challenging it saying, well, couldn't we simplify this or reduce the scope of this and make it so that we can test this in all the ways we need to test it and ensure quality here and ensure reproducibility and and things are gonna work the way it expected. It takes a lot of effort to do that, and it's really complicated to remove complexity from code. So that's when you get exposed to greatness, when you see people doing stuff like that and are involved in that conversation about the sheer volume of thought and emotive expression sometimes that goes into analyzing that and making it better. You're now being exposed to that greatness. You're absorbing that that perspective and that point of view of other people and saying, there's a reason they're asking for this. And what is that reason? And I'm gonna go analyze why they're saying this. And you start thinking about that.

Ben Wilson [00:34:29]:
And when the next thing you build, you have that in your mind. Like, I need to keep this concise and short and clean.

Michael Berk [00:34:39]:
Do you have an example for when you were exposed to greatness? I mean Ideally, a fun and saucy 1.

Ben Wilson [00:34:54]:
It's pretty much, like, my entire working time in engineering at Databricks is that, you know, every day you get to see that. Every PR that gets filed from your peers where they're working on a new feature, That's not to say that everything that people produce is perfect, but you're seeing directionally people making decisions that make sense for what we're trying to do and thinking through and debating and coming to that consensus where our like, we're building the thing that needs to get built in the way that makes it testable, maintainable, extensible, and highly collaborative. So that's when you see great things being built, and it's like every day is like that. Even if it's a day spent reviewing people's designs, for the things they're gonna work on next quarter and helping out, like, answering people's questions, That collaborative nature is greatness to me. I have many examples over the years of not greatness, at places that I've worked.

Michael Berk [00:36:06]:
Yeah. Like what?

Ben Wilson [00:36:09]:
Somebody, like, trying to kick off a new project, and they effectively go into golem mode where with their project and their code, everything's in their head of the design that they wanna do. They go off and build this abomination and then just try to push it to prod. Or if there's, like, 0 guardrails at the company, nobody's involved. There's no adult in the room ever. They're allowed to just deploy this thing and have it be, like, customer facing. And then the rest of the team is like, what is going like, why is this so broken? Like, it's supposed to work this way, and, like, it's just filled with bugs. Or why do we have to spin up this VM that's, you know, 200 times bigger than any other VM that we're using? What's going on here? You find out, like, oh, it's full of memory leaks, and I was like, all the, like, garbage collection isn't working properly here. And you're like, who built this? And you're like, you look back and you see, it's like, oh, this is 1 1 guy, or this 1 lady who built this, and they just pushed to to master.

Ben Wilson [00:37:21]:
Like, there was no review at all. I've also worked places where there is no concept of main branch because there's no repository. Like, this code that is running somewhere, is pushed to some FTP server and is pulled every hour to be executed as a script. And you look at the script that's being run, and you're like, this code is, like, really bad. There's there's no comments. There's no syntax consistency here. We have, you know, functions defined in here. They're thousands of lines long, and they're doing hundreds of different things.

Ben Wilson [00:38:04]:
There's no way to debug what's going on here. And then you look, and you're like, okay. I see you have a a logger. That's cool. And then there's just a bunch of if statements in here. Like, if logger mode is debug, print this statement. Like, this is your debugging mode? Like, what a nightmare. And I've had to maintain stuff like that and rewrite that from scratch because I'm just like, I'm not if I'm on the hook for maintaining this when I'm on call, yeah, I'm I'm not playing this game.

Michael Berk [00:38:37]:
1 of my favorite movies is ex machina, and the without spoiling it, the there's a super genius that goes into a he's it's like a very actually, a lot of parallels to Google that, like, this guy founded the Google of this fictitious world, and then he goes into this really, really nice, place in like, house in a, like, Alaska looking type of type of environment. And, basically, he huddles down and tries to build artificial intelligence. He General general general intelligence. Yeah. Yeah. AGI. And he, like, gets drunk by himself. He creates an AI to, like, dance with and do some other things with.

Michael Berk [00:39:22]:
He, like, has his own green juicer. He has, like, all of his food delivered and basically doesn't talk to anyone. And there's this stereotype of this dude in a basement just typing away building artificial intelligence, and then suddenly it's there. But what I've been learning is that innovation is highly collaborative, and there's very rarely this genius that goes off and builds something. There are a lot of those stories, but, they're sort of touted as the standard when they're typically not. No. So, Ben, I'm curious. Why do you think that is? Because I've seen it time and time again where it's like, there's 1 dude or 1 girl that went and built this thing, and they're solely responsible for it.

Ben Wilson [00:40:04]:
Because we as as a species have an affinity to believe in the divine. Right? It's very easy to convince humans of of divine beings even if they are human. And if somebody exhibits the properties of something that is superhuman in capabilities, That's a story that we like to hear. You think, like, maybe I could be like that 1 day. Back here in reality, when we're talking about implementation of a sufficiently complex system, My question to you, you're now you know, you've you've got some PRs under your belt in MLflow, and you've seen enough of the source code and see how it works superficially at its service when you're you're interacting with APIs, you know, like, this isn't that complicated. Like, it I can log a run. I can log prams and metrics. And now that you've seen the internals of how some of this stuff works, I don't know.

Ben Wilson [00:41:11]:
I I haven't given you anything to work on it in the back end, like server stuff. But when you're looking at that aspect of the project, do you think 1 person could build all that? Like, the state that it is right now, do you think 1 person could just sit down for, like, 6 months and build all that?

Michael Berk [00:41:30]:
I mean, like, the obvious answer is no. But well, they definitely couldn't do it in a vacuum because it's clear that a lot of the design decisions and product build decisions were informed by customer demands and open source demand. So definitely not without the demand. And then could someone, like, settle on this architecture? Probably. I don't I mean, you were trying to get me to say no. Right?

Ben Wilson [00:41:59]:
I don't know. I just wanna know you like, get your take on it.

Michael Berk [00:42:04]:
It definitely wouldn't happen. Could it happen? Maybe.

Ben Wilson [00:42:11]:
So if it were 1 person doing it and knowing that what you just said, the direction of feature implementation and, like, the state that it's in now with however many freaking millions of downloads a week it has now. If you did that in open source, right, you just pushed it to open source, you got a whole bunch of people interested in the project, and they started filing issues for, like, feature requests. Do you think 1 person could handle all that? No. Definitely not. Yeah. Because, like, what would happen in that situation where it's 1 person interfacing effectively with tens of thousands of of users who have all these ideas? You would have to be the gatekeeper, and probably make a lot of people annoyed. But if you didn't wanna make anybody annoyed, you would build all the features. Right? Yeah.

Ben Wilson [00:43:15]:
And then if if that was the case, the product would definitely not feel like it does right now. If every request came in, we just built. We don't do that. Right? Could we? Maybe. It wouldn't work anymore. It it would be fundamentally broken, or it would be unmaintainable. Right? If you built everything that everybody ever asked for, you would be left with the state of a code base where you're like, I can't even test this. Like, all of these integrations and all of these features, it's so complex that it's it's now too difficult to modify anything.

Ben Wilson [00:43:57]:
Even in, like, developer APIs, you can't do anything. So the collaborative effort and a group of people working on something is always going to succeed. And you look at at some of the advanced Gen AI stuff that's being worked on now that the team here at Databricks, you know, the ex Mosaic team, that's not, like, 1 genius working on their own, just building stuff. That's a team of geniuses all working together, collaborating with different teams, with different people, getting feedback, going into previews, testing things out, having people, like, bug bash it. It takes a huge village to make something like that successful. And a lot of very, very smart, very, very humble people that are willing to listen to all that feedback and then filter out what is the most critical thing to work on at what time.

Michael Berk [00:44:55]:
Yeah. And just like providing a proof to the theorem, if you think about the distribution of intelligence of most people, let's say, the smart people in the world, there's very rarely, if ever, going to be 1 person that is better than 5 pretty smart per people. Just, like, distributionally. And so this, like, genius that goes off and builds AI, that just doesn't make sense if you think about the distribution of intelligence. So that's at least how I've been thinking about it. Like, 5 eights is greater than 112.

Ben Wilson [00:45:37]:
Yeah. I mean, it's good to have all of them. I'd rather have all 6. And I'd rather have that that 12 ranking person be somebody who's thinking outside the box and coming up with maybe that first prototype that then everybody else who's willing to to come in and work on that and build it into something amazing with the guidance of that that genius intellect. That's how really awesome products get built. By the way, that's Matej Zaharya, creator of Spark, creator of MLflow. He built prototypes, works on them with a team of people, collaborated, listened to people's feedback, came up with his own ideas, implemented stuff, and then handed it off when we have a a full functioning team that can continue to maintain that, build additional features on it, change things when it it no longer works. So having all of those people collaborate together is a recipe for success.

Ben Wilson [00:46:40]:
That is greatness. It's the people at the right time is what you need as a foundation. And if you've never seen that, and that that's what really brought this conversation to a head the other day. You sent me some code. We're not naming names, but you're like, this is the widely considered to be the finest engineer at this company that I'm interacting with right now. And that and then you pasted some code for me. And I looked at it, and I'm like, yeah. It's it's it's pretty bad.

Ben Wilson [00:47:14]:
But I wasn't super surprised because the nature of that company, like, what business they're in, and they're not in the software business. So they're not gonna be, like, they're not gonna be attracting top talent from anywhere that is involved with software. So to them and to people that are looking at it, who have never worked with those top talent people or been in the room when that stuff is being built to themselves, like, to to that peer group, none of them are wrong if somebody's like this amazing code, and you're gonna have consensus from all of them. They'd be like, yeah. This is really well done. Look look at how how there's, like, comments on every line and and and look at, like, this clever thing that they did with the list comprehension. This is super cool. And you show that to somebody who's building legit software at a big tech company.

Ben Wilson [00:48:11]:
They look at that, and they're like, they're looking at other things. Like, what's with the naming convention here? I don't even know what this function does by looking at the name. What's with all these acronyms here? Like, this makes no sense. I can't read this code. I have to now look at every implementation line to understand what is going on. And then you look at the length of it. You know, like, hang on. There's, like, 7 different things happening here that have nothing to do with 1 another.

Ben Wilson [00:48:41]:
This is more like a public API that should be calling private APIs. But instead of that, it's just 1 function that has all the logic in it. So I can't test any of these things to verify what could go wrong. I can only test the output of this main function. Yeah. So when I need to debug this, I now have to come in and edit code to debug, which is the worst. Yeah.

Michael Berk [00:49:12]:
And you alluded to something that's really, really important, which is it's a culture thing. It's not that the guy is not smart. It's not that the guy or girl or it could be a dog for this case. It's not that it's it's bad intentions or anything like that. It's the lack of the intern saying, I don't get it. Can you make this simpler? It's the lack of the manager saying, is that maintainable? It's the lack of this collaboration and sort of, greatness culture that as you alluded to just isn't part of the business's core competency. And, yeah, yeah, the code works the first time. Is it impossible to read? Is it impossible to extend? Is it error prone? Yes.

Michael Berk [00:49:54]:
But that's not reflection of the person. It's a reflection of the culture. And, because they're senior, they should probably have more stakes in influencing the culture. But if you've never been exposed to good code, I don't know.

Ben Wilson [00:50:09]:
I mean, it is that person's fault. They wrote it. They should know better, for being a Probably. For having the job title that they do, they should know better, because everything that was in that code sample has been written about ad nauseam in books about, like, first principle software development. Like, here's rules that have been learned over the last 50 years of writing code. It's somebody with hubris who legitimately believes that they're what they're producing is good, that they don't even need to get a peer review from it. They just wrote it, checked it in, and they're like, good to go. I'm a principal engineer or whatever their job title is.

Ben Wilson [00:50:58]:
Something that doesn't equate to their capabilities. So it's a failing of, you're right, it's a failing of the organization in their hiring practices. Nobody screened the person that they brought in at that level or when doing promotions didn't screen that person's capabilities properly to say, is this person capable of operating at a level commensurate with their job title. And for these companies that are not tech related, but they're giving out large salaries to people that are technically not qualified to do what they're doing. And then they're wondering why they have to hire consultants to come and help them. Because the people that they hired and gave these lofty job titles to don't really know what they're doing.

Michael Berk [00:51:47]:
Yeah. Yeah. Yes. 100%. I was about to go into some more complaining, but I feel like

Ben Wilson [00:51:55]:
we should

Michael Berk [00:51:55]:
be anonymous.

Ben Wilson [00:51:56]:
But, I mean, I've also interfaced with people at companies like that that they're tech adjacent or that they Mhmm. Like, don't even like, tech is not their core competency at all. And you end up interacting with a team, and you're like, man, everybody here is they're really good. Like, they're building maintainable stuff. They have the right way of thinking of it. And, you know, like, I would never would have involved in, like, data science or ML or in software engineering. I never would have thought to apply to a company like this, because what they do is has nothing to do with tech. Right.

Ben Wilson [00:52:38]:
But then there's this 1 person, usually like a CTO or a VP, who has took it upon themselves to enforce that culture at the company. And by the way, that's the job of a CTO. That's their whole reason for existence is to make their technical organization more competent, more qualified, better at what they do, more, like, have a higher velocity. And unfortunately, a lot of CTOs get hired in those positions because they used to be an engineer somewhere and have some sort of street cred, and then somebody some board was like, yeah. You seem competent. We'll we'll make you the CTO. And you look at them, you're like, yeah. But the person doesn't know it.

Ben Wilson [00:53:23]:
Like, they may know a lot about software engineering or computer science or data science or something, but they have no idea how to build a team or how to get make a team get better over time. So they're not a leader. Unfortunately, that's what that role is for. Right. Inspire your team. Make them better.

Michael Berk [00:53:45]:
Mhmm. Alright. Let's conclude with, an actionable tip for people to either learn about greatness or start implementing greatness at their own organizations. Do you have 1?

Ben Wilson [00:54:00]:
Read books about about, what it takes at those those vaunted tech companies. What did they learn over time? Many, many books have been written about this topic. Learn about what it was like, and you can some of the stuff you can read on people's personal blogs and stuff online. I've read stuff from, like, the early days of Yahoo. Now people kinda like, Yahoo, don't they do, like, news and stock market stuff? Yahoo back in 2004, that was the Internet. Like, that was your gateway to the Internet. Somebody drank their milkshake, by the name of Google, but during those that, you know, first Wait.

Michael Berk [00:54:51]:
Was that a There Will Be Blood reference? Yes. Hell, yeah. That's my favorite

Ben Wilson [00:54:56]:
movie. I drink your milk drink. When you look at the first, you know, 12 to 15 years of of Yahoo, that's where the innovators were. They were building crazy stuff. They invented effectively the modern distributed computing stack. They came up with Hadoop. Right? Mhmm. At which open the open the gates for all modern data usage and all modern application of machine learning and data science.

Ben Wilson [00:55:26]:
Without that, it would not have proliferated like it has. And lots of geniuses work there, and they worked during that innovative period and did some amazing stuff. Once a competitor came along who was doing what they did better, faster, cheaper, more reliably, the brain trust left. We were like, once you start losing that that core group of people who were the innovators, they all go to other places and innovate yet again because it's all about the people. It's not about the company or the or the tech. When that happens, yeah, the company just becomes nothing. You know? Yeah. It's sad, but it happens all the time in tech.

Ben Wilson [00:56:13]:
So I would learn about those stories and what it was that they focused on. What, what was important to them at that time? Who did they wanna be around? How were they thinking about building product And what skills did they develop as a team? Wouldn't pay attention to the tech stack. It's irrelevant. It changes all the time anyway. And for a team that's I've seen teams go off the rails because they're like, our main focus for this year for continuing training is to become experts in Scala. It's like, yeah. Cool. You know, if you're writing in Scala, get better at it.

Ben Wilson [00:56:54]:
You know? Learn how to write more effective tests. Learn how to do, you know, an entire back end in that, if that's your thing. Nobody cares. It's not gonna teach you, like, fundamentally how to become a better organization. But if it's more like, what are our processes that we are doing? How do we approach product development? How do we approach improvements to things that already exist? And what are where are our skill gaps? Like, who do we not have or what expert do we not have? How do we get that person? How do we build an environment where everybody is super pumped and excited to be working on this stuff and identifying the places where it's like, yeah, everybody hates this thing or everybody's annoyed by this thing because that's how to how to kill that culture is all the great people leaving before the more junior people have absorbed enough of their knowledge and experience to become great themselves.

Michael Berk [00:57:53]:
Got it. So sort of the 1 liner is give a shit

Ben Wilson [00:57:57]:
and try? And learn from the past. Learn from others.

Michael Berk [00:58:01]:
Mhmm.

Ben Wilson [00:58:02]:
Yeah. There's so many postmortems out there about entire organizations and about tech teams. People have written online, YouTube videos of people talking about it. You know, you can find a whole a lot of information about what works and what doesn't work from a a team perspective or, you know, an engineering organization perspective, give you insights into where should we be focusing our time. Right.

Michael Berk [00:58:28]:
Alright. My tip is simple. If you are looking to try stuff out, find people to try it out with. Typically, there's a few people on your team that are gonna be more growth oriented, more interested in experimentation, more interested in learning. Find them and try it out. And then if you have a small group that has a tried and true method, you can expand upon it. But trying to work in isolation is really tough.

Ben Wilson [00:58:54]:
It's a recipe for disaster. There's no worse echo chamber than the 1 inside your own head.

Michael Berk [00:58:59]:
Yeah. Exactly. And also for creating momentum to change culture, 1 person can rarely do that unless you're the the manager.

Ben Wilson [00:59:09]:
So Yeah. Yeah. I mean, I have seen inmates running the asylum work before. It's pretty rare. But if a whole group of people, all, you know, unanimously agree to directionally changing how things are done, you can convince management.

Michael Berk [00:59:25]:
Exactly. But 1 person almost never can unless they are the manager. Right. Yep. Cool. Couple more things. Timing of a company is important, when you're trying to learn about greatness. It's arguably more important than the company name itself.

Michael Berk [00:59:42]:
And then also greatness is implicit in the culture. There's rarely a, like, to do list. It's something that you sort of absorb by being there. Or there's a to do list to some degree, though. But, typically, you can absorb a lot of the implicit ways of going about problem solving a lot faster and a lot more efficiently than reading it in a book.

Ben Wilson [01:00:02]:
Mhmm. Alright. We're right at

Michael Berk [01:00:05]:
the hour, Mark. Perfect. Anything else, Ben?

Ben Wilson [01:00:07]:
No. This was fun. This was a good talk.

Michael Berk [01:00:10]:
Yeah. Cool. Well, until next time, it's been Michael Burke, and my cohost Ben Wilson. Have a good day, everyone.

Ben Wilson [01:00:16]:
We'll catch you next time.