Michael and Ben share their insights on being called in to fix issues in production systems at the last minute. They stress the importance of asking questions to understand the context and navigate the political landscape, and caution against providing half-baked solutions. They also discuss the significance of understanding project goals, documenting decision-making processes, and providing guidance to the team to avoid building unnecessary and difficult-to-maintain systems. Stay tuned as they share their experiences and valuable advice for navigating complex projects and delivering meaningful solutions.

Michael Berk [00:00:09]:
Welcome back to another episode of Adventures in Machine Learning. I'm one of your hosts, Michael Burke, and I do data engineering and machine learning at Databricks. And I'm joined by my wonderful cohost.

Ben Wilson [00:00:19]:
Ben Wilson. I do quarterly planning, at Databricks.

Michael Berk [00:00:24]:
And today, we are gonna be talking about some things that I've been running to in the field once again. We really enjoy these panelist episodes because we get to sort of take a step back, reflect on our current day to day. I get to complain, and Ben gets to provide wisdom on how I should proceed. So everybody wins, and hopefully you, the listener, are winning as well because these are real things that we face in production systems. Today, we're gonna be talking about being brought in on the 11th hour to a dumpster fire. Let's say that you are a consultant and you are tasked with resolving an issue that, is just asinine. Like, there's so much tech debt. There's so much organizational burden via this implementation, and your job is to basically make it work.

Michael Berk [00:01:14]:
That's what you've been tasked with. But the real solution is not doing this at all. So before we kick it off with a a real use case, Ben, I was wondering if you could clarify how you think about navigating the political landscape when you're brought in to fix something even though you know proceeding with a fix is just not the solution. The solution is taking a step back and rebuilding somewhat from scratch.

Ben Wilson [00:01:40]:
In a word, carefully. That that basically sums up my last couple of years, when I was in the field that Databricks was being brought in at 11:59 PM before the the midnight release. And people realize, oh, no. This is totally broken. Let's let's get somebody from the vendor who knows what the heck they're doing to tell us how to fix it. And the best way is to ask a lot of questions. Be annoying. Not not ignorant questions about, like, why are you doing this? It's more like asking questions about the project, and you're gonna get you're always gonna and this is applicable to if you're, you know, a new hire at a company working in engineering or you're starting a new role somewhere, or you're just coming in to work on a project for the first time that you're not the one involved in the design or discussion of, like, why things are are being built.

Ben Wilson [00:02:49]:
It's always good to ask questions to understand context, because maybe there were legitimate reasons why these decisions were made. In my experience, if you're being brought in to fix stuff, usually, either nobody ever did that in the first place. Like, the no design was ever done. It was never discussed or thought about. It's a culmination of a series of cripplingly bad ignorant decisions that were made because they seem like a good idea at the time. And if you compound enough of those over a long enough period of time and get a culture within the company and a promised, you know, suite of functionality and features associated with something you're working on that everybody thinks that they need, you're now in a place where you don't even realize what you were doing or why this was being built. You just know people need it, and you gotta make it happen. So asking questions about what is going on here, why do people need these features, And where are the original designs for this? Asking questions like that will ex either it should expose, you know, what you're dealing with.

Ben Wilson [00:04:12]:
Like, was there a design? Did anybody put any thought into this? Or is has this departed so far from the original design that there's no way that this would have worked in the first place?

Michael Berk [00:04:26]:
And it sounds like these questions are Socratic in nature, so they'll get the people to reflect on their decisions. And it also allows you to collect information. How do you balance asking too many questions with too few questions?

Ben Wilson [00:04:42]:
Oh, man. I I don't really I I never cared if I was making people annoyed. I am, by nature, somewhat annoying when I start asking questions, sometimes intentionally so, because it's self serving because it gives me context not only into, usually, I don't care about why they, like, decided things on a project. If you're at that point, you're being called in so late on on onto something, you already know that this was a bad idea. Like, whatever they're trying to do is it's it wasn't simple enough for them to do themselves, so they need an air quote expert. That means you you developed your way into a into a a corner, and you've bitten off more than you can chew, which is never good. So those questions are to to read the room, basically. So you're trying to understand fundamentally what are the personalities that you're dealing with.

Ben Wilson [00:05:49]:
Are people open about answering these questions? Maybe they'll be open to discussing alternatives. Are they are they polite and nice and constructive with their answers, and honest? If there's a lot of, you know what? I don't know why we did this and but we should talk about that and figure that out All the way to the other extreme, which is just complete hostility, you know, people raising their voices, getting really annoyed with you. That lets you know who you're dealing with, whether it's even worth your time to attempt to do this. Or do you need to, you know, get all of those people that are in the room out of the room and start talking to management? Is management clueless about what's going on? Then it start then it's time to escalate to executive management and say, listen. This is not gonna work. I don't know what your people told you, but nobody seems to wanna tell me what we're doing here or why we're doing it. And I don't think that they know, and that's scary. So in in the interest of your business and your budget, let's have a a little bit of a work stop here for a couple hours and get to the root of what's going on here.

Ben Wilson [00:07:09]:
So there's always a a way to navigate navigate to a point where you're going to get some answers that are productive. You may make lifelong enemies with people that are in that initial first meeting room. People may hate you and never wanna see you again. You could get your rental car keyed on the parking lot, has happened to me, as a consultant. Oh, yeah. But so, you know, things can happen like that, but it's important not to not to shy away from that because it really is helping out that team and helping out the people that wanna do a good job and and want to have something that they can be proud of and maybe is reduced in features, but it's something that they can maintain and not want to look for the the nearest exit, to find another job. And the company will be successful. I mean, that's that should be the goal of any consultant, really, in my opinion.

Michael Berk [00:08:21]:
Right. Yeah. One one anecdote from my recent projects, I was working with an extremely hostile, customer, and I was working on something that was very out of my wheelhouse. And they basically provided some unreasonable requests that are just not possible to deliver without, like, building a second Databricks. And so because it was outside of my wheelhouse, I took a step back and talked to the relevant people within Databricks to come in at a higher level and have this sort of heart to heart with upper management and executives. And it worked out really well. They seemed like they connected really well, communicated really well, and brought back a reasonable statement of work that was actually I I mean, I still think it's really stupid, but it's deliverable and it's scoped and, we're not being held to unreasonable requirements. So if you're not comfortable in the specific subject matter, sometimes it's a good idea to rely on the people around you and management within your organization to come in and have that card conversation.

Michael Berk [00:09:22]:
But, yeah, as Ben said, sort of BS ing and and providing half baked solutions, well, it might make you friends with the people. It's not the best thing for the organization. So there's definitely a trade off.

Ben Wilson [00:09:37]:
Yeah. Like, you wanna talk about dumpster fires. Coming into a room and not understanding, like, what people's requirements are that are set on paper or what they're asking in a meeting and saying, like, yeah. Yeah. Totally. We can do that Without knowing if it's even possible, you just effectively issued, like, legally binding agreements, to that company. Whether it would uphold in a in a civil court or not is much to be debated. But for the in the eyes of of management, they'll take that as, hey.

Ben Wilson [00:10:19]:
You promised that this works, and it's not possible. So go fix this, or now we have to build this. So it's really dangerous. And, yeah, I've seen it way too many times with people doing stuff like that. I've been in those meetings and had to, you know, raise my hand and be like, hey. I don't think that's possible. We need to go and discuss this and research this. I don't have perfect context on on this functionality, but I I don't feel that this is a a good way to approach this.

Ben Wilson [00:10:55]:
Can we think about some alternatives? Can we brainstorm really quick? I know a lot of people hate that term. But the scariest thing that I've that I have ever seen is going into a room with people who are talking about building like, fixing the final stages of of something that they're about to ship to production, and nobody in the room understands, like, what the initial design was and what the alternatives were. Like, nobody ever even did it. They were just like, this seems like a great idea. Let's do this. And then 2 weeks later, another person chimes in and says, hey. Let's let's build this as well. That'll be cool.

Ben Wilson [00:11:42]:
Fast forward 2 years, you've got this this weird code base that does all the things, and then you allow all your users to use it in all of these different myriad ways. So now when you have to deploy this thing, it's either ludicrously expensive or it's so complicated that all it is is support ticket generation factory. Like, your users are gonna constantly be hitting issues. You're gonna have, like, environmental problems with with, like, where your code is executing. You can't keep track of of state of things. And then you just assume that, you know, your vendor is gonna was built for the thing that you're trying to make them do. And a lot of times, it's not. So it the ignorance part of it and the the lack of planning can really blow up in your face if nobody has clearly defined, like, this is what this can do.

Ben Wilson [00:12:46]:
This is what this must do. This is what this can do, and here's a huge long list of stuff that this will never do. And the reason we're never gonna do these is because it's, you know, ridiculous engineering burden or it's just gonna cost so much money to implement this. We should not we should be upfront with our users to say, we're never gonna do this, or we can't support this use case. Let's let's focus on the things that we can do really well, and let's approach this from a maintainable state.

Michael Berk [00:13:17]:
Sounds like you're speaking from experience. Do you mind providing some anonymized examples of some of the more crazy dumpster fires you saw?

Ben Wilson [00:13:29]:
I mean, the thing that we were talking about before we started recording, I won't even tell you the industry, but let's imagine that supervised learning projects, you know, traditional ML, and people want this thing to go super fast, Like, Sonic. Gotta go fast. So they have this this requirement on SLA that I I don't know where they got it from. They didn't know where they got it from. It it's just something that somebody wrote down, and they need to adhere to it. And it was 30 millisecond response time for model prediction. Cool. Seems completely arbitrary.

Ben Wilson [00:14:13]:
So I asked. I was like, oh, what website or what feature on your website or your mobile app or something is this is this being used on? And I get a bunch of, like, you know, smirks and, you know, smug little looks on people's faces. They're like, we don't we're not deploying this to a website. Like, okay. Where does the 30 milliseconds come from then? Because that's kind of, like, web transaction, you know, latencies that you wanna make sure that, hey. Your your site is super fast and snappy, and there's not, like, you know, some dom element that's taken forever to load on the page in order to get something to to be surfaced. I was like, okay. What is the use case here? They're like, well, it's it's predictions that'll be, you know, generated on demand, and then it'll be written to, you know, a database.

Ben Wilson [00:15:12]:
Okay. That's cool. Once again, I ask, why 30 milliseconds? Like, how do you how do humans or systems interact with this data? They're like, well, it's the database. Yep. That's that's where it goes, but how do humans actually interact with this? And what data are we writing? So they showed me some examples, and and in the data, I noticed, like, oh, there's there's a user ID in this. And I'm like, okay. This seems like web traffic sort of information, or it's something that's being consumed from a website. So I start asking more and more questions, and, eventually, I get to the the root of it.

Ben Wilson [00:15:56]:
And it's it's a dataset that will be used in Tableau for analysts and, you know, business specialists to interact with. And their request came from somebody wanting to, some executive complained enough times about how slow it was when they clicked on the refresh button on Tableau for the latest updates or the system, you know, that they're they're in charge of to show the predictions. And that made me start asking even more questions. Like, with the user ID that's in there, you're storing this in Oracle. And I imagine that the executive that was complaining about this was joining to a bunch of other tables, some sort of BI report that you're generating, and your predictions are embedded in that. And I was like, yeah. Yeah. That's exactly exactly right.

Ben Wilson [00:16:58]:
It's like, well, if the user ID is being used to join, are you are you calculating the index every time that you write data, and are you doing, like, batch transactions to update that? Like, how does all of that set up? You know, like, well, no. We don't we don't create an index. Well, if that's the key that you're using to join on, how is that on an RDBMS system gonna be performing if it's not an indexed value? Like, well, we have this other table that, you know, has the indexes and our our our DBAs maintain that. Like, well, how fast is it to to put that in Tableau? And they're like, well, the that's what he compares, you know, the performance with. That one's super fast, and then our predictions aren't. Like, yeah, because it's a full table scan that you're doing with any join that you do to this. That's why it's slow. Reducing your latency to 30 milliseconds is not gonna change the Tableau performance, and it's just gonna incur all this ridiculous cost.

Ben Wilson [00:18:00]:
So what are we trying to do here? The LDR, by the end of the engagement, it was a cron job. It was doing batch predictions on data, and the company and the executive staff were told that there's gonna be a 12 hour delay in order to save the company $850,000 per year in online ML inference costs by just not having this thing running in real time. Because it wasn't used by anything that needs real time. And that can be that that decision should have been made at the start of the project, you know, 6 months before I ever walked into that that room. And it wasn't because I have this vast knowledge and wisdom of, like, how all this stuff should be done. I just started asking questions saying, like, did we was this designed? Like, what what was what's the point of this entire pipeline? Why was this done? And then you start asking, like, well, where's the document that you use to weigh all of the the pros and cons of proposed approaches here? Your your design document, like, where is it? And then people just looking around at each other and panic. They're like, we we don't do that. We don't have design docs.

Ben Wilson [00:19:25]:
It's like, well, why would you build anything if you don't know what you're building and why you're building it? Well, somebody told us to build it. Okay. So it's on you as a team and specifically talking to the ML engineer lead, in a little side to side, 1 on 1 chat after lunch saying, listen. You need to you need to think about what you're you're asking your team to do and what these major projects you know, design them out, give guidance to your team. Because if you don't do that, they're gonna build all sorts of crazy stuff. And then, unfortunately, they're gonna have to maintain this. So every time this thing breaks, they gotta, you know, fix it or investigate why it broke. And you've never needed to build any of this to begin with.

Ben Wilson [00:20:16]:
So, yeah, I've had dozens of those conversations when I was working in the field.

Michael Berk [00:20:24]:
How do you think about protecting people who made bad decisions in the past? Do you just let the truth speak for itself? Because in that case, someone probably is gonna get fired. Right?

Ben Wilson [00:20:37]:
No. Nobody should get fired for that. I mean, if if nobody knows that you're supposed to do that and nobody from executive management is enforcing that culture, Like, this is how we develop things at this company. If the manager doesn't do like, know that this is how this should be done. Maybe they, you know, were a homegrown manager at that company, and they've never really built production scale software projects before. You can't you shouldn't fault that person for being ignorant. Like, they didn't know. They don't have that experience.

Ben Wilson [00:21:14]:
They didn't have a friend. They they could ask and be like, is this how I should be thinking about, like, ML projects or data science projects? It really should be an opportunity to learn and grow for the people involved and the company in general, like that department, and have those talks with with executives who are in charge of these departments. Be like, hey. This isn't a fault of your people. I mean, technically, it's a fault of the executive for not forcing their people to embrace widely accepted software development practices. There like, many, many, many books have been written on this subject. This is not new. Not just about software.

Ben Wilson [00:22:00]:
This is, like, any sort of business project that you're undertaking in Indian industry. There's ways to approach that. Maybe there's not a a project manager that actually knows what they're doing that's involved in this. Maybe there's not one there at all. So it's an opportunity to say, here's some ways to prevent this in the future. Learn from this. Grow from this. And here's some some tried and true ways to build projects like this.

Ben Wilson [00:22:27]:
It's a methodology that big tech companies and big engineering companies throughout the last, you know, 100 years since the rise of industrialization. Like, people have learned these lessons the hard way. You can learn from them and think about how to tackle projects in the simplest way possible, minimizing complexity, maximizing maintainability, and making it so that people can enjoy and be proud of the things that they built because they solve the problems they're set up to solve. They don't solve anything else, and they're as simple as they possibly can be made.

Michael Berk [00:23:04]:
This this reminds me of the most recent engagement that I wrapped that was very successful. It involved, basically optimizing, quote, unquote, pipelines. That was the stated statement of work for a machine learning, stack. And what it ended up being is, yeah, I optimize all the pipelines, but I was able to delegate a lot of the work to the customers so that they could actually learn how to do it themselves. So alright. This is the Spark UI. This is how you should go about finding if there's spill. If there's spill, that's probably bad.

Michael Berk [00:23:36]:
These are 3 ways you can resolve it. Is there data skew? There's, like, a checklist when you're optimizing a job that you typically wanna do. And so teaching the customer is is very much part of, my role as a resident solutions architect. But also one of the interesting things was their team lead was actually quite technical, probably the most technical on the team, and also was very hands on and interested in what we were doing, looking to improve process And their code was really, really bad, and it was awesome to see that they were open to refactoring and and adding practices and policies within their organization to ensure code quality. So adhering to, like, general design guidelines, ensuring that there isn't commented out code in production environments, just basic things and all that was facilitated via code reviews. And so, yes, you're often given an initial statement of work. And even if you're not a consultant, you're given an initial task. But, typically, there's a lot of opportunity where you can improve other processes.

Michael Berk [00:24:47]:
And especially if you're a consultant, you bring a fresh perspective, and you don't have all that cultural bias of being in an organization for 3 years or whatever. So, there's a lot that can be done in terms of influencing culture and influencing processes, and it's super valuable. It's I would argue it's more valuable than than building the implementation you set out to do in a in a bunch of use cases.

Ben Wilson [00:25:10]:
Definitely. Could not agree more. And I think that the root cause of how a lot of this stuff happens is people in technical positions, particularly with software. And I see it more with with the data science side of the house than I do in the software side of the house. But when somebody has a problem that's presented to them, and I don't know if this is just how people that get into data science and computer science, like, they have a certain skill set and mentality and personality. I like to solve problems. I like puzzles. I like figuring things out.

Ben Wilson [00:25:48]:
So people with that mentality, when they when they hear a problem, you know, our brains, for people that are like this, instantly start thinking about a solution. Like, how would I solve this? How would I fix this this, you know, puzzle? And because nerd brains behave that way, it's very easy to just move directly into that mode of thought and then go off and build something. You know, you think like, oh, this is the problem that I'm given. Mentally, you're thinking about how to solve that, and then you just sit in front of a keyboard and write that implementation. The problem with that approach is and the thing that that people in data science, I think, need to learn something from software engineers, like senior software engineers, people that have learned that that's a bad thing to do. You have to suppress that urge and and sit back and start before you've read a single line of code. Think about the what and the why. At first, it's why.

Ben Wilson [00:27:05]:
Like, why am I being asked to build this or solve this problem? And taking a higher level, you know, just stepping back and thinking about the why of the problem or the project, we'll start uncovering questions that you want to ask to the person who asked you to build it, which starts flushing out requirements, like core requirements of, hey. This is what are what's being asked for. And after that, you can start amassing a, you know, a list of here's the the features, the things that this thing needs to do. And that is completely divorced from, oh, do we have the data for this, or what algorithm I am I gonna use, or even what language should I write this in? Doesn't matter. You know, you think about the project itself at the start and about what is being requested, and you don't trash people's ideas. You don't bully people into submission. You just wanna get everybody's story. And this is you know, project managers do this.

Ben Wilson [00:28:07]:
It's like what they do as a a core competency. But everybody in a technical position, particularly if you're a lead, should be thinking about this, about project work, if they're saying, like, I I need to have as much context as possible so that I can ensure that I'm building what people really wanna use and making sure that this is something that they're actually going to use. Because if you build stuff that they don't ask for or you built something that has all the features because you thought it would be cool or you thought it was a good idea, Your end user is gonna look at that and be like, I don't even know how to use this, or I only use this one thing that I originally asked for. Everything else is just, yeah, it's there, but I don't use it. And that's wasted effort to build a bunch of junk that nobody cares about. And it also increases your burden from on maintenance. You now have to support, you know, making sure that stuff still works over time that nobody's using. That's bad.

Ben Wilson [00:29:08]:
Like, throw the code away. Get rid of it. Delete it. It's not needed. So going through that process and read some books is all I can say, like, on this process. There's a lot of them that have been written about how to figure out what to build. Yeah. And use the methodology that's out there.

Ben Wilson [00:29:29]:
You know, the Moscow method. It's super, super effective. List it down. But then when you build that design document of what it is that you're gonna be working on, it's not a 1 person show or a 2 or 3 person show. It is written in such a way that people at multiple levels of experience as well as multiple context. So it should be written in a way that anybody who has an understanding of that one question of what this thing is should be do you know, like, building, they should be able to read that document and understand it. So it's not an engineering design. It's not filled with a bunch of stupid boxes on a on a slide that's, like, arrows pointing to stuff and, like, here's this service connecting to the service.

Ben Wilson [00:30:21]:
That's a design. That that's an engineering design. That's later. You know, your project design is, can I make it so that my end users can review this and tell me if I'm on the right track? And if they're like, yeah. All those must haves, 100% agree. That's exactly what we want. And one of those cans, can you put that on the must? Because we really want that. Or you get the feedback of, hey.

Ben Wilson [00:30:47]:
4 out of the 12 must haves, we don't care about. You can move those to won't have. Like, they're not important, which is that's the best news you can get. You're like, there's less work. There's less code to maintain. That's less stuff that our team has to do, and we can go do another project.

Michael Berk [00:31:05]:
Right. Yeah. One thing that really struck home for me with what you just said is, well, 2 things actually. 1st is I think a lot of people fail to think long term and fail to think big picture. And this is typical if you're a low level engineer. A, you don't have a lot of practice thinking in that way, and then, b, you don't have a lot of context. So when we're thinking about what to build, alright, if I have a sprint and I just wanna maximize my output in that sprint, that is very, very different from wanting to maximize the team velocity and productivity over a 5 year period. If you have to think about maintenance, one sprint doesn't impact maintenance that much, but 5 years will dramatically define how your design looks.

Michael Berk [00:31:50]:
So, thinking long term and having context when you're making these decisions is super important. And typically, you have to be senior to do that. And by senior, I mean, have a lot of people in the room have a lot of context. And having experience making these decisions is also helpful. And then the second thing so I love learning. And throughout my life, I've taken a bunch of classes and read a bunch of books. And one of the more influential classes that I ever took was by this woman named Leila Acarglu, acarglu. Long story short, she has this, school and, basically rebranded and repurposed a lot of these decision criteria that we've been talking about.

Michael Berk [00:32:36]:
She calls it the disruptive design method. But one thing that Ben hinted at was people jumping to conclusions too quickly. So she outlines these three phases of mining, landscaping, and building. Fancy terms, but, essentially, what it means is mining is stay in the space where you're just gathering facts. Landscaping is the space where you start mapping information between facts and and finding relationships. And then 3rd, you build. And, typically, more than, like, 70% of your time is spent in mining and landscaping because you really get a And once you have a very strong lay of the land, then you can go about prototyping and building things out. And this has been absolutely game changing for me.

Michael Berk [00:33:25]:
The amount of times that I've been in a room and seen someone provide a problem and have people start providing solutions without really knowing what the problem is, what's the source, why the problem arises, all those things, they lead to really crappy solutions. Sometimes you'll get lucky and it'll save you a little bit of time, but 90% of the time you'll have to refactor. And again, over a long term, it adds the amount of work that you have to produce. So, staying in the in the problem space and not even letting your mind enter the solution space is really, really valuable. And tangibly, like, if solutions will come up, write it down. Don't spend any time on it. Go back to the problem space. Ask questions.

Michael Berk [00:34:04]:
Look for relationships. Look for more facts. Save, prototyping and building for the last stage once you really understand the problem.

Ben Wilson [00:34:11]:
Yeah. And one final thing before we get a wrap up for today's session. A bit of advice for enterprise people. You work for a company that has more than 5 foot 1,000 employees. You are enterprise, not Star Trek Variety, but you're a big company. And in my experience, these huge companies love to do decision by committee, which drives me nuts. It always used to when I was in these, like, oh, we're gonna have a brainstorming meeting about this one project instead of, like, brainstorming for a quarter or the next year of things that we could work on. That works in a group environment.

Ben Wilson [00:34:51]:
But if you're talking about requirements gathering for a a discrete project, that should not be ever done with a room full of humans. It will go nowhere, and all you'll get is a, you know, a bunch of, like, Post it notes on the wall or a whiteboard full full of just a bunch of crazy ideas. It works so much more efficiently if you take one person and give them the task of go figure this out, ask anybody you need of opinions and questions, and then send this document out to everybody. So that gives guardrails to something. So if the page is blank and you have a group of people that all are coming up with ideas, you're gonna waste so much time and energy doing that. You could be in that room for days of, like, oh, we're gonna have another brainstorming session about this project tomorrow, and we're gonna do this for 2 weeks. What a waste of time. And everybody that's in that room could be doing a design document on a project independently, and you'd have, you know, 15 design docs out of the time that it took you to build 1.

Ben Wilson [00:36:04]:
That's that one document's gonna be crap, by the way. So the principle is that one person who does that first draft, it doesn't mean that they they nail it the first time. Doesn't mean like, hey. This is perfect. We can now go build this. It's they come up with a a set of ideas that they could be good. They could be terrible, but it gives everybody guardrails and context within the confines of some work that's already been done so that you can correct it, add to it, or remove stuff from it instead of just a blank slate where anybody can add anything. So I highly advise people in, in project development engineering type stuff.

Ben Wilson [00:36:51]:
Try to do that. Get a senior person to to try it out first if you don't do stuff like that. And then have everybody on the team comment, and then have your your users comment.

Michael Berk [00:37:03]:
Nice. Cool. Alright. Well, I'll wrap. We talked about how to sort of manage broken infrastructure, especially when you're brought in as an external consultant into a politically charged situation with a deadline. Some tips for navigating this is asking a lot of Socratic questions, both for yourself to gather information, but also to get others to think about the first principles that they've leveraged to to proceed, and also understanding the political landscape. Is someone under deadline? Is their manager really mad at them? Are is this just not a collaborative environment? Is it very collaborative? That helps you proceed from a a social standpoint. When thinking about how to design and develop things, think long term.

Michael Berk [00:37:42]:
Don't just think in terms of a sprint, and use the Moscow method, which is must, should, could, and will not have. And then don't do decision by committee. Typically assign 1 person to be in charge of requirements gathering and putting together a design doc and then review by committee. Anything else?

Ben Wilson [00:37:58]:
No, it's

Michael Berk [00:37:59]:
a good wrap up. Cool. Until next time, it's been Michael Burke and my co host. Ben Olson. Have a good day, everyone. See you next time.