Combating Burnout in Machine Learning: Strategies for Balance and Collaboration - ML 178

In this episode, Ben and Michael explore burnout, particularly in machine learning and data science. They highlight that burnout stems from exhaustion, cynicism, and inefficiency and can be caused by repetitive tasks, overwhelming workloads, or being in the wrong role. They also tackle strategies to combat burnout, including collaborating with others, mentoring, shifting focus between tasks, and hiring more people to distribute the workload. A key takeaway is the importance of knowledge sharing and not hoarding tasks for job security, as this can lead to burnout and inefficiency. They also discuss managing burnout and its components, particularly exhaustion, cynicism, and inefficiency, through personal experiences. Finally, they talk about how burnout can lead to inefficiency and physical manifestations, like a lack of motivation to engage in activities outside of work.

Show Notes

In this episode, Ben and Michael explore burnout, particularly in machine learning and data science. They highlight that burnout stems from exhaustion, cynicism, and inefficiency and can be caused by repetitive tasks, overwhelming workloads, or being in the wrong role. They also tackle strategies to combat burnout, including collaborating with others, mentoring, shifting focus between tasks, and hiring more people to distribute the workload. A key takeaway is the importance of knowledge sharing and not hoarding tasks for job security, as this can lead to burnout and inefficiency. They also discuss managing burnout and its components, particularly exhaustion, cynicism, and inefficiency, through personal experiences. Finally, they talk about how burnout can lead to inefficiency and physical manifestations, like a lack of motivation to engage in activities outside of work.

Socials 

Transcript


Welcome back to another episode of Adventures in Machine Learning. I'm one of your hosts, Michael Burke, and I do data engineering and machine learning at Databricks. And I'm joined by my eloquent, beautiful, and lovely cohost. Ben Wilson. I write GitHub Actions workflows for integration tests at Databricks.

Nice. Today, we are going to be doing a panelist episode, and we're gonna be talking about a very interesting topic that's near and dear to both of our hearts. It's called burnout. It's a cliche word, and there's a lot of really, really great advice, on the Internet. Ben and I have been reviewing it in-depth for the past 30 minutes.

So at a high level, it's just gonna be giving tips and tricks and reviewing some stuff that we found on the Internet. So I'm gonna start us off with actually the best piece of advice that I've seen ever on Reddit. First one is realize that today can be your last day here on Earth. We all die someday. Wake up every day and look at yourself in the mirror and say, what would I do if today was my last day on Earth?

One day you will actually die. So much the best of it. What are your thoughts? I mean, they're not wrong. That's definitely a true statement that can be applied to everything in existence.

Right? I always kind of, like, inwardly cringe when I hear an answer like that of, like, hey. And the overused cliche term of, like, carpi diem. Like, you gotta just seize the day every day. It's like, that's very easy to say from an I ivory tower when you're not spending your day in the great capitalist machine that is modern society where you need to make a paycheck and you happen to be somewhat good or maybe just moderately good at something that you've been doing for a while that you're just so tired of.

It's really hard to look on the bright side if your entire daily, like, work life is just dealing with frustration. Yeah. So you don't look at yourself in the mirror every day, give yourself a pep talk, and then go thank the lord that you can write code. I don't believe I've ever done that in my entire life. Okay.

Maybe I should. May maybe I should start doing that. No. No. No.

Okay. So directly into my own eyes and, like, point directly at myself and be like, you need to enjoy today. Seize the moment. Yeah. So, let's go back, to defining burnout.

We looked at a Harvard Business Review article, and it cited 3 components. The first is exhaustion. The second is cynicism, and then the third is inefficiency. Is that genuinely how you think about burnout? And if so, anything to elaborate on?

I think there's different burnouts can happen depending on your core personality, the job you're doing, and the level of complexity. And I've experienced burnout in multiple different industries for different reasons, and I think everybody is susceptible to it to some certain degree. And there's just sort of certain jobs that aren't for everyone. But then those jobs that might not be for you, there's somebody who's, like, almost perfect for it, and that's, like, their ideal job. And but if you're for the frustration aspect of burnout, being in the wrong role is gonna burn you out, and you're gonna hate it eventually.

Like, no matter how long it takes or how long you're doing it, eventually, you're gonna burn out if if you don't really like the job. But then there's another special type of burnout, which is you're in a great job that you really love and you thrive at. It's just the velocity of that and the amount of work and the consistent, like, never ending slog of doing something even if you really enjoy it and you're really good at it. There's only so much you can do. And when you hit a breaking point of, like, okay, I I love this.

I'm excited about doing this, but it feels like my brain isn't working as well as it used to. And that's a sign of, like, the the high level tech burnout that I've seen. It's not just in software. I've seen that in, like, other engineering roles I've had in the past. Got it.

So this is a machine learning podcast. Typically, in the machine learning space are those types of roles, whether it be internal facing or external facing. What are some common symptoms that you see of burnout? I can I can tell you what I experienced, like, why I'm no longer a data scientist? Yeah.

I mean, it's like the first time that you do any sort of applied ML for a new novel use case or at least when I was doing stuff. I would get excited about it. Like, this is something cool and something new to learn, and I get to think through and and research what other people have done and and kind of apply it to the business and talk to a bunch of people and create a product that is hopefully solving some problems and then improve that over time. And at a certain point, when you make something successful, other people are gonna ask for a version of it for them, and it may work or it may not. But if you build something for, like let's say we're we're at a company and we're helping out the sales team, and this company has, within the sales department, 40 different main product lines.

And your first project is working with the top revenue generating product line, and you build something that makes their job like, build some sort of automation or predictive modeling that makes their job so much more effective, and sales goes up, and it's a huge success. What's gonna happen is you probably wanna work on something different, like a new area of, like, oh, I helped that team out. Super cool. And now I'm gonna go work with, like, the fraud team or something, or I'm gonna work with inventory team, and build a novel implementation that's gonna help those teams out. But in reality, what happens is your next 3 years is building individual implementations for each of the separate sales teams.

And after, like, the 10th or 12th one of them, you're just like, I don't wanna do this anymore. This is so boring. But I can't abstract this to make one meta model that solves the like, everything for all of the teams. It just doesn't work. I need to make bespoke implementations because business logic is different, and maybe the products and and what you're trying to sell is different.

So it becomes very boring. That's super interesting. I have follow-up questions unrelated to burnout, but we'll try to stick on the burnout path for now. What techniques did you use to avoid burnout in that space, or is it just inevitable? I tried to be clever and do the abstraction stuff.

I was like, oh, I bet I could figure out a way to kind of automate having to make more of these. And I had convinced myself that it would it would work. And I've I spent a bunch of time delving into, okay, how am I gonna build a framework for this? And I'm gonna write all that code only to find out it's, like, what a waste of time. Like, it would have been faster for me to just do an like, in the amount of time that I wasted doing that, I could have done 3 of the implementations.

So I stopped doing that and just started banging out the implementations as fast as I could. But there's a point at which you're like, okay. I can't do this alone. There's too many requests coming in. So you go to your boss and say, like, we need to hire more people.

Let's get some people that are familiar with this type of modeling. You go hire other people, but then you're kind of signing them up for the same burnout. So you it's all about balancing, like, making sure that one person isn't working on the same sort of project for too long and making sure that they have interesting novel things to work on. So I would canvas other teams that we could talk to while maybe I spend my morning prepping for a model training run of a new idea for, you know, some hypothesis that I had that would solve this modeling problem for this business unit. And then after lunch or during lunch, I'd, like, sit and talk with a completely different department and wanna get their ideas.

So my brain has kind of shifted focus to think about something that's new. And it kinda gives my brain a break from the tedium of, okay, how many iterations do I need to do for this feature engineering test that I'm gonna do after lunch? And, you know, creating a to do list in my head and then on paper of all the things I need to do that are just, like, checking boxes. And I'm okay. Ran this experiment, then I ran this one, then I ran this one.

Stuff like that just bores me. Like, doing that day in, day out for weeks weeks weeks on end. Interesting. I I thought I was teeing you up for a different, answer. The answer that I do now, and it's because you told me how to do it or, like, to do it, is teach other people to do it.

Like Mhmm. Adding value to your org can be done in a variety of different ways. And if you're this I don't know what the word is, but if you're the only person that can implement something that adds value in this space, yeah, you have a ton of job security, but you're really not disseminating the knowledge and adding as much value for the org as you could be. Instead, the higher ROI thing is teaching other people to be like mini use in this space or have some of the skill sets so that they can expand and scale. And then, you are, by definition, not doing the repetitive task.

You are teaching how to do the repetitive task, which is a bit of a break. That said, that then becomes repetitive. I mean, it can. I don't I never get tired of doing that, though. I don't know why.

But it's all about it's all about, like, how many humans you have available. So the story I was telling, that was just me as the only data scientist at a company doing that, so there wasn't anybody to, like, pass this off. But when hiring more people, yeah, the first thing is, like, not I would never do something, and I'm sure you can confirm from you and I working together. I never just take an implementation and be like, hey, dude. Go build this, but for this other thing.

It's more like, hey. Here's the problem statement that we need. What do you come up with? Because I can learn something from the other person as well. They might think of this think of a solution that is completely different than what I was thinking of and could subvert my bias about what an optimal solution is.

So I see all of those interactions when teaching somebody or I don't really see it as teaching. I see it as just collaborating. And I may have more context about the problem than they do, but they're gonna approach it in a a way that my brain wasn't thinking of, and I love that experience. That's why I love mentoring. I love that, like, back and forth.

We're seeing what somebody comes up with and be like, that was super clever. I'm using that in the future sort of thing. Mhmm. I think people who don't do that, who assume that they know everything and don't ever wanna pass on information, or they're like, I'm gonna intentionally make this complicated. I'm the only one that can be called to fix this.

That's the dumbest thing you can do as a professional. Like, the most stupid thing. Because at some point, you're gonna get a manager who actually knows what they're doing that's gonna come in and look at what you're doing, and they feel like, okay. I see what you did. Now make it so that everybody else can contribute to this.

And now you've just created mountains of work for yourself and are probably just gonna quit because it's too much. And you've also you're being called out on your BS. Like, oh, job security, we don't play that game here, and you should never approach problems like that. So force like, forcibly doing the mentoring thing and and collaboration thing, that ensures that you're not building something that is unmaintainable by other people, because you're getting clear signal immediately. Like, did we come up with something that other people can just build on top of?

Get tested out. You're working with your mentee and asking them to build a new integration for it, and you're like, yep. That was successful, and they did it in, like, 2 days, and this is awesome. So yeah. Now they can go and get somebody else to do another couple of those as well.

Yeah. It's an interesting chain of knowledge transfer. How much do you think is lost at each level of the tree in terms of skill, competence, and knowledge? I think there are there are losses that happen, but there's also gains that don't happen that couldn't retrain trace back to that original person. So if you were the first person who implemented something and then pass it on to one person, they pass it on to another, and now they're doing implementations.

When you get further away from the original implementation and people start working together and building something that might deviate from the original plan, If everybody's sort of checking each other's work and making sure that they're doing it thoroughly and properly, after a certain number of time and certain number of people touching it, it's almost inevitable that the product is gonna become better than it ever was originally. So it's gonna surpass the abilities of the person who initially developed the initial implementation. I can't even count how many times that's happened with stuff I built, and I love it when I see it. Just like, wow. This project is so much better with all of these other people working on it.

And I learned a bunch of stuff along the way by seeing how they've changed what I did, and it delights me when somebody just, like, deletes an entire module that I wrote and rewrites it. I'm like, now that's properly done. They come with fresh eyes, fresh perspective. Maybe they're a better software engineer, or maybe they're they're just able to see the whole picture now that all of the other components have been built into it that I didn't have that vision when building that first thing. But I love seeing that.

It's it's awesome. Yeah. And it's interesting to think about how the skill set this is this is really going into the weeds, but I think it might be cool. So the way that I think about a lot of things is from an ecology perspective just because I've liked nature and studied it in school, etcetera. And when you have a food chain, you have, producers, so like grass, trees, etcetera.

They don't actually need another living organism to produce energy. And then each step of the food chain, you lose 90% of the energy that was originally in the prior level. Yeah. That said, each of the higher levels, so primary consumer, secondary consumer, all the way up to, like, sharks, tigers, bears, there's really cool adaptations and sort of modifications of that base energy. So tying it back to knowledge, you start off with, hey.

How do I build a forecasting model for a sales use case or whatever the example is? Mhmm. That exact knowledge might you might lose a lot of components of that as you go down the food chain or down the knowledge chain, but the creativity and the, sort of modification of that starting point via each human's own customization leads to really, really cool results. Yeah. And I'm fine with being grass.

That is totally fine. So that I can see a a bear catch a fish eventually, right, in midair as it's jumping through a stream. And that's kinda how I see it is that sort of ex Do you enjoy being grass? I think the first you asked that of any professional software engineer, they'd say yes. Anybody who's, like, building framework infrastructure code that other people are using and improving and for definitely any open source developer who's out there who has a who has the goal of building a community that other people can contribute to.

Yeah. Like, people are everyone that I've met, including my own self, love that. Hurt. Okay. Cool.

Just curious. So going back to the Harvard Business Review, we've talked about the the components. This can take a variety of forms. I wanna chat about cynicism. How do you manage that?

I did not manage that well when I was younger. And what older me now realizes looking back, and I realized this many, many years ago, was I was just doing the wrong work. And I was doing things, not like doing bad work intentionally or anything, but I was when I would get really bored with things and I would project outward or think about everything but myself. Like, man, this job sucks, and I hate doing these things. And what really changed it for me was, like, a couple of people in management that I had in a couple of really good companies that I worked for who kind of changed my perspective on it more by demonstrating how they do things.

And I'd sit there and kinda watch them, like, how can you enjoy like, I know how good you are and how smart you are. How can you enjoy doing this menial task? And kinda watch them and be like, I don't know if they enjoy it, but they enjoy how well they do it. And they thoroughly enjoy it more when somebody gets value out of what they've what they're producing. And by shifting my mindset to that to be like, I don't really care what I'm building.

And, certainly, today, I don't care what I build. The only thing I care about is is somebody gonna get value out of it? I could be doing the most boring, you know, unsexy software development stuff. I'm writing bash scripts that move files around. Anybody can do that.

But am I doing it in a way that is actually gonna be solving a problem that somebody has? And I'm making it easier for other people to use this. And if and if I am and people are providing the feedback, like, thanks for building this super annoying thing, and I'm glad you built it, not me sort of thing. That makes me feel good. Alright.

So for context, Ben started at Databricks as an RSA and then went to principal in record time and then moved over to software engineering and is now the TL for the open source team. Ben, do you think you add more value as a software engineer or as an RSA? That's an interesting question. I'd say anything that I anything involving a keyboard right now add more value, just because of blast radius. There's, like, more people using it.

But I don't know. Depends on what value you're talking about. Fair. Like, are we talking about the bottom line of of our company of, like, sales? I mean, yeah, I guess we we would have to get into definitions.

The the the the basis of that question is you said you like providing value. That's the thing you optimize for. Did you make the shift to engineering because you thought it was cool or because you thought you'd add more value or something else? Definitely the value aspect. I thought that I could be able to work on things and also learn more of the craft that I was really interested in in that role versus what I was doing in the field at the time when I made that transition.

Got it. Okay. Interesting. So maybe the value optimization is consistent. That was the intention.

Like Mhmm. And it it was, like, partially, like, self serving. I'd Right. But it wasn't a I didn't approach it from a position of, like, I deserve to do this job, so I'm gonna tell somebody I wanna go and do this. It was, I know nothing.

I want to learn from people who know how to do this really well, and I'm interested in this. And it's an opportunity to make myself feel really stupid, and I love feeling really stupid. So that's why I pursued it. And also thinking, like, if I'm if I turn out to be somewhat okay at this, then maybe I could build some stuff that a lot of people wanna use. Got it.

How much of that transition was motivated by burnout? Quite a bit. Cool. But it was more of a burnout in the fact that I was doing a lot of things that were very similar every day, and each of those things that were that I was being asked to do or just getting involved with to try to, you know, help people out and provide value for customers, It wasn't things that appeal to me because it wasn't based in objective fact. I don't deal well with subjectivity, and a lot of, like, the, like, the interactions I was having weren't based on any sort of, like, scientific methodology.

It's like, okay. We tested this and this doesn't work, and here's what we need to fix this, and we evaluated this. And it was more like, I don't like this. And, like, why? I just don't like it.

It would be cooler if it did this. Like, I'm not a product. Like, I I don't make those decisions, but how can I help you and, like, move this forward? And it was a lot of lot of interacting with certain types of companies that didn't make objective decisions. It's more like gut feeling type things.

And I just kinda got burned out of having discussions like that over and over. I didn't feel like I was adding value to those discussions. Got it. Interesting. And now with software, it's a lot more objective.

Like, there's a right and a wrong. Obviously, there's gradients of better and worse, but pretty factual. For probably not as much as you think. When you're talking about implementation, you're either you either did it well enough to make it work for what it was trying to do or it's broken and you need to fix it. So that yeah.

That is objective. Like, does this run? Yes or no? Does this solve the problem that it's trying to solve? Yes or no?

But there's a huge amount of activity that comes about guessing whether this is something that people want when you're doing greenfield development. Like, bug fixing is one thing or building onto an established product is another thing. But when you're going into, like, hey. We're creating this new feature that does x, y, and z. You have no idea if people are gonna even use it.

They have no idea. And sometimes you got a sleeper thing that goes out where you'd you're kinda thinking that this was gonna be something that was big. You work on it. You release it, and you you're looking at instrumentation. You're, like, looking at metrics.

And you're, like, okay. We had 10 people use it this last week. Cool. 4 months later, how many API calls this past week? 12.

Well, it went up, and then the next week, oh, there's one API call. Should we just kill this with fire? Like, this sucks. And then you forget to kill it with fire, and you check a year later, and you look, and you're like, 84,732 API calls last week. What the hell is going on?

Who's using this? Fun fact, that happened to me 2 days ago when checking metric usage of something, package that I built two and a half years ago. For sure. It's Diviner. I was like, who's using this?

Yeah. Right. Alright. We'll chat after. So, yeah, sometimes stuff like that happens, like and then you're like, oh, no.

Do I need to, like, add features to this? But nobody's asking for anything. So, yeah, you never know. Like, so that's there's, like, this unknown subjectivity aspect of, like, is this good? Should we should we keep this around, or is this just a waste of time?

So there is that uncertainty Heard. Okay. Cool. Did you ever get inefficient when you were burnt out? Oh, for sure.

Yeah. You that look like? I mean, I think there's physical manifestations to burnout. It's not just that, like, mental fog that that a lot of people get where you're just like, I gotta do this again. You know, living in a state of perpetual annoyance or frustration is not good, like, mentally.

Mhmm. But also it physically, it isn't good either because, at least, personally, it makes me less motivated to, like, work out, like, something less motivated to be interested in things that are not focused around that negativity of frustration. So there's ways that I've handled that even if we're doing some sort of crunch right now, where we're like, hey. We got a deadline. We gotta, like, get this thing out, and there's a lot of stuff you have to do in a short period of time.

It's very important to have, like, a separation between, you know, your time on keyboard too when you're not doing that. Like, consciously not dwelling on it, not thinking about it, and also hobbies. Get as many hobbies as you can hold on to and make them interesting, time consuming, and something that feels like you're making progress on. Do hobbies do you try to optimize your hobbies to be different from your work? For sure.

Like, I don't I no longer code outside of working hours. I used to, but I have zero desire to do that anymore. Yeah. An example was I started to learn to produce music, and it was way too similar to my job. And that's like, you're using your hands on a different type of keyboard, but watching YouTube videos and, like, teaching yourself something was just, like, too triggering.

So because it used the same part of your brain and really didn't refresh. So now I do the opposite of that. Yeah. I I keep my guitars right next to my desk. So sometimes I'll take a break midway through the day and play, like, 10 minutes of guitar.

Or, if I wanna learn a new song, I'll do that in the evening and just kinda focus on that. I'll also do stuff like play video games. Like, yeah, it is involving a a mouse and keyboard, but it's completely different, you know, part of your brain that's involved in that. It's just like, oh, I'm doing this thing that is creative in a different means and or is just killing demon hordes, you know, whatever you're big into Diablo for. Yeah.

It's a it's a way to check out. And then also making sure that all like, set a schedule with gym time. Like, hey. 3 days a week, here's my routines that I'm doing and keeping track of how that progresses as well. Yeah.

I I have a question. Curious if you've experienced this as well. So I completely agree with literally everything you've said. And I've noticed that the human brain is pretty holistic. And if one area of requirement is lacking, all the areas suffer.

Yeah. So to put it tangibly, like, if I have friends, I have professional life, I have health, and I have, I don't know, hobbies or something else, like personal things. If any of those quadrants are lacking, all of the others suffer. And then likewise, if I can level up any of these quadrants, all of the levels level up. All of the quadrants level up.

What are your thoughts? I have potentially controversial take. I don't remember where I first heard it, but I definitely definitely didn't come up with this theory, but I'm gonna co opt it. And it's it's the burner, Not burnout, but burn earth, like, theory. So imagine a stove, like a gas stove in a house, and you have 4 stations on that stove.

But there's a fixed width diameter of flow rate of gas that can come into that stove. Mhmm. And the 4 different burners are your health, the health of your relationships and family, the health of your relationships to friends and community, and then your work. So those are the 4 burn like, 4 burners that you can change the flow rate to. If you turn all of them to maximum, no burner will go above medium.

So you can never get the high bright, you know, crazy flame. But if you believe that them down. Do you believe that that there's a fixed Oh, yeah. For sure. Yeah.

Yeah. I think I don't. I think 2 plus 2 can equal 5. Not in my experience. Interesting.

There's only so many time like, there's like, the the theory of that gas flow rate is you can change the gas. That's actually so what I think what you're talking about is you can change the gas type that you're using. So you can get the color to change on it to make it more appealing. You can get it to be more efficient, but you can't get the flame to be bigger because the limitation that you have is hours of daylight or hours in a day where you're not sleeping. Uh-oh.

Because time is the the ultimate fixed, you know, quantity. And is it you I think you would have a different respect like, perspective if you had a couple of kids. Because they take up a lot of time in your day, and you need to spend that time with your kids, or else you're just gonna be, like, a checked out parent and just not good for their well-being, typically. But there's certain points in life where you have to actually turn one of those burners way down or completely off, but there's there's side effects to doing that. So if you turn the work one way down I've I've worked with people that are like that.

Like, they have, like, 5 kids. They're constantly doing stuff out of work with, like, friends and their family, and it's, like, social life is the their reason of living. And they seem very happy with that. But then a rift happens at the job, and they're the 1st person to get canned. So they get fired, which is unfortunate.

Just like, man, that sucks. I really like that person. But I can understand why the company did that because they do the bare minimum and are frequently leaving early and not really contributing that much to the team. More often than not in high-tech, scenarios, I see people turn the work all the way up. And then other things, they try to turn those up, but they don't have enough gas left because they're working 16, 17 hour days.

And on the weekends, they're too freaking tired to go and hang out with their friends and family. So that that ends up suffering, or they're just, like, physically and emotionally burnt out from working an 80 hour work week, you know, 16 weeks in a row, and they just they've hit a wall. So I think there's a finite gas amount that can be applied to that stove. Okay. Fuck.

Gears are turning. Alright. First thing, objectively, that seems right. But I still have experienced other areas of my life. As they improve, they reinforce each other.

And Oh, of course. Yeah. So then I'm taking that as okay. You're saying gas is time. Like, you have 24 hours in a day?

The volume of gas is time. What you're talking about of gas. Is the quality of the gas. So you're removing, like, water vapor from it because you're like, oh, I'm I'm better at my job, or I'm better at my hobbies, or I'm better at interacting with friends and family. So I feel better as a person, and everything kind of benefits.

You're just removing impurities from your gas. I have a question. Yeah. I have never in my job been okay. For the types of jobs that we do, we need to type stuff on a keyboard, and that's, like, 80% of it, then talk to people for 20% or whatever whatever the split is.

Typing the right keys doesn't take that much time. Getting to understanding what the right key should be, that's the hard part. So design, testing, exploration, figuring stuff out, rewriting. If you knew from the outset the correct key keys to type and you had, like, a screen on one side and the typing screen on the other, how many hours a day would it take for you to do all of your typing work? You're just copying code or docs or whatever.

Interesting question. I don't think I've ever thought about that. If I had no changes to make on any code that I wrote and I didn't need to do design Mhmm. And I knew the correct answer of how to implement something, I would my day would probably be an hour development and probably 3 hours of meetings, and then it would have, like, tons of extra time. So that's the asymptote.

Right? That's the the true threshold of the amount of time that your job takes if you were a omnipotent genius. Yeah. But it's it's a sliding scale. So if that's my work output and it only takes me an hour to to write 6,000 lines of code and it's flawless, I now can work on 6 times more stuff.

Right. And that's up to you. Like, the the job description determines that your current required work would take 1 hour. And, of course, you can now do extra. But if you were perfect at your job, you would have 1 hour of work.

No. I'd still have 10 hours because I would be doing 10 x the amount. Because you want to, though. No. That's determined by needs of the business.

Like, the faster we get features out, the more customers we get, the happier our customers are. So I would be doing that. Right. But the point is, like, if you could do you're you're currently doing well at your job, let's say, hypothetically. If you can do all of your job in 3 plus 1 hours, that's the exact same output, but much less input.

Right? Yeah. But but I wouldn't cap my output at the level. So you would be, by definition, doing extra. Like, if you're doing good enough now okay.

Cool. Yeah. There's always a backlog. You've seen that. Yeah.

I have seen. Yes. So this this has actually been a really interesting paradigm shift for how I do work is thinking about that asymptote and thinking about how I can get closer to that asymptote. Because, again, the right answer isn't doesn't take that much time. Reaching the right answer takes so much time, and that is skill.

That is intensity. That is quality of the gas. And also wisdom. So Yeah. Oh, yeah.

And luck and all this other stuff. Yeah. And it's also a different job role. So when I was doing your job, I approached it in the same way, which is, like, the faster I can get to an answer and unblock a customer or bang out some code for a customer. I'm on this fixed time scale of consulting hours.

It's like they paid for this amount of time for me to do this stuff. And my goal was always, can I get all of this done in a 10th of the time that they purchased my time? Mhmm. And then I can ask them, what else do you want me to do? Or, hey.

Do you wanna just do it like a hackathon? Or, like, let's build something new together. And people love that. They're like, thanks for doing all the annoying work, and now we can, like, do some awesome stuff that we wanted to do, but we didn't know how to do it. I'm like, yeah.

Rock on. Let's do it. And that's that was fun for me. It was getting able to do that stuff. But on, like, where I am now, we have that perpetual backlog that, like, there's in order to clear the backlog, we would need to quadruple the size of the team in order to get it done in a year.

And that's just the nature of our business. And there's always ideas and things that we wanna do and paper cuts that need fixing, and you're always limited by time. But it's not limited by people's competency. It's just there's so much work. I struggle to understand the line between competency and, like, efficiency of output.

Like, you there's there's a threshold to the human capabilities. The asymptote is one screen has the answers, one screen has the you typing, and maybe you can copy and paste to make the asymptote even, like, lower. Maybe it's 30 minutes if you have a copy and paste feature. But the question is, like and I I think the fun of this job is how quickly can you reach that answer? Well, we don't know what the answer is.

That's the problem. We know how to implement something. We don't know if we're implementing the right thing. And the only way to know that is to get feedback. So we have to release something and then either watch how people are using it or how they're not using it, or we actually just have to go out and ask them and say, like, did we build something dumb here?

Like, do you like this? Sometimes you get the response of, no. This is amazing. We love this. Can we get, like, these 5 things added to it?

Like, you know, backlogs, we or is or it's more like, yeah. It's okay. We tried it. Or it's like, yeah. We tried it, and it doesn't work, and here's why, and you suck at life.

And then we go and fix that as mean like, as quickly as we possibly can. What we're not looking for is, oh, that's a thing. We had no idea. Or, it's okay. That mean, like, the, it's okay.

It's like, okay. They're they're being nice, and this is totally stupid, and we shouldn't have built this, or it's missing the mark on what we thought. We thought this was something that people cared about, and they they give 0 shits about it. Right. So maybe we we just delete it.

Like, we don't need it. Mhmm. Nobody was asking for that. But when you get the response of, yeah, this is awesome, but we need these things in order to make this more useful, we've just amplified the amount of work that we have to do, and it there's no longer a right answer on how to do that. Some things there are, like, yeah, we kinda know that it needs to do this, but we might have to go back and rework a bunch of stuff to make it work in that way so that the amount of work increases for all of those things.

It's easier when it's like, yeah, we love it, but this is broken, and here's what broke with what we're trying to do. That's more cut and dry. And, usually, we can bang out those fixes in minutes, get emerged. We're good. But it's more the, it'd be cool if it did this, that that requires a lot of thought.

Got it. Okay. Cool. Tying it back to the burnout conversation, and the the stove analogy, which I think we've I think this is still generally on topic. But, to summarize the point, I think it's really valuable to have, areas of your life that reinforce your professional life so that the quality of the gas is a lot higher.

Because if there's burnout, that's typically a function of the amount of time you're spending. If you can do the same amount of work in less time, a, it's more fun. B, you have more time to do other things, and generally, there's less burnout. Yeah. Sound reasonable?

Yeah. If you're if you're producing something, I'd say that's very reasonable. Versus? Versus if you're just helping people. Oh, god.

Okay. You're just there to answer questions. Like, that can be a a different sort of burnout Yeah. In a technical capacity. Oh, you're the expert.

You should be able to answer this stuff. You know, like, okay. Do I just answer this for 837 people, or do I write a blog about it? Or do I write documentation and just point people there? And that that's a way to alleviate that repetitive nature of, like, answering the same question over and over.

I don't think anybody can endure that for too long before just, like, starting to feel like they're losing patients. Yeah. That's an interesting point. I was chatting with a couple miscellaneous humans. Doesn't matter who or what.

But, they were asking some pretty simple questions, and I was explaining, via Socratic method, like, how I was thinking about the problem. And I started getting frustrated, and I've been doing that recently, and I'm trying to not. And a really valuable reframe that I've been leveraging is how can I be so good that this person knows exactly what they need to do and has the issue with intuition to proceed? That now is an interesting challenge if if they're like, what is, I don't know, SQL? And you're like, it's this thing, and then how do I write a select statement?

You do this. Instead, be so good that they will become self sufficient as well as learn the knowledge that they needed from this meeting. Did you ever try that as a reframe? Yeah. I tried a bunch of different approaches.

One of them was, I was like, oh, all we need is, like, a master guide that answers the the 400 most frequently asked questions. And it it got so big that nobody looked at it. It was like, okay. This is like an ebook now, and it's it gets out of date fair like, fairly quickly because you know how our company is, releasing features all the time or changing functionality. So I gave up on that, and then I was like, maybe I could do like, what are the questions that people struggle with that I can't answer quickly and that I can't just, like, point to a slide deck or something where they can get that top level information and move on once they grok this concept.

So I did a couple of those and sent and I would use them as prepackaged things. I would send it out sometimes as a preface to our initial meeting. Like, hey. Here's this cool guide that might give you a a primer on what it is that we're gonna be talking about. And some some people are like, this was awesome.

Thanks for doing that. I'm like, yeah. Sure. Not knowing like, they don't know that I've already sent it out to a 117 different companies before meetings, but it saved me a lot of time and effort. And sometimes people really enjoyed it.

Other times people are like, yeah. I knew all that stuff in there. I was like, well, I didn't wanna make any assumptions. Yeah. And then that morphed into a couple of long form engagements that I was having with customers where they're really struggling not on the modeling part.

Like, they knew how to write code to generate a model, and they knew how to do all the feature engineering stuff, and they were excellent technicians about data science operations. And what they wanted to talk to me about was mostly about, like, API usages. And they're like, well, we wanna deploy this model and then do this AB test, and we need to know, like, the best way to do that in Databricks and with MLflow. And and then they collect the data after we built that. And they're like, how do we do this statistical processing of of comparison?

And that's when it started to dawn on me a lot of times that, like, we're not seeing any benefits to these models being deployed or you look at usage statistics of this deployed endpoint. I'm like, is something wrong here? Like, why are there only 4 requests to this model in the last 24 hours? And come to find out nobody in the company is using it. Nobody cares.

Mhmm. They build this thing that's gonna be integrated to a Tableau dashboard, and you look at the usage of this thing. These 4 data scientists spent 16 weeks building this thing, and one person's used it in a week. And I start having these conversations with them about why did you come and get me to help you with this? Like, what was what happened in the 3 months leading up to me showing up here that made you think that you needed somebody who knew how to do this to help you out?

Who did you talk to before starting this project? What department is your point of contact for the actual owner of this? And I started realizing that a lot of these teams were just building stuff that they thought was gonna be a good idea without telling anybody. And that's why I wrote that book. Was it and that's what that book is about.

It's like, how do you figure out what to build? And there's some tech stuff in there about, like, showing examples of things, but mostly, it's that's the most important part of data science. And once I realized that and sunk a lot of my time into answering that question, and the only way that I knew how to answer it, which was by writing a book on the the topic, because it's very long form to explain that. Then I felt a sort of a cathartis, you know, cathartis of doing that. Like, I I think I did an okay job of answering that question.

And then in the future, I wasn't even the one making the the suggestions. It was, like, salespeople. It was like, hey. Buy this dude's book before you start doing data science stuff. Mhmm.

And then getting the feedback from that and, like, random thousands of people randomly on LinkedIn being like, I got your book, and it would it changed how we did stuff. I was like, that's why I wrote it. Awesome. Glad to hear it. Yeah.

It it's, machine learning engineering in action. Right? Machine learning engineering in action. Yeah. Yeah.

Yeah. Listeners, it's very good. Like, if you haven't it's it's the key thing is it's different from existing textbooks. Like, it it it does walk you through, like, what linear regression is. Actually, I don't think it does.

It does some, like, time series stuff Yeah. It's time series stuff. But the decision making criteria, if you can internalize some of those lessons, you don't have to use them per se. But if you understand the logic, it really helps. So highly recommend.

But, also, Ben gives me 5¢ per copy per royalty. So, okay. Cool. I had one more topic I wanted to ask about before we can just sort of rapid fire and or summarize. I'm still sort of early in my career, and the past several years have been absolutely game changing in terms of understanding myself and what I like and don't like.

And the one of the recommendations, we also were looking through Reddit before this, aside from be in nature, take PTO, focus on wellness, and then realize you're gonna die one day, there's another that suggests energizes and drains reflection, which is basically what exercise energizes you and what exercise drains you. Mhmm. How do you I mean, yeah, you can, like, go and create a list that's not that hard. But do you have just general thoughts on how to acquire this knowledge? And also, does it change over time?

I think you need to explore a bunch of things and do self reflection. I don't write stuff like this down, but I do think about what my emotional state is when I'm doing certain things in a job. Mhmm. And I had done that in the past and try to think, like, what? Okay.

I I feel annoyed right now. Why? What was it about this, like, the last two hours of work that annoyed me so much? And realizing that, okay, it was this I'm not gonna bring up specifics here, but, like, this one aspect of what I used to do used to really aggravate me. And I would just try to think about how can I solve this one thing so that I it's not as painful?

And is it, like, oh, I was dragged into a 2 hour long meeting that had no point whatsoever. And it just became this free for all of people just dropping anecdotal evidence without any factual basis of things that or just random thoughts that pop into their head and nothing comes out of this thing. And getting pulled into those meetings and after getting off of them, I was like, man, that sucked. I hated that. So I started doing things that made it so that that was impossible to happen.

We're, like, saying, hey. If if I'm gonna come on this call, here's the 27 questions I want answered before the call starts, and here's the thing the 10 things based on those answers that we're gonna talk about, and we're gonna go through those 10 things. And then we're gonna have 5 minutes at the end. And when that 5 minutes is up to do random discussions, I'm going to hang up the the video call, and I'm gonna leave and go do something else. And setting that stage for people, people started to kind of under like, appreciate that in some ways.

Other people were like, it's super cool how we we did this asynchronously at first. And then that 2 hour long meeting, it took 11 minutes, and everybody got all their questions answered. This is awesome. Like, the customer really appreciated that. I'm like, I really appreciated it.

Mhmm. So bringing that structure in order to discussions reduced a ton of stress for me so I could go do other things. And it's all about identifying that stuff and then coming up with proposals that might work for everybody. Right. That's crystal clear.

How do you balance seeming like a curmudgeon with efficiency? It's all about ruthless efficiency. I don't editorialize when making those those requests because it's irrelevant, and it just makes me seem like a jerk. So if I if it seems short, you can always swing something like that to the efficiency thing. Like, hey.

I know everybody's really busy. Here's a bunch of things that we need to, like, discuss before the meeting so that we can all spend the least amount of time possible discussing this. We can really hone in on what the problem here is. Most people read something like that, and they're like, yeah. Sounds like a plan to me.

I've got other stuff I need to do too. Yeah. Versus, hey, everybody. How was your weekend? What are we talking about today?

Oh, yeah. Hey, Bob. Didn't you have that weird thing that happened in that notebook? Oh, yeah. Can you bring that up?

Yeah. It's it's screen share. No. It's not that notebook. It's this other one.

Oh, jeez. Yeah. Well, I I didn't run it, so I gotta start a cluster now. It's like, why am I here? Like, you could have screen captured that before this meeting.

Taking that stack trace, saved it in a file, and then we can talk about that. Mhmm. So stuff like that just drove me insane when people were just doing stuff like that. It's like, do you have nothing better to do with your time? Yeah.

No. It's it's a really good call out. I've started doing that as well, and there've been a couple times where I'm just like, I'm not gonna go to these meetings. Sorry. If you have issues with it, let me know.

Yeah. And people usually don't care. But sometimes, you're talking about for the meetings you do need to be attending. There's a I've developed a sort of shift in my brain where it's we're friends, and then we're in work mode. And friends, I will I I enjoy chatting.

Like, I'm down to, oh, your kid hit a home run-in baseball. That's that's actually very boring, but there's some interesting things in people's personal lives. Oh, of course. And so when we're in that mode, I there's no time. Like, I'm or time limit.

But once we enter work mode, it's like, we are going to do this as efficiently as possible in the most direct and concise way because Yeah. That's how we're gonna operate. And people usually love that sort of dichotomy of come in, chat, be like, these are the 6 things that I've done. What are the 6 things you've done? Everybody's prepared.

We wrap the meeting 5 minutes early. Cool. Yep. So Yeah. And everybody appreciates that, particularly about technical stuff.

Mhmm. Because it you see us do that on the engineering side quite a bit when we have these meetings with people in the field. We're sticking to a script because we don't want scope creep. Yep. If it's if it's, like, sort of this unbounded, you know, discussion where people people are gonna be people, which ideas pop into people's heads, like, oh, it'd be really cool if we did this thing.

There are meetings for that, brainstorming sessions that are specifically for that thing. But when you're talking about something like, hey. Here's the status of this feature that's being built. We don't wanna go into that brainstorming mode or go into ideation or, oh, it could do this thing. And a lot of times, we'll cut those conversations off.

Like, we're gonna talk about this offline. Nobody talks about it offline. Yeah. It's more it's a polite way of saying, please stop. Like, we're committed to building this in this way.

Here's the status of what's going on. Mhmm. And that if like, that sort of ruthless efficiency is there to protect a team's time. Right. To make sure that because if you if you're working on a project and it also was like that with when I was in the field as well, you're talking to a customer about a deliverable and somebody just starts ideating.

Like, oh, it'd be really cool if we had this extra feature or we if we could do this thing with that. And I would always just slap that down in a very polite professional way. Sometimes having to explain, we can't deal with the scope creep right now, or this is gonna introduce this complexity to this deliverable. So if we if I choose to do that, please tell me the other three things that we can drop from the project in order to fulfill this request. Mhmm.

And when you present things in that way, people instantly stop. They're like, oh, no. No. No. We we can't drop anything.

Like Yeah. Alright. Moving on. Let's talk about the next topic. And making it all about that finite resource of time and human energy, usually stops conversations like that.

But I I didn't know to do all that stuff at first. So I would just sit there and stew and be annoyed. Like, why why are there, like, 5 people on this call trying to increase the scope of this project by 10 x? None of this is gonna have like, we don't have time to do all this stuff. Mhmm.

And if you don't know to say no to that stuff, that magic word, no, you don't know when to use that or to bring up the point that, hey. You you've got 5 weeks of my time, and right now, I'm booked 2, 5 weeks. And anything more that you want, you're either gonna have to buy more of my time at this rate or you're gonna have to let something else go. Mhmm. And people that don't know how to say that and don't know how know when to do that, a lot of times, they'll just agree to all that stuff.

And they take on extra work or try to, like, oh, I think I'm smart enough to just, like, hack this around and get this done, and I can deliver all what they're asking for. And then it turned out that day before the delivery, 5 weeks later, they are very burnt out because they've overworked themselves. Yeah. Or they built an abomination that doesn't really meet any of the needs. Oops.

I've been there. I've done that. Learn from it. Yeah. It's crazy.

Like, Databricks has 3 months of onboarding and training, but the soft skills, there's there's a playbook. And the most frequently used one like, scope creep happens probably 1 out of 3 to 5 projects, at least. And the phrase, I can do that, but what do you want us to remove from the initial SOW is magic. It's it's literally a silver bullet. And I I really wish we had, like, a doc of, like, the 5, like, phrases to reuse over and over.

Because it's also like, it's correct. Like, you are putting the customer first, the project first. You're focusing on value. And people who tend to be a bit ADD with with their goals and the goals of their team, sometimes they need a bit of steering and focusing them back on the first principles isn't always, like a core skill of an RSA. It should be though.

Yeah. When you talk about a project plan being sacrosanct for consulting Mhmm. And that trade offs are entirely possible and should be welcomed, but you put it in the terms of finite resources of time or money. You're like, yeah. We can do this, but it's gonna extend the project by 3 weeks.

So it's gonna cost you another $120,000. Mhmm. Is that accept is that how much this feature is worth to you? And sometimes you'll get a response of, like, there's no way this takes 3 weeks. And you better be right on, like, the extension Mhmm.

Of when your estimates are, but you get that over time. You understand how complex stuff is. And you just list it out or you say, like, hey. We'll come up with a proposal after this that, like, here's the statement of work modified, and they get that. And then a lot of times they're like, yeah.

We don't have that kind of budget for this one feature. You know, like, okay. So let's stick to the original plan. Or they do, and that's And then they say, everybody wins. Win win.

Yeah. But it's all about that, like, objective measure of what is the level of effort required to to do that. I didn't know that starting out, so I would get very frustrated early on in consulting. I was just a yes man. I was like, yeah.

I'll do that. That sounds cool. I don't know how to do that, but I'll do it. And you find out, like, why did I agree to this? This is way beyond my capabilities to figure this out in that amount of time.

Yeah. Guess I'm pulling an all nighter. Yep. Yeah. Cool.

So we're almost at time. For the final exercise before I summarize, let's take turns naming tips and or pieces of advice and or just wisdom that we've gotten over the past years, for how to deal with burnout. And we'll do 3 each. That sound good to you? Yeah.

Cool. You wanna kick us off? Yeah. First one, get a hobby that's not related to your job. Can you define related?

If you're spending all day at a computer figuring out technical problems, don't do that as a hobby. Mhmm. Like, your hobby shouldn't be, oh, I'm gonna contribute to open source projects. That's just doing more of your job. Like, figure out how to do open source projects during your workday.

Work on things that your company cares about. Or if that is something that really in like, gives you a sincere sense of satisfaction, time box it, and then have another hobby that has nothing to do with computers. Like, go pick up an instrument. Go learn photography. Like, do something that's interesting that makes you go out and touch grass.

You know? It's very important. Heard. Yeah. Just to piggyback really, really quickly, feel how it makes you feel.

If you feel drained by the activity, it's that simple. Like, if you spend a 10 hour day coding and you're like, alright. I'm gonna code some more, maybe that does energize you. But just be pay attention to how it makes you feel. Yep.

My pro tip number 1 is reframe. Make up a reality where you're interested. Like, it's that simple. That's a good one. My second one is, don't be afraid to bail.

So if you're doing something that the core tasks of what it is that you're doing, not in your company, but analyze whether it is your company's version of that role that you don't like, or is it that role no matter what company it is? Either one of those things. Either think about, hey. I've talked to people or I understand how this this role is at this other company, and I know it's different. Go try to work for them.

Or is it I understand that this is pretty universal across this entire profession. Go find another profession. Like, life is way too short to sit there and just suffer in misery if you, like, fundamentally hate the nature of your job. I've done it 5 times now. Like, just completely change industry and job entirely.

Like, what I do now has nothing to do with any other job that I've ever done. Or just pop smoke and vote with your feet if it's not working for you. How long do you think this job is gonna last? Oh, I don't know. I've been doing it a while.

Yeah. It's still fun. Heard. Okay. Cool.

On mine, I think context switching is essential, and the most detrimental and energy draining thing for me is thinking that something is complete and then having it not be complete. So, of course, things will break. Of course, there will be bugs, but really focus on excellence and successfully wrapping stuff so that when you call something done, it frees up the mental space to work on something else. Yeah. That's a great one.

100%. My final one is don't neglect your health. It's really easy, particularly in tech, to work too many hours, to not go out and touch grass, but to eat poorly and not exercise. And it might seem like, oh, you know, I see a lot of, like, San Francisco dude bros, like, encoding. They're all in great shape.

They do that not in order to get attention from people. They do it because they know that it's healthy for them, and they wanna live a long time, and they wanna feel good. So the more physically fit you are, particularly in a desk job, the better you're gonna do at your job, the more efficient you're gonna be, the better sleep you're gonna get, the more energized you're gonna be for after work. So, yeah, never never neglect that. Heard.

There's, like, that are conglomerating. I'm gonna try to put them all into 1. So we've talked a lot about getting different inputs to your brain and different styles of input. One thing that I try to optimize is inputs that give you perspective. Not to be morbid, but, like, if a family member dies, suddenly your job is not important.

If, like, there's an economic crash, suddenly your job is not important. It's a job. Like, chill. And so with that, there are ways that you can get reminded of that. I personally do a lot of charity and, like, nonprofit work, and it's a nice way to remind yourself that retail company x is not they actually don't matter at all.

Right. And helping humanity, like, smiling at people, being present, that's actually what matters. So food for thought. Yeah. That's an excellent one.

Perspective is everything. It is. Don't get caught up in corporate America or corporate, whatever country you'll you live in that you're listening from. Yeah. I remember, Ben's old manager, Alex, I was chatting with him.

He's, now a skip skip manager for me. And, I I was like, damn. I'm kinda burnt out. These these projects are boring. And he was like to the he in a the nicest way possible, he was like, shut up.

Your life is great. Have perspective. And I was like, now now that you put it like that, that does make sense. I'll I'll I'll be happier. And it actually worked.

Yeah. Exactly. Like, it it's really easy, I think, to convince yourself that you're suffering through stuff that that seems like, if you have something that you're doing that you really enjoy, and then you have too much of something that you don't enjoy that you have to do, it can like, our brains are just programmed to just wanna do the thing that we wanna do. Mhmm. And sometimes we'll rush through that stuff that we don't wanna do or just dread it.

Like, oh, man. I got all these meetings that I gotta sit through. Yeah. Sometimes just having a perspective shift and be like, can I make the most out of this stuff? Mhmm.

And sometimes I try to, like if I know I'm gonna be in a meeting that I don't really wanna be in, or I don't do it so much anymore because I don't have nonessential meetings anymore, but I used to have a ton of them. I would think of interesting things that I could do, like, during that meeting while still paying attention. Mhmm. Can I get somebody to say the same obscure word 8 times in this call? Like, do things to entertain myself while still paying attention to what's going on, or can I use the same very obscure word 20 times in this conversation?

I don't keep tally. Me and one of the account executives do meows. Whoever can say more meows during a meeting wins. Yeah. From Super Troopers or whatever?

Yeah. I think. Yeah. But it's better if you pick a really obscure word from actual language and just what? Just some synonym of a common word that is has been Oh.

17th century. Got it. And just use that every time in place of this other word and see if anybody notices. Turd. Okay.

So that's our 7th tip for burning. Find ways to make boring stuff entertaining, and then you will enjoy yourself more. Yeah. Danger nut. Yeah.

Cool. Alright. So I'll quickly summarize, not those tips, but some other things that stood out at least to me. If you're successful in your projects, that will lead to repetition and demand for you building the same thing and scaling it. So, a, if you're a manager or a direct report, try to get diversity of projects for either counterpart.

You can also try to automate a lot of this work. And then finally, if you have people who can benefit from this skill and can scale out via humans, try to teach them the skill. Burnout is specific to the person and their desires. There's no specific playbook for what energizes and what, drains. And that said, hobbies are a great way to have an outlet.

And then final point, be the person that brings efficiency. There are ways to do it without seeming like a dick, but overall, everybody will appreciate it, especially yourself. Yep. Anything else? That was good.

Cool. Alright. Well, until next time. It's been Michael Burke. I'm my cohost.

Ben Olson. And have a good day, everyone. We'll catch you next time.
Album Art
Combating Burnout in Machine Learning: Strategies for Balance and Collaboration - ML 178
0:00
Playback Speed: