From Hack to Holistic: Firsthand Experiences and Knowledgeable Interactions in DevOps - DevOps 171

David Linthicum is an Enterprise Technology Thought Leader and Visionary, Cloud Computing and Generative AI Expert/Pioneer, Educator, and Business Leader. They delve into the world of DevOps and explore various aspects of it. They discuss the benefits of working for a small consulting firm, the importance of learning from experienced individuals, and the risks of over-reliance on AI in software development. They also touch on the emerging discipline of FinOps, the integration of security into every stage of the software development process, and the challenges of building efficient and secure systems.

Hosted by:

Jonathan Hall

Special Guests:

David Linthicum

RSS Spotify Apple Podcasts YouTube Amazon Music

Show Notes

On YouTube

From Hack to Holistic: Firsthand Experiences and Knowledgeable Interactions in DevOps - DevOps 171

Socials

Picks

David - The Dead South
Jonathan - Modern Software Engineering

Transcript

Jonathan (00:01.09)

Hello everybody, welcome to another exciting episode of Adventures in DevOps. I am your host for today, Jonathan Hall, and I'm excited to have guest David Linthicum, let me try that again. And I'm excited to have with me David Linthicum. Did I say that right the second time? All right. I just tripped over myself, okay. I need some confidence. Well, David, thanks for joining. Would you take a moment and tell our listeners a little bit about who you are and your background and what you do?

David Linthicum (00:14.553)

You got it exactly right. I think you got it right the first time too. Just need confidence.

David Linthicum (00:27.809)

Yeah, Dave Lenticombe. I'm the chief cloud strategy officer at Deloitte. My career is mostly serial CTO jobs. I'm a CTO of about a half a dozen companies. Started and sold in many instances. Wrote 17 books on computing. Just finished up my last book called Insider's Guide to Cloud Computing. It's the purple cover you see behind me. And I do a lot of speaking, thought leadership topics on cloud DevOps, AI.

technology is my primary focus, work with clients as well as doing thought leadership activities.

Jonathan (01:03.838)

Nice. So you said that AI is your primary focus?

David Linthicum (01:07.893)

AI is one of them. It's been, I would say it's my primary focus for the last two years. Suddenly, it's certainly as it exists in cloud. I actually just did a course on LinkedIn learning on AI, generative AI in the cloud and how it affects that and also how it affects things with DevOps, what's working, what's not, where people are making mistakes, things like that. My focus is really kind of on the pragmatic use of the technology. Not really kind of a hype-driven person. So in other words, I think hype can get us in trouble.

Jonathan (01:12.962)

OK.

Jonathan (01:31.543)

Mm-hmm.

David Linthicum (01:37.425)

managed by magazine, that kind of stuff. It's looking at the core advantages of the technology you can bring and figure out when it's a good idea to jump into the mix of it. And certainly AI is an example of that. We all used JetGPT and it's kind of cool and we're trying to integrate it in different applications and different aspects of processes. Certainly DevOps is a big contender for that technology now. But in many instances, it's overkill. It's something you shouldn't use.

just because it doesn't have the advantages you think it would have.

Jonathan (02:10.206)

Interesting. Well, I'm really interested in talking to you then about some of the impacts that AI can have on DevOps and maybe the other way around too. Where would you like to start? I mean, I have several questions already off the top of my head.

Jonathan (02:26.494)

Maybe just at a really high level, where do you see the intersection between DevOps and AI? And we'll go from there.

David Linthicum (02:33.869)

Well, there's sub-functions that we can automate better with AI. Certainly observability and the ability to get into the operational aspect of it. Certainly the AI ops things are all over that. And there's real benefits there because it's able to discern patterns amongst large amounts of data. And I think one of the problems we have in operations, we have a tendency to be drowned in noise, our information that comes off of these systems. So your ability to set up a system is able to call through.

Jonathan (02:46.965)

Mm-hmm.

Jonathan (02:57.538)

Mm-hmm.

David Linthicum (03:03.725)

you know, all the log files, all the log information, and come up or create what's actually going on. So in other words, when an application's about to die, it's eating too much memory, you know, those sorts of things where the patterns may be fairly minor, we don't see them. So the AI capabilities are able to spot that. And I think that's a huge value and I've seen that work in practice.

Jonathan (03:22.423)

Mm-hmm.

David Linthicum (03:32.865)

Other aspects would be the code systems, the ability to generate auto code. Obviously, people are leveraging Gen.ai, and for good reason, to generate codes and things like that, to the install scripts. It's really great for that. In many instances, we may be missing some inefficiency there that we could do if we've done it in a manual process.

Jonathan (03:55.235)

Mm-hmm.

David Linthicum (03:58.113)

So I'm seeing lots of opportunities where this has a tremendous amount of benefits, certainly on the operation side and on the development and design side, it certainly can be a useful tool, but it's something that probably can't be relied upon as much as we think. And I think everybody's out there basically trying to leverage these auto coders and all these various systems to bring more productivity to the DevOps processes. And I think those are great, but in many, many times are misapplied.

Jonathan (04:10.915)

Mm-hmm.

Jonathan (04:28.382)

Let's talk about that. I mean, what do you see? I mean, I can imagine, but I'd like to hear from someone like you who knows what how do you see it being misapplied? What are some common maybe anti patterns?

David Linthicum (04:38.529)

Yeah, that's a great question. Well, I think the efficiency of the applications can be a problem. So in other words, when you're dealing with AI-generated code, you're typically dealing with code that already exists in another place that's been basically as nothing more than a mirror of existing data. So in other words, it looks at billions of lines of applications and discerns a pattern, and then comes up with a pattern as the best way to develop that application.

The problem is when we build systems, normally they're purpose-built for the particular platforms that we're on. That's why we have cloud native capabilities and building for containers and things like that. And in doing so, we have to think through the efficiencies and the engineering of the application, down to the nth degree. And so in many instances, what's happening is people are generating the majority of the code, they're leveraging the code, it compiles clean, they put it to the development process, it seems to pass the smelt.

all the security tests and smoke tests and all that stuff when we go into deployment, but it could be 75% less efficient at leveraging the resources of something that's going to be more properly engineered. In other words, something that humans are looking at and probably even making minor changes in how these things work. In missing that, we're leaving money on the table and we're leaving efficiency on the table. So we get to this state, which drives me nuts as an architect, it works.

OK, it works, but it's costing us 150% more money every month. And by the way, if you looked at 2022, people were right upset about the cloud bills that they were getting. And so that just makes things worse. Things are more expensive. And that's because we're missing opportunities for doing engineering. Now, we do have AI tools that actually analyze that code to look at some of the efficiencies of the stuff. But there's some inefficiencies in doing that as well. So my big concern is that.

Jonathan (06:06.761)

Mm-hmm.

Jonathan (06:16.366)

Mm-hmm.

David Linthicum (06:31.789)

DevOps engineers and developers specifically may take the code as the single source of truth when in essence it's a starting point to get to some good engineering discipline to do much better at making that thing efficient.

Jonathan (06:38.711)

Mm-hmm.

Jonathan (06:46.346)

I think that's a good point. I mean, I've listened to several people doing like creative writing or copywriting using chat GPT, and they just consider it a rough draft. You know, use chat GPT to spit out something quick, but then you gotta go in and clean it up. It sounds like you're saying basically the same thing for code, which makes perfect sense to me. I don't trust the average of stack overflow on all of the internet to write good code. They might write usable code, but it's not gonna necessarily be good or performant code.

David Linthicum (07:16.725)

Yeah, I think that's the core function. And so the thing is, if you're a good engineer, a good software engineer, you need to understand the differences. Like I said, the problem now, and it's like in many instances, we were doing it manually. Normally we took the time to do it right, because we had to go through and manually code the thing anyway. Now in the world of AI auto-generation stuff, where even non-coders are coding, because of this technology, we're missing some of the engineering disciplines out of that.

and therefore we're missing the efficiencies that systems can bring. And so we're deploying things into production that is cost is using 20 times the CPUs, a hundred times the storage systems, 20 times the amount of memory that's needed because there's a few inefficiencies that aren't built into the systems. And so, and that's because that the AI system is only building things based on previous builds, based on...

Jonathan (07:45.032)

Mm-hmm.

David Linthicum (08:11.289)

previous coding, previous IP, previous systems, and it only understands how to do it in those ways. It's not necessarily, and I'm sure it'll get there at certain times, understand how to do efficiency and disciplines and doing engineering of the system. And that's where I think we have a bit of a crisis right now in the fact that people are using this as a crutch, but aren't necessarily getting to an end state of efficiency where they should be. And so that doesn't mean don't use it.

I think it's great technology for operations, for example, things like that, and certainly security. But your ability to depend on it 100% for engineering the system, the technology is typically not there, and you're just going to end up with something that works that's incredibly inefficient.

Jonathan (08:41.545)

Mm-hmm.

Jonathan (08:57.126)

Mm-hmm.

What other concerns are there? I mean, I can imagine that you just mentioned security. I can imagine that this could go both ways with security. I imagine, I've heard of like copilot spitting out credentials and crap like that. You know, that's not ideal. But I can also imagine just insecure code being generated in some cases. What have you seen in that regard?

David Linthicum (09:15.138)

Now, oops.

David Linthicum (09:22.733)

Yeah, that's a great point. And it's another issue. And that kind of goes to engineering. I look at security as something that has to be engineered to the application. People have a tendency to want to bolt security on as an afterthought. But if the application is not built and engineered to provide a certain level of security, it's going to be innately insecure. And so if we're, and again, to the point you just made, what I think is spot on, if we're letting these things generate code for us and we're not necessarily changing the systems,

Normally there are a lot of security vulnerabilities in there along with the inefficiencies. In other words, we're not using the resources correctly. And we're also exposing things and resources that shouldn't be exposed, credentialing, things like that. It needs to be, again, hand engineered so we have the right level of security. Now, the good news there is if you're AI lazy, which I think a lot of developers are becoming,

Jonathan (10:18.384)

Hehehehe

David Linthicum (10:20.161)

You can do code scanners to figure out where those things are. And they're fairly good and spot and change some of those systems to remove some of those vulnerabilities. But it still takes the mind of a good engineer to go through it and make sure that the security systems are as optimal as they can be. And so I've seen things that are hugely insecure, hugely inefficient. They pass all the code scans, the security scans, things like that. They go through the security testing mechanisms as part of the tool chain.

Jonathan (10:38.541)

Mm-hmm.

David Linthicum (10:50.273)

And when the things are deployed, there's so many vulnerabilities in there. The thing is just laid, it's just hacked to death on the first week of operation. And people are concerned because there's so many things that can be wrong in terms of security around that application that it's impossible for scanners to really kind of figure out where all of those things are. And so we may reduce them by 80%, but that 20% is where the vulnerabilities remain. And I think that's a...

That's a part that people are missing. So we need to be careful as we're leveraging this technology to make sure that we're spending time to do the purpose-built engineering within the system. Human beings are still looking at it and making sure these things are as secure as they possibly can be. Because again, we get to the point, it works, it's deployed. And it passes security scans, things like that. But there's...

So many vulnerabilities there, it's a problem. The other aspect of this is we're getting attacked by Gen.ai itself. And so in other words, the people out there that are doing, and I'm not nowhere near a cybersecurity guru, but there's systems out there that are leveraging Gen.ai to do token spoofing and things like that, where we're unable to, it's able to adapt and respond.

Jonathan (11:51.542)

Mm-hmm.

David Linthicum (12:10.797)

with different databases that keeps trying things and goes through a hierarchy of trying with a knowledge-based engine that's building over time. And the defenses there have to be a level three times as much as they are now. Your ability to kind of spot these things in terms of behaviors that are coming after you and understand that the behaviors aren't gonna be static as they've been in the past. In other words, things are repeatable. They're gonna change over time and evolve over time, keep trying different things, and even seem like different entities, and it could be different entities.

So now we have to, and I wrote about this in my InfoWorld blog, we have to wrestle around how we're going to set up this AI technology to defend against AI technology. I guess spy versus fi kind of thing. And part of the engineering process is to set your application up to be dynamic enough to understand when those vulnerabilities are gonna change and exist. And so in other words, we're putting, we're getting into.

Jonathan (12:51.468)

Mm-hmm.

David Linthicum (13:06.381)

very geeky engineering stuff now. So we have a certain amount of things that are statically coded in the system, but a lot of stuff is dynamically set up. So we're putting volatility into a configuration layer where it's able to auto-config around changes in the threats. So in other words, it sees different kinds of threats and different kinds of attack vectors starting to emerge. And we're able to, number one, do the core functionality application and have the hooks in there for security. But actually the security system sits as a separate

in a separate application space, not necessarily a security engine unto itself, that's able to adapt and respond to different threats that are changing directly from the application. And so, you know, mind blown and also it causes a lot more money and work and delays and getting things done. And so that's the trade off. You know, it's I always tell my clients, do you want it security? You want it fast. You know, we can't do both, ultimately. And so we need to find to strike a balance. And I think

Jonathan (13:56.877)

Mm-hmm.

Jonathan (14:00.631)

Mm-hmm.

David Linthicum (14:04.605)

engineers need to drive this. So DevOps engineers specifically need to understand how this technology can be applied and also how the threat statuses are going to change over time and understand the vision for that and build that into the way in which we're building and deploying applications. And it is going to mean 25% more, and best I can figure that's what it's going to cost us, 25% more resources and human beings, and by the way, needing talent that we don't seem to have, that's able to...

Jonathan (14:31.968)

Mm-hmm.

David Linthicum (14:33.781)

In essence, engineer these things to be completely, to be as protected as they can be, again, by this dynamic engineering and leveraging AI in terms of defense, as well as something that's coming after you. So, and the other thing I mentioned is part of that dynamic system is you're leveraging things like gen AI and the ability to adapt and respond to different threats that are coming in as they start changing over time.

and then leveraging security systems as a mechanism of doing that in the encryption identity access management and credentialing and all that stuff we have to deal with but it gets to a problem where Your ability to put these systems into a common domain is really kind of the best way to engineer that So we're not doing the one-off for every application out there We're doing a common system that's leveraging the security

systems that's protecting a group of applications and a group of databases. And it's also leveraging a security system. So in other words, we're building none of the application itself, which should be engineered correctly and have the hooks in it for security. But we're putting another application out into, I guess you'd call it kind of a middleware thing, out into its own domain that's able to provide common protection services using security systems, using encryption, identity access management, things like that.

but it's separate and distinct from the application. So we're getting into the engineering part. It's into the fact that if we're gonna get to a next level of security, we have to set up this application domain, which is separate and distinct from the core applications. And because it's going to be serving lots of applications. I just almost confused myself with that one. But I mean, that's kind of where you get into it. And so if you look at some of the engineering techniques.

Jonathan (16:11.714)

Hehehehehehehe

David Linthicum (16:18.605)

that we're seeing out there, and there's some brilliant minds out there that are working on this. It's the ability to build something like that as a standard framework that's gonna be associated with groups of applications that's leveraging different technologies. So, used to be we had the hooks in there for security, and we added the security system on top of it, and the application would leverage those hooks to get to whatever security services that we had. Now it's the ability to get to another application space that's getting to the security services on your behalf.

I mean, there's this notion of super cloud, meta cloud out there and things like that. It's the ability to build this thing in kind of a cross platform way. Now, a lot of people are asking the question, shouldn't that be a product? I don't think it can be a product because it's gonna have to be bespoke for every application use case out there. And they're always gonna be a bit different and they're always gonna have to be purpose built for the application levels that you're looking at. We are gonna have security managers and encryption systems and ID and access management systems.

Token management systems things like that will exist cross-domain But we're gonna have to figure out how to engineer this stuff ourselves and you know, maybe gen AI will play a role in that But again, I'm a little suspicious of getting to something. That's that level of engineering and that level of importance For a gen I AI system to get right the first time So even if you do a base layer a base code gen you're gonna have to go back and engineer the system So it's so it's compatible or working the way you want it to work and in many instances that's rewriting the whole thing

So you might as well do it from scratch to begin with in many instances. So I send people back to the drawing board many times when they, all these huge things, and you look at the inefficiencies that are in it, when you do a code review, and they have to go back and redo it.

Jonathan (17:47.297)

Right.

Jonathan (17:57.45)

I wonder if you could share any sort of horror stories about a specific example. You don't need to name names of businesses, but a specific example where JNAI performed terribly or had a security vulnerability or something like that.

David Linthicum (18:01.271)

Thank you.

David Linthicum (18:12.653)

Yeah, it's, we just get the nail in the head is if people are really dependent on auto coding, auto coding, copilot, for example, things like that, not picking on that, but those sorts of things, the ability to leverage GNI to generate code, normally the vulnerabilities are gonna be there. They're just not spotted in the system because they normally can pass testing. So, you know, my role is typically coming in and auditing those things. In other words, looking at someone who's done a tool chain and essence grading.

Jonathan (18:20.45)

Mm-hmm.

Jonathan (18:32.397)

Mm-hmm.

David Linthicum (18:40.173)

and looking at the different aspects and how they're doing the developed design deployment and all those security testing and things like that, it's able to bypass those tests in many instances because it's learned from those tests. In other words, the same knowledge base that those security testing systems are leveraging are also leveraged by the auto coders, by the AI coders, and therefore they know how to avoid them.

So they get through the test, but they're missing a great deal of vulnerabilities they're not necessarily being tested for. And again, the danger there is the attackers are going to change up their stance and change up their vectors. And the code is going to be vulnerable from those ways, because we're missing it, because it's overly dependent on the AI system. So you asked for an example of that.

David Linthicum (19:34.513)

was building something like this, didn't realize that the exposures were there, were very happy with the way in which it went through the security testing as part of their DevOps chain. But the stuff that went into production, when we audited it and look at the vulnerabilities that are in the system, they were amazingly numerous to the point where I was just surprised that they weren't hacked at the time, to the point they were off the scale in terms of risk. Security is really kind of a measure of risk. We're trying to get to risk. We can never get to zero risk. We're trying

Jonathan (20:03.415)

Right.

David Linthicum (20:05.105)

And they didn't really understand. And there's a lot of stuff at stake in terms of health data, things like that, things that are protected by law. And they could have been held responsible if the vulnerability was there and it was exploited. The funny thing about that is once those things were patched up and fixed and took some time to do that, they were attacked. So in other words, they just got that door closed just before this tremendous attack came in.

Jonathan (20:13.696)

Mm-hmm.

David Linthicum (20:33.769)

and went after their systems. And so, you know, that's an instance of that. Ransomware is another exposed systems and this always drives me nuts because people all they have to do is back the thing up. But the ability to in essence put hooks inside of code that's already deployed and the ability to deploy a ransomware attack within someone's application. I thought that was kind of a neat trick and.

making it happen. But again, they do that by exploiting the vulnerabilities. In many instances, those vulnerabilities are put in there because they're missing bits and pieces out of the AI code that is being generated. They're just not looking at it with a critical eye. And all these things are starting to occur. I think that to be honest with you, I think the not necessarily this year, but 2024 is going to be the year of breaches. Just because I think so many people out there are unprotected.

Jonathan (21:27.103)

Mm-hmm.

David Linthicum (21:30.433)

don't really understand what's exposed and why it's exposed. In many instances, because these autocoders have exposed them, they've built applications that have vulnerabilities in them. And then we're going to have huge amounts of a gen AI based attacks coming into the system. You kind of see it now.

not going to be able to keep up. We don't have the talent around both on the engineering side of the applications as well as even the security operation side to keep, you know, play whack-a-mole with these things that are coming into the systems. And we're just going to end up, you know, huge amounts of attack to the point that I think some of the companies are going to end up disconnecting from the networks just because they're so vulnerable and the vulnerabilities have been built into the systems. And the breachers are getting so good at exploiting these tools that are almost free now.

Jonathan (22:12.866)

Hmm.

David Linthicum (22:22.317)

So it's kind of funny, my chat GPT API can be your chat GPT API, and we're seeing that now where people are leveraging the same entity to, in essence, create both an attack and create a response. So it's an odd world. So we're going to have to think differently in how we do security. I think that we're moving into a point where we...

Jonathan (22:31.34)

Mm-hmm.

David Linthicum (22:47.349)

we're always dealing with security as kind of bolt-on kind of things. In other words, it's certainly built into the application, but it has to be systemically a part of each part of the process, the design process, you have to design it to be secure, development process for sure, and then the deployment process, which needs to consider how you're gonna put it into operations, and then the operational response of the systems, how you're gonna put observability around the systems, things like that. So.

Jonathan (22:51.32)

Mm-hmm.

David Linthicum (23:11.501)

we're keeping an eye on the application that's running in the space, and that's in essence where the protection is. So the protection is worth considering the design. We certainly use AIH to make that happen. Certainly in the development and the deployment and, but in operations, and it's all part of the, it's all part of the chain. So I don't think people think about that now. They kind of think about security as an afterthought. Once they get into the coding phase, sometimes when they get into the operational phase, so though.

put an application into a protected sandbox space that they're protecting with some security system and that's just not good enough anymore. The application itself has to be engineered to be secure. That's a bummer for people that want to do less work, but I don't think we're gonna get any way around it based on the vulnerabilities we're seeing out there and their ability to exploit AI to come get us.

Jonathan (23:41.964)

Mm.

Jonathan (23:52.554)

Right?

Jonathan (23:55.853)

Yeah.

Jonathan (24:01.206)

What advice can you offer to somebody who hasn't yet been thinking about this? Maybe this is the first time they're hearing about the risks involved in Gen. AI. Where do they start?

David Linthicum (24:12.269)

Well, I mean, get audited or an expert opinion on what you're doing. So, have another set of eyes. There's lots of boutique firms out there that do this and people, the independent consultants that just go from domain to domain to domain and they're more helpful because they've seen lots of stuff. They're not working for one company, they're working for 20 and they understand where their vulnerabilities are, how to review code and how to come up with a current response.

Jonathan (24:17.965)

Mm-hmm.

David Linthicum (24:41.705)

see where you are. In other words, how bad things are and how the processes can be fixed. And we have kind of two levels of response there. Number one, how do we fix the existing applications that have these inefficiencies and vulnerabilities that are already in production? Very important. But also how do we fix the tool chain and the process? So we're building this discipline into every step that we're thinking about efficiencies and security and governance even, you know, as we're building and deploying these applications. And I think that's where the...

the key problem comes in. Because normally people are budgeted for a certain amount of money. And if you show up and say, you got this stuff wrong, you're going to have to fix it. And it's going to cost you maybe $3 million over 10 months to get it done with some extra help and developers and different tool sets and things like that. That may be something they don't do just because of the budgets, which is weird to me because you're going to have the mother of all.

budgetary problems once you get a breach and you know people customers start going away, but So that's what I would do and then the thing is have a systematic way of improving things over time So in other words look at where your vulnerabilities are where your as this state is your to be state needs to be How you get to that to be state as quickly as you can but doing so and it's in a Systematic way and then the other thing is create a culture where people are asking questions

The big problem here is we're not dealing with technology, we're dealing with the people aspect of it. They don't have a cultural understanding of how to do this stuff in a continuous way. They're not asking the questions, they're not having, I hate to use the word committees, but we're not looking at this as a group in terms of how we're continuously improving some of this stuff. And that means put in there as well, because guess what's gonna happen? They're gonna fix this instance of it, and then a year later they're gonna have the same amount of problems because none of the developers and the DevOps engineers and...

Jonathan (26:10.118)

Mm-hmm.

David Linthicum (26:32.877)

you know, operations specialist, things like that, really kind of asked the questions that need to be asked. I mean, the changes, it suggested that the changes need to be made. So, and by the way, that's the biggest aspect of all of this. It's not the technology. Technology is easy. We can solve most problems with technology, but it's the ability to get the people aspect done and the ability not to solve this problem just one time, but you know, teach a person, you know, teach a man to fish and they continue to, they continue to improve the process.

Jonathan (26:45.954)

Mm-hmm.

Right.

David Linthicum (27:02.017)

So I guess that's what people always say about DevOps. It really kind of comes down to people at the end of the day. And you think about this, it really kind of boils down to a people issue.

Jonathan (27:14.319)

Mm-hmm. What advice can you offer to practitioners, maybe somebody earlier in their career who wants to become the expert who does these audits or helps to fix the security stuff, how do you start on that journey?

David Linthicum (27:26.337)

Yeah, I'd go work for a small consulting firm that has pretty agile processes and will get you out and exposed to many different domains. Lots of them out there. And normally there may be 20 to 30 people, extremely talented, and they're usually extremely talented because they're a small, no bureaucracy organization that's really able to keep developers happy. And I found that in my years of CTOs that you want to...

Do less bureaucracy, less overhead, and keep more people happy. And then learn from the people around you. That's the best thing to do. I don't think you can read a book about this stuff, to be honest with you. I mean, I wrote a few articles about it, and I see a few things that are written. Certainly IEEE articles and things like that would have a tendency to be more complex than to be useful. But you're going to have to understand this from people who are really kind of understanding the tricks of the trade.

and normally not writing this stuff down. So going through and following them through the process and start aligning yourself with the most brilliant people in the organization and make sure that they're involving you in each stage of the process. And you have to go that through about a year to get a good base level of understanding. And by the way, that's understanding that you're coming out of, you know how to do development, you know how to do design, you know how to do testing, things like that. You have basic DevOps skills, things like that. But all these tricks and hacks

are really kind of where the game is played and won. And that only comes from people, and that only comes from the experiences in using this stuff. And so, and like I said, not a lot of it's written down. I see a few things, a few tidbits I may pick up, but as far as the details of it, I have to drag someone who knows what they're doing to these domains to get that work done, because they do know the unwritten understanding of how to do this stuff correctly.

So that's it. And then the other thing would just be work in as many domains as you can. And so in other words, work for different clients and take on different roles and opportunities and keep focused on a particular aspect of it. Understand holistically what you're looking to do because then that's where your problem needs to be fixed. But also understanding the details and how these things operate. I think that's where we're missing the piece. We're relying too much on the automation aspect of it and not as much on the engineering.

David Linthicum (29:48.506)

aspect of it and that's where the deficiencies are coming in.

Jonathan (29:53.414)

It seems, I've certainly seen the pattern all over the DevOps landscape of people focusing on the automation and forgetting to build a process that works first before they automate it. And they just, they try to jump straight to the end of the process and they end up automating crap. And it sounds like you're talking about the exact same thing.

David Linthicum (30:11.137)

Right. Yeah.

Yeah, the exact same thing. And also, if you look at foundational, we have a supply chain issue. We have to teach college universities and even training organizations to training people in this stuff. So if everybody has to find out organically how this stuff works in 2023, we've done something wrong. And if there's a common complaint, and I actually teach at LSU, teach a DevOps course there, if there's a common complaint is not having up-to-date

Jonathan (30:25.439)

Mm-hmm.

David Linthicum (30:41.945)

talent creation capability at the university level is able to feed into the companies because they're normally not getting it at the companies. They may be sent off to some training, but it's all OJT and a lot of people aren't organically learning. So the ability to kind of combine that with the auto die deck, who is who I like to hire someone who's a self learner and a continuous learner versus someone who needs to have training pushed upon them.

And that's normally where people can find success. It's just you don't find too many people out there that are self-learning and continuous learning, things like that. They like to learn an aspect about their job and then they focus on that. And if they don't have to explore more out of that because their job doesn't entail that, that's fine with them. But the thing is that limits their value and also limits the value, the amount of value they're able to bring therefore the amount of money they're able to earn. And also I can't imagine doing that. It'd be uninteresting to me just doing the same thing over and over again. So...

Jonathan (31:34.534)

I agree, yeah.

David Linthicum (31:35.745)

Your ability to play many roles and certainly with the DevOps engineers and developers, things like that, understand how databases work, spend some time on that. I know developers that understand databases just know how to call an API, data comes out and data is pushed in, how to engineer those systems, how to deal with performance tuning, how to deal with platform tuning issues. You know, all these things that really were kind of foundational to me. I'm 61 years old, so I've been in this business a long time and we programmed everything at the assembler level. So it was very.

Jonathan (32:05.068)

Hmm.

David Linthicum (32:05.221)

primitive in what we did, but it's not necessarily foundational to things out there. And now I'm hearing kind of scary talk, like, why would we learn how to code? We have AI systems that are able to code for us. It's never going to be the case. We're going to have many of the system components that are built using AI, and they're going to be handing tools to reuse basic different IP components and different coding standards and to reflect different behaviors and building APIs at light speed, things like that. But we're not necessarily...

Jonathan (32:15.395)

Yeah.

David Linthicum (32:34.433)

going to have the skills to do this in a holistic way. And I think that's where we're missing the boat right now in the IT industry in general, I think. It applies to many different spaces.

Jonathan (32:48.663)

Mm-hmm. Yeah. One frustration I have is with the sort of just the education system as it relates to software development. You know, and I fell into the same trap. I went for a CS degree, not realizing that it was more theoretical than practical. And then I got a job programming and didn't understand the first thing about version control and how to work with product people. You know, all the 80 percent of the job I didn't understand. I knew how to write code.

Kind of. Not very good code, but I need the syntax. It sounds like you're talking about a similar problem, not specifically about code though, but just the broader sense of the IT landscape and DevOps. A lot of these things aren't being taught or they aren't being taught broadly, and there's a huge skills deficit just across the board, I think.

David Linthicum (33:39.585)

Yeah, and I think it's getting worse as time goes on. You would think that would get better as people understand that we have better technology and things like that, but I'm just not seeing the discipline and the engineering vibe and how people are building systems. They're not building systems to be productized under themselves to function in a highly efficient way. They're building systems to, in essence, get to the requirements of the application and they move on. And they're getting away with that because there's no one's questioning that. So...

Jonathan (33:44.857)

Mm-hmm.

David Linthicum (34:08.453)

In other words, if you put somebody in production and it works and it asks the behaviors you're looking to get out of the system, people are fine with that because you're getting to the end goal and you spent the least amount of money to get there. That's probably how with cloud right now is people are migrating to the cloud doing lift and shift and not changing anything and the thing's terrible and running in the cloud. So the ability to kind of think through what that is and what changes need to be made really is a discipline that needs to be put into the culture of the organization. I do see it every once in a while. I do see people who kind of understand that.

and then get into the proper engineering of systems. But for the most part, we're able to get away with that stuff. And we, as human beings, have a tendency to get away when we can get away with, with the least amount of time and effort and things like that. So I'm not sure how to fix that. I think it comes down to a cultural issue. And I think it comes down to a training issue. Even you look at other cultures in other countries, they have different ways that they approach engineering disciplines and things like that based on the culture of that particular company that they operate in.

Jonathan (34:38.921)

Mm-hmm.

Jonathan (34:47.927)

Mm-hmm.

David Linthicum (35:07.361)

Now we're seeing that in a microcosm of a company and people are kind of going to adapt to what they think the culture is going to be acceptable and they're not asking the proper questions. In many instances, they get punished for asking the questions because questions lead to problems which lead to effort to fix the problems which leads to money and spending which leads to overages on budgetary control. In the United States, we go for quarter on quarter growth. We really don't look at what we're generating in terms of value.

Jonathan (35:19.458)

Mm-hmm.

Jonathan (35:34.645)

Mm-hmm.

David Linthicum (35:35.269)

And the other, like I said, the core thing would be, and I mentioned it earlier, think in terms of product development. In other words, how we're building and engineering products. We're not just building a one-off application that's gonna be used by a few people. We should be building this thing as a product that can be productized, that is gonna have version control, is gonna have a roadmap, is gonna have a life unto itself, and therefore it's able to bring more value back to the business. Also, it's gonna be engineered properly.

because they're engineering as a true product. And I never understood the difference between the two. When I was a product development, I was a CTO and building technology. In other words, programming products I had to run on multiple platforms. And then I worked in an environment which is the corporate world. We're building applications just to work with a known group of users. And it should be engineered the same. You should put the same time and effort that it gets to building that application as you're building a product, as you're putting it into production.

I'm sorry, putting into an application development stuff. And it's a culture that we don't hear a lot about. And we don't hear people think in terms of building products. They think in terms of getting to the end state. I think it's going to be a case where people have to touch the stove. I think there's going to be more inefficiencies that occur in 2024 and 2025. And

Jonathan (36:44.642)

Mm-hmm.

David Linthicum (36:55.125)

One of the nice things about good and bad things are running on cloud. The good things is you don't have to own the infrastructure. The bad things, they make you pay for using the infrastructure you're able to use and do so per drink. So therefore, efficiencies, vulnerabilities, things like that are going to become under scrutiny. Also, we have FinOps in place, financial operations, which is going to be part of the DevOps chain in terms of making applications that are most efficient from a cost standpoint. We're going to be able to understand and get metrics off of that in terms of efficiencies. People have to pay for inefficiencies, by the way. They're quick to fix them.

And we're just going to have lots of different data that comes at us next year, where we realized that a lot of the stuff that we produced was crap and we're going to have to get back out there and start redoing it and fixing it. And the governance of these systems, the observability stuff, how we're running in operations, we just didn't see it in 2020, 2021, 2022 as we moved in this mass migration into the pandemic and then massive application migration into the cloud.

we're gonna have to see it now because the bills are coming in and they're just out of the budget. And so efficiency really kind of comes down to a business decision that you need to make, even though there's lots of other benefits from it. And certainly the vulnerabilities and efficiency we talked about. But it's gonna have to be a different cultural perspective. And I think we're gonna get big bills, as I mentioned earlier, and people are gonna respond to that with suddenly, efficiency, vulnerabilities, everything we're talking about this podcast become a priority.

Jonathan (37:57.451)

Hmph.

Jonathan (38:02.985)

Mm-hmm.

David Linthicum (38:21.849)

culture to get to efficiencies and proper engineering and all that kind of stuff. And it's going to just kind of change the shape and the way in which we do development.

Jonathan (38:22.381)

Mm-hmm.

Jonathan (38:33.062)

Lately, there's been a little bit of a maybe a shift in focus. You know, I've heard, you know, DHH recently posted about moving off of the cloud to private infrastructure. Amazon had a popular blog post about switching from serverless to different infrastructure to cut costs on their own hardware. So we started to see this. Everybody was jumping into the cloud for the last decade or so.

Now we're starting to see, I think, people pushing the brakes a little bit on that. They're realizing the cost and to some extent that it's harder to secure that. What would you say to the people?

David Linthicum (39:13.532)

to secure things in the cloud or secure things on premise?

Jonathan (39:16.87)

Well, I think I see people being afraid of both directions. Certainly people feel like when they go to the cloud that they have less control because it's not within their four walls. Whether that's true is maybe a more complicated discussion. But I'm just curious, what do you see with regard to this? I don't know if it's fair to call it a trend yet, but at least some of the rhetoric about pushing back on the cloud, what are your thoughts in that area?

David Linthicum (39:46.273)

Yeah, it's called repatriation. And I wrote one of the most read blogs that I wrote this year for InfoWorld was 2023, the year repatriation, because I saw it as a pattern. And what occurred in the last 10 years, the hardware prices have dropped substantially, data center spaces have dropped substantially. So it makes in many instances a better business case to run something on premise, whether it's in a private cloud or even legacy infrastructure than it does on a public cloud provider. For security, I would argue and...

Jonathan (39:56.598)

Mm-hmm.

David Linthicum (40:14.925)

that since we're spending so much R&D dollars on cloud-based systems, they have a tendency to be more secure than some of the on-premises. And I think that's kind of illogical to a lot of people because, well, this is next to me and this one's 2,000 miles away, but that can be the case if you use those systems correctly. However, when you look at the costs, and that's what's driving a lot of this, things were moved into the cloud as quickly as they could. The engineering wasn't being done. The efficiencies weren't corrected there, and they weren't leveraging cloud-native technologies to make it happen.

Jonathan (40:32.223)

Mm-hmm.

David Linthicum (40:45.189)

credit stuff on premise and put it in the cloud, and it just became credit stuff in the cloud. And it burned too many resources, and they got these $100,000 cloud bills when they expected $10,000 cloud bills. And so that's when they started asking the question, why did we make this move? The thing runs as well on premise. By the way, the cost of storage on premise has gone down 120% in the last five years. And certainly the hardware it's able to run, why don't we just put it back?

Jonathan (40:49.696)

Mm-hmm.

David Linthicum (41:10.761)

on premise for now until we figure something else out, even if it's just going to be a holding pattern to end up moving into the cloud. But my take on this is you're looking for platform optimization and you should never be a cloud enthusiast. This is about optimal platform enthusiast. In many instances, that's going to be on premise and sometimes in the cloud.

Jonathan (41:25.378)

Hehehe

David Linthicum (41:35.713)

I would make a case there's some reasons to move to the cloud because of where the technology is migrating. You're finding that security systems are kind of suffering on on-premise because they're not putting as much investment in them. So in many instances, you're going to be more vulnerable on some of the on-premise systems, but you're not going to be a million dollars more vulnerable. You'll be able to fix those things if you're able to run them on-premise for a greatly reduced price. And I think that's a good point.

Jonathan (42:00.791)

Mm-hmm.

David Linthicum (42:01.869)

That's what people are going through right now. So they're making a painful decision. I can't imagine going to the CEO and saying, hey, remember how we moved 100 applications or 1,000 applications in the cloud? Hey, we're moving them back. And by the way, remember the data center space? We decided to lease out, we're gonna have to lease some more. But they're doing so for logical reasons. And I applaud them for doing it because it's politically, it's gotta be a very tough thing to do to kind of look at the fact that we made the mistake. But what happened during the pandemic.

Jonathan (42:12.502)

Mm-hmm.

Jonathan (42:17.131)

Yeah.

Mm-hmm.

Jonathan (42:24.307)

Mm-hmm.

David Linthicum (42:28.829)

I saw this occurring in slow motion. We went from like 55 miles an hour and people who are migrating to the cloud to 70 miles an hour and people who are migrating to the cloud. There wasn't a lot of thought and planning that went into the applications that moved into the cloud. People moved too quickly. They had the expectation that the cloud was gonna be better for remote work environments and for supporting systems that weren't as vulnerable. Some of my clients, for example, couldn't get into their data centers because they were quarantined. But in doing so.

They made moves and decisions that weren't necessarily to the benefit of the business. Now they're hitting the reset button on some of that stuff. Not everything, but many instances. A lot of people are doing, I think the wise thing as well, is modernizing in place. When they had migrated stuff into the cloud, then they're doing the modernization of the system where they are building in the efficiencies and removing some of the vulnerabilities and going through...

Jonathan (43:14.12)

Mm-hmm.

David Linthicum (43:22.965)

going through a development and design operation to redo some of those existing system. That's a tough thing to do. It's gonna cost you some money. Some people are moving to cloud native systems, containerization and Kubernetes. And there's reasons to do that, but it's gonna be incredibly expensive for many instances to move that. And then some people are, like I said, pushing it back on premise. The dumbest thing to do right now is to leave a inefficient application on a platform that's costing you more money than it should. So either you're gonna fix it in place or move it back to where it came from.

Jonathan (43:47.918)

Mm-hmm.

David Linthicum (43:51.525)

to get access to cheaper hardware platforms, things like that, or maybe even move it over to another cloud. People are accepting these $100,000 bills now. It's just kind of the way of the world in terms of we're on the cloud now, so we're with the cool kids. That's a good way to run your business into the ground because therefore they're spending money and they're not getting the value back from you as an IT pro. That's kind of the goal of what you need to do.

Jonathan (44:14.338)

Yep, yep. Good.

Jonathan (44:20.738)

What else would you like to discuss? Is there anything else I should have asked about that I haven't? Any other topics you want to touch on?

David Linthicum (44:26.597)

We can talk about FinOps, which I'm part of a DevOps. I've been focusing on that. I have a course out on LinkedIn learning that addresses that. I think that's an interesting topic. And then.

Jonathan (44:34.164)

Okay.

Jonathan (44:38.942)

Okay. So yeah, let's cut back in then to whoever's editing. You can cut out that little interlude there. So tell me, I understand that you have a course out recently on FinOps. Let's talk about that a little bit. What does the course cover, who's it for?

David Linthicum (44:54.273)

Yeah, about a few months ago, and it's for it's just basically a basic course on what FedOps is and how to leverage it both within a cloud environment as well as well as a DevOps environment. And it's a kind of a rethinking in the way in which we're dealing with cost management in terms of cloud systems. So, you know, it's recently it's three, four years ago, we would get a bill, you know, it shows how much storage we use and how much compute we used and we would have to pay the bill.

But we didn't know what purpose, what application, what reason it was done. And we really kind of had no way of understanding what was efficient and not efficient in making these things work. So Phenops came along really kind of financial operations and there's a Phenops org out there that's kind of creating the base level of understanding. Lots of companies are in there with their own Phenops tools where we're able to put accountability and control and governance around.

cloud spending. So if we're, and that can be across a single cloud environment or multi-cloud environment, even dealing with private clouds and legacy systems, so we know because we're spending money on that stuff as well. So the beauty of that is we finally get accountability and understanding as to what resources are being burned and for what purpose, and therefore are able to adapt to control and optimize those resources in ways we just didn't do before. And even like part of the DevOps process would be

building and engineering applications to be good FNOPs citizens. And so they're able to produce information that's needed to understand how much resources they're consuming and therefore how much they're spending and the ability to adjust those resources as needed. So what happened was it just basically shined a light in the corner and saw the cockroaches. So we just had no visibility. It was too coarse grained. We were, people who were developing systems were.

over provisioning in many instances, and that's a common issue with DevOps. People, when they deploy stuff, if they can deploy, they can provision five servers where only two are needed, and they just leave them running forever. And they have no understanding what they're paying for the system, because normally you're not seeing the bill. They're just the developers sort of pushing this into production. Now we understand it can see it and can make adjustments as to what resources are being deployed and what those resources are doing.

Jonathan (47:03.05)

Mm-hmm.

David Linthicum (47:10.769)

And the FENOPs provides us with not only the ability to see this information in a way that's going to be useful to us, who, what, when, where, why, all those sorts of things, but the ability to have the automation techniques to auto shut these things down, to give people and provision them with a certain number of resources that they're allocated for and budgeted for, and also accountability of these resources. So if someone is burning through $200,000 of storage when they really just need 10.

And normally for no good reason, you're able to see those and adjust them and spot them. So we're seeing huge interest in this right now, and there's certainly huge interest in that course, in the fact that people are seeing this as a discipline and something that was missing from the core cloud computing stuff. And I always thought this was going to be the case because cloud, at the end of the day, is a utility just like our electricity and water, which we're paying for.

And it's a complex set of resources. And it's coming from different cloud providers that have different pricing models, different ways of provisioning systems, different ways of billing for these systems. So the idea is that we're going to put management and operational observability around those systems. And not only that, so we can see where the issues are, but the ability to have automated approaches to optimizing these systems automatically behind the scenes. In other words, the humans don't have to do them. We're moving humans from the character.

from the whole process and get to a point where it's going to be near optimized, not optimized completely, but the ability to kind of have the fact that if we're spending $100,000 on our cloud bill, that we're using every bit of that $100,000 for pragmatic purpose in running these applications and supporting the storage systems. And we didn't know that before, and in many, most instances that wasn't happening. They were spending three, four times, sometimes 10 times the amount of money they needed to spend because they didn't have these controls.

and disciplines in place. The other thing is moving into multi-cloud environments we're deploying. Google, Amazon, Microsoft, and many instances, 95% of the people out there deploy, leverage more than one public cloud. Those things are all different, how they bill are different, and so therefore we can have one common billing platform and monitoring platform, observability platform across those platforms. In the olden days, meaning last year, we had to look at the native tools.

David Linthicum (49:27.977)

and the accounting systems and the billing systems for each of the platforms, Amazon, Microsoft, and Google, and somehow make an understanding as to what's going on and make that happen. Now, and it's incredibly complex unto itself. No one had a good handle on it. Now we're able to see those stuff in context of each other. Great thing about that is, hey, guess what? Storage is cheaper on Google than it is on Amazon or vice versa, and therefore I'm able to allocate as I start moving into different, deploying different applications, they're able to pick storage systems.

they're going to be much more cost efficient based on the fact that I can see the differences in number one, the resources that I need and the pricing of these things. And the FnOPS tool will actually make the decision for you as to where the best resource exists for the most cost efficiencies and how it should be allocated. So in other words, you're requesting something to the FnOPS tool, and you're not just saying, I need storage on Google, I'm just saying I need storage. And it's allocating the storage on the system that you need, returning the resource.

Jonathan (50:20.17)

Mm-hmm.

David Linthicum (50:24.845)

to you that you're leveraging for the application. And then it participates in the monitoring observability pool as it moves forward. And it may change efficiencies as well. It could report back that said, well, we've leveraged this storage and the prices have gone up and we're able to move to this other storage for a certain amount of money, certain amount of testing. And this is something we should do to reduce our bill. So it's becoming smarter in leveraging resources, very complex resources in better ways. And also,

getting fin ops in the hands of people that can do the most good, namely DevOps engineers and developers, operators, things like that, the ones that are actually spending the money.

Jonathan (51:02.83)

Mm-hmm. Good. Yeah, there's a lot of room. I've seen it at various startups I've worked with and I've been just talking to other people. There's a lot of room to improve in that area. So just like you said, so many people who are increasing the bill don't have any visibility into that bill at all and don't realize that what they're doing is just burning money in some cases.

David Linthicum (51:30.337)

Yeah, and some of the cloud providers would come back and they wouldn't have any granularity into what the bill was. So you had no line items and understanding what users were building something, what regions were running it. They would just go, storage, 80,000 bucks, pay us. And I think that was acceptable at the time because people were just really kind of getting into the whole on-demand model and didn't understand they could optimize that model, which they're understanding now after they got the big bills that we saw last year. So in many instances, cloud was...

Jonathan (51:57.911)

Mm-hmm.

David Linthicum (51:59.493)

Cloud systems are shut down because they couldn't afford them. And so had to figure other stuff out. And so we had the repatriation we just talked about. And so now we just have visibility, better understanding, observability across these various systems, governance, and again, accountability. I think that's the big thing is people were just abusing the fact that they could allocate resources that they didn't have to go buy and install in a data center. It was pretty much auto-magical for them and just got drunk with power, I guess.

Jonathan (52:15.927)

Mm-hmm.

David Linthicum (52:29.401)

provision too many resources. And also many people, when he asked the developers, you know, I said, how much do you think that storage system costs? And they would quote me something that was probably a thousand times cheaper than it actually was. So they didn't understand how much it was actually costing because again, they didn't see the bill. You had the accounting system and the financial organization saw the bill that came in, they paid it and there was no visibility from the people who are actually consuming this stuff. I think that's the biggest change that's occurring.

Jonathan (52:56.554)

Yeah, yeah, great. Well, hey, David, how can people reach out to you if they're interested in following you whenever you have a new book come out or otherwise your blog posts, how can they get in touch with you and follow you?

David Linthicum (53:08.717)

Yeah, well, I'm on LinkedIn. Check me out there. I'm more active on that. My book, Insider's Guide to Cloud Computing's out there. It seems to be doing pretty well. I have 50 plus courses on LinkedIn learning, all cloud and DevOps related. If you're going to Louisiana State University, take my DevOps courses. I teach out there as well. And follow me on the, and make sure to follow my InfoWorld blog. I post there twice a week. I've done that for 12 years.

I'm focused on cloud computing, DevOps, AI, how that technology is evolving. Kind of more of a pundit in the space. I take a pragmatic use of the technology and so you reach out to me if you need me.

Jonathan (53:36.482)

Nice.

Jonathan (53:48.834)

Very cool. Well, as we usually do on this show, we like to end with some picks. Do you have anything that you would like to recommend our audience check out?

David Linthicum (53:58.081)

Yeah, I think the Dead South is a group which is kind of punk rock meets bluegrass. They have a video called In Hell I'll Be in Good Company out on YouTube that has about a billion hits out there. It's just I kind of found them to be an amazing group with an eclectic array of things and I just get excited when they come out with a new song. So I'm a big fan with those and I think you should check them out.

Jonathan (54:25.014)

cool. I will definitely check them out. I haven't heard of them, but I see them. They're on my browser page now. So after we hit stop, I will at least listen to one of their songs and see how I like it. Great. So I'm going to pick something a little more along technology lines. I just finished reading the book, modern software engineering by Dave Farley. It's a great book. I think I mentioned it last week too, but I recommend anybody read it or even listen to it. It's actually one of those books you can even listen to an audio book, which is what I did.

There are a few diagrams you miss out on, but it's still easy to follow along. If you really need the code examples to understand the concept, you can go read the PDF that comes with it later. So I...

David Linthicum (55:04.239)

What was the fundamental lesson on the book? I'm always interested in hearing that. That you took away, it was like, mm-hmm.

Jonathan (55:07.97)

The fundamental lesson of the book, I would say it helps recognize when we're using the scientific method and how we're using it to do software engineering. Basically, we aren't doing random stuff to see what sticks to the wall. We shouldn't be anyway. We should have a hypothesis, aim to achieve it in some way, and then test our assumptions and see if they're validated.

And then, so I would call it the underlying premise of the book, and then most of the book is, what are the practical things we can do to accomplish that, to implement that sort of process?

David Linthicum (55:45.741)

Yeah, that's nice. I like that. It's good pragmatic advice, things like that. I'll check out that book myself and it's on tape as well.

Jonathan (55:51.798)

Yep, I listen to the audiobook myself. It's easier for me to do that for some books because I can listen on the bicycle or when watching the kids or whatever.

David Linthicum (55:59.253)

Yeah, I'm recording my book now on tape and it's tedious process. It's about two hours a chapter. I didn't, I didn't know it was going to be as hard as it is.

Jonathan (56:04.57)

Yeah. Well, good luck to you on that. Well, thanks, David, for coming on. It's been a pleasure. Interesting conversation. Come back any time. All right. Thanks, everyone. See you next week.

David Linthicum (56:08.121)

Thank you.

David Linthicum (56:15.695)

Anytime. Thank you very much.

From Hack to Holistic: Firsthand Experiences and Knowledgeable Interactions in DevOps - DevOps 171

0:00

55:56

Playback Speed:

Show Notes

On YouTube

Sponsors

Socials

Picks

Transcript