Peter Elger and Eóin Shanaghy join Charles Max Wood to dive into what Artificial Intelligence and Machine Learning related services are available for people to use. Peter and Eóin are experts in AWS and explain what is provided in its services, but easily extrapolate to other clouds. If you're trying to implement Artificial Intelligence algorithms, you may want to use or modify an algorithm already built and provided to you.

Hey, everybody, and welcome back to another episode of Adventures in Machine Learning. I'm here with Owen Shaughnessy and Peter Eljer. You guys are the authors of the AI as a Service book for Manning. And, yeah, just chatting with you briefly, it sounds like you're also AWS consultants, so you know kinda what their raft of services are. And, yeah, you're experts in helping people set up AI systems that live in the cloud, which is is really, really interesting and using kind of the prebuilt systems that they provide you because, yeah, I'm I'm not a math whiz that can invent this stuff and it sounds like you're not either.

So, yeah, real quick before we get going too far, how do people connect with you all if they have questions once we're done talking? Yeah. Sure. You can get us on LinkedIn or Twitter or you can come you can mail us directly. We're always open.

We have some open source up on GitHub as well, so, you can you can come and, take a look at that and and message us there as well. Very cool. We'll we'll put links to all that in the show notes. If you wanna just drop it in the chat, that's how we get that in there. And, yeah, let's let's go ahead and talk about AI as a service because yeah.

This sounds like kind of my AI implementation is, oh, somebody else has built this, and I'm I'm just gonna, like, look smart by using it. So what's out there? Let's just start there. What's out there and available for people to use? Yeah.

So there's there's a ton of services. So I suppose the rationale for writing a book, both myself and Owen have been in the software field for more years than we we care to remember. And we've both been very excited by the rise of of AI technology, but not every not everybody gets to work on self driving cars or write train, you know, image recognition algorithms. But what's Oh, I was doing all that stuff last week. So Of course you are, mate.

Yeah. Yeah. Of course you are. It's not me. Yeah.

But what's become what what became very clear at the time we started writing this book, which was in, 2017 when we first started writing it, was that the one constant in our industry, and there aren't many constants, but one of the constants is the force of commoditization. Right? Everything commoditizes. You've been in developer for long enough. You'll you'll have seen this.

And AI bit was starting to commoditize then and is commoditizing further now. So, really, for us, the the aim was to take some of the knowledge that we have of AI machine learning. We both worked it on various different AI projects in the past, not to the self driving car level, but to realize that that that commoditization is happening and that the the technology and the techniques are now within the range of, you know, most developers. So we want to provide an engineering guide for those folks to how they can get started with with these AI systems and apply them to their work to solve, you know, business problems and problems in their day to day work. Over the course of writing the book, the scope and range of of AI services available from Amazon, because that's what we focus on, has grown just in 3 years since we started writing the book.

So we have to include some more more in there. Owen, maybe you wanna talk a bit about the range of services that are available today? Mhmm. Yeah. Sure.

I mean, I I'd kind of echo what Peter said and also say, you know, if if you're interested in getting into machine learning let's face it. A lot of people are because it's it's just a buzzword, and you go googling around for tutorials. A lot of what you'll see is kind of Python tutorials and using TensorFlow or PyTorch, some machine learning framework and, you know, helps you get up and running with with deep learning, which is really interesting and really cool. And we all like playing with kind of deep technical toys, but when you actually have to do something real with us, you know, you actually have to figure out how to manage your data, how to run your algorithms, how to deploy that and maintain it. And that suddenly you've got a massive set of tasks on hand.

So, yeah, what we really wanted to do is say, okay. That's really a lot a lot of that kind of field of expertise, you know, learning doing Python machine learning with those deep learning frameworks. It's re it, you know, it has its place in a lot of innovation and a lot of academic academia. But for a lot of business users or a lot of, you know, your 90% of developers just wanna figure out how they can actually make hook this into their systems. Uh-huh.

So it's about, like, you know, what can you use off the shelf without having to go writing assembly code? You know? Because that's that's what it's kinda like when you're when you're fine tuning deep learning models. You know? You're down at the the at the depths that 90% of people don't necessarily wanna be doing.

And, you know, it's not gonna be efficient cost efficient to do it that way. At the same time, you know, companies like Amazon and Google, Microsoft, they've got access to massive volumes of data to train and fine tune algorithms. And that's what, you know, that's what they use to build things like Alexa, and then they give it to you as a service in terms of Lex or Poly or, AWS Transcribe, you know, services for doing chatbots, for doing, speech recognition, speech synthesis. And then, you know, it did there's many other services besides something where you could look at forecast for predictive analytics, personalized for, you know, recommendations, and then you've got, recognition for recognizing features of images, features of videos. There's a huge amount of cool stuff you can do there.

So I I'd recommend for people playing with the loop machine learning. It doesn't necessarily have to be about, you know, getting a GPU machine and spending quite a lot of money, you know, figuring out how to train deep learning algorithms. There's a lot of there's a lot that you can learn. A lot you can treat very quickly by playing around with some of those AWS services. Makes sense.

And it sounds it sounds really interesting just to dive into some of these. I will also point out that, you know, and it's not it's not necessarily calling anything out on AWS or anything, but I've, over the last few years, been invited out to several Microsoft events. Right? And they talk cognitive services, and they offer I know that that I've heard them talk about some of the things that you mentioned. I think Google Cloud Platform and Oracle Cloud also offer some of these same services.

So wherever your stuff lives, there's probably some form of this you can go look at and see what they have to offer. And I've also seen people do multi cloud or cross cloud stuff. Right? So if you find, oh, well, you know, they don't have the predictive modeling at one cloud, but they have it over at Amazon. You can actually push your data across the Internet to Amazon.

Let them do what they're good at and then push it back. And so it's it's great to just see all of this innovation in this space and go, okay, I don't have to go invent this. I have to kind of have a cursory or maybe I'm wrong, you know, maybe you do need to have a deeper knowledge of the the algorithms. But I I kinda have to know how it works and what I want out of it, but I have to be an expert. Yeah.

So maybe I'll just and you're you're right. Okay? But there's a couple of points to follow-up on there. Right? So first of all, yeah, most of the cloud providers have very similar services in terms of like image recognition, natural language comprehension, and so on and so forth.

Right? If some like I said, one provider is specifically ahead in a particular area and you need that, then a hybrid cloud approach is fine. I would just say just be wary of data transfer costs if you're doing Right. High volume, high scale. Yeah.

The second That makes sense. Yeah. Sure. Right? Yeah.

But if, you know, if if you have the money to spend on it and you're making money from from the the solution you're providing, then, you know, that's just a a business. Right? The other thing, yeah, you're right about understanding You don't necessarily understand all the gubbins underneath. Sorry. That's a European term.

If you're American listening, I will. You don't necessarily need to understand how the algorithms work, but you need do need to be able to interpret the results. So by that, I mean, for example, let's take natural language comprehension. Let's say we're doing something like sentiment analysis. So we feed some text into machine learning AI algorithm through an API call.

We get a response back. And typically, you're gonna get a response back that says it's positive, negative, or mixed or whatever sentiment, but it's all the key thing it's gonna return to you as well is a confidence level on each of those results. So your job as as an implementer of of these types of systems is to understand the confidence level that you score that you get back and interpret that specifically for your domain. So by that, I mean, if you don't if you just want it to be generally positive sentiment and then you can say, well, if it's an 80% score, that's fine. I'll go with it.

If it's more mission critical, then you might need to to tighten that up. It might need to be much higher positive score before you're prepared to take a you know, to to allow it to be processed automatically. And how do you handle with the the mixed sentiment stuff? Are you gonna have some kind of cue system where you you punch stuff that is not, you know, below your confidence interval that a human is then gonna look at? So it's all about understanding and interpreting the results that are coming back and making sure you're making the appropriate decisions for the particular business domain that you're solving the problem for.

Yeah. That makes sense. So I guess my question there because I I really appreciate you, first of all, you clarifying these points, how do I go about getting the expertise to be able to interpret the results? Because it seems like some of these algorithms, it's fairly straightforward. Right?

It'll give you a yes or a no, or it'll give you something really, really clear. But, yeah, you're talking about sentiment analysis, and, yeah, it's it's 80% or 70%. You know, how how do I know what's good enough? Is it trial and error? Is there some education I can I think it's go get somewhere?

It's it's more about involving the the the business stakeholders in those decisions. Right? So it's like Okay. Look, we are going this is what we we're gonna get back. If we, you know, if we put the confidence level at 99%, then that means we're gonna get a whole bunch that need to go for human processing.

Right? Because it's not but if we drop the level down, there's gonna be less human processing involved, but what's the upshot of an error? Right? So if the upshot of an error is well, we send a message, a marketing message out, and it and it might go to, like, 5% of the wrong people, probably not not not a big deal. Right?

But if you if that's then some kind of contract checking or something like that, then there are deeper business ramifications if you get a wrong interpretation. So it's really it's really a business analysis decision rather than a than a technical decision, I would say, most of the time. I gotcha. That makes sense. That makes a lot of sense.

Now the people that I'm trying to talk to on this show are mostly implementers. Right? So they're they're going, okay. How do I build my thing? Right?

And you kind of get things started in your book with building a serverless image recognition system. And and so I'm a little curious if we can just, you know, for the next 10 minutes or so, just talk about, okay, let's say that this is something people wanna build or maybe there's some other algorithm or system that they wanna hook in on a serverless type of system. How do they go about doing that? Like, how do they go about finding all of the pieces and then putting them all together to make this work? Sure.

Well, they could they could win a copy of the book. I know you have some, codes to give away there. Right? Yeah. Yeah.

We're we're gonna do a contest. I'll announce it at the end. Yeah. Sure. So so really, I suppose the the other thread to to what we do as as a as a company and as developers is we're big believers in serverless.

And, again, that is the, you know, the the march of commoditization is everyone's gonna go to serverless. So to get started with with serverless and there's a lot of great tutorials around. There's, you know, there's a bunch of books around you can you can you can learn. But, really, the key thing is if you can understand how to call an API Mhmm. But at the fundamental level, if you can call an API, and you can understand how to use a framework, so we would use something like the serverless framework.

Yep. I love the serverless framework. Yeah. It's it's awesome. So easy.

Yeah. Exactly. Right? If you can do that, you understand understand a bit of YAML, you can understand how to use the service framework and you can make API calls and you can get started. It's kind of that simple.

And most developers, will be able to do that. Right? Yeah. One thing I'd point out here is that unlike a lot of machine learning books, we deliberately decided to use JavaScript as a language for the book. And this is kind of against the grain.

You're welcome. I'm glad it's appreciated because, you know, it was a surprise to a lot of people that a machine learning book would would use JavaScript because Python just seems to be, you know, the lingua franca of machine learning if you like. But, you know, at the same time, there's a lot of web developers out there. For for machine learning developers or data scientists, they're probably gonna choose a different path anyway. So this is more about, like, web application developers, Node.

Js developers, people who are familiar with, you know, full stack developers, making it accessible to people like that. Essentially, like myself and Peter, because that's our bread and butter for the last decade. So, that's why we chose JavaScript. And, you know, it was I think it was really vindicated in in terms of, you know, it it's it's gotten good feedback from the users and it also hasn't made any difference in term any negative difference in terms of the applications we were able to build. Now the AWS SDK works equally as as well in JavaScript as it does with Boto 3 and Python.

What if you're not gonna be doing any of the data analytics yourself, if you're using a managed service, it really doesn't matter which language you're you're using for your back end. Yeah. I was actually going to point that out and ask, right, that exact question because yeah. I mean, I don't I'm not proficient with Python. I'm sure I could pick it up.

I've been programming for 15 years. Right? But the reason that I see a lot of people using Python is because it has all these algorithms and, you know, has the math libraries and stuff that that are built in, and they're pretty fast. And so at the end of the day, it gets all this work done. But if I'm outsourcing that to a service, then, yeah, why can't I write the glue code in whatever programming language I want?

Yeah. It was just So Yeah. I mean, like, with regard to Python, I mean, both both myself and Owen do a fair bit of coding in Python as well. And I I kinda like Python. And you're right.

It does have those very efficient math libraries, which are great for data science and machine learning if you're doing it the kinda lower level. Right. But the whole point of this book is to say you don't need to do it at the low level. If you wanna get business results, there's stuff there for you. You can just call it.

And my preferred language is just happens to be Node. Js and JavaScript. Right? Yep. Yeah.

I grew up doing Ruby and Ruby on Rails, but JavaScript's kind of the next language I would reach for. I was gonna joke, you know what? I'm gonna go ahead and write the glue code in Eiffel or something. But, I think I think you can if you want. I thought it was great language.

That was, he was that. Burnet something you wrote that book? John Mayer. Yeah. And he introduced the concept of, class and variance and, and all of that kind of stuff.

Good language. Yep. So so yeah. So But I think most most of the time, the fact of it is we end up processing JSON more often than not in these applications. And that's where we spend, you know, 80% of our time as developers if we if we're gonna be honest about it.

So, why not JavaScript? You know, it's it's definitely the best language for processing JSON. Yeah. Yeah. Well, just about any language that I've ever even looked at has a well put together JSON library.

So I I think you're pretty safe there. So you set up the serverless, and then you just I can't remember exactly how AWS authorizes all this stuff, but you essentially yeah. You just get your authorization for the AI systems or the machine learning systems, and then you just make the API calls and that's it. The the permission model for each service is is very if you're familiar with permissions for buckets or so on, it's very similar permission model. All you do in your serverless YAML code is to make sure you give the appropriate permissions for the Lambda function to call Right.

The API in the same way you would with any other service and and off you go. Now one other thing that I'm curious about is do you put the data processing in that same serverless function, or do you wind up having a separate process for that? I suppose the answer is it depends. Sorry. You guys really are consultants, aren't you?

We are. Yeah. How did you know? It it depends on your application, right, and the volumes of data and stuff that you're you're trying to work with. You can certainly so I guess some of the APIs need to work in a can work in a request response way.

So you pull something like recognition on on an image. So you you're point essentially, what you're doing, you're uploading an image to a bucket. You're pointing recognition of that image, and then, you you know, almost immediately, you're gonna get a a return back with your set of labels on it, let's say. Right? So that can be done in line.

For others other things, let's say you have a larger document that you want to do some NLP analysis on, then typically what you're gonna do is your Lambda function will trigger a more batch type API within AWS, so comprehend, which is the natural language service, will be triggers, and then you would basically wait for wait for responses. So you might call another API that says, okay. It's gonna tell you, yeah, I'm processing, I'm processing, I'm processing, and then eventually, it's gonna finish, and then you you take your results. Yeah. That makes sense.

That makes a lot of sense. How do you go about well, let me back up because I guess I guess the other thing is is you could use something like Apache Spark or you could pull together some other data pipeline system. That Oh, yeah. Right? And then and then you have the cleaned up data, and then you can just hand it off.

Right? And then from there, it triggers like we're talking about. Yeah. Like, if you've got a a kinda core system with Spark or something like that, then you can certainly employ Lambda around the edges Yeah. Orchestration and and or, you know, interfacing to these, these these API jobs.

Yep. Absolutely. So how do you go about making sure that this stuff is secure? I mean, some of the some of this is gonna be, like, securing your serverless function like you would do on any serverless function. Right?

But are there any other concerns with this for security outside of just how serverless works? I think, generally, with this model, we're using AWS with services like this, kinda forces you into a a more restricted minimal privilege Right. Kind of pattern anyway. Unless, you you know, certainly with IAM, with identity access management in AWS, you can you could you can allow STAR to for in terms of, you know, wildcard actions on wildcard resources. But that's clearly not the best practice and minimal privileges is a better practice as as a lot of people doing anything on AWS will know.

What it means is that you get very fine and great access for each function, for each service it's trying to access. And because you're we're, you know, we're advocating for a serverless pattern anyway. Every deployable service with its set of Lambda functions has a very clear minimal set of privileges for an ex very specific set of resources, which makes it much easier than trying to deploy, you know, a monolithic application, which, you know, has a certain benefits for some people. But at the same time, trying to trying to get fine grade security and access control is is much more difficult. Whereas, you know, when you when everything is deployed as tiny fragments, really Mhmm.

You've you've got much more control. And then you just need to think about, I suppose, the security at the boundary. You know, if it's if you have a SaaS application with an API gateway at the front, then you need to start controlling web traffic, which is, you know, it's it's own set of concerns, but not unique to this use case. You know, you need to be thinking about quote quotas and rate limiting and DDoS prevention, authorization, API keys potentially. Been there, done that.

Yeah. And then it makes sense. Right? As far as this is a tiny right? It's a serverless application, so it's just a tiny bit of JavaScript.

It's very well defined needs, and so, yeah, I'll just go turn those on and not turn on the rest of them in my permissions. Yeah. So what have you built with this stuff? Good question. Fair bit, actually.

We've done some fairly interesting stuff. Actually, we had to reengineer something with Rekognition. That was quite an interesting case. So as we were starting to write this book, Rekognition, for example, wasn't available. And we had a client that was needed to do some some it was working in the KYC space.

So it was a need to kinda scan, in Back up just for a second. So I have the I don't know if I'm gonna say benefit, but I work for a company that is in the financial space. And so if you don't know what KYC is, KYC is know your customer. And it's effectively I had to go through all this training, and I'm a developer. Right?

So I don't even have to do KYC. But it's effectively use your tools and techniques to know who your customer is, what their business is, and understand what they're doing to prevent things like money laundering and stuff like that. Right? So 99% of the people you're gonna talk to and do the k y c work on, you're gonna find out that they're exactly what they say they are. But they wanna make sure that they are exactly what they say they are.

Exactly. Yeah. Sorry. Thank thanks for that, Charles. Anyway, we what we this particular piece of the work was to recognize information from various documents like utility bills and so on.

Now when we started that, so comprehend was available, but what wasn't available was was Textract. So we actually had to go build using an open source OCR library, go and automate the scan of the document, pick out do some math to pick out the bits and pieces in the document that looked the right the right shape and feed those through our own recognizer to extract the required information. Then about 6 months after we did that, Amazon released Textract. So what we could have done what took us, you know, good 6 weeks to put together could have been done in a in a few days. Oh, wow.

But that's just an example of, you know, if you if you haven't got a burning need, if you wait maybe, maybe 3 months, you might find that what you need is gonna be released anyway. Right? That's interesting. Maybe you wanna talk about the, some of the IoT, agritech stuff, Owen. You guys work on some projects.

Yeah. We do. We're lucky. What are what are the interesting ones? So there's a there's a not always another level down.

So I suppose the question is and the other we get asked it quite a lot. It's what happens when the managed services don't fit your use case exactly? And, you know, that's that's not an uncommon occurrence. You know, these things are designed for general purpose, image recognition, you know, recognizing faces and photographs, for example. Mhmm.

But we do have a customer who had a very bespoke, kinda, agricultural IoT application where they had cameras in out in the field measuring grass growth. And for that purse purpose, we collaborated with them building a custom machine learning model, which we then employed SageMaker to deploy. So SageMaker is an AWS service that allows you to deploy custom models or actually, you can you can go to the marketplace and pick a pre trained model as well. In this case, it was it was a pre it was a trained model that was trained on a kind of on premise infrastructure. So it was trained on a bunch of GPU machines.

But then when it comes to deploying it, you know, you don't necessarily need that bunch of GPU instances and you don't want to be maintaining it or swapping out disks on it or, you know, managing capacity. So the there's a couple of ways you can do that. And the first way we approached it was to use SageMaker. And SageMaker allows you to just take that model, upload it to an s 3 bucket, and then when you need an endpoint, you can just bring up SageMaker endpoint. So you get a HTTP endpoint that you can just you can post an image to, and it'll give you back an inference result.

So it's it it'll give you back a measurement results. That was that was really nice. And to be honest, it's that that was a that was a really interesting project to work on, and we were able to get that up and running in, you know, a week or 2 actually. It was very straightforward to do. But I think it's actually even easier these days because one of the one of the things we would have liked to have done at the at the time was just use Lambda to run that machine learning inference.

Because it's you know, while it's nice to play with these code toys like SageMaker and these machine learning services, my favorite way to deploy any kind of code is with a Lambda function because there's so little you have to do. Uh-huh. And you can also scale it really instantly and really, really horizontally. So at the time, you know, Lambda had certain restrictions around deployable unit size and memory size. But now for the last couple of weeks, we've been able to deploy 10 gigabyte, Docker images to Lambda.

Wow. So Wow. That's awesome. Exactly. Right?

I mean, this is I mean, I I keep describing that as a game changer. But for applications like this, it really is because Well, if you need it. GPU yet. Lambda doesn't do GPU yet, but you don't for inference, like, when you're just uploading an image or a document and you're trying to get a score from it, Lambda is perfectly fine. Previously, you know, you might have been constrained by the image, the the 250 megabyte deployable code size.

Or if you can deploy a 10 gigabyte container, then you can run it locally, develop it locally, upload it as a Lambda function, and you can, you know, get access to 10 gigs of RAM as well. So there's a Wow. All of a sudden, there's there's a huge amount you can do without even touching any of the machine learning specific services. Right. So why is it 10 gig?

Is that just all of the, I guess, all of the numbers you need that after having trained it to store all that or is it something else It's definitely on, on on Lambda now. Oh, okay. Yeah. For for images. So if you're deploying a ZIP code as a ZIP, it's still 250 megabytes.

Right. But if now you can just do Docker build. So for people who are used to using container tools and container images, You can do docker build. They give you the tools to run those contain or sorry, image based Lambdas locally as well. But you upload them to your container repository to ECR, and then you can run them as from Lambda.

And, you know, the runtime model is obviously very different to using Kubernetes or something else where you're running typically something long running and it's responding to requests, multiple requests concurrently. When with Lambda, you're going to respond to single events. Every container just serves a single event at a time. But the fact that you can deploy 10 gigabytes of images, you know, it's rare. I would be I've normally struggled to come up with a a way to fill that 10 gigabytes.

But the fact that you can go up to 1 or 2 gigabytes is makes a real difference because you could some of these Python libraries especially are really heavy. You know, you can get up to Yeah. Half half a gigabyte pretty quickly. And I spent too much time in the past trying to, you know, strip down. Yeah.

No. I misunderstood. I thought I thought you said you had deployed a 10 gig image, and I was like, it's like, wow. What do you have to put in there in order to need that? But yeah.

I haven't built many models that are that size, but, you know, some people have. I think some of the examples I've seen of people have done, you know, close to 10 gigabytes, and they run, you know, in under a second, which is I don't know how they do it actually, but I I have done an example where I've I've got I took some I took an open source x like, computer aided diagnostics algorithm that from GitHub for many for scanning chest X rays. And the image size was about 2 and a half gigs, but I was able to run 2,000 concurrent instances of it in a couple of seconds and go through 100 of thousands of x rays and get the results and pull those images from s 3 and get the results into DynamoDB. And it was all done in under a couple of minutes. I think it was 90 seconds or something to process a record set that was 100 of thousands of images.

And, you know, the beauty of that is the beauty of that is, yeah, there's nothing there before you run it. 2 minutes later, it's all done and you still don't have anything running. Right. I mean, that's that's great from a cost point of view and a flexibility point of view. But as a developer, it means that as you're iterating on that container image, you don't have to wait for clusters to scale up and scale down.

You know, you have immediate feedback on how well your code is performing. So if you need to run hundreds of cycles in order to optimize your algorithm, That saves you a huge amount of time and developer time is, you know, it's costly. Yep. Yep. Yeah.

I've I've had this discussion with people before where, yeah, they're like, well, you know, we'll just throw more infrastructure at it. And then somebody goes, well, that's gonna cost us several 1,000 more dollars. And then somebody else points out, well, we go through that paying a developer for a week. Right? Yeah.

And this is saving us 3 developers. So, you know, it's easily worth the cost. And so yeah. And and, yeah. I I love the example.

It's like it's like just it's amazing when you really think about it. Right? And what you would have had to do before something like this. Yeah. We're really bad at considering that part of the equation.

Intuitively Yeah. At including developer time. Because we kinda love to be busy and we love to be solving hard technical problems. That's kind of a an instinct we need to kind of fight against. Yep.

As if we can free ourselves up, then we can get rid of a lot of the firefighting headaches that we typically manage to, create for ourselves. Yeah. Well, the other end of that is also that, at least in my case, I like tinkering with this stuff. Right? And so it's like, oh, this would take me 3 weeks to do the other way.

And it sounds like fun. Right? But from the business standpoint, yeah, it's it's much better to have pay me to spend my time working on the other problems that can't be solved in a matter of minutes on AWS. Yeah. Exactly.

I mean Sure. Your point is to focus on on delivering, you know, delivering the solution for the business and what kind of all of these other stuff also to the side. Right? Right. So I I guess the other thing that I'm wondering about is that I know that, at least Amazon, they charge you for traffic, like, for bandwidth.

They charge you for storage, and they charge you for compute. You know, so, like, how many processors and RAM and all that stuff you have on your resources. So we were talking about how cheap it is compared to developer time, but how do you keep the cost down when you're putting something like this together? Right? Are there things that you can do to make it so that you can run as efficiently as possible?

Yeah. I mean, there are. It's you you I I suppose the other thing to be aware of is that when you're using these machine learning APIs, there's a there's a cost for using those those APIs. Right? So you'll you'll build the model's a bit like, Lambda in the sense that, you know, for Lambda, you'll build for I think you're billed at the millisecond interval for for usage, whereas reach each inference, you're billed for each inference.

So if you call an image recognition, you'll build for that inference. It looks you know, and for things like, comprehend, you'll build I think it's it's per per call, but also per volume of data processed as well. So, you know, it's fine when you're doing it at small scale if you're just running a few a few inference jobs. It's not gonna be a huge cost. Just to keep an eye on those if you're starting to to process these things at scale because the cost will add up at scale.

Right? Mhmm. But even so, you're not managing your own infrastructure and so on. So, typically, the the Right. Is is is pretty good.

Right. Yeah. It's I mean, you so sorry to cut across the Charles, but yeah. You you it's something to be aware of. Understand the billing model before you get too deep into it for sure.

But just as we as we just said, you know, compare it to your your total cost of ownership and your developer time and what's the alternative? You know, if you're gonna build if you're gonna try and save money building your own custom infrastructure, because, you know, you gotta take all the costs into account. But, you know, Peter's right. It's it's built per unit. So the some of these APIs can get seemingly very expensive if your data volumes and request volumes are really large.

So at that point, it's worthwhile kinda taking a step back, looking at your cost comparison and talking to your talking to AWS about, you know, how to how to keep that as low as possible. Because, you know, if you're doing a Twitter scale image recognition on using an AWS service, then all of a sudden you're into, you know, something the average credit card on Amazon isn't gonna isn't gonna handle. Yep. Yeah. That makes sense.

But at the same time, what I find is when you get to Twitter scale, the other thing is is that your concerns change. Right? And so you're gonna probably be focused on some things that are a little bit different from, you know, what your AWS bill is unless it's just way higher than you expected it to be. Yeah. Absolutely.

I suppose the thing is just be cautious not to have stuff running away in the background that you don't turn off. Right? Right. Yep. Now, you know, typically, that's gonna happen with with with, virtual instances or whatever where you just get to turn stuff off.

But if you've got a deployment that's pulling stuff from an s three bucket, it's constantly processing it. You know, just just be cautious of those kinds of things. Yep. Yeah. Absolutely.

Always put billing billing alerts in place. So you at least, you know, cost overruns do happen. And typically, you know, they happen you know, if you're using AWS extensively, they happen from time to time. But the question is, can you can you spot them early enough so that they don't make a really any kind of substantial difference? Mhmm.

So if you're spotting them at the end of the day as opposed to at the end of the month, that's that's a massive difference. So, yeah, that there's a bit of groundwork to put in place there, but it's just good common sense to to have those billing alerts in place. Because, you know, we're we kinda have a fairly good understanding of the billing model from many of the services, but, you know, there's a there's a huge amount of variation in there. And keeping it all in your head is quite difficult. So everybody needs a, you know, at I I suppose an alarm bell that they can they can see going off that they can, you know, review things pretty quickly and and scale things down as necessary.

I turned my video off because it's starting to get choppy and I I think it's my Internet connection here. So Oh, that's it. Should we do the same? Yeah. That'll probably help.

We're just about done anyway. So so if you wanted to just kind of wrap all this up and pull all these ideas together, I kinda have a vague idea of how to do it, but you guys have been talking about this stuff and considering this stuff a lot longer than I have. So what's kind of the core message here? What's the what's the core idea, and how do we pull all this stuff together for a listener? Yeah.

I guess the so the the core message is that AI is no longer the the, the domain of guys with PhDs. It's well within the scope of any developer who can call an API. So really to to be effective with it, we we, you know, treat it as an it's now an engineering problem. Right? Mhmm.

It's no longer a research problem. So it's more about application of the technology. The quickest way to to get up to speed and start applying this technology to to business problems you might have is to start at the kind of service level, so make API calls. If that doesn't quite solve your problem, there's a middle ground. A lot of these services can be cross trained.

So that's a really important point is that you don't have to go from, well, Comprehend doesn't solve my problem, therefore, I need to go and employ a team of data scientists to solve my problem. There is a middle ground of actually cross training existing services to fit your specific domain. So start start at the kind of high make service core level, look across training, and then if that still doesn't solve your problem, then you need to go back to services like SageMaker where you train your own your own your own models. But it's all about looking at the business challenge, fitting a service or services to that problem, and then understanding and interpreting the results. Great.

Yeah. That makes sense. And it makes it sound so much more approachable than, yeah, kind of the PhD level stuff that I think a lot of people really do kind of put machine learning into because it really wasn't that long ago where it was in that domain. Yeah. Absolutely.

But, you know, exponential technology growth and all of that. Right? And I think over the coming over the coming years, we're just gonna see more and more, commoditization of AI and and more and more growth in the in the kind of range and capability of these services. So, you know, as a as a piece of career advice, I'd really advise people to to just get up to speed with some of these services because the chances are sooner or later you're gonna need to apply them. Yeah.

That makes sense. One other thing that just came to mind while you were talking about that, you know, with the commoditization of things like machine learning or AI, It's and it really came out when you said the exponential growth of technology. And it seems like, at least over the last few years, society at large has really scrambled to try and keep a pace with how we think about technology. And, you know, with this exponential growth, it's just gonna get harder and harder to do that. So how do we how do we keep on top of what machine learning is capable of?

And how do we start thinking about some of the problems that might be posed by some of the capabilities that are out there that may or may not have positive or negative repercussions for our society? I know I'm getting into philosophy now instead of technology, but I'm curious what your take is. It's it's a huge question. Right? It it really is a huge question.

And you're right. There are wide societal implications for this technology. Right? And it's you know, there have always been societal implications for from technology. Right?

If you go right back to industrial revolution, you know, the Luddites put out of work by the, you know, the new, billing machines or whatever it was. Right? So that continues. Now in the past, of course, what's what's happened is that the new technology has created new new jobs just in a different shape and allowed people to kind of move across. I I think this time it's a bit different.

Right? You're you're starting to replace you know, previously, you were replacing a lot of of manual labor. Now you're you're replacing kind of intellectual labor, white collar type workers. Right? Mhmm.

So I think that's a that's a different context, and it's a it's a hard context to grapple with. And, honestly, if you wanna get deep into politics and so on, I don't think that our current our current economic models and and ways of organizing society are actually fit to cope with the change, right now. Right. And and it's interesting to talk about too because we don't we don't actually know where this is going to take us per se. Right?

I mean, you even look at at what the Internet did, you know, when we're 25 years past sort of the wide adoption of the Internet, and nobody could have predicted where it was gonna go with social media, with the kinds of devices we're connecting with, and then how people use it, and and how that affects our society. I mean, I see people that I've known for years years years fighting with other people that I've known for years years years, or with fighting with other people that I've known for years years years or with me for expressing an opinion on Twitter or Facebook that I I never would have dreamed that they would be treating each other the way that they are or, that we'd be having the kinds of conversations about the kinds of things we're having conversations about because of the the advances in technology. And that's just over 20, 25 years. You know, I think this is gonna come upon us a whole lot more quickly, And I don't think anybody really knows what it's all gonna mean and what it's gonna lead us to. You know, it it may wind up being a breakthrough in an area that nobody's really even thinking about using tech technology like machine learning on right now, and that changes everything for everybody in in meaningful ways.

Yeah. No. I I think so. I I think it's it's very hard to to, you know, we can probably see 1 or 2 years out, but if you wind forward 10 years. Yeah.

The way that technology is growing, it it is difficult to see, you know, ultimately Yes. We'll have our reptilian brains. Right? You know? And we still have the you know?

Each of us has this kind of, emotional reactive way of dealing with things. And as a society, I think I think we struggle to deal with these kinds of huge kind of sea changes. Just look at something like climate change, for example. Right? A present threat to to humanity, but something that we're we don't seem to be able to deal with.

And maybe that's because we're we're more tuned to deal with immediate threats rather than kind of thinking strategically about threats to us. And I think it's probably similar with with AI. All I can say is that it it the change is gonna come. I don't think there's any way to stop it. And the the reason I say that is because, you know, the the ability to automate intelligence and intelligence itself is the key value is is so valuable to to our societies that we're just not gonna stop.

Right? And even if the even if the open societies were to were to try and put a limit on the technology, there are other societies that are perhaps not quite so democratic that would pursue the technology anyway. So I don't Pandora's box is open. You can't stop it. What you can try and do is is adjust the trajectory in flight to the benefit of all.

But I I how you do that, I'm afraid, is probably a little above my pay grade. No. I agree. And it's it's really interesting just from that standpoint. Yeah.

You know, we try and influence the trajectory to the best outcome. But the reality is is that, yeah, these kinds of technological changes I mean, even down to, you know, you brought up the industrial revolution. Nobody really understood what it was going to mean until after it was over. And so we can try and steer the ship. We don't actually know what the entire landscape looks like ahead of us.

And if we turn, what what we're gonna be headed toward. So, you know, we've kinda hope for the best. I think most people who are working on this are working on it because they see the potential. And I think they're working on it for the good of humanity or at least for the good of the industry or the good of the company they work for. But at the end of the day, yeah, it's it's gonna have some far reaching implications, and it's it's always interesting to dive into.

But, yeah, that's we don't we don't really have time to go much deeper, on this. But it's all it's always so fascinating just to see where people are at. It is. Why don't we come back in a couple of years and, we can review? Hey.

There we go. Alright. Yep. Alright. Well, let's go ahead and get into pics.

Now I don't know if you had a chance to listen to the other episodes, but pics are essentially shout outs about things that you're enjoying or things that are making your life better or or stuff like that. Right? So we've had people pick, like, TV shows and movies that they've enjoyed or activities that they enjoy doing. Some people actually pick technology stuff because they're excited about it, so it really can be anything you want. But I'll go ahead and go first just so that we can, so you can get an idea of what I'm talking about here.

One pick I have, and I think I picked this a few times just because I've been listening to this book series since, like, September. And it's a book it's a book series that I listened to when I was in high school or not listened to. I actually read them in high school. I've been listening to them on Audible now. And it's The Wheel of Time by Robert Jordan and Brandon Sanderson.

And, I'm I'm on, like, the last next to last book, so I had to throw in Brandon Sanderson because he he had to cowrite those last three books because Robert Jordan passed away. But, yeah, I've really, really enjoyed these books, just fantasy novels in general. I'm sorry if I sound funny. I have a little bit of a cold. It's actually lingering stuff from COVID 19.

So anyway Oh, feel better. Oh, well, I'm I think I'm over it. It's just yeah. I've I still have a little bit of the congestion. So anyway but yeah.

So I've been really, really enjoying that. I've also picked up a few podcasts that I've really enjoyed. They're about podcasting, and so, I'm just gonna throw them out. I don't know how interesting they are going to be to most people, but just to put it out there so people can find them. I think my favorite one honestly is Changemakers with Jamie Atkinson and Gina Suzanne, and that's just a terrific, terrific show.

They kinda get into the business of podcasting a bunch, and and I enjoy that. Another one I actually appeared on this show, it's the Podcast Domination Show with Luis Diaz. And then podcast talent coach is another one. That's Eric Johnson. I interviewed all of those folks actually for my podcast growth summit December, but their shows are just terrific.

And so if you're interested in podcasting, those are some to check out for sure. And then the last one I'm also gonna throw out there for podcasters is the, Buzzcast, and that's put out by Buzzsprout, which is one of the hosting companies you can host your, podcast on. I think most of the technology folks are have, passing familiarity with Justin Jackson, and so they wind up going with transistor.fm, which is another great solution for hosting your podcasts. And I think they have a podcast too, but I haven't actually listened to it. But I know the guys over at Buzzsprout, they do a great job, and their show, they really dive into some of the podcasting stuff.

And the way that they look at podcasting is very in line with what the way I look at podcasting. So I'm gonna pick them. Actually, I I should should also mention that their CTO or director of engineering, I can't remember exactly what it is, Tom Rossi, was a panelist briefly on Ruby Rogues because their system's built on Ruby on Rails. So anyway but, yeah, I'll I'll put links to all of those in the chats. They wind up in the show notes.

And, yeah, if you have any questions about podcasting and how it can help your career, I'm also just gonna throw out that, I am putting together some coaching for career career planning. Typically, I'm looking at, more senior folks. But if you want help planning your career and then building it up using podcasting and a few other tools that I'm putting together, then you can go to devchat.tv/nextlevel. And honestly, it's not gonna be a sales call. What it really is gonna be is just me asking you questions about where you wanna end up and then kinda getting a feel for how we could get you there.

So, you know, because because I don't actually have a product to sell there yet. So anyway, those are my picks. We'll put all those in the show notes like I said. And, Peter, why don't you go ahead and throw out some picks for us? Yeah.

Sure. Okay. So I've recently been pretty fascinated with the Wolfram Physics project. I don't know if you're aware of that. So I'm Yeah.

Cool stuff. Super cool. So my background is actually physics. I I have I did physics over my major and I worked in fusion research for a number of years. I'm sorry.

There you go. But, you know, I stopped being in physics, you know, a long time ago and have been in in engineering and software, ever since. But, it always struck me that there was, you know, to to get another layer down in physics was what was needed. And I hadn't seen any kind of really fresh thinking in it until I came across the the the Wolfram Physics project, which is, Stephen Wolfram of Mathematica fame whose view is that the universe itself is computational. And that all of the things Interesting.

Around us are emergent complexity from very basic computational processes. And he claims that they're at the point of kind of deriving things like, special relativity and quantum mechanics, or areas of quantum mechanics at least from these these low level kind of computational simulations. So I'm I'm actually fascinated by that, and I'm definitely worth checking out. My second pick is gonna be physics as well. I'm a big fan of the PBS Space Time channel.

If you haven't seen it, it's kind of physics, and the universe, explained in a in a really kind of interesting and and way that the kind of layman can pick up. So it covers everything from kind of warp drives down to quantum mechanics and stuff like that, so endlessly fascinating. My last 2 really quickly, 3 blue one brown. If you haven't seen that channel, it explains concepts in maths, AI, data science really, really well, with really good really good animation. And then my final one is is one letter center.

I I play guitar and you know what? I I haven't played guitar for a number of years. And then when COVID hit, I picked my guitar up again. And I picked up this channel, guy called Troy Grady. It's called Cracking the Code.

It's fascinating. And he actually goes in and and looks in incredible detail at the the actual mechanics of playing guitar. So it's really helped me improve my playing during COVID. So those are my picks. Awesome.

Owen, do you have some picks for us? Yeah. I do. Actually, what I've got is a couple of kind of, working from home ergonomics picks because, you know, a lot of people are probably may still be kind of figuring out their home working from home setup. And about a year ago, I ended up in hospital with a back spasm and I realized that kind of sitting sitting down all the time, isn't that great for me.

So right after that happened, you know, I was getting some physiotherapy and stuff. And I also decided to try out a sit stand desk. So I on on the recommendation of another 4th year guy, Greg, I got the IKEA Idesen sit stand desk. So this is a motorized it's really rugged, stable motorized desk. And, I've used it for almost a year now, and I found that it's made a massive difference.

And, you know, I've just I've just felt better, in general. And just I I feel like I don't end up with kind of hip, tension or any back problems since I definitely recommend that one. It's a bit of an investment, you know, if especially if you're if you're paying for your own one. It's about $600, €500. But, you know, it connects with Bluetooth to your phone and can can measure how long you're standing up for if you're into that sort of thing.

I definitely recommend that one. The other one I have is, last week, actually, I've kinda in the in the spirit of making kind of for added improvements to my work setup, I changed my keyboard. I found that I ended up with kind of wrist tension as well. I don't know if it's RSI or some kind of carpet carpal tunnel, but I do get fatigue from using either my laptop keyboard or just a standard keyboard. So last week, I took delivery of the Kinesis free Freestyle 2 keyboard.

I think a lot of people might be familiar with this one. But, it's a it's a split keyboard. And it also I also got the, extension pack that allows me to tilt each half of the split keyboard up to 15 degrees, I think it is. So I'm still getting used to it. So half of my emails are full of typos at the moment, but I think it's definitely it's definitely much more comfortable to use.

And so I'm pretty happy with that. Yep. Makes sense. I'm gonna just pile on that a little bit, because I I ran into that same problem for a while. I actually wound up with a giant cyst on my left wrist just from all of the moving my arm around.

Right? It didn't hurt. The doctor said that it wasn't dangerous unless it didn't go away, but I was also having, pain, actual pain in my forearms and wrists and hands. And then I would have numbness when I woke up in the morning in my hands, and it kinda freaked me out. And what I found is just the $3 foam keyboard bumper that goes in front of your keyboard to push your wrists up and then wearing, wrist, wrist braces at night solved a lot of that for me.

When I went to the doctor, he also gave me a set of exercises that I could do just with, like, a can of soup, right, in my hand, and then just move it around certain ways and and exercise or stretch it certain ways really helps. So if you're experiencing any of that stuff, I highly recommend you go to the doctor because a lot of the stuff that they're gonna give you is not invasive. It doesn't require surgery. They're not gonna recommend that, and you can actually just go and solve a lot of that with some pretty inexpensive and easy to do routines. So yeah.

Anyway and I love my sit stand desk, but I'm gonna let yours stand. I'll I'll pick that on another show. Okay. Yeah. I'm interested to know, but, yeah, I'm really happy with this one.

Good deal. Alright. Well, once again, if people wanna connect with you guys on the Internet, where do they find you? They can find me I you can find me on Twitter. I'm Owen s.

So e o I n s. Alright. Peter? You can find me on Twitter at palger, p e l g e r. Awesome.

And we'll make sure that those wind up in the show notes too, so people just click through and follow you. Thanks again for coming. This has been really terrific, and it's it's been a little bit different flavor, I think, than a lot of the other shows. We're actually digging into, like, the deep bowels of how these algorithms and stuff work. But at the same time, I feel like this makes it really approachable for people because it's like, don't have to be a genius to get into this.

I can just go implement something interesting or fun or something for work. And at the end of the day, I can use the skills I have, and then I can go deeper as I need to. Yeah. Hope hopefully, your audience find it interesting. You know, it's, it's it's, for us, it's, it's it's been fascinating to write the book.

It's always hard work. Right? But, ultimately, we really enjoy doing it. You know? And if it helps people to, to move their careers forward and get on with this technology, then then so much the better.

Yep. Absolutely. Alright. Well, thanks again. We're gonna wrap it up, and until next time, folks.

Max out.