Ethan Garofolo is the author of Practical Microservices with Pragmatic Programmers. He starts out debunking the ideas behind pulling parts of a monolith into a different services and change function calls into HTTP calls. Instead, it's an approach that keeps things moving for development teams that solves several productivity issues. He breaks down the ways to move functionality around and which approaches make sense for breaking your application up into pieces that are easy to work on and approachable for multiple teams.

Panel

Guest

Sponsors

Links

Picks

Special Guest: Ethan Garofolo.

Sponsored By:

STEVE_EDWARDS: Hello everybody and welcome to another exciting and thrilling episode of JavaScript Jabber. My name is Steve Edwards, the host with the Facebook Radio and the Voice for Being a Mime, but I am still your host. And today on our panel, we have the lovely Amy Knight. 

AIMEE_KNIGHT: Hello from Nashville. 

STEVE_EDWARDS: Hello Amy and AJ O'Neil. 

AJ_O’NEAL: Yo, yo, yo. Coming at you live from under desk Urevo treadmill. 

STEVE_EDWARDS: How's that working for you? You burning calories? 

AJ_O’NEAL: 28 so far in the past 31 minutes. So walking pretty slow. 

STEVE_EDWARDS: A little calorie. 

AJ_O’NEAL: Get used to it. Right on. 

STEVE_EDWARDS: And our guest today is Ethan Garfolo. I think I got that right with the silent. 

ETHAN_GAROFOLO: Oh, right. Yeah, you did. Excited to be here. 

STEVE_EDWARDS: Good. So for starters, I want you to tell us who you are, why you're famous, and just a little bit about yourself and your background. 

ETHAN_GAROFOLO: Right on. I don't know that I'd say famous, but I've been a software developer since, I don't know, 2005, 2006. Live in Utah right now in Provo. Claim Wisconsin as my heritage. Lived in Texas for a while. And in the past few years been focusing on microservice architectures, particularly event-driven ones. I wrote a book about it with the Pragmatic Programmers. It's called Practical Microservices. And I like cake decorating and running long distances at a slow pace. 

STEVE_EDWARDS: Not at the same time, right? 

ETHAN_GAROFOLO: No. Although this treadmill thing that AJ has going on is inspiring me here. Maybe there's a way to blend the two. 

 

Did you work your tail off to get that senior developer gig just to realize that senior dev doesn't actually mean dream job? I've been there too. My first senior developer job was at a place where all of our and all the failures were ours. The second one was a great place to continue to learn and grow only for it to go under due to poor management. And now I get job offers from great places to work all the time. Not only that, but the last job interview I actually sat in was a discussion about how much my podcast had helped the people interviewing me. If you're looking for a way to get into your dream job, then join our Dev Heroes Accelerator. Not only will we help you get the kind of exposure that makes you attractive to your dream employer but you'll be able to ask them for top dollar as well. Check it out at devherosexcelerator.com. 

 

STEVE_EDWARDS: Oh yeah, you could get a little cart with the cake on it and sort of decorate while you're running. 

ETHAN_GAROFOLO: Yeah. 

STEVE_EDWARDS: Boy, that would be a TV show in the making. So when you say long distances, what's long distances? Because, you know, just for reference for me, two and a half to three miles is considered long distance when it comes to running. 

ETHAN_GAROFOLO: Right on, yeah, so marathon and above. 

STEVE_EDWARDS: Oh, so you do like ultra marathons, like a hundred mile type stuff, or is 

ETHAN_GAROFOLO:  that not that far, not that far yet this year. It's training for a 50 K 50 K. 

STEVE_EDWARDS: So that works out to about what 20 miles. All right. 

ETHAN_GAROFOLO: A Little over marathon. Gosh, I can't do the math in my head. Let's see. 50 K. 

STEVE_EDWARDS: Oh, that's keyless. I'm thinking the wrong conversion. 1.6. That's right. Not 2.2. 2.2 is kilograms to pounds. 

ETHAN_GAROFOLO: 31 and some change. 

STEVE_EDWARDS: Okay. Well, people like you all never understand. That's way too much pain and running for me, but more power to you. So for today's topic, we're going to talk about, I think we discussed event driven programming and microservices or event driven microservices. What exactly is your specialty that we're discussing? 

ETHAN_GAROFOLO: Yeah. So that, so let's see to sum it all up. I think it was my opinion that on the blogosphere, there's a lot of not so helpful advice as regards microservices. And most of it boils down to take your existing monolith and take one of the tables out, put it on another server behind an HTTP interface. And I would contend that doing that doesn't fundamentally change anything. And all it does is change function calls into HTTP requests. And so what I got into and discovered through the help of some brilliant people who guided me, is that there's a better way. And what we're really trying to do with a microservices architecture, like the chief goal of it is to help keep development teams productive in the long run. I don't know how it's been for y'all, but I've been on so many projects where when it's greenfield, it's awesome. Lots of progress, getting lots of things done rapidly delivering value. But inevitably comes that productivity wall where things just get really complicated. I can't reliably make a change in one part of the system without fear of breaking another part. And that's usually about when I fire up the printer in the olden days. I guess it'd just be make sure LinkedIn is up to date and start looking for a new job. And I've just found that there's a better way to do it that doesn't lead to those kinds of problems. 

STEVE_EDWARDS: Now, when you talk about printer, you weren't around long enough for the dot matrix printer, right? 

ETHAN_GAROFOLO: I was. 

STEVE_EDWARDS: Oh, okay, yes. 

ETHAN_GAROFOLO: Yeah, I appreciate that that was your assumption that I didn't know what one was. 

STEVE_EDWARDS: But, okay. I don't know. Well, you said 2005. Well, that's right. You were around before you came a web developer. So, yeah, 

ETHAN_GAROFOLO: indeed. Yes. 

STEVE_EDWARDS: And I'm still having images of faxes with the slimy paper that curled up and made it really hard. And so flashbacks here. 

AJ_O’NEAL: Um, we learn what you have to say, then we won't have to find a new job. 

ETHAN_GAROFOLO: At least not because the tech becomes so hard to work on. I can't speak for other aspects that are part of the job.

STEVE_EDWARDS: So, okay, so let's talk about microservices. So my understanding from hearing other discussions on microservices is that I guess ideally you could have them be completely different tool sets. So say you're working with front end, a large front-end project that has multiple pieces that need to be put together. So in theory, you could have something done in Vue versus something done in React versus something done in Angular versus Vanilla JS. Is that correct?

ETHAN_GAROFOLO: Yes, it is true that with the architecture, you can blend different technologies like that. I wouldn't say that's the chief aim of it, but it is something that falls out from it. 

STEVE_EDWARDS: Right. Yeah. It's not a goal to have that, but I guess a benefit, shall we say, in terms of flexibility that you can do things like that? 

ETHAN_GAROFOLO: I do consider that kind of flexibility a benefit. Yes. 

STEVE_EDWARDS: Okay. AJ, while I'm running on my head, you got any questions? 

AJ_O’NEAL: Ethan, we need a really strong lead in here. Take us away, man. 

ETHAN_GAROFOLO: Okay. So let's see, the way that microservices help the development of a system are that the architecture helps manage coupling a lot better. It's really easy to get into some esoteric definitions here, so let's try that. But you might have heard of afferent and efferent coupling and afference is when... 

AJ_O’NEAL: Absolutely not. 

ETHAN_GAROFOLO: Absolutely not. Okay, great. So given that we're all totally up to speed on our Latin prefixes, those words probably don't need definitions, but just kidding. So afferent coupling is when something is dependent upon and efferent is when something depends on something else. An example of that, Lodash has a very high degree of afference. It is at least the last time I checked the most dependent upon package on NPM. So what that means in practice is that Lodash is not very free to change. Does that make sense so far? Low dash makes a major change. That's like internet breaking level stuff. 

AJ_O’NEAL: I would just say, let's not use those words anymore because I just, it gets high dependency. Yeah. High dependency. 

ETHAN_GAROFOLO: Okay. Fair enough. When lots of things depend on you, you're not free to change anymore. However, that tends to generally not be a problem for low dash because it solves very generic problems, right? Like map. I guess that's on the JavaScript arrays now, but it wasn't always. And the map function is defined. That definition is not going to change what you do with the iterator. Sure. That's going to change, but what map means that has a definition. So those are a couple of coupling metrics. And then there are a few others that appear at the systems level. For example, temporal and spatial coupling, the temporal and spatial coupling ideas I was introduced to by a guy named Rudy DeHaan and the inbound and outbound coupling, you could say, a guy named Scott Bellware introduced me to those. I'm sure we'll talk more about Scott and the Eventide project later. But anyway, so temporal coupling is suppose I am a web server and you call into me, but for me to handle that request, I need to make a call to web server B. Web server B's processing time appears to you as my processing time. So if I have to call B, B has to call C, C has to call D, all of that time together makes me look slow to you. And that's temporal coupling. Spatial coupling then is kind of the same idea, but with regard to availability. So if you call me and I have to call B and B is down, well, then I'm going to look down to you as well. And so holding those in our head, let's talk about monoliths. I've heard a lot of not so useful definitions of monolith like, if it runs on a single server, it's a monolith. If it uses Docker, it's not a monolith. If it uses Ruby on Rails, it must be a monolith. And those don't get to the core of the issue. And this is something I learned from Scott is that monoliths are a data model. And so if you imagine the user's table in basically every app you've ever worked on, look at the data on there and ask like, what do these data have in common besides an identifier linking them. There's a local company here that has a really good example of this. Not because they did anything wrong per se, but they just open sourced their user stable and I could provide a link to it. But they have on there a column for a time zone and then storage quota. And I think borrowing ideas from Rudy DeHaan again, what operation in my system requires transactional consistency between my time zone and my storage quota. And Udi will go on to say, try to form a requirement that requires those two to be consistent with each other. For example, if I'm in GMT minus seven, then I can only have seven gigabytes of storage in the system. And that's it's nonsensical. Like I don't know how to come up with a requirement where I need those two pieces of data to be transactional and consistent with one another, which tells me that 

AJ_O’NEAL: I know, I know, 

ETHAN_GAROFOLO: oh, I would love this. 

AJ_O’NEAL: There's a new buzzword. I forget what it is, but it's. Gosh, I wish I could remember the buzzword. I might have to come up with it. I'll give you the definition. I'll see if I can find the buzzword again. But it's when you advantage some people and disadvantage other people based on the bias that you want to introduce into the system. So like you say, we're gonna give people in Indonesia more storage, but we're gonna give people in America less storage and they're gonna have to pay more for it. 

ETHAN_GAROFOLO: Okay. That may be a legit reason to put those two together. I don't think that's- 

AJ_O’NEAL: I don't think so. I don't think so, but people do stuff like that. It's a real thing that's happening right now. 

ETHAN_GAROFOLO: Okay. I don't think that was the intent when the user's table got put into place there. And so, like the way that I grew up modeling things in web development was this very noun-based thing. Like I have a user, therefore there's a user table and any data even remotely related to a user goes into there. And the trouble with that is if I start grouping unrelated data together, I'm going to start grouping unrelated behaviors together because behaviors follow the data and so that's how you end up with the user class in a traditional MVC product. I'm, I would bet that most MVC projects have a very large user class. I've seen them thousands of lines long and that's not from comments in whitespace. And so that it's that data model that is the core of the issue. And so if I then take a user's table that's like that and just put it on a different server, and now the way that I communicate with it is via an HTTP call. My data model remains the same. I still have a monolith only now it's distributed, uh, giving me a distributed monolith, so now I have the worst parts of a distributed system with the worst parts of a monolithic design typical win-win scenario. 

STEVE_EDWARDS: Yes. Amy, you had something to add in there? 

AIMEE_KNIGHT: Yeah, so we've talked about this before. And I feel like one part when we talk about microservices that is often ignored is the infrastructure component, which really to me is the reason why you would, I mean, I guess the decoupling makes it better for working in the system, but the infrastructure component is really like the the driver for the end user and that a lot of times you can like scale better that way. So I was curious if we could like talk about that for a moment. 

ETHAN_GAROFOLO: For sure. Yeah. So to then break up the model if the data model like that would enable the kinds of infrastructure things that you're talking about, because I'm probably going to end up having smaller pieces than I would if I have the big user table. If I were to take the business processes whose data is recorded in a user's table and break them up into independent things, then I guess by definition, they would be smaller. And so now I can take advantage of infrastructure, things like that. But I wouldn't say necessarily that the infrastructure pieces are the chief aim of it. Cause I could design a system and I'm guessing we'll get into that as we continue to talk of how to design a system this way, but I could design one that breaks up that monolithic data model and I could still deploy it as a single unit running in a single operating system process. The way that I imagine this, the deployment story is a separate one from the software modeling story. And so... 

AIMEE_KNIGHT: Yeah, I guess I say like, so it's kind of the way that I envision it. And we are talking about, like, you know, buzzwords and stuff like that. And I feel like sometimes people don't make the distinction between microservices and I hate using these words, but service-oriented architecture, whereas it sounds like to me, if we're only talking about the software components, we're more talking about service-oriented architecture. 

ETHAN_GAROFOLO: Can you develop that further? 

AIMEE_KNIGHT: Yeah, so, you know, at least in, you know, like what I have come to know, but, you know, I'm open to, you know, other people's ideas and stuff like that, but at least from my understanding, microservices, like you're saying, is more the infrastructure component, whereas if we're talking about just the services themselves, we're talking about what they say is called service-oriented architecture. Because with infrastructure components, you want to be able to scale things independently of each other. 

ETHAN_GAROFOLO: I see. Okay. So, as it was explained to me, because I didn't come up with any of these ideas myself. 

AIMEE_KNIGHT: Yeah, that's why I think everybody has like a very unique, there's, I don't know, I should look on Wikipedia, but there's definitely like very, I don't know, I feel like the terms are often like kind of like mangled together and not necessarily broken apart. 

ETHAN_GAROFOLO: They sure are. Yeah, that's one of the most difficult things about it. When I was introduced to the concepts, my background was as a web developer. And so I thought that microservices was basically what I was already doing kind of a thing. Like just using more servers. But there is actually a history to it. And it was like a shot in the, or a two by four in the face realizing like, oh my goodness, this has a precise thing to it. So microservices was born out of service-oriented architecture. I was just starting school when it was a big thing, but I've heard microservices described by folks who were there for both as so the good parts. And the distinction between microservices and SOA is a lot to do with the communication channel. And so in the SOA days, you have these very complicated message broker type things, enterprise service bus, very intelligent routers, where there, if you do a distributed system, particularly a message-based one, there are now new technical issues that you have to deal with, that you don't have to deal with, with other communication technologies. And in the early days of SOA, folks were trying to make it so that as a developer of part of the system, you didn't have to worry about those parts. So things like how do I handle seeing the same message more than once? What happens if my connectivity goes down? Stuff like that. And it wasn't working so well. And as Scott Bellware likes to say it a lot, that's because computers are subject to the laws of the physical universe. As an example, if I'm a message broker, take Kafka, for example, and you retrieve a message from Kafka, you have to send an acknowledgement back to Kafka and that acknowledgement is itself another message and that could get lost. And so if Kafka never sees your acknowledgement, you're gonna see the same message again later. And if you're not prepared to handle seeing the same message twice, you get interesting results up to and including sending the $10 million wire transfer twice. So we don't want to do that sort of thing. So what microservices did was okay, the smart central broker thing is not going to work so well. So let's do a dumb central communication thing. And so Martin Fowler has a really good article about this, where he makes the distinction between smart pipes and dumb endpoints and versus dumb pipes and smart endpoints. So, so I had a lot of the smart pipes. So the, the communication mechanism itself tried to be very intelligent and handle all the concerns of being in a distributed environment. And then the dumb endpoints just wrote the code as if none of this was going on. And so the microservices approach was, okay, let's make the communication channel simple, very simple, not attempting to do all of this, but now the endpoints need to be aware that they are in a distributed or message-based environment like that. So microservices was born out of service-oriented architecture. And so I'm guessing that's why we'd still see a lot of similarities in the ideas. 

AIMEE_KNIGHT: I appreciate that history lesson. That was good. 

ETHAN_GAROFOLO: It was like, you know, the animated GIF of the mind blown guy. That's how it was for me when it was explained to me as well. It's like, oh, wow, huh. I'm sure I got part of it wrong, but that's, that's the difference. 

AIMEE_KNIGHT: No, that was good. So I feel like, you know, maybe the distinction, you know, certain depending on where you are in the stack of everything that it takes to get a system up in an environment. Maybe your focus, you know, depends on like, you know, what you do mostly day to day. So 

ETHAN_GAROFOLO: that sounds reasonable. People view things. 

AIMEE_KNIGHT: Yeah. 

STEVE_EDWARDS: So the discussion up to this point is as I'm listening to it, mostly focused on your backend server based architectures, I guess, and services and so on. When I, as a web developer. When I hear the term monolith, I think of building a website, you know, with like a WordPress or Drupal or, you know, Juma pick your CMS where everything is handled and taken care of in that one stack. So you've got your database, you got your, your backend code, you've got your templating language, whether it's, you know, a twig or PHP template, whatever your choice of tool is, but it's all there within that one application. And then with JavaScript frameworks coming along and other tools on the front. And then you started seeing things split apart where you started with like the mean stack, Mongo express, Angular JS and node for instance. And then it's the ME RN stack, rethrow and react. Now it's ME asterisk M or either so many tools out there. They're sort of break up the monolith. And then interestingly enough, you start seeing people coming back to the monolith type architecture with something like inertia JS, where you have, you know, inertia sits in the middle between your front and your backend. So it's sort of like the old-school monolith but with a little more flexibility. So I'm just curious to see your thoughts on that from a web development standpoint. And if that's something you deal with very much or if you're focused more on backend servers, infrastructure type of architectures. 

ETHAN_GAROFOLO: Yeah, that's a great question. So I am less familiar with a lot of the browser side JavaScript technologies. I've used React and Angular before. But let's see, I'm trying to think how best to answer this question. So let me try another story and see if that answers your question or if I'm just saying a bunch of words. I'll let you judge what I've done here. So one of the other big things that the distinction between front end and backend. Having grown up again as a web developer, to me front end was what ran in a browser or potentially on a mobile client and backend was the server sitting in my data center. And I've since learned that it's a much bigger- 

AJ_O’NEAL: Wait, hold on. When you were a kid, you had a data center? 

ETHAN_GAROFOLO: Yes. No, I didn't. I suppose I should be more precise, but no, I don't know. So-That was the distinction. If it ran on the server, it was backend. If it ran in the browser, it was front end. And what shocked me was that it's a bigger world than that. And so the distinction that I'll make, so a microservice, I tend to not call them microservices anymore. It's a useful marketing term, but it's not very precise. And Amy, as you were saying, there's the 101 different definitions of what that word is. So what I'll say is autonomous component or just component for short. The way that it operates, I alluded to the dumb pipe earlier. There's a piece of technology that broadly speaking would be called a message store and a component is pulling the message store to get a batch of messages to process. So it's not real-time. It pulls, gets a batch of messages, goes through them one at a time. It's kicking off other messages to record the work that it has done. And those operate in what we call batch mode. And those are the backend pieces. Now users still need to interact with a website. And so the portions that handle that are the user interactive or the front end portion of the system. And so what I thought of as traditional front end and backend is actually all front end, so what's running in the user's browser, as well as the portions within my own personal data center that I own and operate myself are front end pieces because they're dealing with the user world. So to then big story, lots of words, uh, to bring it back to your question, Steve, thinking of an example like WordPress. Um, I haven't actually looked at the data model of WordPress to know how it operates. I'm assuming it is monolithic in nature and gosh, am I getting anywhere close to answering your question?

STEVE_EDWARDS: Well, I didn't so much have a question. It was more sort of a discussion jumping cord. I guess I'm curious to see how you see, you mentioned, you know, obviously your background as a web developer. So I was just curious to see how you see microservices in terms of a website stack and the, the, uh, pieces made to build a website. 

ETHAN_GAROFOLO: Yeah. So, okay. Then situating the website in an overall architecture done this way. The website is a front end portion, even the server that runs it. And it's, however, you want to set it up so that your users can interact with your system is fine. I suppose there are different architectures you can do for those front end pieces that make it more maintainable in the long run as well. But as to whether or not an overall system is microservice based or a distributed monolith. I don't know necessarily that the choice of front-end technology has a huge bearing on that. 

STEVE_EDWARDS: Okay. Well, let's look at it this way. So one of the, and maybe this is a difference in what we're defining as a microservice versus maybe say components, breaking something into components. To me, microservices are and components are interchangeable because you're taking them and piecing them together. Let's say you do a really large website. Drupal world was weather.com where they were using Angular, first AngularJS and then Angular two in combination with Drupal and different templates and piecing all of the things that you see on the front page together as different, you know, components and different services. So I guess the point I'm making is to, when I hear microservices, that's sort of what comes to mind is different components written. They could be in completely different languages, but you're basically piecing them together to make one big display or one big picture, one big, you know, however you want to phrase it. 

ETHAN_GAROFOLO: Okay. Yeah, I would just say that the parts that your users see aren't the microservices. And so if you did have, say, a, like Amazon's UI, I think is an example of this, what you see on the product page at Amazon, that is not all coming from the same web server. Eventually, I think it's decomposed like that. And so, those various web servers backing them though, are still not microservices themselves. Those are just API endpoints or web servers. 

STEVE_EDWARDS: Okay. So what else have we not discussed about microservices? I guess maybe that you cover that you think are important. 

ETHAN_GAROFOLO: Definitely the modeling of it. And then also there are some tools that are useful for doing all of this. The thing that I found the most difficult in moving towards microservice architectures, and it's something I still have to check myself on, is the how to model the system is very different. So early on we were talking about say the users table, and so a lot of modeling is like, oh, I have users in my system, therefore I have a users table. I have products in my system, therefore I have a products table. But what the, what a message driven, autonomous microservices architecture is going to push towards is, no, no, no, don't model the things, model the processes in your system. Those processes will kick off data. Let me give some examples. One that I borrowed from the Eventide project on a number of occasions is consider products. Products have a price and there are business rules around how to change the price. Products have inventory and there are business processes around when inventory gets added to or subtracted from. And products also have their description. And those are three distinct processes. And so putting them in the products table is now blurring the lines of those processes. And that's where the terrible effects of coupling kick in and make the project hard to work on in the long term. Instead, what if we modeled it as a separate entire thing for the inventory? So if I were not doing message-based thing. I might have three tables for that. I might have the product inventory table, the product price table, and the product catalog table. Now those processes are separate from one another. They don't share any data other than a product identifier, and that switch to thinking in terms of processes rather than these noun-based entities in the system is really hard. It gets me some really cool benefits though. Prud is a great word in our industry. And depending on who you ask it, it's either create, update, delete, or create, read, update, delete. Whatever the case may be, think of a business domain like, I don't know, banking, right? Banking is this rich domain with this history and these processes. Obviously there's the withdrawing and depositing of money into an account, but there are processes now, especially in the modern day, know your customer laws and fraud detection and overdraft detection and all this kind of stuff. And bankers have been working on this for years. They have this rich vocabulary to describe all of this. And then we as developers, we come along and we're like, hey, take that whole domain that you have and reduce it to three verbs, create, update, and delete. And I think a lot gets lost in that translation, particularly our ability as developers to talk with these subject matter experts. And so when I then switched to modeling by processes, particularly with asynchronous messages, like commands and events, I can start modeling the system using the same language that the subject matter experts use. They could look at my model and understand what's going on without having to speak tech speak like we do. We come to where they are and meet them in a familiar territory. And I think that's a beautiful thing.

AJ_O’NEAL: Now you've got a couple of like go-to examples for this that you put on slides and whatnot, don't you? 

ETHAN_GAROFOLO: I do, yeah. 

AJ_O’NEAL: Could you go into that a little bit to give us a clearer picture of some specifics of how that vocabulary translates for a product or process? 

ETHAN_GAROFOLO: Sure. Yeah. Let's continue talking about bank accounts. AJ, if you walked into a financial institution today and the result is that you walked out of there with an account at the financial institution, How would you describe what you did? 

AJ_O’NEAL: Well, OK, so because we're talking about CRUD, I was going to say I created an account, but actually we say I opened an account. 

ETHAN_GAROFOLO: Exactly, right. And that one might not be the most powerful example, but you start to see the picture of it. When a user registers for your site, did we really just create that user? 

AJ_O’NEAL: As far as I'm aware, the store brought them to us. 

ETHAN_GAROFOLO: OK, well then the event would be stork brought child or something. In either case, I think user created or creating a user is a very bold metaphysical statement and one that would land us in hot water with the child online protection act or whatever it is. Copa, I think. So yeah, so those are some examples trying to think like with in the real estate domain. I'm going off the top of my head here now. I don't know, you wouldn't like create a home purchase. You would just purchase a home. Yeah, I don't know. Is that helping? Is that a long, 

AJ_O’NEAL: no, that's super clear. That's super clear. Like, yeah, but okay. The thing is in our systems, we like to have, you know, we like to keep it dry. Now one way to do that is to make everything different. Like, you know, the average PHP application or the other way to do that is to make things very similar and then reduce the amount of, of copying. So dry is, don't repeat yourself. Right. So, if I am naming everything arbitrarily different, like I, you know, I have an open account method and I have a, what was the other thing that you just said? Like purchase product method and all that, rather than create, create, create, create. Then I can't do like meta programming and have some generic thing that generates all of my crud methods. So you're not, you're creating problems for me, Ethan.

ETHAN_GAROFOLO: Sure. Yeah. I don't think dry was ever about avoiding typing. And I would also counter with it's never been the case for me. I realize I am one person with one perspective, but I've never been slowed down working on a website or project at all by my typing speed. 

AJ_O’NEAL: That happened to me once. 

ETHAN_GAROFOLO: Did it? 

AJ_O’NEAL: Yeah. It was like two months ago. I actually knew what I needed to do. And the problem was I couldn't type it out fast enough. I spent three months building a framework. No, I'm just kidding about the framework part, not about the actually having one occasion where my typing was slowing me down. 

 

Hey, folks, if you love this podcast and would like to support the show, or if you wish you could listen without the sponsorship messages, then you're in luck. We're setting up new premium podcast feeds where you can get all of the episodes released after Christmas 2020 without the ads. Signing up will help us pay for editing and production. And you can go sign up at devchat.tv slash premium. 

 

ETHAN_GAROFOLO: Right on. Yeah, it's true. There will be all these computer-generated things like that. And in practice, I haven't seen that as an issue. There's a little more boilerplate. We could talk about more about how the systems are built, of course, but I don't know, I don't feel boilerplate. So anyway, yes, I can. 

AJ_O’NEAL: So you don't consider that boilerplate to have to sign like, assign all of those names differently and 

ETHAN_GAROFOLO: Oh, no, no, no, I would say it is, but I don't fear it. It's totally fine. 

AJ_O’NEAL: You don't fear. 

ETHAN_GAROFOLO: Yeah. Yeah. Did I say feel? I wasn't sure. Right. 

AJ_O’NEAL: I'm not sure what you said. That's why I continued with the. 

ETHAN_GAROFOLO: Okay. Yeah. Just reducing the number of characters. I don't know. It's not an aim I go for because I don't know, just getting abstractions wrong is so much more costly than not having the abstraction in the first place. And dry is a lot more about do things in the same way than it is avoid characters going into text files. And so, yes, I will have very unique names for all of these messages that I'm defining in this system. Great. I mean, the business process that I'm modeling has all these unique names for things, and I'm trying to model an actual process and not just reduce how much typing that I need to do. So, 

AJ_O’NEAL: paid by the hour, don't you? 

ETHAN_GAROFOLO: No comment. Anyway. Yeah. So let's see. So trying to install back then to how the overall flow of it works. So we talked earlier about spatial and temporal coupling where we might've developed that more was, okay, well, what properties does a system have? If I want to avoid those kinds of coupling. And a lot of it comes down to let's remove these synchronous calls from one piece to another, and instead do that with asynchronous messaging. And that is where the event driven nature of this comes from. That communication pattern is called PubSub and really it's PubSub that is the amazing thing that makes the projects easier to work on. Um, have you all heard of the event sourcing before? 

STEVE_EDWARDS: I've heard of it. I've heard the term, but I've never quite grok'd it, shall we say. 

AJ_O’NEAL: You've explained it to me once, but then I got muddled with other people saying it. Now I don't know what it is anymore. 

ETHAN_GAROFOLO: Okay. That's fair. So, all right. So if we've removed all of these synchronous calls between things and replace them with asynchronous messages. We now have these messages. What if we saved them and called them our authoritative state? So in a traditional setup, if I create a bank account, I will insert a row into a database that now, great, now I've created a bank account. If I were to withdraw money, maybe I would update a balance column in that table. This of course is not how financial institutions work, but we do the analog all the time when we're doing credit-based stuff, which actually gets me to a better example. Oh man, here, let's talk some fun stuff. All right. You all familiar with the concept of a finite state machine? 

AJ_O’NEAL: But of course, 

STEVE_EDWARDS: I understand finite and I understand state and I understand machine, but putting in the three of three of them together, not so much. 

AJ_O’NEAL: I don't think most of our listeners would know what that is. I think that's more of a computer science topic that is super useful to understand how to design systems but rarely spoken about as such. 

STEVE_EDWARDS: Now we would all understand an infinite state machine. No, I'm kidding. 

ETHAN_GAROFOLO: And an infinite and probability drive. There you go. Yes. 

AJ_O’NEAL: Okay, I did get married. 

ETHAN_GAROFOLO: Yeah, I did too. I feel you on that one. So that is infinite and probability in my case. So let's, this is a lot easier to do with visuals, but finite state machine, like AJ said, is a way to model a system. 

AJ_O’NEAL: Just turn the camera back on. 

ETHAN_GAROFOLO: Oh.Is there like screen sharing? 

AJ_O’NEAL: No, but it might help the conversation and you can use your hands and then I'll explain what you're doing with your hands. 

ETHAN_GAROFOLO: Okay, all right. So you might have seen these kinds of diagrams before. Let me whip one up real quick here. 

AJ_O’NEAL: Currently, he's using his hands to draw on what appears to be a desk with marker and some sort of parchment. 

ETHAN_GAROFOLO: Okay, so there is a really simple one. These all have a starting state, some sort of a transition. And then a record of a particular state. So you could think of this as like a turnstile as an example I like to use. It can be locked. And then if you insert a coin into it, it moves to the unlocked state. If I were doing that, I'd be missing some transitions here. So if it's locked. 

AJ_O’NEAL: What he did was he drew a diagram, a flow state diagram that only had one flow, no yes or no choices. That's what we're up to so far. A dark circle for the starting point some other circles for the only states you can go to and then an end circle. That's where we're at. So you can visualize this in your head. 

ETHAN_GAROFOLO: Yeah, all right, here's a better one. So think there are two circles, one that says locked, one that says unlocked. From the unlocked circle, there is an arrow with push assigned to it. That arrow takes you back to the locked state. And there's an arrow with insert coin, which transitions you to the unlocked state. Within the unlocked state, there is an insert. 

AJ_O’NEAL: I don't know of a turnstile where I insert a coin. When I thought of turnstile, I thought like I go into the bank and they have that thing that turns around or maybe like at the amusement park when I'm exiting, it counts me. 

ETHAN_GAROFOLO: Yeah. Or you could imagine like old school subways. Yeah. So I think that's a good point. I think that's a good point.

AJ_O’NEAL: None of us are that old, Ethan. 

STEVE_EDWARDS: I know what you're talking about. 

ETHAN_GAROFOLO: Okay. Yeah. So if it's locked and I just push on it, I remain locked. So if I were doing traditional CRUD and I were modeling a turnstile, I would just store whether or not the turnstile is locked or unlocked. That's all I would store. If I'm event sourcing, what I'm going to save are the transitions. So if I told you the turnstile is locked. Could you tell me the sequence of events that led to it being locked? 

AJ_O’NEAL: Well, in that picture, yeah. 

ETHAN_GAROFOLO: Could you? 

AJ_O’NEAL: Well, I guess not because there's an infinite number of ways it could be locked. It could have been unlocked and then relocked and then unlocked and relocked, any number. 

ETHAN_GAROFOLO: Yeah, exactly. I could have arrived at the locked turnstile and pushed on the bar. I could have arrived at the locked turnstile and pushed on the bar twice. I could even, hear me out here have arrived at the lock turnstile and pushed on it three times and it would still be locked. And if all I'm storing is that it's locked, I lose all of that detail. 

AJ_O’NEAL: Oh, like a traditional database crud app. 

ETHAN_GAROFOLO: Exactly. Yeah, I lose all of that detail. So if I'm event sourcing though, what I would save is pushed, coin inserted, coin inserted, pushed. And that would tell me that interesting insight I might be able to glean from actually having that history is like, huh. Maybe our signage is not telling people that they need to insert one and only one coin to go through this thing. Greg Young, who is a big speaker in this space as well, he likes to use the example of e-commerce where if all I'm storing is some purchase thing and I'm missing the events for items being added to a cart and items removed from the cart, I miss out on some really big insights. Like, why do people remove items from the cart after they've viewed the checkout page? Well, probably they saw the total. They're like, I can't spend that much money right now. So they remove those items from the cart. Well, if I had events tracking all of those things rather than just, Oh, the cart has now three items in it instead of five. I can do a report and be like, Hey, wow, this person removed this item from the cart. What if we sent them a coupon for 25% off the item if they bought it right now? 

AJ_O’NEAL: And that's just the targeting for the wind. 

ETHAN_GAROFOLO: It's just information that's lost entirely. And so as I'm defining. I like this example because it clearly clearly that's not fair for me to say it illustrates the difference. If I'm doing credit, I'm just storing the state. If I'm doing event based stuff. I'm concerned about the transitions from the transitions. I can reconstruct the state and so the pub sub communication is what makes the architecture special. If I've already got these things defined, I might as well save them. And that's all event sourcing is is I'm these events that I already have, I'm going to save them and call them the authoritative state in the system. And if I want to get the current state, I can do that. I just get all the events and run them through a process called a projection. And I can get that current state, but yeah, that's all event sourcing is. 

AJ_O’NEAL: It's now you call it a projection. I'm pretty sure I call it a function. 

ETHAN_GAROFOLO: A projection is made up of a number of functions, but yeah, you could call it a function in that more abstract sense of it. 

STEVE_EDWARDS: So how is this a finite state machine?

ETHAN_GAROFOLO: There's a finite number of states in which this can exist. 

STEVE_EDWARDS: Oh, okay, gotcha. 

AJ_O’NEAL: So the turnstile is in a finite number of states and there are a finite number of ways for it to arrive at those states. You forgot to add the exception. Where do the exceptions occur? Where do the unhandled errors happen here? Person jumps over turnstile. 

ETHAN_GAROFOLO: Yeah, well, I mean, if you haven't accounted for it in your model, then you would need to build a more complicated model than this one. Although I would say, that is not part of this process. You might have some cameras observing the turnstile that can detect when someone has jumped over it, but I wouldn't say that's the job of the turnstile to handle that case in the same way, if I'm doing banking, it's not the accounts job to manage overdrafts. You could have something observing the account that handles the overdrafts. 

AJ_O’NEAL: And if I'm, uh, accepting users, it's not the user microservices job to apply purchase power parity to how much storage they get on their account. 

ETHAN_GAROFOLO: Indeed, yes. In fact, in a system like this, I bet there would be no user microservice. 

AJ_O’NEAL: Wait, wait, wait, wait, wait, wait. What happens to Okta and the other one that's more popular, Auth0 and- 

ETHAN_GAROFOLO: Okta just bought Auth0. 

AJ_O’NEAL: How in the world does that happen? They're like the new guy. 

ETHAN_GAROFOLO: Billions of dollars, that's how it happens. 

AJ_O’NEAL: They just came in yesterday.Like no one had even heard of them until six months ago. 

AIMEE_KNIGHT: What are you talking about? Octa or off zero? 

AJ_O’NEAL: Octa. Yeah. 

ETHAN_GAROFOLO: Oh yeah. Sorry. 

AIMEE_KNIGHT: I haven't heard about Octa for a while. 

AJ_O’NEAL: Well, yeah, but I mean like, like you go to the average developer, everybody heard of zero. Like you are the average developer. Nobody's heard of Octa. Like, you know, I learned about it maybe two years ago because they were like this, you know, hot startup that was like cutting sweet deals with people to you know, acquire customers or get people over to them. Somebody was a part of the company that decided to move over to them or something like that. But I mean, like, and then I never heard anything about him again until like six months ago. I mean, I just, just what are they, who are they to come out of nowhere and steal an incumbent's business that isn't, doesn't even go with the model of how business works. The incumbent wins unless they suck really bad. Maybe off zero does. I don't know. 

ETHAN_GAROFOLO: I'm certainly not going to comment on that because I don't know enough to anyway, but yeah. So what happens to them? Something like that could still very well exist in the system. It just wouldn't be a microservice. That's a user interactive concern authentication. 

AJ_O’NEAL: I don't understand. 

ETHAN_GAROFOLO: So. Do you have concern what? 

AJ_O’NEAL: Yeah. So I don't understand the terminology and I can't infer because you just broke my brain, so I don't know what you're saying. 

ETHAN_GAROFOLO: That's totally fair. So, um, the components, again, the microservices are operating by getting batches of messages to process and they are back end systems. Whereas front end port, sorry, not back end systems back end portions of the system. And there are the front end portions of the system and the front end ones are the ones that deal with the outside world. And so user authentication is one of those concerns dealing with the outside world.

AJ_O’NEAL: And that's kind of what you're saying here is like the authentication piece is the piece that breaks the system. And you actually do need that to be part of a more, uh, shall we say traditional type of, of workflow because it doesn't, it doesn't work. It doesn't work doesn't lend itself well or maybe isn't even, uh, possible in the event source model of thinking 

ETHAN_GAROFOLO: for sure. Yeah. And those, those immediate real time pieces still absolutely matter in the architecture. Someone has to be at the front door, authorizing, or sorry, I don't want to confuse authentication and authorization, but letting people in. According to specific rules, like how do you signed up and so forth. So Um, where I was getting at with the comment that there wouldn't be a user's microservice is because have you ever usured before what business process is called users. 

AJ_O’NEAL: Hey, John, when you finish usuring that, would you go make me a copy and grab a cup of coffee? 

ETHAN_GAROFOLO: Yeah, exactly. There's a whole bunch of different processes that involve users like registration. 

AJ_O’NEAL: Well, wait, wait, wait, wait. We could call it, we could call it the it's the registration system and the authentication system or the verification system the checks identifying system. 

ETHAN_GAROFOLO: Yeah, you could have all of those. I don't think authentication lends its well to the batch execution of a microservice, that like grab a bunch of messages and process them because that needs to happen in relative real-time. 

AJ_O’NEAL: No, you got it all wrong. You have people sign in all throughout the day, and then at midnight, you batch process all the sign-ins, and you let them know in the morning whether they're in or not. 

ETHAN_GAROFOLO: Yeah, I suppose you could build a system that way. 

AJ_O’NEAL: It's the future. People are going to love it. 

ETHAN_GAROFOLO: It's happening right here. So dear listeners, as you're hearing this, you heard it here first. 

AJ_O’NEAL: The funny thing is somebody's going to come in and be like, yeah, I used one of those systems the other day. Uh, no case in point, IRS.gov. When you sign up for an EIN number, the, the website is down from 5 PM Eastern time until 9 AM Eastern time. And then it comes back up in the morning. 

ETHAN_GAROFOLO: Yep. Yeah. Okay. And if there's an error, does it show you the lines of code where the error happened? 

AJ_O’NEAL: Probably if you put an apostrophe in a form field, but you have to wait until business hours to do that. 

ETHAN_GAROFOLO: Oh, fair enough. I wonder why it's done that way. Oh, yeah. Whatever. It's how they run the system. That's fine. That makes it slightly better than the phone system, though, that they use. 

AJ_O’NEAL: OK, let's take 30 seconds to go on this rabbit hole. What?

ETHAN_GAROFOLO: No, I'm just saying that I've not had good luck with calling into the IRS before. 

AJ_O’NEAL: Oh, so their website is open during business hours, but their phone, not so much. 

ETHAN_GAROFOLO: Yeah. And waiting till the next day is a slight improvement upon the phone system speed. It's so easy to pick on the IRS because don't buy the way. 

AJ_O’NEAL: Are you doing a Rubik's cube? 

ETHAN_GAROFOLO: Oh, sorry. No, it's this little, it's, it's, it's really loud. I'll stop it. 

AJ_O’NEAL: Yeah it's loud. 

STEVE_EDWARDS: Okay, so before we get on to picks, Ethan, anything else we missed that you'd like to cover quickly? 

ETHAN_GAROFOLO: Yeah, so I'm modeling the system in terms of processes and microservices are responding to commands, which are a type of message that are requests to do things that ideally the, well, not ideally that, in a lot of cases, the front end pieces we'll be putting into there. It picks them up. It does, carries out the business logic of whatever process it represents, kicks off other events to show that it happens. So in the case of like a bank account, you know, there'd be an open event, a withdrawn event, a deposited event, potentially a closed event one day. And then if I want to know the balance of the account, I just go through all of those events in order and apply them in that projection process. Does that take a lot of time and practice? Not so much because these events are organized into what are called streams and for an individual account or individual thing, entity in the system, you tend to not have a bazillion events in the same stream. Then, right, those events are not particularly useful for building user interfaces unless you're building an audit screen. And so there's another piece in the system called an aggregator whose job is to consume events. 

AJ_O’NEAL: So hold on, let's slow down just one second. 

ETHAN_GAROFOLO: Yeah, okay. So the basic flow of operation in a system built this way is there are the, the applications, the front end pieces. And I keep using the bank account example, because it's a good one that the event-type project put together. 

AJ_O’NEAL: Yeah, let's keep going with it. 

ETHAN_GAROFOLO: Okay. So maybe I show up and we're going to skip things like, no, your customer and all that. Just make it very simple. But I show up and I send a request to some server to open an account. What it will do is write a message called a command to a stream for the account that I want to open. And gosh, there's so much background here. So streams, what are they? They are groupings of messages related to the same thing. And thing of course is the technical term there. But for example, suppose I had account with identifier 123, there will be a stream where all of the commands for that account go. There will be a stream where all of the events for that account go and so the web server puts 

AJ_O’NEAL: this sounds like I almost like I've got a table for every user. I mean, like if I'm using my old terminology here. 

ETHAN_GAROFOLO: Yeah. Let me. Hmm. 

AJ_O’NEAL: It feels like a lot. It feels like, Oh gosh, there's a ton of stuff going on there. 

ETHAN_GAROFOLO: Yeah. 

AJ_O’NEAL: Partitions. 

ETHAN_GAROFOLO: It ends up not being as many moving pieces as it sounds like. So this, uh, now to get into specific technologies backing this, there is a great message store called message DB, which is effectively a Postgres schema with some server functions that make it behave as a message store and in there, there's a single table called messages with a column called stream name. And so to make a new stream, I just write something with a stream name value that hasn't been written before. So there's no real ceremony to it. It's not like doing a new topic in Kafka or something so a lot easier to work with from that perspective in my opinion. So if I have account one, two, three, I would have a stream account dash one, two, three, I usually use you ads for identifiers, but they don't fit great in slides and no one wants to hear me read a full you in here. So I'm going to say one, two, three. So in stream account dash one, two, three, I'm going to put every event related to the account. It's probably going to have an open event first then some number of withdrawn and deposited events in there. And then if I ever close it, it will have a closed event. Let's say there will be an account and this is a convention that emerges account colon command dash one, two, three is the stream where I'd find all of the commands pertaining to that account. So request comes in to open an account. The web server puts a command into that account command stream. And then the account component is running. And it is looking for every account command. So 

AJ_O’NEAL: you mentioned aggregators briefly before, and I think that's what you're talking about right now, but I don't know if we actually expanded that at all. 

ETHAN_GAROFOLO: We didn't know. I am talking about the account component right now. So that command that comes in is a request to open an account. It's saying, I want to open this account. So thing that manages that please open the account and it's the account component that owns. That process that decides whether or not an account is going to get opened. And what is it going to look for in a simple example? Will it has the account already been opened? If it has, then I don't want to open it again. And so that command comes in the open imperative mood name. Their command comes in, it processes it. What it's going to do is in our simple business rules here, we're going to say an account can always be opened unless it has already been opened. And so what it's going to do is it's going to go to the corresponding stream where all of the events would live the account dash 123 and project all of those to get a current state of the command. Is that making sense so far? This is also much easier to do with visuals and slides and whatnot. But if it opens the account, what it's going to do is write an opened past tense event to that account 123 stream. So the first time it does that, it's going to go to that stream. Well, there probably aren't any events in that stream right now. So it knows this has never been opened before. Okay. My business will say, go ahead and open the account. If it finds an opened event, then that resulting projection is going to show that. And it'll be like, Oh, this account has already been opened. I'm not going to open that. 

AJ_O’NEAL: So that sounds like it would take a really long time to search through. However many messages to find this event. 

ETHAN_GAROFOLO: Yeah, that is a, that's a key thing here. It's a big part of the distinction between a message store and a message broker. This is where something like Kafka is not a good choice for a message store. Kafka has topics, which if we take the account-123 example, that is a stream. We could logically say that there is another stream just called account that has every message related to every account ever. And that is roughly what a topic in Kafka is. So if I need to get the current state of a particular account, Kafka topic is not well suited to that because I have to go through the entire, every event for every account ever to find that. With a message store, I can get just the events related to account one, two, three. Now that's one piece of it. To your question, what if that stream has a lot of events in it? There are shortcuts, not shortcuts, but I'm gonna try to say it. Performance optimizations we can do as I'm projecting these what these are foundational rules. I forgot to mention Events are immutable. We don't change them once they've been written and the streams that they reside in are append-only so I don't remove events from them I could only add new events to the stream so if given those constraints if I do this projection process for events one through ten I get some answer now I want to incorporate event 11 into that answer. What is the difference between starting at the first one and going all the way through 11 or starting at the result I had at number 10 and applying just the one? 

AJ_O’NEAL: I guess you're shortcutting the state machine. 

ETHAN_GAROFOLO: You're saying,  time it takes is the difference. The, the answer has to be the same because I can't change the messages that came before it. So to do it all the way back from the beginning, I'm going to in the intermediate arrive at the one through 10 and then just apply the next one. So if I just save that 10 off somewhere, when I go to do the projection, I can start from that location and then project from that point going forward. That's called snapshotting. It's a performance optimization, not strictly needed, but it is quite useful if you get large streams, which in the case of a 20 year old account would be a situation where that would be good. So yeah, these, these commands are coming in. The components are producing events to show the work that they've done. That's the job of a component. It owns a particular business process in this example, the life cycle of an account and it responds to messages. Now you want to see the balance in your bank account, probably sometimes. And so if they just showed you a list of all the transactions and said, Hey, AJ, have fun doing the math. That's not going to be so much fun. That's not a very useful view. Although you do want a view of the transactions. You want to know the balance right now. So that's where these pieces called aggregators come into place. So there's events that the account is kicking off because this is pub sub. Anything is free to observe those. Not everything is free to write to account streams. Only the account component gets to do that. But these pieces called aggregators, they observe these events and maybe. We talked in the beginning, maybe there is an accounts table with a column that has the balance in it. Well, I'm saving deposit and withdraw, sorry, let me, deposited and withdrawn events to account streams. I can then take those and aggregate them into a balance column in a table that the thing that you're looking at can query to get a quick response to give to you. So what I'm doing is I'm separating how I write the data from how I read it that has been branded as CQRS in some times, some places. 

AJ_O’NEAL: Remind me what that means. 

ETHAN_GAROFOLO: Yeah. It's command query responsibility segregation. 

AJ_O’NEAL: The idea that I can write data before. 

ETHAN_GAROFOLO: Okay. Yeah. I can write data one way and read it one or more other ways. 

AJ_O’NEAL: I mean, this sounds like what you're talking about generally just sounds like a pattern that you're probably going to end up in as you as you scale a system, like one, really, there are so many ways upon which like you gave it the shopping cart example, the bank account example, like you really do want to have an event history, but you also want to have something that's quick and you know, it almost makes it feel like I could divide it's like if I have a crud system, I'm just missing the other half of the system. You know, because if I have a crud system, I have the the caching part, but what I don't have is the history part and for a lot of applications, especially where anything needs audited or security or anything like that, it sounds like this is. A way to go, which again, sorry for, I'm always going to distract and go down right by holes, but like what, what is the ideal. Like set of, of what are the key indicators that I really should be investigating this event pattern? 

ETHAN_GAROFOLO: Are you building a software system?

AJ_O’NEAL: Well, no, I mean, 

ETHAN_GAROFOLO: the answer to that is yes. 

AJ_O’NEAL: Because so many people do along just fine. They run their business for years. They have minimal tech, you know, but so I'm giving something a little more thoughtful than that, if you don't mind. 

ETHAN_GAROFOLO: Yeah. So when you make a change in a file, are you afraid of breaking the whole system? 

AJ_O’NEAL: That's a really good, 

ETHAN_GAROFOLO: if your business doubled, could you handle it? Particularly if you're in a regulated industry. Could you reconstruct the state of your system from your audit logs? If you can't, then you don't have a court admissible audit log. You have a few records of things that may have happened. 

AJ_O’NEAL: Election, election. 

ETHAN_GAROFOLO: So those are some things I look at. So honest to goodness, like when I'm approaching a new system, my default is to think of it in these terms. To me, the standard CRUD monolith is the special case that needs to justify itself. As far as a whole system design would go, but does that include the rabbit hole? 

AJ_O’NEAL: Yeah, for me it does. Like those were great. Those were great. 

ETHAN_GAROFOLO: Okay. Yeah. So, okay. So I've got the events for the withdrawals and the deposits and all of that. Not super useful for a particular user interfaces. So the aggregator's job is to consume those events and to build what is called view data. And that is the read side of the system. So when I want to show a response to you that application goes to view data to get that response to render that to you. And it's the job of an aggregator to do that. So aggregators operate in the same manner as a component in that they get batches of in this time only events to aggregate. They're part of the backend. That does mean that the view data is eventually consistent, which is why we don't use it to make decisions. Like the account wouldn't, the account component wouldn't query an account table to get the current state because some event may come in from the time that it fetched all the events to project that would change the state of it. And in the middle of that is the message store. There doesn't have to be a single message store in a system, you can't have more than one, of course, having additional ones creates additional operational costs. But, and so the communication that is done through these asynchronous messages, and I actually wanna come back to the rabbit hole that you were talking about. If I just do the crud, I'm missing half the system. Yes, and also you probably modeled it in a different way as well that will lead to the productivity wall that will be coming.

AJ_O’NEAL: Well, the one, the one thing though is like, okay, crud systems. I'm either using Postgres or Mongo. Everybody knows that. Right. Whereas I don't, you know, you, you basically told me maybe Kafka is not the right choice, but I didn't hear like, like, what are my, what are my two options? Like if I'm going to do this event sourcing technique, I'm going to be using X or Y. 

ETHAN_GAROFOLO: Yeah. So message DB is what I would say to start with. And it's effectively Postgres, but with a particular schema, not in the sense that Postgres means schema, but like, so that would be my go-to unless I'm at massive scale, then there is the event store project led by Greg Young. I forget the URL if it's.org or.com, but if you Googled event store, all those one word, it would come up. Those are both good choices for an event store. I tend to lean messageDB because I don't work currently at Facebook scale. And so the familiar battle hardened postgres technology works very well for that kind of a use case, but there is a little bit more to it. Okay. I've got the message store now I need code to interact with it. And that's where some of the challenge comes from, but maybe save these for the picks or I could just repeat them at that point, but I don't know if I do pigs, but. 

AJ_O’NEAL: Yeah, you do, you do. By the way, did I link to the correct message DB? There's two of them. There's one with a hyphen and one without a hyphen. 

ETHAN_GAROFOLO: That looks like the correct one that you just linked to, yes. 

AJ_O’NEAL: Yeah, one's the first result and the other one's like the second result. Okay. So we got these links in the show notes here. 

STEVE_EDWARDS: Alrighty. So looking at the time, we definitely should start wrapping up for today. 

AJ_O’NEAL: All right, let's do it. 

STEVE_EDWARDS: And move into Picks.

 

This episode is sponsored by Sentry. Sentry is the thing that I put into all of my apps first thing. I figure out how to deploy them. I get them up on the web and then I run Sentry on them. And the reason why is because I need to know what's going on in my app all the time. The other thing is, is that sometimes I miss stuff. I'll run things in development, works on my machine. We've all been there, right? And then it gets up into the cloud or up on a server and stuff happens, stuff breaks. I didn't configure it right. AWS credentials, something like that, right? And so I need to get the error reporting back. But the other thing is, and this is something that my users typically don't give me information on, is I need to know if it's performing well, right? I need to know if it's slowing down because I don't want them getting lost into the Twitterverse because my app isn't fast enough. So I put Sentry in, I get all of the information about what's going right and what's going wrong. And then I can go in and I can fix the issues right away. So if you have an app that's running slow, you have an app that's having errors, you have an app that you're just getting started with, go check it out. Sentry.io slash four. That's F O R slash JavaScript and use the code JS jabber for three months of their base team plan. 

 

STEVE_EDWARDS: So, AJ, I know you always have multiple picks, so let's start with you. 

AJ_O’NEAL: Okay. Fine. So we'll see if I get through these quickly. If so, I'll add some more, but there's this thing about, so the pyramids, nobody knows how the pyramids were made. There's a lot of theories. None of them make any sense. I watched this thing disclaimer. It gets a little bit kind of like ancient aliens in some parts, like where it's like, okay, you guys are just totally off the rails, but don't throw the baby out with the bad bath water. It's called the movie. Great pyramid K 2019. And it starts off a little too aspirational, like we're changing history with our new discoveries. But the guy just makes a lot of claims that they just seem like simpler. It seems simpler to say that with materials that were readily available in the area, the Egyptians learned how to make cement and pour and mold it. Then to say like all these other crazy things that can't easily be replicated with modern technology, like his method with the cement stuff. It's like, yeah. I could totally believe that Egyptians would have known how to do that because they worked with the earth a lot. They probably discovered cement. Okay, cool. I'm on board anyway, and then there's a couple other things, but it's like a three-hour long documentary. Some of its totally bunk like this whole thing about the meter and water droplets, but a lot of it and also like this thing about the the that perhaps they discovered magnification and glass somehow, which that one seems a little bit further out there, but still I don't know more believable than a lot of the other theories. Anyway, so that that's what I'll pick. 

STEVE_EDWARDS: Hey AJ real quick. I know this goes back away. Are you familiar with Steven Wright? I end up comedian. 

AJ_O’NEAL: Yeah, 

STEVE_EDWARDS: according to him. They were financed by a guy named Eddie But uh, that's all I know. That's a line from one of his first, uh, Albums I have a pony even though I don't 

AJ_O’NEAL: I don't get The joke they were financed by a guy named Eddie. 

STEVE_EDWARDS: Oh, it's part of a bigger joke I'll have to send you a link later, but it's uh, if you're into Steven Wright you get it Ethan's laughing quite hard. So I'm guessing he's a Steven Wright fan. 

AJ_O’NEAL: I've listened to a little bit of Steven Wright, but not much. Like I think I picked up one of his, uh, DVDs at the $5 Ben at like the flea market or something. 

STEVE_EDWARDS: Oh yeah. I hate it when I put please falls asleep during the day, because that means it's going to be up all night. 

AJ_O’NEAL: Yeah. Anyway, I think, I think either he said that or you said that or somebody. Yeah. Anyway. So another thing I'm going to pick is I love Postgres. I think it's great. I've got a Postgres cheat sheet for just like how to install it in a, in a friendly way that will work in Docker, that will work on Linux, that will work on Mac, don't have something that works on Windows, but then like your basic commands to just get the database set up and then get a user, which are just like, there's so many bad tutorials out there. So that's webinstall.dev slash Postgres is where that's at. And then Jim quick, if you are a person like me, so I'm six foot five, I'm 600, or not 625. 265 pounds or so. And there's like my, sometimes my fingers get tingly. Like I basically, I realized I gotta take care of myself. I'm already at a disadvantage being tall and I'm not like hugely overweight, but I'm significantly overweight. And so just a number of things came together and I started, I realized for a long time I needed to make some lifestyle changes, but I just was too lazy and just couldn't couldn't fathom how to do it. And so it's been brewing for, I don't know, probably a couple of years just as a backburner thought. And I was I was talking with the guy that cuts my hair and he was telling me how he's just been feeling so much better lately and yada yada and kind of telling me the same story that I've heard with other people except that, you know, he's a guy that's a little bit overweight that doesn't have everything in his life to get, you know, a little bit more where I'm at and a little bit less like CEO, you know, and, and so gave me a little bit more confidence to like, okay, if he can, if he can get a morning routine together, if he can start exercising, if he can start meditating and get all this stuff and he's feeling happier and he doesn't feel like, oh, I hate doing it every time. He's like, yeah, I've actually learned to enjoy it. Kind of gave me a feeling like, okay, maybe I can do this too. So I'm changing some lifestyle habits. And so Jim quick has a lot of just really good stuff about being productive and whatnot. And he has this video on 10 morning habits, which I'm linking. And basically, I went from somebody who, well, I mean, we'll see how things go over the, over the coming weeks, but from somebody that was, you know, sleeping in a lot, staying up way too late, you know, pulling all nighters, which I probably still will do once in a while. But, but, you know, but it's not good for me. And, and just the very idea of getting up early, like at the crack of dawn, just wasn't fathomable to me, but, um, I, I watched this video and I'm trying to build a morning routine that I can look forward to and celebrate. And the first step is not to say, okay, I'm going to change my lifestyle. The first step is to say, well, what if as an experiment to tonight, I go to bed early and tomorrow I have the mental attitude of I'm going to wake up early, not because the alarm's going off, but because I, I went to bed early and I got, I know I'm getting good sleep. So I have got this mental attitude of, okay, it's possible for me. Well, be well rested in the morning. And then like have the mental attitude of like, okay, no, I'm not going to go jog for an hour. Could I jog for one minute, just one minute. And you know, with that, could I accomplish that? And so with some of these kind of like a tiny habit type things that I'm starting, I, I feel like it's accomplishable and like, maybe, maybe I can change my lifestyle, you know, one week at a time, just a couple of things added to the routine until hopefully, you know, maybe in six months, we'll hear that I'm feeling fit. And I've got my whole day scheduled and I'm a changed person, but, but at least just a few changes in the morning. Like this one's really hard, but taking a cold shower gives you like an extra 10 minutes in the morning, because you get out really quick. Um, but you know, just, just a couple of things like that. So if you're in that situation and you know that you need some change and, you know, you see all these really super Uber successful people and you're looking for like more of an average man to give you a little confidence. I hope that I can inspire you a little bit and check out some gym quick stuff. It's KWIK and there's a link here in the show notes to the 10 morning habits. That's all. 

STEVE_EDWARDS: That's all only 10 minutes worth of pics. We appreciate it. So what's your picks for today Aimee? 

AIMEE_KNIGHT: Oh, man. I gotta hurry now. I'm gonna go with. Hopefully this is not a duplicate pick. I don't think I've picked it yet. Three mindsets to avoid as a senior software developer. I like the advice in here. Just, I don't know, stuff that I think newer developers kind of have by default, but you can lose in time. And I personally don't ever want to lose these things. And I enjoy working with people that never lose these things. So that is my pick for today. 

AJ_O’NEAL: Would it be cheating for you to say what those three things are?

AIMEE_KNIGHT: It's a couple of things. Let's see. One of them is I'm self-taught and I need no one. Another one I'm better than younger programmers. New tech is just recycled old tech. Actually those were the three things. 

AJ_O’NEAL: So two truths and a lie. Which one wasn't true? 

AIMEE_KNIGHT: I know it's fun. I actually, when I read this, I thought of you AJ. Anyways, that's going to be my pick. 

AJ_O’NEAL: Oh, wait. So wait, wait, which one is that?

AIMEE_KNIGHT: New tech is just recyclable tech. 

AJ_O’NEAL: Yeah. That's that is true. 

AIMEE_KNIGHT: I find myself sometimes I slipped into this. So this is a good reminder. Like, you know, get a, there's, there's trade-offs everywhere and just keep it open. Mine. 

AJ_O’NEAL: Sometimes you got to be open to the idea that it's just old stuff. 

AIMEE_KNIGHT: Yeah. There you go. 

AJ_O’NEAL: Anyway, 

AIMEE_KNIGHT: I'll drop a link. 

STEVE_EDWARDS: So I'll go next. Oh my God.One pick today, and this is actually a newsletter. It's called bites by you. It's by Tyler McGinnis. He has a site called UI.dev and he sends out a newsletter periodically. I think it's weekly, but I really like it. You know, the content, the things that he, uh, links to different articles, different things going on in the, in the JavaScript community, but I love the sense of humor that he's got in the way he writes, so I always look forward to, uh, to getting his newsletters because he's got a real dry sense of humor, sort of like mine. So, uh, I'll put a link to the, uh, it was just a bites.dev is the link for the newsletter, but I'll put that in the show notes. Ethan, your turn. What do you have for us? 

ETHAN_GAROFOLO: Well, a few tech ones and a non-related tech one. First one, message DB. So a Postgres implementation of a message store born out of the event type project, which is my second pick and I'll put a link to it in the chat here. But the Event-Tied project is a Ruby based toolkit for building systems in the way that I described. It's fantastic. The documentation side in particular has a lot of conceptual things in there that won't just help you do better at microservices but be better at development in general. So I can't speak highly enough about the project and the folks behind it. I've progressed a lot of my career because I got to know them. So mad props there. So then also there is a node port of Eventide called Gearshaft from a guy named Mario and I'm sure I will butcher his last name, Pareja, Pareja, I'm not sure. He's done a node implementation and just like Eventide, it's used in production systems, financial ones included that move billions of dollars a year. Gearshaft's documentation isn't so well developed. Mario is working on that and he welcomes any contributions. If you like board games, there's a series of games called Unlock, kind of escape room theme. The family really enjoys those. Those are a blast. And then of course, my book, Practical Microservices, I think it's a good pick. So I'll put a link to that as well. 

STEVE_EDWARDS: Well, if you think it's a good pick, that's all that matters. We'll just laugh at you behind your back. Okay. 

ETHAN_GAROFOLO: Fair enough. Yeah.

STEVE_EDWARDS: Alrighty, well, that is going to wrap it up today. You got sort of a long episode today, but lots of good info. So thank you to Ethan for coming on the show and subjecting yourself to AJ. Thank you to AJ for questions and rabbit holes. And we will see everybody next time on JavaScript Jabber. 

 

Bandwidth for this segment is provided by Cashfly, the world's fastest CDN. Deliver your content fast with Cashfly. Visit c-a-c-h-e-f-l-y.com to learn more.