We talk about the Pros and Cons of using a database directly vs using an abstraction layer, common mistakes, optimistic concurrency, and a nice tangent into programming concurrency models to top it off.

Sponsors

Panel

Special Guest

Links

Picks

Steve

Aimee

AJ

Chuck

Valeri Karpov

Follow JavaScript Jabber on Twitter: @JSJabber

Special Guest: Valeri Karpov.

CHARLES MAX_WOOD: Hey everybody and welcome to another episode of JavaScript Jabber. This week on our panel, we have AJ O'Neill.

AJ_O’NEAL: Yo, yo, yo, coming at you from no longer stormy and cold, pleasant growth, just slightly cold, pleasant growth. 

CHARLES MAX_WOOD: Amy Knight. 

AIMEE_KNIGHT: Hey, hey, from Nashville. I don't ever have anything exciting. Nashville's Nashville. Don't associate me with the music industry here. Not me. I'm just here because my family goes by. 

CHARLES MAX_WOOD: Steve Edwards. 

STEVE_EDWARDS Hello from a very, very smoky, so bad you can't go out of your house Portland area due to wildfires. 

CHARLES MAX_WOOD: Yeah, I've heard that. I'm Charles Max Wood from Devchat.tv and that's Valeri Karpov. 

VALERI_KARPOV: Hi everyone, coming to you live from sunny and hot Miami Beach, Florida. 

CHARLES MAX_WOOD: You know, there are days that I just hate people from places like Miami Beach. It's cold here. 

 

Hey folks, one of the things that I find that really makes a difference for people being happy in their job is working in a place that makes a difference. And there's a terrific company out there that's looking to hire full-stack developer just like you, and that's Faithlife. Their average tenure is five years. I mean five years, that's forever in developer years. Usually I see people changing jobs every one to two years. People are sticking around because they're great. They have a great values-based culture and they are hiring developers in the United States. They're looking for full-stack developers who can do C-sharp or JavaScript on the backend and React on the front end. Go check them out at devchat.tv slash faithlife. That's devchat.tv slash faithlife.

 

CHARLES MAX_WOOD: Anyway, you want to remind people who you are. We've had you on like four episodes, I think, plus one. 

VALERI_KARPOV: Yeah, this is my fifth episode. I was also on JavaScript, or my JavaScript story. Let's see here. Well, you might know me best as the maintainer of Mongoose, the popular ODM for Node.js and MongoDB, which is the topic of this episode. I'm also a principal engineer at a tech startup called Booster Fuels. I guess I don't think we're really a startup anymore. We're well into our series C round. But we do gas delivery in the Bay Area and Dallas, Fort Worth, primarily now to fleets. So if you run that, if you run like a landscaping company, you have a bunch of employees with Ford F-150s, you park overnight, we come and fill up your gas, your F-150s gas tanks and everyone else's. And I'm also a blogger, thecobarbarian.com and masteringjs.io. 

CHARLES MAX_WOOD: Nice. I don't think I've heard of Mastering JS. 

VALERI_KARPOV: I just started that about a year ago. I have like it seems to be kind of shorter, more bite-sized tutorials and also ranked better on Google. I found like in Code Barbarian, when I was vlogging on the codebarbarian.com, I found that kind of the blog posts that like got the most traffic and the most positive feedback were kind of like shorter, more bite-sized ones that kind of attacked some more basic topics as opposed to like the really critical thought pieces where I was, where I really put a lot of effort in. So I started thinking maybe I should start like a third different blog where I'm writing more about like, okay, these are kind of common questions that people are asking. Let me put out like a good answer for it. So I don't have to refer back to kind of questionable stack overflow answers every time I have this question. 

STEVE_EDWARDS So you send out master JS weekly, is that you? 

VALERI_KARPOV: Yes. 

STEVE_EDWARDS Okay, yeah, I get that. Good stuff. 

VALERI_KARPOV: Oh, you're on it, awesome. 

STEVE_EDWARDS I'm on it like sticky on tape. 

VALERI_KARPOV: I appreciate it. Did you like the Trello email or the Headless app?

STEVE_EDWARDS Oh, yeah, I saw that. I haven't had too much chance to dig into it. But yeah, I did see that looked interesting. 

VALERI_KARPOV: You should check that out. I think it's one of the best emails I've written recently. 

CHARLES MAX_WOOD: Awesome. Well, let's let's dive in and talk about Mongoose. Now, we have had Joe Carlson from MongoDB on a couple of times. And last time he talked about MongoDB. So assume that our listeners have listened to or maybe they hit pause, went downloaded that episode again and listened to it again and they have an idea of what MongoDB is and how to think about setting up a schema. How does Mongoose fit into this mess? 

VALERI_KARPOV: So Mongoose is an ODM, kind of like an ORM equivalent for MongoDB. It uses the MongoDB Node.js driver under the hood, but it adds a lot of functionality on top of it that you would expect from an ORM. Think like schema validation, plugin, middleware. Yeah, that's all those things that you would expect from an active record. Mongoose provides that for Node.js and MongoDB. The big difference though between an ODM and MongoDB terms versus an ORM and conventional SQL is that Mongoose doesn't really transform the data for you in terms of how it's stored in MongoDB. So if you have, say, an object in active record. What the object looks like in Ruby is going to probably be pretty different from what's actually stored in the database. So you'll have a bunch of different tables and one object goes into a bunch of different tables. With Mongoose, you have a model and the structure of your model matches the data that's in your database. So that's something that addresses kind of one of the big criticisms of ORMs, which is trying to figure out what the hell the ORM is doing to your data and how it's storing it in the database.

AJ_O’NEAL: Okay. So RM ODM, we've got some other acronyms in there. What's all that? 

VALERI_KARPOV: ORM object relational mapping, ODM object document mapping. So a document in MongoDB is like the fundamental unit of what you store. You can think of a document that's just like a JavaScript object basically. So it can have, um, it can have a raise and have nested object. It doesn't necessarily line up with kind of like the rows and columns layout that you expect from SQL. 

STEVE_EDWARDS So isn't there an O and G in there too? What was that? 

VALERI_KARPOV: I know WTF. I know BBQ. 

STEVE_EDWARDS And a WTF. Yeah. Okay. Got it. 

CHARLES MAX_WOOD: So yeah, of course I've been reading Brandon Sanderson's The Stormlight Archives and they wind up facing odium, which is the one in that. That's what I keep hearing. So anyway, good book. 

VALERI_KARPOV: You're gonna have to send me that book. I want to read it now. 

AIMEE_KNIGHT: I guess that brings... 

VALERI_KARPOV: Is that gonna be one of your picks? 

CHARLES MAX_WOOD: What was that? 

VALERI_KARPOV: Is that gonna be one of your picks? Or are we still doing picks? 

CHARLES MAX_WOOD: Yeah, I'll make it a pick. That's fine.

AIMEE_KNIGHT: I guess that brings me to a question. So when I use MongoDB way back when I actually didn't use Mongoose, I just used like the wrapper that MongoDB has. But when I did Rails, you know, obviously I'm going to use, you know, use all of that because there's like, that's, that's the whole point of Rails, like all the magic that you get. So if you don't get the magic that you have in Rails with Mongoose, why use it? 

VALERI_KARPOV: Because Mongoose offers a lot of functionality, kind of like gives you kind of like a halfway point between those two, right? You don't necessarily have to be on one extreme or the other. Like I did a lot of development in Rails between 2010 and 2012. And I like Rails, it's a great project. Just my problem was always like, if I wanted to do something like not quite exactly what was documented, it was always a bit of a problem. Like the flexibility wasn't there. And then the project would quickly go off the Rails, unintended. I probably make that joke every single time I talk about Rails. Oh no, I mean no, Rails is a great project. It wasn't exactly like my favorite developer experience, but a lot of people are really productive with it. So lots of respect to Rails. No offense, man. Just Mongoose kind of gives you, it's not like, so Mongoose isn't a full kind of Rails replacement. It more focuses on like, kind of like the active record part of Rails and handles a lot of the things that MongoDB either doesn't handle for you or didn't handle for you previously. So schema validation is a big one. Making sure that the data that's going into MongoDB is validated and passed into the correct types. For example, I guess the most basic example is MongoDB object ID. So a MongoDB document has an underscore ID property that is default unique identifier. And then object ID in Node.js is an object. It's not a string, it's not a number. It's basically like a 24 byte buffer, or is it 12? It's actually 12, and then the hex string representation is 24, I forget exactly, but that's besides the point. 

AIMEE_KNIGHT: I remember doing a lot of this by hand. It's coming back to me. Ugh. 

VALERI_KARPOV: Yeah, so like, but typically, you know, you're getting in some JSON data or like an object ID from a query string. It's gonna be represented as 24 hex characters. So you need to be able to convert it into a, into an object ID object before you actually query it with Mongoose. And that becomes really annoying if you have an array of strings that look like object ID or an object that contains deeply nested properties of arrays of nested objects that contain, that contain ID properties. So Mongoose handles all that for you. You just define a schema once. You say that, okay, you know, my array is an array of objects that have a property test that is an object ID. And now every time you access the query, whether it's whether you pass in a string, Mongoose will know and automatically convert myarray.zero.test into an object ID and throw an error if you can't. 

AIMEE_KNIGHT: I'm going to have to message my bootcamp instructor and ask them if he did this on purpose. Cause he's also the one who forces to use Vim instead of a, you know, like ID. 

VALERI_KARPOV: Real editor. Sounds like he wanted you to do all the hard work and teach you character. 

AIMEE_KNIGHT: Yeah, 

STEVE_EDWARDS it's almost a built character. 

AIMEE_KNIGHT: I liked it. I like it. 

VALERI_KARPOV: I mean, in my first job out of college, I spent two years writing C++ and vanilla vim, that was pretty fun. I'd really get myself used to vim. So it was helpful because I had to deal with some huge files, but I'm not going to get too deep into vim leads.

CHARLES MAX_WOOD: So you're talking about some of the, I guess, things that are gotchas or are, you know, not as easily done if you just use the MongoJS driver. In general, what is the benefit of an ORM or an ODM? I mean, I've been doing rails for years and years and years. I just actually got a full-time job writing rails again, you know, and so I see some of the benefits. But yeah, I'm curious what your take is, because there are definitely some nice things that some of these systems do for you or do you, depending on how you look at it.

VALERI_KARPOV: Yeah, schema validation is one thing. Plugins and the plugin ecosystem is another thing I'm a big fan of. Another thing that I've been thinking a lot more about and working more with lately is middleware. Middleware is something that is like a very powerful paradigm that the MongoDB driver by itself doesn't provide. It's an instance of... Have you guys read anything about aspect-oriented programming or anything like that before? Or does that ring a bell? 

AJ_O’NEAL: It's been a while. 

VALERI_KARPOV: Yeah. So middleware or just like the general practice of aspect-oriented programming is about being able to hook into function calls. So you kind of visualize your system as a sequence of function calls where you say you're calling like find one, insert one, insert many, all that. And you're able to kind of pipe into this stream of calls and make changes to the calls as they come in, make changes to the results of the calls, do some logging or side effects, things like that. So that's what middleware is all about. Middleware in Mongoose is about saying, okay, every time I save a user, I want to do this one thing. That's especially important with MongoDB because MongoDB is all, one of the main benefits of using MongoDB is denormalization or the ability to denormalize properties from, say. One idea is, say, if you want to render a user with all their blog posts, or at least their top five most popular blog posts quickly, instead of fetching the user and their five most popular blog posts every time. You store an array of their most popular blog posts, like the name and the link to the blog post or whatever. That can kind of save you a bunch. That can make your app a lot faster and a lot more reliable in the sense that you don't need to execute multiple queries in order to fetch the same page over and over again. So handling denormalization without update anomalies is something that Middle Letter is really good for. 

STEVE_EDWARDS So is that denormalization handled at query time or at storage time? 

VALERI_KARPOV: storage time. 

STEVE_EDWARDS Okay. So my experience, I have quite a bit of experience with using Apache solar, both as a search engine, as well as a data store, as a base data store for a website, for a large website. And so that's one of the things that you try to do with solar is denormalize, get everything in one record so that you're not making multiple queries. You know, I got this, now I got this ID, now I got to go run another query and get that. I've seen people use solar in a relational structure, believe it or not.

But yeah, it's the same idea as I understand it. And correct me if I'm wrong, but you know, you try to get everything in one record so that when you query that record, everything's there and you're not making multiple queries or in the case of the database, well, multiple queries to your database. 

VALERI_KARPOV: Yeah, similar to that. The mnemonic or like the kind of catchphrase that I use for that is store what you query for or just store what you actually use. Good example for that is, so user documents. If you want to display a user's top five blog post and- like do that every single time you display a user, whether it's their profile or whether you have a list view of users, then it's a good idea to store the user's blog posts or at least some of them on the user document. On the other hand, something that you should probably, you shouldn't be storing on the user document is their password hash. That's something that actually gets people a lot because you very rarely want to load a user with their password hash. Unless people only do login, if anyone interacts, with a user document that the user themselves, you should probably exclude the user's password hash from the, from the user document, because you're going to be constantly excluding it from the doc in any way. And the, what 

AJ_O’NEAL: NPM in the early days, in the early days of NPM, everything was in a couch DB and all of the user password hashes were public the entire, the entire database. 

VALERI_KARPOV: Wow. I was at a problem.

AJ_O’NEAL: I don't know if it really was a problem or not. It might've been because they might've been using something like MD5, but I don't know if Bitcoin was a big thing yet. So I don't know if we had like hash miners in the same way. Anyway. Yeah. Big companies have made this blunder. That's what a big organizations have made that blunder. 

VALERI_KARPOV: Yeah. It's an easy blunder to make. It's just tempting to store it because it just seems like a user's password is so intuitively tied to the user. But the problem is with the password, you explicitly want to hide it most of the time. And most of the time you don't even, you don't need the user's password hash, unless they're logging in. So that's like the one case where you, uh, you want to use the password hash. So you're better off storing it in a separate collection if you never want it associated with the user. 

CHARLES MAX_WOOD: So then what does that query and that flow look like? Cause I think that gives us an idea of how to think about some of these problems that Mongoose is going to solve for us. 

VALERI_KARPOV: Yeah. So what Mongoose has is a function called populate. Populate is kind of like the equivalent of a left outer join in SQL. So it makes it easy for you to query a user and also throw in their password as well, while having the password stored as a separate document. So you don't explicitly have to hide the password when you load the user most of the time, unless you explicitly opt in to pulling the password documents as well. So denormalization is something that's handled by middleware. That makes it so that you're embedding certain data that doesn't strictly belong to the user document in the user document. And then Populate helps you pull data from separate documents into the document for your application. So Populate ends up executing a separate query under the hood. 

CHARLES MAX_WOOD: That makes sense. So I guess you kind of explained you want to denormalize on things that you're not going to be pulling in every time. And, or maybe I got that reversed. I can never remember which one's normalized and which one's denormalized. 

VALERI_KARPOV: Yeah, you want to kind of denormalize if you don't need that every time. You always want associated with a user. I think of the things that like a user really needs, the things that you can't really represent the user in your system without that's beyond the user, things that you always want to display. So like, you know, password or not password, profile picture, blog posts, or some other sort of information that's associated with the user that you always want to display links, things like that. And then the things like, say like their password or like all their blog posts, as opposed to just the ones that you want to kind of highlight when you display their profile those should also be separate documents. 

CHARLES MAX_WOOD: So I guess the other thing that I'm wondering about then is most popular blog posts that could change over time. Right. So in a system like what we're talking about here, do you just, you know, say you have blog posts, visit count in another collection. Do you just run a job periodically that updates that or? 

VALERI_KARPOV: Yeah, you can do a job. You can also do kind of, if there are, you can also make it middleware as well, where every time you increment the page views, per month or something like that, or the total page views collection. You can also go and update the user. If you wanna be very, very clashed, if you wanna be like really up to date, but for the most part, these sort of like, in SQL parlance, this would be called an update anomaly where the actual most popular blog post doesn't line up with what's in the user collection. But most of the time, the difference is not too bad. And it's also something that you can fix later with a job. So one of the things I've also been thinking about more lately is classifying potential problems in the system. And so the inconsistency of, oh, the user's most popular blog posts aren't quite up to date, but they're going to be updated in a day anyway, it's an easy problem to solve that's also not particularly pressing. Whereas on the opposite end of the spectrum, you have issues like, oh, the system is experiencing that's kind of like a hard, urgent problem. And you want to avoid the hard, urgent problems as much as evenly possible. 

CHARLES MAX_WOOD: Right. So what kinds of things are normal to come up in, you know, in these kinds of systems, you know, you're talking about, yeah, the hard, urgent problems like performance, for example, I mean, how do you go about identifying those and solving those? Because, I mean, I usually run into those when I either have some weird case or when I have like too many of something, right. That I'm querying or certain classes of performance issues or things like that. How do you keep track of that stuff and then how do you solve for it so that your users maybe don't even see it once they update? 

VALERI_KARPOV: Yeah. So performance is one issue. Another issue that I frequently, that frequently comes up as a, it's kind of like a hard and also urgent problem is just errors that prevent people from doing their jobs. So that's another one that I think of there. But in terms of identifying performance issues, one of the things that I, that Mongoose helps with there is Mongoose tries its best to kind of make like one operation the same as one operation for MongoDB. And it kind of encourages you to avoid some potential pitfalls with MongoDB. So for example, MongoDB has its own kind of equivalent to, not necessarily equivalent, but they have a operator called $lookup that lets you execute kind of like a left outer join as part of their aggregation framework. Dollar lookup works great, but it also runs into some performance issues sometimes. The issue that pops up with dollar lookup is that, let's say you're querying for 10 users and you need to also look up those users' log posts, right? If you have an index miss on the query that's basically pulls an individual user's log post. You're going to get O of N squared performance degradation because dollar lookup under the hood executes a separate lookup for each of those 10 users through the blog post collection. So if you have 10 users, you're doing 10 scans through the blog post collection. If the blog post collection is huge, that can be a serious performance issue. With Mongoose, instead of globbing it together into one complex dollar lookup operation that's a little bit hard to inspect. It makes it into two separate queries and have like two queries are easier to, uh, to think about reason about than one aggregation query, at least in my experience, because it's just a query. Either it doesn't table scan or it doesn't scan certain number of documents. Makes it, uh, makes it a little bit easier to dig into. Does that make sense? 

AJ_O’NEAL: So then you're bounded by the number of IDs that you want to look up at that time. Right. Which would make sense because you're going to be paging data anyway. 

VALERI_KARPOV: Yeah. Yeah. So if you're doing so with Mongoose, if you're doing like loading one document at a time, right now, it actually does do a separate query for each document that you pull out. We actually have a pull request open to improve that right now to make it so that we go through every batch that you load. Or that's a MongoDB driver loads underneath the hood. So you don't really have to, you don't really have to execute a separate query for each individual user to fetch their blog post. But if you're doing pagination, like loading 10 users at a time, that ends up being only one query for those 10 users blog posts. 

AJ_O’NEAL: So I didn't quite understand. Can you, or can you not have multiple IDs or multiple indexes so that lookups are fast? 

VALERI_KARPOV: That's intense. The problem is, is like sometimes in production you miss indexes. It's like, and other times you don't have indexes on a collection yet because the collection hasn't gotten big enough to justify the, uh, justify the index costs. But then the potential is if your dollar looks up, if you're doing a dollar lookup from a collection that is really big into that collection, then all of a sudden you can start having performance issues because now your performance degrading as O of N squared. 

AJ_O’NEAL: And this, I mean, that's the same problem with SQL databases. So no, this is just the nature of databases. Like you've got to figure out your indexes. Things are slow. Things like table scans, which in this case, dollar sign lookup and indexes are good culprits to look for. 

VALERI_KARPOV: Yeah. Indexes are important. And if your indexes are perfect, then it's not really a problem. It's just an issue that I've run into at least a couple of times at my day job where we started using dollar lookup pretty early when it came out. It was released. I want to say MongoDB. I forget which version, 3.4, 3.6. So we started using it pretty early and made it kind of like our de facto choice for relationships. And then we quickly, within kind of like six to eight months, decided that that was a bad idea, but still haven't changed it yet. And we decided it was a bad idea precisely because we ran into a case where, okay, we were looking, we were querying into a collection that was indexed, but the dollar lookup was slow because it was, because the dollar lookup was looking into a collection. that only had about a thousand or so documents, but because it was executed at a table scan for multiple documents from the local collection that the Dollar Lookup was looking up, or that the Dollar Lookup collection was using. I meant that was not very well explained because we were scanning through a lot of documents in the local collection, even though the foreign collection wasn't huge, the fact that we needed to scroll through a thousand or two thousand documents every time was causing some serious performance issues. Now, normally in my experience, unless you're using $LOCEP, you really don't need indexes on a MongoDB collection if it has less than 10,000 documents. And probably if it has less than 100,000, you can get away with it. It's just like in my mind, like 100,000 is like the cutoff where you should be using indexes unless you're on like ultra high-end hardware or anything like that. My experience is just kind of low to mid-tier VMs. So your experience may differ. 

AJ_O’NEAL: Well, what about a case like..You know, say, let's go back to the user object. Like I've got an email, which is ephemeral, you know, people change their email address. And then I've got a user ID, which is permanent. Like no matter how many, you know, linked accounts I have through my email and Facebook and GitHub or whatever, I'm still going to have the same user ID, but it becomes very, very convenient to be able to look up a user by their Facebook ID or by their email or by their user ID or by their Google ID, et cetera. Or are you just saying like, no, it's really not that big of a deal. It's fine just to do table scans on all the other IDs. 

VALERI_KARPOV: Oh, so in that case, email would be a candidate for denormalization. So like, let's say you want to search for blog posts by the author's email address. Their email is a candidate for denormalization because the assumption is that the user's email will change infrequently. If it changes at all. So many cases, you're not even allowed to change your email address on an account, you need to create a new one. And even if you do allow it, people don't change their email terribly frequently, or at least much less frequently than people read their blog posts. 

AJ_O’NEAL: It's not, it's not my point. Wasn't that people change their email address. My point is that there's multiple identifiers for a user. Like my Facebook ID is an identifier. My email address is an identifier. My GitHub ID is an identifier. Yeah. Like when I click social sign in. Like I've got five different options that I might sign in and it's all the same, but all resolves to the same account at the end or, and there's, there's other instances like that where you have more than one identifier and one might be public and another might be private or intersystem or pairwise, et cetera. I was just bringing up the use case of you do have multiple IDs for things. 

VALERI_KARPOV: Yeah, exactly. And SQL like. Kind of similar to MongoDB, there is kind of a notion of like the primary identifier though. So the primary identifier is the document, which is MongoDB as in documents underscore ID. There's, that's the one that you usually use for relationships. I guess you could in theory use the OAuth ID as well. That's just not something I do. I mean, in general, like the way that I like to structure it is like, we just have an authentication method model or authentication method collection. That's store that's basically a map. Each document is a map from a user's ID, like the user primary identifier to the, uh, to the authentication, uh, the OOP provider, like Facebook, Google, and then the, uh, the OOP ID and whatever often you need. 

AJ_O’NEAL: So rather than having more indexes on a document, you just create more documents where those documents are just the one ID linking to the other ID. 

VALERI_KARPOV: Yeah. That's what I like to do. I get people don't like logging in is not the most frequent task when you're working within most apps. You're more likely to just log in. And then you do a bunch of other stuff that usually involves reading a lot of data. 

AJ_O’NEAL: Right, right, right. 

 

Leveling up is important. I spend at least an hour every day learning ways I can improve my business or take a break and listen to a good book. If you're looking to level up, I recommend you start out with the 12 week year as a system to plan out where you want to end up and how to get the results you want. You can get it free by going to audible trial.com slash code. That's audible trial.com slash code. 

 

CHARLES MAX_WOOD: So what are you working on now? Like what, what's coming in the next version of Mongoose? 

VALERI_KARPOV: So right now the, uh, the big thing that we're working on is officially supported type script bindings. We don't have a, Mongoose doesn't have an index.dfts file. And that's been, that has been source of a lot of frustration for a lot of people. So we're finally biting the bullet and making that happen. That'll be our 5.11 release, along with a bunch of other smaller features. We just released version 5.10, which included, the big features there were, I want to say, I forgot my blog post already. Let's see, optimistic concurrency was a big one. Another big one was, let me actually just pull up the list real quick. What was the, what were the highlights? How did I forget the improved transaction support? I guess have a more generic topic to talk about would be, it would be optimistic concurrency. Have the particulars of Mongoose transactions might not be as interesting. What do you think? 

AIMEE_KNIGHT: I was about to say, can you get into that? I'm curious about this. I don't think I've heard it before. 

VALERI_KARPOV: Optimistic concurrency? 

AIMEE_KNIGHT: No, I don't think I have. 

VALERI_KARPOV: Okay. Let's say you have two copies of the same underlying document. Let's say you do find one and you load, let's say you load my user document, and then another Bay request API handler also loads the same document. So now you have two request handlers that have kind of two copies of the same underlying document that's in the database. They both make some changes and then they both save the changes to the database, right? So they could in theory change the document in incompatible ways. If you have validation that assumes that the document that you have in memory is the same as the document in your database, you get, you can end up with an inconsistent state. So the example that I like to use is let's say you're building like an Airbnb clone and you have basically, you have like a home documents, which is like a house that someone can rent and you, um, and you want people to basically put their submit their houses for approval. Cause you want to make sure that the listing looks good. So you have a list of photos for the house and you wanna make sure that any house that's listed has at least two photos because otherwise it's just gonna look bad. So how that could go badly in Mongoose is one request handler loads the documents, changes the document status to approved and then saves it. And then another, but in the meantime, another request handler came in and deleted all the photos. So both changes end up in the database at the same time, but neither is aware of the other. So what optimistic concurrency does is it tracks an individual version on the document and throws an error if the document has changed underneath you when you're trying to save it. So if you're using optimistic concurrency, one of the two saves would throw an error because the document changed underneath them. The reason why this concurrency is optimistic is that it assumes that there will be very few conflicts because it's a bad user experience to have a throw-in error. And if you expect a lot of conflicts and a lot of contention, you might want to have something that retries or repoles the data or something like that. 

AIMEE_KNIGHT: Okay. This makes sense. 

VALERI_KARPOV: It just assumes that there won't be very many conflicts. So when there is a conflict, we're just going to throw an error and call it a day. 

CHARLES MAX_WOOD: So one other thing that I'm curious about. AJ loves TypeScript. He tells us that every week. 

VALERI_KARPOV: I can imagine. Okay, AJ. 

AJ_O’NEAL: Deno, Deno, man. No, are you redoing, are you redoing Mongoose and TypeScript for Deno? 

VALERI_KARPOV: No, not planning on it right now. 

AJ_O’NEAL: Well, maybe you need to change your plans. Maybe you need to get on the bus, man. Stop screwing around with that mansy-pansy language JavaScript. Start fooling around with a real language like TypeScript that's got types and an interpreter that doesn't need source maps anymore. 

VALERI_KARPOV: Yeah. Source maps I don't like. I tried playing around with Dino. There were a few blockers that prevented me from using Mongoose with Dino, mostly just because of, I seem to remember something with like crypto or Node.js is crypto not working right with Dino. And the MongoDB driver relies very heavily on the Node.js is built in crypto library. So that just wasn't gonna work unless I use a different MongoDB driver. So that was my problem right there.

AJ_O’NEAL: I got to check on what it's like today because 1.4 was just released yesterday. But yeah, as, as of 1.0, the crypto libraries for Dino are not all complete yet. That, that's one of the things that I, I found too, but goodness knows, I hope people don't bring node libraries over to Dino that would just be that, that would destroy the whole, like the beauty of Dino is that it actually makes a decision on purpose to be TypeScript. And if you start bringing in JavaScript crap over to Dino and Dino just becomes a dumpster fire like node, well, let's just stick with node then. 

VALERI_KARPOV: Oh, okay. So what are, uh, what are some of your concerns about node? 

AJ_O’NEAL: Are you asking in response to the dumpster fire thing? 

VALERI_KARPOV: Yeah, I'm just, uh, I'm just in general, uh, curious. No one has its problems. I don't disagree. I'm just curious, like what you think the most, uh, the most we call egregious ones are.

AJ_O’NEAL: I think just a complete lack of leadership from the very beginning, just a lot, a very, a very like lackadaisical laissez faire, like anything goes, like we don't need to make decisions, the community will sort it out. And, you know, even though it got investment and attention from Microsoft, it's never worked well on windows. I'm glad to say that a bug that I opened like five years ago, finally got closed this last month, which that was awesome, a windows related bug, the event loop would just drop if you very simple sequence of like you do a console log and an HTTP request and, and then open standard in. And if you did them in the right order, really simple, basic stuff. But if you did it in the right order, your program would just stop running. Just, just stopped. Anyway, it's just that it was never, it was never a community of JavaScript developers, you know, from day one, the Ruby people stormed in and were like, screw JavaScript, we've got coffee script. And then after that, it turned into other languages and then it turned into TypeScript. And so just like, I don't know, it was like a baby that was thrown to the wolves, no sooner than it was born. And so it's not that I think that there's a problem with Node as a technology, although, I mean, obviously its concurrency model is limited. It works really great for most of what we're doing on single CPU deployments, as opposed to multiple CPU deployments. You know, so it it works really well in most cases, but only being able to work on a single CPU obviously has limitations and makes it unfit for certain use cases. But really it's just the, I wish, I wish that it had strong leadership like Python or go where the smart people that knew they were smart made intelligent decisions on purpose, included things in the standard library on purpose and you know, just made it a really pleasurable language to work with. Whereas over time it. It just became, it's become more difficult to work with rather than easier because of all the, it's the layers of extraction and complexity and stuff that's been added to it over time. So, I mean, I still use, I love working in Node myself. I write just plain vanilla JavaScript in it. And you know, obviously, that's not something everyone's going to do, but for me, that makes it pleasurable. And then I use go for, for other things, but I feel like it's kind of a tangent to the episode there, but that's the answer to your question. 

VALERI_KARPOV: No, that's a fair point. I did a lot of work with Go, I think around 2013, 2014. I actually didn't like Go very much. I thought JavaScript or Node was a lot better for a couple of reasons. Like number one, don't really like Go routine. Number two, I didn't really like how, at least in 2013, Go installed dependencies globally, so it made it difficult to have multiple projects with multiple incompatible versions of the same Go dependency. I also didn't like kind of how there wasn't like a standardized package source for packages. You kind of just needed to see it downloaded from GitHub. I don't know. It's been a while since I worked with Go. But for one thing, I, one of the things I love most about JavaScript is ironically enough, the concurrency model. Like I think the event loop is good enough for the vast majority of use cases. And like I haven't done like some pretty complex clustering algorithms in JavaScript. Oh, don't tell my college professors that still have a heart attack. But I mean, it's one of those things where like, okay, you know, you're, if you're doing a complex calculation, you just need to remember to await every once in a while to basically yield on yield on the main thread and let other stuff do work. 

AJ_O’NEAL: So I don't want to take up any more time on this other than to dispel one common myth because everybody had this problem that worked with go in the early days back before the package management. And yes it sucked, but it is absolutely the best thing that exists out there today. There is no package management system that is better than go. And I will, I'll link to a video as one of the picks. I go, I think it's go with versions and I advise you to take a look at it. It is the most intelligent design ever around package management. It took them a long time to get there, but once they did, it's bar none. It is absolutely the best. 

VALERI_KARPOV: I'll take a look. I mean, one of the things that I really liked about go unrelated to that, that I thought was really well designed was their JSON on marshalling support. I'm not sure if that's changed since 2013 because I haven't really touched Go in like four or five years. But that was one thing that I found kind of really delightful in Go. And I have like written a couple of JavaScript libraries to kind of simulate that. And to a large extent Mongoose can do things like that but it's not quite as good. Or this impact isn't quite as neat as Go. So really big fan of Go's JSON on marshalling. 

CHARLES MAX_WOOD: All right. Well, I'm gonna push this over the edge to Picks unless there's something else that we want to dive into here. 

VALERI_KARPOV: Nothing, nothing for me. 

CHARLES MAX_WOOD: All right. I didn't hear anyone else pipe up. So let's go ahead and do some Picks. 

 

Have you thought about learning to do native iOS development? Are you using Swift at work? Or maybe you've considered writing applications for macOS. We have a podcast that covers all of that called iFreaks. We have a new panel and a lot of exciting things to talk about. So come check us out at ifreakshow.com. 

 

CHARLES MAX_WOOD: AJ, why don't you just start us off with Picks? 

AJ_O’NEAL: All right. So I've got an overload that's accumulated. So I'm just going to have to run through some here. Graph QRL it is curl for graph QL. I haven't actually used it that much myself. Well, I haven't used it at all yet. I just, I was looking for something like it after working with graph QL playground. I was like, there's gotta be something, you know, that, that is more like curl. And it's at, I just thought the name graph QRL would be exactly what someone would call it. And so it is QRL and it gives you, I don't think people understand how awesome the terminal is and how great the auto completion can be. And it's a model for what the web should be like. And I mean that in all seriousness, I wish more web pages worked as well as great command line apps, but yeah. So graph QRL that's with Q URL for those that didn't get to catch that. Also we've been having these, what I call micro power outages. I think the technical term might be brown out, but it's just all summer long, the power goes out for like a second. And I don't know what it is, if it had been the heat when we were having like a hundred degree heat waves or whatever. But then we had a storm come through last week and the power was going out like every 15 minutes to an hour for several hours. So I finally got myself an APC UPS, just a cheap one. It was like 60 bucks for my iMac. And it has, it has USB, so it plugs in and then you can go into the power settings and then a new option magically appears that is. UPS settings and you can go in and have it so that if you've got some sort of desktop computer and it works with Windows too of course. It has like special driver software with a Mac, no driver needed. It's just plug and play. So it's an APC UPS and I'll link to that. It was cheap, 60 bucks and lasts like 20 minutes or something. I mean it was basically the smallest amount of time that something can last that has a USB port. And then I picked up another one for the mount, the router, the modem and the and the access point, which is just a cyber power. And it does not have like the USB stuff and it only lasts like five or 10 minutes in it, but it was 40 bucks. So for a hundred bucks now, I don't have to worry about these little micro power outages that are happening all the time. Then I'm also gonna pick, so Google Lighthouse overstepped, but in a way that I think was necessary. So, you know, a lot of these companies have normalized this problematic behavior of creating large huge bundled libraries and Google specifically picked on Moment.js and gave them the thumbs down. So if you use Google Lighthouse, Moment.js is recommended as do not use, and then it recommends other libraries instead, which I think like that's, that's, that feels kind of, you know, and then 

VALERI_KARPOV: I agree wholeheartedly. And I think I saw this morning that Moment actually like put a note on their homepage, that Moment is now done, quote unquote. 

AJ_O’NEAL: Yeah. Deprecated. 

VALERI_KARPOV: Like legacy maintenance mode. 

AJ_O’NEAL: Yeah. But so here's the thing, like on the one hand, I think that it was an overstepping of bounds, but on the other hand, I think that they kind of created this problem with, you know, stuff like angular, for example, where it's like, Hey, let's include two megabytes of JavaScript on the page. Not kidding. Like two megabytes. Right. And so I think for them to kind of, I, so I feel good and bad about it at the same time, but that was, I think that the result will be good, although I think that. You know, interpersonally it's bad. I think that the result of having, yeah, people being more conscientious about the size of their webpage and the complexity of their bundles. Well, I think it will end up having benefits overall, although it does make me cringe and feel kind of, you know, feels dirty to have Google do that, but whatever. 

VALERI_KARPOV: What I found kind of most frustrating about that, I also kind of gave me the heebie-jeebies even though I'm like, okay, like I understand why people don't want to use Moment. My thought there was though that just like, you know, the, the cost of save the savings that you get from switching from moment to Luxem is very small compared to the benefits that you get from switching from react to preact at least by default with the default web pack configs. So like why pick on moments as opposed to say reactor Angular? So that kind of was what felt a little like a disingenuous to me. 

AJ_O’NEAL: And I, yeah, I don't disagree with that. So I got last two. And then I whittled down most of this mountain I've been building up. Frog chess is like checkers, but kind of with a twist. It's, it's only got two rules. You have to jump a frog every turn or you lose. So you have to jump a frog every turn or you lose. And if you jump out of bounds as your last jump, then that frog dies. And I think those are the only two rules. Well, and then every jump is jumping over one adjacent frog. You can't move one square over. You have to jump over a frog and then that frog of course is removed from the board. But anyway, frog chess, it's like checkers, but it's an interesting spin. The rules are a little bit more. It's not the rules are more sophisticated, but the strategy becomes more sophisticated by binary cocoa. Love binary cocoa. They have tons of great physical games as well as some digital games. So check them out. And then last but not least, I finally figured out a way to get Windows subsystem for Linux to install from the command line so that you don't because you know, the old saying a picture costs a thousand words. It is so difficult to explain to somebody how to go through a bunch of different menu systems and da da da da da da da. It was not, the result is simple. It's like ends up being five different PowerShell commands and you have to reboot in between them because Windows you still can't an installation of something without the prior thing being in. You can't just say, okay, install all these things, then reboot. No, you have to install half of them, reboot, install the other half, whatever. But so I've got that single command and the cheat sheet for how it works in case you want to replicate it and integrate it into your own system of sorts up on webinstall.dev slash WSL. So hopefully next time you're trying to explain to somebody how to get Linux on their Windows computer rather than linking them to six different blog posts with 400 pictures and lots of confusing things. You can just be like copy and paste this line, put it in the terminal, reboot, copy and paste it again. You're done. You've got Linux installed. Now you can access it from the start menu. 

VALERI_KARPOV: If you want some inspiration for using WSL, Scott Hanselman gave a really great talk on JSLA last month about that. It was really good. And just look up one of his talks on WSL. He really puts on a great live coding show. 

AJ_O’NEAL: I will do it. Thank you. 

CHARLES MAX_WOOD: Nice. All right, Amy, what are your picks?

AIMEE_KNIGHT: Okay, I'm going to go with another more DevOps-y kind of thing since that's a lot of my world now. But I always love different people who do code cartoons and stuff like that. But I think this girl is in DevRel at Google, but she's called herself the Cloud Girl. And she has basically different cartoons for all the different services in GCP, which is a Google Cloud platform. So pretty cool. I like it. That's going to be it for me. 

CHARLES MAX_WOOD: All right. Steve, what are your picks?

STEVE_EDWARDS First of all, I want to say AJ, a picture is worth a thousand words. It doesn't cost a thousand words. Minor difference. 

AJ_O’NEAL: Unless you're writing documentation. 

VALERI_KARPOV: Yes. 

STEVE_EDWARDS Well, then it costs you more than a thousand words. Probably it's worth a thousand words to the people reading it, but it probably costs you about 10,000 in cuss words. Right? Anyway, so hopefully this will still be a little bit relevant or maybe not by the time this episode comes out, but as of mid September here in Oregon, half of our state is burning down. We've had some pretty significant wildfires through Clackamas County, Marion County, and down farther south. We're like number one in the FEDS list of important fires in the region. Started out on Labor Day, actually, we had some incredibly dry, hot winds. And within about two or three hours, we had three big fires, three or four big fires take off and just go nuts. I belonged to one of the fire districts and was out on the front lines for probably three days, parts of three days helping fight. So there is a nonprofit that's associated with our fire district here in Clackamas County called the Clackamas Emergency Services Foundation. And I created a website for them using Next. And I've blogged about it on dev.to. And one of the things that we're doing is we have a PayPal option on there. It's a donate button to donate. And there's a specific option for wildfire relief. There's been, it will go to the foundation that we distribute them in terms of supplies and hotels and, you know, any number of ways to help out the community. We've got right now the number just went down to eight dead in the state and probably about 50 people unaccounted for. We burned close to 900,000 acres in a week, which is more than double the annual amount of acreage burned. And this is just, it's unprecedented in our state that the number and the strength of an acreage of wildfires burned. So my pick is just our little website, CESF, Pacamist Emergency Services Foundation dot US little donate button if you want to donate. I'm sure by the time this comes out, we'll still be looking for funds to help those around us in our community that have been displaced and affected by the wildfires. 

VALERI_KARPOV: That's really terrible. And yeah, I just wanted to say thanks for your service. And I really, I really admire your bravery. That's, it's easy to get on the podcast. It's hard to go fight a, hard to go out and fight a real live wildfire. 

STEVE_EDWARDS Yeah. It's, it's a, yeah, that first night was absolutely nuts. Monday night, I was on between 6 p.m. and 3 a.m. By the time I got to 3 a.m., I was on my fourth fire. Three of them we'd put out and the fourth one turned out to be one of the bigger ones that took a number of days to get control of. And that was just our part of the state. So yeah, there's a lot of guys that have been out there a lot longer, you know, a week at a time, 7200 hours at a time. And it made it more challenging because you have a lot of resources out in the wildfires, but you still had fires in town. And so we had a lot of guys that were on fires with much less, much fewer resources than normally for a house fire or a, you know, apartment fire or something like that. So we, with the federal help and state help from the state fire marshal that we've gotten in since Wednesday, that's allowed us to go back and, and go back to our base, you know, coverage as there's more resources helping, but it, it took a few days to get help. And just because everybody was already stretched so thin anyway, especially with what's going on in Northern and Southern California and their fires down there. 

CHARLES MAX_WOOD: Yeah. That fire stuff is nuts. Well, hopefully, hopefully the air gets better and everybody stays safe. 

STEVE_EDWARDS Yeah, we had a so smoky right now that the air index was off the charts. I mean, they're saying just don't go outside. I can go outside and be outside for a while and my smoke clothes will smell like smoke. And we had, fortunately the weather's cooled down. We really haven't gotten any wind, unfortunately, blow the smoke out. We're supposed to get some rain here hopefully soon, but after the first couple of days, then we got this nice inversion. So these clouds came down and sat on top of all the smoke and kept everything in, at least in the valley area here. So it's starting to get a little better and hopefully over the next few days, things will go back to normal. But even a lot of businesses like restaurants and stuff have closed down just because of smoke so bad. You don't even wanna go outside because it's like smoking a pack of cigarettes. 

CHARLES MAX_WOOD: Yeah, makes sense. All right, well, I'm gonna throw in some picks. So one thing that I've been working on, this is podcasting related more than programming related is the Podcast Growth Summit. It's gonna be held the second week of November. It's going to be free and then you can buy an all access pass if you want to get access to the videos after the day of release basically. So each day videos will go up for 24 hours and then they'll be taken down unless you're by the pass. I'm looking to do something very similar for most valuable dev. So if you're interested in either of those most valuable dev, it's going to be focused more along the lines of, you know, if you're a junior to mid level or maybe even a senior, some seniors run into this too or you're competent at your job, you're comfortable there, but you're not quite sure what you need to be doing in order to move ahead to the next thing. So it may be, I don't know what to learn next, or it could be, how do I take my career to the next level? Should I be speaking at users groups or speaking at conferences or doing any number of other things? I'm gonna be bringing in a number of experts to talk about their journeys through a lot of that and what they do to get past that, right? And I'm just going to do a bunch of interviews and we'll just post them as part of the deal. So anyway, those are going to be at podcastplaybook.co and mostvaluable.dev. And then I mentioned the Stormlight Archives by Brandon Sanderson. And the next book is actually coming out next month. So keep an eye out for that. They were going to do a big launch party here in Salt Lake and with COVID, I think they canceled it. So that was my chance to meet Brandon Sanderson. Anyway, a good friend of mine was going to be running part of that live event. So anyway, next one, I guess, usually when he does the new book, he does a lunch party here in Utah cause he lives here. So, and those are well worth going to. They're fun. So anyway, I'm going to shout out about that. And then, yeah, I don't know if I have anything else to really shout out about. I did start a full-time job to kind of pay the bills around here, but we are still looking for sponsors on some of the shows and things like that. So if you have any ideas, feel free to reach out to me at Chuck at devchat.tv and Yeah, with that, we'll let Val Valeri do some picks. 

VALERI_KARPOV: Yeah. Thanks Chuck. So I guess number one pick, I just came out with a new ebook called mastering mongoose, I took about eight years of experience building apps with mongoose and about six years of experience as maintainer and kind of distilled it into 170 pages, took careful time to make some just opinionated enough in terms of which mongoose fundamentals are important and sufficiently unopinionated on which, uh, which web server framework or front end framework you should use. So check it out. There's a little discount for JavaScript JavaScript listeners. Link should be with the pic. I guess second pic, a brilliant, it's our discussion about, about go lang and go on marshalling. There's a, there's a small library that I wrote called archetype that this is what booster uses for, for validating all data coming into our API and it does something inspired by or it works kind of similar to Go's on marshalling, or at least it's the closest thing I've seen in JavaScript to Go's on marshalling. It also has kind of a Mongoose-inspired syntax, so check it out. And finally, I've been, third pick, I've been reading Neil Stephenson's Cryptonomicon this summer. It's a long book, but it is a very fun read. A lot of hilarious jokes, a lot of, had a lot of really cool commentary on cryptocurrency, so worth checking out if you have a few months to kill. 

CHARLES MAX_WOOD: Nice. And Valeri, if people want to connect with you, where do they find you? 

VALERI_KARPOV: You can find me on Twitter. Let's see here, at code underscore barbarian. You can also follow Mongoose on Twitter at MongooseJS. Our website is MongooseJS.com. And you can also find my JavaScript tutorials on masteringjs.io. 

CHARLES MAX_WOOD: Awesome. All right, well, we'll go ahead and wrap this up. Thanks again for coming and talking to us. 

VALERI_KARPOV: Always great to be here. Thanks for having me. It's always great to chat with you guys. Amy's always great. AJ always keeps me on my toes. 

AIMEE_KNIGHT: I was like, please, please take other people off. Thank you, though. 

VALERI_KARPOV: AJ, always fun debating with you. I know we disagree on quite a number of things, but I really respect your opinions. And you always have interesting stuff for me to look at. 

AJ_O’NEAL: Thanks. I don't feel like we could disagree that much. 

VALERI_KARPOV: I mean, I'm not the world's biggest TypeScript fan or Go fan or Dino fan. 

AJ_O’NEAL: So I'm not, I'm not a big TypeScript fan myself, but I, I'm a big fan that Dino has chosen a side and something to stick with that. I think that it's better to, to have a platform that actually supports its language a hundred percent and you know, and sticks to it rather than have a platform that's kind of Willy nilly and the whole source map thing. 

VALERI_KARPOV: Yeah. 

AJ_O’NEAL: So I think that go is much better than TypeScript as far as type languages go. But I think that TypeScript is, I'm glad that Dino uses TypeScript. I think it's a new opportunity. I think it could move the web forward. 

VALERI_KARPOV: Oh, hopefully next time I'm on, we'll have a chance to base something like that. But after we get to see a little bit more of where Dino goes in the near future, following closely. 

AJ_O’NEAL: And I call Node a dumpster fire with all the love in my heart. I love that. 

CHARLES MAX_WOOD: All right. We should just wrap it up here. Till next time folks, Max out. 

AIMEE_KNIGHT: Bye.

AJ_O’NEAL: Adios.