Homebrew Unleashed: Diving into the Fast and Efficient Packaging Process - RUBY 628
Mike McQuaid is the CTO and cofounder at Workbrew. They dive into the world of Homebrew, an open-source package manager for macOS and Linux. They explore the history and development of Homebrew, from its origins in the Ruby community to its evolution into a widely-used tool for installing and managing software.
Hosted by:
Charles Max Wood
Special Guests:
Mike McQuaid
Show Notes
Mike McQuaid is the CTO and cofounder at Workbrew. They dive into the world of Homebrew, an open-source package manager for macOS and Linux. They explore the history and development of Homebrew, from its origins in the Ruby community to its evolution into a widely-used tool for installing and managing software.
The conversation delves into the intricacies of building and maintaining packages, the introduction of binary packages and a new JSON API, and the creation of Workbrew, a company focused on commercializing features for Homebrew. They also touch on the latest developments in Ruby, the differences between Homebrew Cask and Homebrew Core, and the complexities of handling a large number of packages in Homebrew. Join them for an insightful and engaging discussion on all things Homebrew and software development.
Sponsors
Links
Socials
- mikemcquaid.com
- LinkedIn: Mike McQuaid
Picks
- Charles - Fire Tower
- Mike - Machine Vendetta
- Mike - Halestorm
Transcript
CHARLES: Hey, welcome back to another episode of the Ruby Rogues podcast. This week I'm talking to Mike McQuade. Mike is one of the maintainers of Homebrew. Uh, if you have a Mac, you know, and if you don't have a Mac, then we'll, we'll get into what it is, but yeah, cool stuff. Uh, calling in from Scotland. Is that what you said?
MIKE: Yep. Edinburgh and Scotland.
CHARLES: Very cool. I have some Scottish in me. I have a whole bunch of other European countries in me too. But anyway, do you want to just jump in and let us know what else people ought to know about you before we get rolling?
Hey folks, this is Charles Maxwood. I've been talking to a whole bunch of people that want to update their resume and find a better job. And I figure, well, why not just share my resume? So if you go to topendevs.com slash resume, enter your name and email address then you'll get a copy of the resume that I use, that I've used through freelancing, through most of my career, as I've kind of refined it and tweaked it to get me the jobs that I want. Like I said, topendevs.com slash resume will get you that. And you can just kind of use the formatting. It comes in Word and Pages formats, and you can just fill it in from there.
MIKE: Sure. So yeah, as Charles nicely said, I'm Mike McQuade. He said my name perfectly, which is not always something you can rely on, which is very nice. So I'm currently working on Humbrew in my volunteer life, spare time sort of stuff. Worked on it for 15 years. Since 2009, I got involved. I think I was the third person to get a commit bill on the project.
CHARLES: Oh, wow.
MIKE: I'm still standing after that time. And I'm like the project leader, which basically means I'm sort of an elected like figurehead slash tech lead, et cetera. Um, so yeah, so in my kind of outside of my homebrew life, work wise, um, I'm working right now, started a company last year with some ex GitHub people called workbrew where we're working on commercializing some stuff around homebrew and basically building the features that kind of more medium large companies need that homebrew does not have so that they can work in the way they want to work while Hubroot plays nicely with that. And then before I was working here, I worked at GitHub for 10 years. I left as like a principal engineer last year and I was bounced all over the place there, but particularly I guess focused on like developer experience stuff and open source and scratching my own itches basically.
CHARLES: Very cool. Yeah, I've known a few people who've worked at GitHub. I also remember way back in the day when Chris and Tom put their heads together and started the thing as kind of a project in a restaurant or something. Yeah, it's kind of funny to look at where it all wound up with Microsoft buying them and everything else. Let's just kind of start with the fundamentals here. I did kind of mention that if you have a Mac, you probably know what Homebrew is, but do you want to give people a rundown of what it is?
MIKE: Sure, so Homebrew is a open source package manager for Mac OS primarily and also increasingly Linux as well, interestingly.
CHARLES: Oh, really?
MIKE: Could talk a little bit more about that layer, because that's sometimes confusing to particularly Mac people. But yeah, so basically, if you write Brew, which is the kind of shorthand, brew install, and then pretty much any application you would want, and you can install that on your Mac. So I believe we have some degree of evidence that about 30 million people and or bots make use of Homebrew and it's, yeah, in our opinion, like the best way of installing open source software on your Mac. Well, you may not know as well with Homebrew, it started off just being like building everything from source package manager for like open source tools, W get, curl, whatever. Whereas nowadays you can also install like upstream binary. So like say like Chrome, which I, I'm a diehard Safari guy. So I'm streaming in Chrome today cause I had my usual sadness of learning the website didn't support Safari. So I fired up Chrome, but I, I installed Chrome by just having Brew install Google Chrome, and then it sticks everything in my applications folder, sets it all up and all that type of things. And there's a long tail of like ecosystems where you can have like. Brew bundle is another favorite of mine where you can have like declarative ways of telling homebrew what to do and all this type of stuff. So yeah, that's basically, and my, uh, my non-tech explanation of homebrew is like a app store for open source in your terminal.
CHARLES: Right. Yeah, you just brew install whatever you want and off it runs and does its thing. Um, in fact, a lot of tutorials, if you need a particular piece of the puzzle, a lot of the gems and things that I use in Ruby, it'll, it'll basically say, go brew, install your blah, blah, blah. Right. And then, and then you can run from there. So yeah, it's definitely something that I use and I reach for, and I don't even think anymore that, oh, gee, somebody actually made this, right. It's just part of life. So anyway, so yeah, so I'm kind of curious as we kind of get into this a little bit, and there's a lot to talk about, right? Because I do want to get into work brew and what that's all about. But before we do that, for those of us who have used Homebrew for a long time and are excited to see, you know, maybe where it came from and what all is entailed in it, where did it come from? You said you weren't in it from necessarily the beginning, but you got to commit pretty early.
MIKE: So yeah,
CHARLES: what's the story there?
MIKE: I think I got involved about, you know, maybe started using it four or five months and started contributing pretty short after when it was first created. It was made in 2009 by a guy called Max Howell, uh, who was in London. I think he was working at LastFM at the time. It's a blast from the past that probably still exists, but not as widely used as it once was. Um, so he,
CHARLES: I have kids younger than homebrew.
MIKE: Yeah, I'm sure. Yeah, I also have kids younger than homebrews. And it's distressing when you meet kids who are the same age as homebrew and how articulate they are. So he basically had played around with all the package managers on the Mac at the time. There was Mac ports and Fink and...
CHARLES: Oh yeah, I remember those. Yeah.
MIKE: Come on, what the other one was.
CHARLES: These Mac ports. But yeah,
MIKE: yeah, Magports is probably the dominant one. And he didn't have a huge amount of like good time using Magports felt that it kind of did too much and he didn't like the approach. So he built Hubrew, which was this kind of beer themed package manager. And yeah,
CHARLES: and the other interesting model is still called casks.
MIKE: Yeah, the packages are still called like formula and casks and it's still got all the beer themes all over the place. But one of the interesting things he did as well is it was, you know, this was i can't remember, you probably know Charles actually, what your GitHub opened to the public, but certainly this was in 2009. So GitHub was like relatively new. I'm pretty sure when Homebrew first got created, the pull request didn't exist yet. But Max from the outset had decided like, hey, I wanna make this thing, but I can't maintain thousands of packages all by myself. So I'm gonna have to kind of open this up and make it a community effort to do that. So yeah, so that's basically what happened and more and more people kind of particularly in the Ruby community. Like I think the Ruby community is one of the early adopters of Homebrew because Homebrew itself was written in and it still is primarily within Ruby. And yeah, basically just it kind of grew. More people got involved, more people contributed. And then, you know, people who contributed lots and ended up doing a bunch of reviews and stuff like that. I guess folks like me ended up kind of becoming maintainers and then the projects just sort of grown and grown over time. So yeah, it went from originally like just max in 2009 to nowadays we've got, I think 30 something maintainers, we probably had like 50 total all time. And then we've had like over 10,000 people contribute to some part of the project, at least once.
CHARLES: Very cool. So. so many questions. So one of them is there are kind of a bunch of different pieces to this, right? You've got the command line interface and then you've got the, hey go grab the thing and how do I know how to build the thing and how do I know where to put the thing and how to, right? And so let's just start with the command line interface. So what tools did you start out working with and has that changed over the last what?
MIKE: years. Yeah, yeah. So homebrew was written originally exclusively in Ruby pretty much. And it's still primarily Ruby. Like the irony in homebrew land compared to maybe other bits of Ruby, where because we're like a CLI and because we're, we are installing a bunch of stuff related to compilers and things, but we can't rely on necessarily being there. And we haven't wanted to ship native binaries for the application itself. When we want to make things faster in homebrew, often they are ported from Ruby to bash. So an increasing amount of,
CHARLES: wow
MIKE: they're kind of very performance intensive bits of homebrew, like the bits where like, you want to be able to run a command and have a response instantly, or you want to be able to have a command that you can run every time you open your shell and it not have too much overhead. So some of that, more of that stuff is kind of ported to like bash and like the updaters in bash and because Homebrew, it still has its own Ruby nowadays because we originally used just whatever Ruby shipped with macOS and that worked all right for a pretty long time. But I think last year we were speaking to our Apple contacts and they theoretically deprecated Ruby, I think a few years ago in the system. And we said, hey, can we get a newer one?
CHARLES: I think it's still there.
MIKE: It's still there, but it's a 2.6. Yeah,
CHARLES: I was going to say it's not a current version.
MIKE: Yeah, pretty ancient. And so, yeah. So we ended up... Mac OS versions and for Linux anyway. So we just like, well, we can roll around for everyone now and just make it mandatory.
CHARLES: Right.
MIKE: And I think it's Ruby 3.1 or two or something like that nowadays. And yeah, and that's a much nicer experience for us and makes everything a bit faster and snappier and stuff. So yeah, so the main package manager is all in Ruby. And then we have, I guess, two main repositories. So there's like homebrew brew, we would call it, which is like yallup.com slash homebrew slash brew. That's the package manager. And that's, like, that's my personal like favorite happy place to live. And I kind of reviewed pretty much every PR that is in that repo. And then there's like homebrew slash homebrew core. And so that's like the core formula, like basically everything in there has to be open source and there's like 6,000 packages or whatever in there. And that again is I think one of the reasons why Ruby was such a good pick originally like is that the, like the files that kind of describe how to install something are known as formula and again, going down the beer theme and that it's like a really nice Ruby DSL that kind of is, I would say fairly, if you're, if you kind of have a basic understanding of almost like, you know, configure make, make install, like how Unix programs are generally built. Like it's, you can read through that and it's pretty obvious how to get started. And if you've ever had the misfortune of having to try and package like RPMs or like Debian packages or whatever, which are very, very robust, arguably better than Homebrew once they're there. But like the actual just process of going from nothing to something that works is very time consuming in my experience. Whereas in Homebrew, you can just, you know, like make a fairly short little thing and then have yourself up and running, installing applications in a pretty quick time. And I think that's why it's grown and stayed big because it's quite easy for people to contribute and modify and change things. And it's just, you know, you can do it all on GitHub nowadays.
CHARLES: Right. So when I was in college, I was a systems administrator on mostly Red Hat systems. And so I've written some fugly bash scripts to do all kinds of things. And I've also done RPM packaging. And, you know, later on, I, yeah, there are a few things that I've had to fiddle with and maybe put into a Debian package. And yeah, that is not a fun process and then you just kind of have to put it out there and then pull it in with your package manager or with some utility that can pull it in and unpack it and do whatever it does. And oh man. Yeah. Homebrew is pretty nice that way. I've also done plenty of stuff with make, which is it's a powerful utility, but it's a little arcane sometimes. And so, yeah, I definitely feel you there. So. Yeah, so it's all written in Ruby. Is it pretty, is it pretty simple or are there a lot of edge cases you have to handle or is that mostly in the packages or the formula or whatever?
MIKE: Yeah, a bit of both. Like, I mean, there's, I think with, with stuff like homebrew is like a really long tale, so I guess with the stuff that Max created in the first place, um, you know, I can't remember how many packages he almost like shipped with to begin with, but, um, it's one of those things where If you have a hundred packages, everything seems pretty simple. If you have a thousand things start to get a bit more complex and then you start getting up to like five, six thousand. It's like, it gets really messy. All the weird and wonderful things that things do. And as a packager of software, you have this kind of weird decision framework where there's often the way the, the people who write their software originally, I guess we would refer to them as upstream and the way they want to release their software is not necessarily the way your users want to consume the software or the way you want to be keeping up to date and stuff. So you're trying to sort of balance these things. And yeah, I mean, the thing I enjoy about it personally is like, I've always been a bit of a kind of generalist software ride. So like with Ruby, particularly when I was doing a lot more of the packaging stuff than I do probably today, you know, in a given week, you could be writing Ruby, C C++, Haskell, like having to play around with like 10, 20 different languages and frameworks, and you kind of develop a sense of how this stuff plays nicely together and what the edge cases are and what you need to watch out for and everything like that. But yeah, I mean, software is a wily beast and packaging it's no less so.
CHARLES: Interesting. So yeah, so what are your dependencies? I know that you have to have the Xcode build tools installed does it pull in the rest of them for me?
MIKE: Yeah, basically, on a Mac, that's all you need. You just need to have the Xcode utilities installed. And then even then, we've tried in the past sometimes to make them not required. But it's pretty hard, given the state of things in Apple land, to go completely without that. And every few years, Apple introduces some new requirement, which requires using a CLI provide in this new open source version and we're kind of back to square one there. So yeah, basically just the Xcode command line tools are the only dependency. And then if you're running Hublone Linux, there's a few other bits and pieces there. But the big one I mentioned before, like Ruby, like we, we build our own kind of statically compiled Ruby, which you can just use anywhere that's auto installed for you and everything.
CHARLES: So I guess, uh, some of the, uh, something else that I'm wondering is are you taking advantage of some of the newer features in Ruby? So for example, you've got YJIT, right? That speeds things up quite a bit. Or the other one is like a fiber scheduler and stuff like that, which gives you some level of concurrency. Or are you digging into any of that? Or is that kind of upcoming when you get to it?
MIKE: Yeah. So we've, we've played around with like bits and pieces around there. Like, so it's, it's funny because homebrew is very much, you know, that there's been this kind of meme in the Ruby community for awhile, that like people get sad that like Ruby is associated almost entirely with Rails most of the time. And like homebrew is pretty much as far away from that as you can get. We're not, we're not even a web app. We're not attached to a database. Like you're running a CLI. So it's funny. It's, um, I guess a lot of the places from where the JITs come from, like it never say never, like it may well make things faster in the future, but right now all of the kind of JITs, different JITs we've tried and the different JIT sessions we've tried like just the way homebrew runs, even if it's doing a relatively complex task, like a, the run times are relatively short. The process only sticks around for, you know, at most probably a few minutes. Um, but then a lot of the time it's just around for a second and there's new data every time, so you can't necessarily cash things super hard. Um, and like basically everywhere where homebrew is slow and like, that's the most common complaint about homebrew is that homebrew is slow and the almost everywhere it's slow, it's slow because of the like IO, basically, like we're either downloading something off the internet, we're writing something to your hard drive or reading something from your hard drive, whatever. And we can maybe do a little bit more parallelization, but again, like in the past, uh, there's been times where I've, and again, some of this may be changing from splitting platters to SSDs or whatever. There's been times where I've ripped out parallelized code that we have in homebrew and it's actually dramatically faster not necessarily that it is parallel. So yeah, we definitely have a decent amount of profiling tools, like we rely on stack, but basically everything you would expect in RubyLand, StackProf, CoProf, all these types of things. But yeah, but we do try and make things as fast as we can and the Jits don't seem to really be helping much for us at the moment, but as I say, maybe in the future.
CHARLES: Yep. So, um, I'm going to move on a little bit to you mentioned homebrew core. So is that the list of casks and formula? So homebrew core is the list of formulas.
MIKE: So we have homebrew cask and homebrew core. So the difference between casks and formula are like homebrew cask used to be at certain like separate project. And that's the way that's what you use for installing like binaries from other people. So like, like Google Chrome or one password or whatever. Like if you're installing that, basically if, if the project is, whether it's open source or not, if the main way that that's delivered is like a pre compiled thing or like a dot app or DMG or dot PGA or whatever, then you're installing that with RoomGruc Cask. Like the essentially to the user, the interface is more or less the same nowadays, like you type brew install Google Chrome or brew install W get. And it's, it looks essentially transparent under the hoods, but yeah, they're, they're managed in separate repos like this was previously like a requirement due to some open source political nonsense. Um, but yeah, so everything in homebrew core is open source software and everything. And homebrew cost has a much looser, uh, requirement basically there. Um, so yeah, they're managed separately. They have roughly the same amount of packages, but then like, again, the, the DSL that's used are kind of slightly different and homebrew cost is a bit more friendly to like proprietary software for GUIs, whatever. And then homebrew core is a bit more friendly to like CLIs and is open source software.
CHARLES: I gotcha. So yeah. And see, I didn't even know you could install some of those binaries with home brew because I always just go to the website and download it and run the installer and right.
MIKE: Yeah.
CHARLES: Or open the DMG and drag it to applications or whatever. So that that's really interesting. Um, as you get into some of the open source stuff in particular, how do you know how to build it? Are you always looking for the make file or is there something else to it? Or..
MIKE: so sometimes there's instructions in like a GitHub repo or whatever, or sometimes it's kind of, you know, it's got a make file and sometimes you just kind of try it out and figure it out and hopefully you can do the right thing. And then two years later, the, the author pops up and they say, Hey, you can build this totally wrong. And it's a, well, you didn't tell us what to do, dude. So like, that's, that's what's going to happen. So yeah, it's, it's always a. Like there's definitely like a certain amount of kind of trends and traits of things that behave pretty similarly. And those projects are very nice to deal with. And then those projects who decide, hey, I'm going to invent my own build system because all the other ones suck. And yeah, those are less fun to deal with.
CHARLES: Right. So essentially, if I do a brew install on something like Google Chrome, it's just going to pull the binary off of wherever and it's going to put it wherever it goes and make sure it's in my path or whatever, whatever it has to do in order for me to be able to run it.
MIKE: Yeah, exactly.
CHARLES: And then for a formula, it's going to go through some kind of build process. And it sounds like you don't like auto detect the build script. You go and you look it up and you add that to the formula and tell it how to build it.
MIKE: Yeah. Exactly. So what we, we try and do like certain bits of auto detection for an end user for the, you know, if you're running Brew and so on your machine, uh, the experience is actually kind of similar because you're, you're still downloading a binary most of the time. If you installed from Brew on like a supported macOS version, which is one of the last three and the kind of default location,
CHARLES: right.
MIKE: Then what it will do is essentially when someone submits a formula or an update to a formula, we have our pull requests on GitHub our continuous integration, we'll go and build that. And then when that works, we then have a process that kind of that build artifact gets uploaded. We call those like bottles, the binary packages. And that was my probably favorite contribution to homebrew is doing that just for the amount of time it's probably saved the world. Because before that, you know, if you wanted to install W get, it would take five minutes and everyone would essentially spend five minutes with their CPU fans in the days before Apple Silicon, like going 100% to spit out essentially the same binary at the end. So
CHARLES: yeah
MIKE: now we kind of have these binary packages, and those are downloaded on your machine and installed, and it's all nice and quick. And also, it's way less error prone, because with the compilation stuff, there's always things where you have some random utility on your machine, and the build script detects that, and then blows up or whatever. So you don't have those same issues with the binaries.
Hey there, this is Charles Maxwood. I'm excited because I wanted to let you know about this thing that I pulled together that I had just, I've been dying to have this for years and I never felt like I could, and then I just realized that there's no reason why I can't. So I'm putting together a book club and we're going to read development, focused books, career books, you know, technical books, whatever. The first book that we're going to do is going to be Clean Architecture by Uncle Bob Martin. If you're not familiar with Clean Code or some of the other stuff that Bob has done check that out. I've also talked to him on the Clean Coders podcast, which is on Top End Devs. But yeah, we're gonna get on. He's gonna show up to some of our meetings. And what I'm thinking is we'll probably have like five or six people part of the conversation along with Bob and I at the same time. And we'll just, so somebody can come on, they can ask their question, and then we'll just rotate people through. So we'll mute one person, unmute another person when it's their turn to come on and be part of the discussion. So.. We'll do that for like an hour, hour and a half. And then the other part of it that I'm putting together is just kind of a meet and greet gather area on Gather Town. And so after the, the meetup and the call, what we'll do is we'll all go over to Gather Town and you can just log in, walk up to a group and have a conversation. And that way we can all kind of get to know each other and, and make friends and, and get to know people across the world. Uh, one thing that I'm finding is that, yeah, the meetups are starting to come back but a lot of people don't have the opportunity to go to a meetup. And I really want to meet you guys and talk to you. So we're going to put all that together. It'll all be part of that book club. You can go to topendevs.com slash book club to be part of it. And I'm looking forward to seeing you there. The first book club meeting will be in December, the beginning of December. We're starting the first week of December. And, um, you'll also be part of the conversation about which book we do next. I have one in mind, but I want to see where everybody's at. So there you go.
CHARLES: Right. So it looks at my setup and says, Oh, I know that this bottle's compatible. So I'm just going to give you the binary instead of making you build it.
MIKE: Totally.
CHARLES: And then if it can't for whatever reason, then it'll do the build step or the build process.
MIKE: Yeah. And I mean, that, that essentially should never happen on, you know, a supported market and a supported location. But yeah, if you're outside of that, then it will fall back to kind of building from source when it can't do that otherwise. But yeah, but most things most of the time will always be you get like a binary package for most users. So,
CHARLES: right. So, just backing up a little bit because you mentioned homebrew core and homebrew casks is kind of the places where these things live. Is homebrew literally going to GitHub and cloning it or like how does it get that list?
MIKE: Yeah, originally, well until very recently, yes, we did that
CHARLES: to get a little wild.
MIKE: Yeah. So the original update or for homebrew was just, it had one repo which was like an MXCL, which is Max Howell's GitHub, MXCL slash homebrew. And that would just be cloned in a directory. And when you ran brew update to like, you know, pull the latest packages, that was just doing a Git pull basically. So over, over time that became more and more sophisticated. As homebrew grew, that became slower and slower, like even the operation of just doing a git fetch on that repo and it's telling you, you don't need anything. That was taking like a couple of seconds. At the time I worked at GitHub and I was getting annoyed with how slow it was. So I actually added a entry to the GitHub API to specifically to make homebrews updates faster. And like it ended up being used by a bunch of other cases and package managers or whatever. It's actually what you could do is you could just like pass the, the hash that you have what's it called, the HTTP ETAG. So you could pass the git commit hash as the ETAG, and then you would get a whatever, I can't call it, the status code that says not modified essentially. So it would do that, and it also wouldn't count against your GitHub rate limit, because that was very heavily cached, that operation. So that basically meant you could get a really, really fast response for GitHub repos, and we still rely on that API to this day. And yeah, so until very recently, that was still how it was done, and we would fetch stuff or whatever. But over time, that was getting slower and slower. And also, GitHub were pretty nice about it all. But I think at some point, I believe there was a dedicated bare metal server that was essentially just 24-7 serving homebrew core to people. Because also, the way that that repo had super long history, loads of contributors didn't. I did a lot of like squash and rebasing. So like not very kind, like history management. And again, like thousands of files in a folder, which we never knew when Humbrew was created is extreme. If you have very deep histories, get gets very angry and the performance goes to bad places pretty quick. Um, so yeah, so as of, I think it was a year and a half, maybe two years ago. Uh, like someone, one of the homebrew maintainers kind of started a first attempt at like a little project to see about, okay, well, we can, we, you can effectively turn pretty much all the homebrew packages into like JSON output anyway.
CHARLES: Right.
MIKE: And when you're just downloading binary packages, you're basically just downloading a thing off the internet and sticking it in, like installing it that way. Anyway, you don't need all the kind of sophisticated build information for every package. So yeah. So as of, yeah, like about a year ago, I think we've migrated entirely to what we call our kind of JSON API, which is downloading just a single JSON file, which provides all the details for those packages. And then you just have that single download, you don't need to have that Git repository on your machine anymore, which saves a bunch of space. And it's like a lot quicker than downloading. And yeah, the implementation of that API is another disgusting thing where so perhaps because I'm Scottish, we are known for such things or just, I don't know. I don't like paying for things and in open source projects,
CHARLES: I think that's universal
MIKE: even less. So our, our JSON API is actually on GitHub pages. And it's, yeah.
CHARLES: Oh cool
MIKE: So it's all like, it's a statically generated API essentially. So like every 15 minutes we like regenerate our packages. And what that means is that it's really, really, really fast and essentially, you know, nigh infinitely scalable for GitHub to stick a CDN in front of that and host it that way and it's also pretty easy for us to kind of download and debug and we're not maintaining some like complex, um, like, you know, when I worked on the API at GitHub, like maintaining the performance on that thing, when you were at the scales we were at was not an easy feat and yeah, and I, I, I saw that I guess working on Brew and I was like, nope, I'm gonna, if I can make this just entirely static, unauthenticated API, then let's do that. So that's the route we went down and it's been working pretty well for us.
CHARLES: That's awesome. I love that. It also makes me think a little bit that sometimes I might be over complicating my life, but trying to make it work the other way, right? The more traditional I'm going to build, I'm going to build an endpoint on rails and it's going to give all this stuff and it's going to check all these things. Instead of just saying, Hey, it doesn't matter if everybody knows this and it'll be way faster and it's not that big a file. So yeah, just go get it.
MIKE: It's funny because I had this idea for over a decade about building essentially static APIs. Because you know, like static site, I've used like Jekyll and GitHub pages for my homepage for a long time. I kind of like that sort of static site builder model and the results are nice and super fast and it's really cheap and easy to host and very non-error prone. Once you've generated the files, you're like, well, nothing can go wrong now compared to like maintaining a Rails app. But yeah, so I always kind of liked the idea of having a self-gap BI. I was pleased that I was actually finally able to do it because in cases where essentially you want to deliver the data to absolutely everyone, the data is the same. It's not like user dependent, whatever. And the data only needs updated, like not very often. Like, you're not updating like second by second from some database, but you're updating it like, you know, a few times an hour. Then, yeah, like this approach has actually been really quite cool and fun. And yeah, I'd encourage others to play out. Uh, play around with the approach and maybe not on get on pages. Cause get on my gang root.
CHARLES: Well, I can imagine though, cause I'm just thinking through maybe what I want to do with building an app for the podcast network, right?
MIKE: Yep.
CHARLES: And yeah, like 99% of the stuff that I'm putting out there, it's going to be the same for everybody and it's all public and right. So I don't have to protect it at all. And so yeah, why not just have a static file out there that it can just go, oh, you want the list of podcasts here, right? Oh, you want this podcast and all of its episodes here, right? Now, granted, some of the podcasts have more than 500 episodes at this point. And so you might get a little longer JSON file, but still it's not terribly large. And. You know, you start getting into it really, when you start pulling down images or audio files, but you know, I can put those in as URLs on the JSON and just make it really wicked fast and then it's, okay, now I have restricted access for memberships and stuff like that. That's where I can, you know, maybe put a check in front of it, but still then just have them pick up a statically generated thing because once you're authenticated, that's all the same too. So, I'm really digging this.
MIKE: Yeah, it's, it's been fun. I think, I don't know that this is the thing I like about working with open source and stuff like Herbrew is that, you know, you get, I guess the thing I've learned in my career is the most interesting projects are the ones often with any sort of constraints, right? Like the projects would like, you know, take as long as you want, you have as many people as you want, do it whatever you want in whatever language you want. Like they're they're often like, maybe I'm just a masochist, but I find them often like less fun. Whereas the fun thing was something like homebrew as well as like you're building homebrews API. I said, well, we, we have like, I guess the way I describe homebrews financial situations, like not enough to pay anyone, but too much for stickers, right? So you have like this like middle ground of money. And as a result, like it's this tricky thing of like, well, I don't really want to put us in a situation where we're paying vast hosting bills. So let's try and figure out a way of doing this for free essentially, but it has to be able to scale to like, you know, millions, if not tens of millions of people like using this thing. So yeah, so like that's been a fun kind of comparison compared to, as I say, like GitHub days where you're scaling to many more people than that. But you also have, particularly post-Microsoft acquisition, like many more resources at your disposal in order to achieve those scales. And you're not worrying quite as much about the bottom line as you go along.
CHARLES: Yeah. The other thing that I just went through my head was... And this isn't JSON, it's XML, but RSS feeds.
MIKE: Oh, yeah.
CHARLES: Yeah.
CHARLES: Yep. So yeah. Um, all right. So we've kind of talked through how the repository works. And, um, let's say that I decided I wanted to build the CLI and it was moderately complex. Are there, are there things that you recommend to me as a Ruby developer as places to get started?
MIKE: Um, I mean, the funny thing is I, I wasn't, you know, I realized this, uh, maybe. Um, I don't know, a controversial take for the podcast, but like, I mean, nowadays I probably wouldn't necessarily reach for Ruby for a CLI. Like it's, it's okay for like doing, doing, you know, I still think it's great for doing web apps, but like, you know, I've, I've been building like a new CLI for some of the stuff we've been doing with Workbrew. And like, Go feels like a more, I was basically weighing up between like Go, Rust and Swift basically.
CHARLES: Oh, okay.
MIKE: the, the options.
CHARLES: I wouldn't have Peg Swift for that.
MIKE: Well, I, yeah, I didn't peg Swift in the end for various reasons, but you know, I ended up going with Go, which I, it's, I don't know, like for me, Go is just, it's good at what it does. It's such a boring language. I don't have any fun when I'm writing Go, but you can ship a single static binary.
CHARLES: Oh, it's wicked fast.
MIKE: Yeah. And it's super fast. And for CLIs, like it's like the responsiveness is really great. I think.. I think that's the thing that maybe I'm just burned for like 15 years of working on a Ruby CLI, but like, there's a certain amount of people will complain about the slowness that it's like literally even like a Ruby interpreter spinning up and saying hello world is like unacceptably slow to some people. Right. And at that level, I can't really do anything to make it it faster. But I guess if you are going down the route of almost like really wanting to get into a root, make it some sort of Ruby CLI or whatever, I guess I would just say like, You know, the beauty of Ruby is that you can start with like a single script,
CHARLES: right
MIKE: You can just have a script, stick the shebang at the top. Like the, for those who don't know, that's the thing where it has, um, like exclamation mark hash, like user bin, I don't know, like user bin Ruby or whatever. And then you can just start writing Ruby code straight away. Right. And, and like, that's, that's how I think is the best way to maybe get started with like CLIs or whatever. It's just like rather than almost like reaching for some option parsing framework or whatever, no, just go and just start writing some Ruby and just read the argv, which is the array of like the arguments you pass into the script and like just read that yourself and just, you know, don't, don't go too fast too quickly and you'll probably have like a good time. Because again, like when you CLIs, if you wanted to distribute them to other people, like if they, you know, you start relying on loads of other gems and stuff, then it becomes a little bit more of a headache to kind of distribute and use these things and keep them up to date and whatever. So like, yeah, I think the beauty of Ruby is like Ruby has like a pretty decent broad standard library. You could do a lot with like a single script and even like an old version of Ruby, like the ones that ship with like Neck OS, like there's still, you know, it's still a powerful, like pleasant language to work in. So,
CHARLES: yeah. Very cool. So yeah, so is there anything new coming in Homebrew or?
MIKE: Yeah, I mean, we're always working on, it's funny because people quite, people ask periodically about like the roadmap for Homebrew and because we're essentially, I mean, maybe except for me now, like essentially almost everyone is like a volunteer. The roadmap for Homebrew is kind of like, well, let's see what people want to work on. Whatever people want to work on, right? And it's like the product management on Homebrew kind of looks more like Nurds attempting to nerds like other maintainers into solving problems that it does like, you know, building a team and assigning them a six month project.
CHARLES: Right.
MIKE: So yeah, I mean, I think like a big thing is like performance, like we're continually trying to kind of eke out better performance where we can. And then yeah, and I guess like I'm looking at it from the kind of workbrew end as well of being like, okay, well, so if, if me and a team of people are actually able to work on homebrew full time or like spend like a decent amount of time putting into homebrew. Like what are the really big impactful things in homebrew that kind of might, people might look at and say, that's just impossible that we could do if we have like a paid team of people who are working on this stuff full time. And I don't want to, you know, I don't want to be too spoilery with that, but yeah, there's definitely, there's definitely some stuff we're exploring where if we see that there's interest and we can kind of pull it off, I think people will really say wow about, but in the short to medium term, I think there's more along the lines of, yeah, just like making homebrew faster, more secure, making the contribution process better, and just trying to do everything we do right now, like better, basically. Like, I mean, I'm a big fan at GitHub and on homebrew of like, when I was at GitHub, that was like reading really, really heavily into automation. So again, like a lot of what we're trying to do on homebrew sometimes is like, we've only got 30 people maintaining the.. the project, but yeah, the throughput of like PRs we have and it's fairly ridiculous. And we just try and have so much automation lean more and more and more heavily into that. And anytime, you know, I catch people doing things that I'm like, a bot could do this, then it's like, right, I try and write some code and get the humans doing the human things and save the bot work for the bots.
CHARLES: So I was gonna ask about Workbrew and I'm gonna ask about it in a second, but you did mention.
MIKE: Yeah.
CHARLES: maintainers. And so if somebody is like, Oh, I've got a great idea for homebrew or, um, you know, I use this all the time and I'd like to contribute back to it. Right. Is there a good way for people to get started with contributing?
MIKE: Yeah. So homebrew, the best way to get started is there's like a contribution section on the readme of the homebrew brew. So if you go there, then you'll, uh, or if we even just search for like contributing to homebrew, like the result, like results are pretty decent. I would say our docs are probably like on the better end for like contribution, like our tooling is probably pretty good. But like, I would say like, it's a really easy project to get started with. And we, we really try to kind of cheerly the people who want to get involved with the project. So like, if you're in any doubt, just try something and we'll help you out. And you can mention me by my GitHub handle or whatever, if you say that it's your first PR and you're struggling and I'll try and do what I can to help you and stuff like that. Um, it's a, yeah, I, as I say, I think it's a really good project to kind of get started with. Um, but as you know, for some of us, at least it's also can be, uh, fairly addictive at times, so be wary from that perspective as well. Yep.
CHARLES: All right. So the other question I have then is, yeah, do you want to talk about work brew, like what it is and where you're heading with that?
MIKE: Yeah. So, I mean, as I mentioned earlier with like homebrew, like I've been working on homebrew for about 15 years and for, you know, probably 10 of those years, there's been people who have been encouraging me at various points to, oh, you know, you should make a company doing stuff around homebrew. And I have been historically kind of down on this idea or fairly cynical, particularly when, I mean, not going to name any names, but you know, there's been various companies who kind of have done open source stuff. And then, you know, they complain down the line when everyone uses their software, but they're not making enough money or whatever it may be. And I get... I guess to me it's like open source itself is not a business model. I know. So like I, you know, I kept, like you mentioned earlier, like you've got, you know, kids younger than homebrew and stuff. Like I also feel like homebrew is kind of my first child and I feel quite precious of it and I wouldn't want to, you know, like destroy it or whatever. So it's, it, yeah, it took a while for me to get to a place that I was like, yeah, actually it feels like a few things have come together. So one is like homebrew feels, I mean, even with the roadmap stuff, like we'd had this conversation a year ago or two years ago, I probably would have said like more stuff on the horizon, but it feels like fairly close to kind of feature complete at this point, and without almost like a big injection of kind of effort and stuff. So that was one part. The other part is like, we've got, I won't go into all the boring details of like open source governance, but you know, but we have an open collective. We have a, like a leadership committee, we have elections every year. We have like ways of getting involved with the project beyond just like current docs and stuff like that. We have money that we're using to like provide stipends to some maintainers and pay for them to kind of come and meet each other in person and stuff. So the project feels like it's in a really good, mature place and it's also in a place where no one party could ever kind of completely take over homebrew even if they wanted to. And so yeah, so, and it's also in the backdrop of this, I guess, like, you know, I left GitHub last year and was sort of looking for a new challenge and something new and you know, again, over the last 10 years, there's been various times where companies come and say, Hey, we'd like, you know, homebrew to do this or that or whatever. And essentially what a lot of homebrew features come down to is like, do the volunteers who are mainly working in their spare time want to build and support this?
CHARLES: Right.
MIKE: And for a lot of like big enterprise stuff, I don't know, Charles, have you ever had the misfortune of reading through like a sample spec or something like that before? But you know, once.. Once you get into this big enterprise tech, it's often not very fun, and it's not what people want to spend their evenings and weekends doing. And I previously worked before my home-room days in an open source company, back when I was a KDE contributor, and this company, a consultant company, and they hired more KDE contributors than anyone else. And I worked and I thought, sweet, I'm going to be able to work on open source most of the time. But I learned pretty quickly that their open source projects were actually not super fun because they were basically stuff that people would pay a consultant company to go and like, hey, we've got this stuff which we can't get anyone to do for fun. So will you do this for money? And then they're like, okay, well, fine. And so with Workbrew, I kind of see it as being a similar thing where what we're trying to do is, you know, there's a lot of talk about sustainability, open source and stuff right now. And I guess that's on my mind. I worked on the original team of people who built get up sponsors and have helped kind of homebrew have that open collective. And to me, like this is the next almost like step in the evolution of like what sustainability could look like for homebrew is having a commercial entity where we're trying to kind of make a sustainable like commercial entity who has an interest in making money in the homebrew ecosystem but not charging money directly for homebrew itself. Homebrew itself will raise and will always remain like free of charge, but we're going to make homebrew better by making workbrew better and be providing services to companies in the homebrew ecosystem through our expertise and by building stuff that tightly integrates with homebrew. So essentially, the version of Workbrew that I like run on my local machine today, it looks and behaves almost exactly the same as homebrew, except it's got a little coffee cup instead of a beer emoji. But it's running in a different way. It's running in a more isolated fashion, a multi-user fashion. It's reporting back to do fleet management so someone could upgrade like if I've got critical vulnerabilities remotely, and so that people can see kind of, oh, like you've, you haven't updated in six months and we've got some problems with that, whatever it may be. So yeah, so essentially that's what we've tried to do. We're trying to build something that provides a better experience for using Homebrew at work. But for most hobbyists, I mean, we very much have the goal that if you, in five or 10 years, if we're successful, even if you've never heard of Workbrew, even if you never interact with Workbrew, you will look at essentially, Roundabout now is being an inflection point where our hubbrew gets really, really good in the next five or 10 years and it gets noticeably better for everyone using it because we're able to kind of get involved and improve the sustainability of the project through the commercialization of this part of the ecosystem.
CHARLES: That's cool. If people want to know a little bit more about what's going on with that, like you have a mailing list or something, or you can pull up to date somehow.
MIKE: Yeah, the best way of getting in touch with us, we've got a contact form on our website. If you go to workbrew.com slash contact, then you can send us a message, and then we can go and get in touch, and we'll send you emails and stuff like that. And you can also see if you go to workbrew.com, our demo of what we have in private beta right now, which is essentially like fleet management for homebrew and kind of integrated with MDM tools. And so that basically like Mac admins and primarily, and, you know, security professional developer experience folks can basically get a nicer experience of using homebrew at work. And so, yeah, so we're recruiting like design partners for that right now, which essentially means like people who will give us lots of feedback while we make the beta kind of more to their requirements. So again, if you're interested in that best way is reaching out through the contact form. And if you have any thoughts, equally as well, if it's easier, if you're one of these people who gets spooked by contact forms, you can send me an email. My email address is pretty easily findable on the internet. So if you do that, or it's just Mike at workbrew.com or at brew.sh if you want my open source email.
CHARLES: Awesome. All right, well, if people wanna connect with you on some other level, where do they find you on the internet?
MIKE: So I'm still, I'm basically read only mainly on Twitter nowadays. I'm Mike McQuade on there, but the best place probably is I'm Mike McQuade at mastodon.social on like mastodon nowadays. Or if you just find my website at mikemcquade.com that's basically got, you know, my, I try and kind of put, if I write blog posts or like do talks and all my contact information is there and stuff as well. So yeah, and You'd be surprised how few people actually just email me to say hi or share their opinions. So feel free to do that for almost anything except for saying my home is broken. Please fix it. In which case,
CHARLES: right. All right. Good deal. Well, I'm going to go ahead and roll this into the picks here.
Have you ever wished that you had a group of people that were just as passionate about writing code as you are? I know I did. I did that for most of my career. I'd go to the meetups. I try and create other opportunities and it was just really hard. Right. The meetups, I got some of that, but they were only like once or twice a month. And it was just really hard to find that group of people that I connected with and really wanted to, you know, talk about code a lot. Right. I mean, I love writing code. I think it's the best. And so I've decided to create this community and create it a worldwide community that we can all jump in and do it. So we're going to have two workshops every week. One of those or two of those every month are going to be Q and A calls, right? Where you can get on, you can ask me. me and another expert questions. The rest of them are gonna be focused on different aspects of career or programming or things like that, right? So it'll go anywhere from like deployments and containers all the way up to managing your 401k and negotiating your benefits package. Well, we'll cover all of it, okay? And then we're also gonna have meetups every month for your particular technology area. So we have shows about JavaScript, React, Angular view and so on. We're gonna have meetups for all of those things. I'm gonna revive the freelancer show. We'll have one about that, right? So you can get started freelancing or continue freelancing if that's where you're at. And I'm working on finding authors who can actually do weekly video tutorials on some thing for 10 minutes. This related, again, to those technology areas so that you can stay current, keep growing. So if you're interested, go to topendevs.com slash sign up and you can get in right now for $39. When we're done, that price is going to go up to $75. And the $39 price gets you access to two calls per week. The full price at $150, which is gonna be $75 over the next few weeks, that price is gonna get you access to all of the calls and all of the tutorials and everything else that we put out. From Topendevs along with member pricing for our remote conferences that are coming up next year. So go check it out. Topendevs.com slash sign up.
CHARLES: Um, and I, we usually have the host go first. So I'll go first. Um, uh, one of the first picks I have, uh, I always pick a board game when I, uh, when I do my picks. And so I'm going to pick a board game called fire tower. And basically it's a, it was a relatively simple game. Let me look it up on board game geek real quick while I tell you how it's played. But, um, essentially all of the players have fire towers in the corners of the board. Uh, so what you're trying to do is you're trying to burn everybody else's tower down before the fire gets to and burns your tower down. And so, um, you can play two to four players. We played it with four people. It was a lot of fun. Uh, board game geek rates at a 1.77. Uh, I tell people that a two is kind of a reasonably complicated game. It's complicated to be fun, but not so complicated that it's hard to pick up. And so, yeah, this is right in there at that level playing time, it says 15 to 30 minutes. I think we played it 45, but it was our first time. Yeah, 10 plus player or 10 plus on age. I honestly think my eight year old could play this game. Right? It's pretty simple. And after she played it once or twice, she'd probably get most of the strategies figured out. Right? Don't have the wind blow toward your tower. Put.. put down the fire tokens in a way that gets it closer to the other people's towers. I mean, right, anyway. So, I'm gonna go ahead and pick that. It was a super fun game, really enjoyed it. One other one that I'm gonna pick is, we were talking about some of the install stuff and Daniel Kehoe, who used to be in the Ruby community, way back in the day, He has a website, mac.install.guide, and it walks through how to install Ruby and a bunch of other stuff, and it pretty heavily refers people to Homebrew. So anyway, I'm going to refer to that just as a resource that people can go check out. And then i had something else and I'm just kind of blanking on it. Anyway, where I talked to Daniel most recently was when I picked up the railscomposer.com domain from him. And so I'm working on that. You can actually watch me build Rails Composer and all of the different Rails engines that I plan on putting together for that on the Rails Clips video series. So just go to topendevs.com and you can find it. I think I also own railsclips.com and I think that goes to the right place. So go check that out. But anyway, yeah, good stuff. Mike, what do you want to pick on the show?
MIKE: Yeah. Okay. I guess I will do three little picks. So one would be, I'm reading a book right now by a guy called Alistair Reynolds and he's a science fiction author. He's written a lot of kind of space opera stuff. I've read all of his books that he's ever written so far and he released a new one a couple of months ago, which I'm 88% of the way through called Machine Vendetta that you Penny Arcade talked about it and you probably don't want to jump straight into Machine Vendetta as there's been a couple of books proceeding it in the series, unless you want to deal with the discussion of hyper pigs in the first paragraph, which may throw you off. My second recommendation would be a band, you may or may not be into their vibe, they're called Aylstorm, like a storm of beer. I guess you can see there's a theme going on. They are a Scottish pirate metal band. I'm seeing them live next Sunday in Scotland, but they play internationally. My kids really like a song from their latest album called Pirate Party, because it has a moment with a man drinking from a shoe which is apparently very funny when you're four and six years old. Uh, but generally I would not recommend most of their songs to children. Um, as they are very heavily expletive written in some of them, uh, but you, but you know, whatever they may work for you or not. And my final recommendation, I guess, on the theme of, um, open source maintenance in particular is therapy. Uh, I've been doing therapy intermittently since 2020 and I've something I've found tremendously helpful personally. Um, with all sorts of things in work, personal life, my relationships with my friends, family, kids, wife, and even on occasion, I have sessions where I talk about a particular difficult person I'm dealing with open source land in a given therapy session. So if you're interested, I would highly recommend it. And I've written a blog post on my website of like a step-by-step guide, if that's your way of doing things of like how to find a therapist. So if you are interested and that's useful to you. Please consider doing that.
CHARLES: Oh man, given the last few months that I've had, I might need that. Anyway, what was the name of the band again? And if you can put a link into the comments, that'd be great.
MIKE: Yeah, yeah. I will drop a link into the band, I will see.
CHARLES: But yeah, we'll go ahead and wrap up here. Um, one other thing I was just going to put out there, this is the thing I forgot is that, uh, my contract work has slowed down a bit. So if you're looking, I'm either looking for a full-time job or a contract, prefer contract, but you know, if you've got an interesting set of problems and you're going to pay me well to fix them, then I am happy to consider full-time work. Um, you can just email me Chuck at topendevs.com. All right. Thanks for coming, Mike. This was really, really interesting.
MIKE: Thanks for having me, Chuck. And have a good rest of your day, everyone.
CHARLES: Yeah, you too. Max out, everybody.
Homebrew Unleashed: Diving into the Fast and Efficient Packaging Process - RUBY 628
0:00
Playback Speed: