JAMES:
David, we’re going to do sound check. Sit down. [Fart noise]
JAMES:
[Chuckles] That works.
[Hosting and bandwidth provided by the Blue Box Group. Check them out at BlueBox.net.] [This podcast is sponsored by New Relic. To track and optimize your application performance, go to RubyRogues.com/NewRelic.]
[This episode is sponsored by JetBrains, makers of RubyMine. If you like having an IDE that provides great inline debugging tools, built-in version control, and intelligent code insight and refactorings, check out RubyMine by going to JetBrains.com/Ruby.]
[This episode is sponsored by the Ultimate DevOps Academy. They will be taking your complete Rails app through the entire automation step, complete configuration management, CI, autodeploy, autoscale, and at the end, you’ll have a top to bottom setup with all the tools for running a fully integrated production environment. The course has a live interactive webcast for instruction, but everything is recorded so you can work through it all at your own pace and save materials for later use. Plus, there will be lots of Q&A, help sessions, labs, et cetera to ensure success for everyone. Go sign up at UltimateDevOps.com]
CHUCK:
Hey everybody and welcome to Episode 116 of the Ruby Rogues Podcast. This week on our panel, we have James Edward Gray.
JAMES:
You will [die].
CHUCK:
Avdi Grimm.
AVDI:
Hello from Austin.
CHUCK:
Josh Susser.
JOSH:
It’s really warm here.
CHUCK:
Katrina Owen.
KATRINA:
Hello from the chilly room in Austin.
CHUCK:
David Brady.
DAVID:
Wow! You guys all sound like you’re in the same room. The acoustics are perfect.
CHUCK:
I’m Charles Max Wood from DevChat.TV. And yes, we are recording from our retreat. We are looking at each other not virtually.
AVDI:
This is wild.
JAMES:
We can see our faces.
JOSH:
Do people know what our retreat is?
JAMES:
Yeah, tell us.
CHUCK:
Yeah, let’s talk about it really quickly. We will be posting the episode that we did at LoneStar Ruby Conference and that kind of explains as well, but we got together to write the Ruby Best Practice Patterns that Kent Beck recommended that we write a year and a half or so ago.
KATRINA:
So, we’ve been here two days and that’s it. The book’s done. [Laughter]
JOSH:
Well, we’ve got to edit it now.
DAVID:
It’s Ruby Fast Practice Patterns, is what we ended up with.
JOSH:
So we all came here.
DAVID:
It’s also the [inaudible].
JOSH:
We all came here to Austin for LoneStar Ruby Conf. We did a panel. Then we ran off south of the river and we’re hiding in a house from the sweltering heat and writing and watching cool movies and eating yummy food.
JAMES:
When Josh says house, he means slash castle. [Laughter]
CHUCK:
Yeah, are you talking about the house that’s the front of the house or the house that’s the back of the house?
JOSH:
I’m talking about the house that’s the top of the house.
DAVID:
This is the house that Emacs built. [Laughter]
JOSH:
Yeah, there is a turret.
CHUCK:
We’ll post videos and stuff and pictures.
JAMES:
Katrina’s sleeping in there.
CHUCK:
She is sleeping in there. We joke around. I want to make sure people know. Yes, she actually slept in the turret. [Laughter]
AVDI:
She does a manly [inaudible].
JOSH:
We did try to go for a walk yesterday and it nearly killed one of us.
AVDI:
And feet melted.
JAMES:
It was mostly successful. And David seems to have survived.
DAVID:
Unfortunately.
JAMES:
He was so new at it.
KATRINA:
We’ll try harder next time.
JOSH:
We only got kicked out of one facility.
JAMES:
That’s true. We were not quite arrested.
CHUCK:
And Dana was gracious not to look at Dave and say, [inaudible].
JAMES:
It’s true.
DAVID:
Well, I walked back in about, I had heat exhaustion and I looked at her right in the face and I said, “I was right and you are not staying vertical.”
JOSH:
So, we’re writing a book. That’s gone well. But we figured, while we’re all in a room together, we should record a podcast.
KATRINA:
So today, we’re going to talk about non-Rails Ruby projects.
CHUCK:
You mean Sinatra?
DAVID:
Or Padrino.
KATRINA:
Or Padrino, or perhaps “command line appliance” applications. JOSH: Raspberry Pi.
KATRINA:
Or scripts.
JAMES:
Raspberry Pi is powerful.
CHUCK:
Wait. Ruby does stuff other than web?
JAMES:
No way. It’s getting complicated. [Laughter]
JOSH:
So, I’m working on something right now that’s not a web thing at all. It’s just a background process that slurps stuff out of one database, massages it around, and puts it into another database. I was like, “I have no idea how to do all this stuff right. Rails always helped my hand and tells me how to put the project together and makes everything wonderful. Wow, it’s a lot of work to make this stuff work right on my own.”
KATRINA:
In particular, if you need threading or daemonizing or that sort of thing. It’s not something you do every day.
JAMES:
Yeah, that’s a good point. It’s unusual.
JOSH:
Even the simple stuff like how do you require all the Ruby files that you need? Rails usually just auto-loads all that for me.
JAMES:
Let’s talk about that. It came up on [inaudible]. There was a lot of discussion about it. Some people thought that you maybe require the file at the top of the hierarchy and then require relative everything from there.
JOSH:
Can we put a pin on that for just a second? We talked a little bit about the kinds of projects. Maybe we can talk a little bit more about that before we get into how we build them up. So I just said, I’m writing a daemon process that’s going to sit there long-term and just do some background processing. I joked about Raspberry Pi, but I guess that’s a serious thing to talk about. So we can talk about that. There’s Sinatra.
AVDI:
Wait, you have a pie here and you haven’t shared?
JAMES:
You were too busy eating Katrina’s cupcakes.
[Laughter]
KATRINA:
We did a whole show on gems. That’s the obvious one that we maybe don’t have to talk about just because we’ve already talked about it.
CHUCK:
Yeah, we’ve talked about command line stuff but I think there are two classes of that and one is where it actually provides you with a repl of some kind, like irb. Then there are the other ones where you make a call from the command line and it does something.
JAMES:
So as part of our work here, as we’ve been working on the book, a couple of times we needed to go into the book that we’re working off of, Smalltalk Best Practice Patterns, and go to the Table of Contents to get a list of patterns. At one point, we use those to create files in a directory, like [inaudible] files that we could flesh out. Another time, we use those to pile issues on GitHub. In the first version, I wrote, literally, on the command line, ruby –ne basically and looked over the content all in the command line. The second time, I made a very small script that just had a small [inaudible].
KATRINA:
Scripting is a big topic that would be nice to hit on.
JOSH:
Right. And then there’s things like rake tasks.
JAMES:
That’s a good point.
CHUCK:
Then there’s the whole devops world, which is an extension of command line, but there’s really a whole lot more to that, where there’s management systems and things like that, would go into it.
JAMES:
That’s a really good point. Recipes and stuff.
AVDI:
Wow. We are going to run overtime. [Laughter]
JAMES:
We’ve got all night. We’ll just keep talking.
DAVID:
That’s right. We’ll get three episodes out of it. [Laughter]
JAMES:
Alright. So I think we’ve made the impression that there’s a pretty diverse ecosystem. We go now to specific issues?
JOSH:
Sure. So you were talking about requiring things.
JAMES:
Requiring. So this discussion was on Parley recently. People were asking questions maybe related to require relative. It ended up leading to what is the proper way to require things in my project? There was some thought that you require the top-level file, the first one in the chain, pretty much, and then require relative everything after that. There was another opinion that require everything, just changing the paths for those lower level things. So they go from the directory to [inaudible]. So it’d be like ‘require lib my project whatever file’. Avdi, you had a lot of great thoughts on that. You want to start this conversation?
AVDI:
I’ve forgotten all of them.
[Laughter]
KATRINA:
I can say what I do and then you can tell me all the ways that you think it’s wrong and we can go from there. I like to put lib on my path and then require just the paths. Not require relative, but require the lower level paths from there.
AVDI:
That seems very reasonable.
KATRINA:
It’s not so bad?
JAMES:
Put lib on your path?
AVDI:
Yeah. I feel like in a lot of non-Rail projects, fairly early on I wind up at setting up some kind of environment file, which is going to be, whether it’s in a rake task or in the main app or whatever or testing, it’s just going to set up a load path.
JAMES:
What’s my database this time?
AVDI:
Right. And then individual files just having require foo/bar. I like the general idea of logical paths.
KATRINA:
What’s a logical path?
AVDI:
To me, you know how in Ruby you say require foo, but you don’t say require foo.rb. And the reason for that is that we normally think of Ruby files as all .rb, but actually a given Ruby implementation might support .so for shared objects on a UNIX system or .dll for shared objects on a Windows system. Theoretically, you could plug other things in there. The stuff that we require is already notionally a logical path, because you’re saying give me this thing, this named thing, but I don’t care which exact file it’s in.
JAMES:
Especially if it’s a standard lib or something, right?
AVDI:
Right. That’s nice because it gives you the opportunity to move from a pure Ruby library to a Cimplemented library later on and not have to change any of the requires because they just work. To me, that’s the idea of a logical path. It looks like a path, but it’s not necessarily referring to one specific place in the file system. It more refers to a name or a category of functionality.
JAMES:
I think I have bad habits according to Avdi. Based on what you just said there, I thought that way too. In the past, what I’ve done is use that logical path to load my code in the beginning, or my gem, or my whatever, just like you do with a standard library. But then from there, I have in the past just require relative everything below that. My thinking there was at that point, the resolution has happened. That name has dissolved to my particular code and that I want the rest of the resources that were related to that particular piece of my code, that particular version of the software, or that particular thing. But on Parley, you said that probably wasn’t the best route.
AVDI:
Well, I don’t feel super strongly about it. I think it’s pretty reasonable. To be honest, especially when I’m doing one-off command line stuff, all bets are off. I’m not real strict about it. When I’m doing command line stuff, I’ll often throw a require relative in there just because it’s the fastest way from point A to point B and I don’t have to worry about making sure that my environment file is always included. But I will say that I’ve occasionally found that using logical paths leads to a little bit more flexibility. The example I gave on the Parley discussion was where you want to basically shim out a piece of a library that’s normally required. I’ve seen cases where libraries will require OS-specific stuff that really shouldn’t be required and it’s not enough to just monkey patch it. You can’t just get it out after the fact. It’s going to break the load when it actually tries to load it the first time. So if they’re using logical paths then you can actually override. By fiddling with Ruby’s load path, you can actually override what gets loaded by providing your own stuff that matches that path that’s found somewhere else in the load path.
JOSH:
That sounds pretty evil.
AVDI:
It’s a last ditch effort. It’s not something you distribute necessarily, but it’s a way to avoid opening up the gem on your hard drive and editing the gem.
DAVID:
Which you also do.
KATRINA:
I do it all the time. [Laughter]
AVDI:
Which I also do, but it’d be nice to avoid. I can’t edit that gem on my disk and then easily give that to somebody else on my team. Whereas I can put a shim like that in as well as some stuff in the environment file that loads the right path into the Ruby load path and I can say, look, this is our workaround for right now, is we’re going to shim this thing out using the load path shenanigans.
JOSH:
I think that that’s an incredibly uncommon case and that being such a low-frequency thing to worry about.
AVDI:
I will say though that the logical path thing is what I think is the norm in gems.
JOSH:
So one of the things that tripped me up recently with requiring was that in Ruby 1.8 land, dot, the current directory, the current working directory for the app, is on the load path.
AVDI:
Right.
JAMES:
That was a security issue.
JOSH:
Yeah, that’s a security issue so they took it out in Ruby 1.9. It’s like putting dot on the path in UNIX. If somebody drops a file in there that they can write and that can be dangerous. I understand why they took that out, but then they put require relative in, which solved all those problems for you and introduced maybe a few others. But I like putting stuff on your load path. Although the approach that I usually take was something that Yehuda Katz showed me where you create a file which is basically the root of your require hierarchy. When you do a gem, within the gem file structure, say you’re doing the refraction gem, in lib you would have refraction.rb that would require all the directories and files that are peers of it. This is so hard, the high-level stuff. I’m just waving my hands. It basically walks down the directory hierarchy within lib and at each level, all it does is require the things that are one level beneath it. Then each of those things, if you have a subdirectory within there, there’s a file that knows how to require all the things within there. Everything you require is explicit. You can control, the order things are required in so that dependencies get sorted out correctly.
AVDI:
That’s important.
JAMES:
I think everybody gets to a point at some point in Ruby where they think, “I know, I’ll be clever, I’ll throw all this in an array and each over it and then build up the paths.” Just don’t do that.
JOSH:
Yeah, you glob file directories.
AVDI:
Right. It seems like a great idea but then you wind up screwing stuff up with load order. I like what you’re saying, Josh, because it’s an object-oriented way of dealing with it. Each directory has its own way of loading stuff beneath it and you’re not doing that all at the top.
JOSH:
Yeah. The nice thing is that if you add a file to a subdirectory somewhere, you only have to add a require statement to one file.
AVDI:
Yeah. I want to say one thing about require relative, and I just figured this out, I think if your require relatives have a lot of dot-dots (..) in them, then I think…
JOSH:
Ooh, yuck.
AVDI:
That’s how I’ve seen it used a lot, and I think that’s a dangerous sign. Because if your require relatives have a lot of dot-dots (..) in them, you’re going to see breakage eventually. You’re going to have paths that break.
JAMES:
Require_ relative’s also a tricky beast too, because it requires the current file path and there can be scenarios where you don’t have that. For example, a Rackup file is read in and then evaled so you actually don’t have that context. If you require relative in there, [inaudible].
KATRINA:
Right. Rackup is the first place I started loading lib onto my path actually. I do use require relative when I give a test suite to the students. I have no idea where they’re going to run this test suite from, so require relative will tell them that this file that they’re going to make that doesn’t exist will be in the same directory as the test file that they’re currently running.
JAMES:
I think that’s totally reasonable. I use require relative to get to my code from my test suite. To me, at that point, I’m not concerned with things like gem load paths and logical paths. I’m just like, “This test file here belongs to that code file over there.”
JOSH:
The scenario I just described with walking down directory hierarchies recursively [inaudible] basically a level at a time. I’ve been using require relative for that. It’s really nice, because if you move stuff around, you just move it around.
AVDI:
I don’t like how long it is.
CHUCK:
It’s a relative peer, it’s not a…
JAMES:
How long require relative is?
AVDI:
Yeah. It’s actually long enough that it feels like something to be used occasionally. JOSH: Well it’s just a method. Create an alias for it.
JAMES:
Avdi hates require relative. That’s the moral of this discussion.
AVDI:
I don’t hate it. [Laughter]
DAVID:
He’s afraid of it.
AVDI:
Yes. I’m afraid of it. There you go.
DAVID:
Okay.
JAMES:
Awesome.
JOSH:
Okay, so require. Is there more to say about requiring?
JAMES:
No.
JOSH:
Chuck, did you…
CHUCK:
No, I was just going to say I’m curious about what other configuration or setup issues you run into where Rails really does just autoload everything.
KATRINA:
Well, autoloading.
JAMES:
Autoloading is a big thing, right? Actually, I have one kind of related to that. I need to beg the core team for this at some point. It seems like in about 80% of all the projects I make out of Rails, at some point, I want to do the name trick of the underscored file to the camel-cased word or whatever. And so, I always end up writing my own underscore and camelcase methods as some kind of utility because they’re in active support and I don’t want the dependency on active support for those two things.
AVDI:
I rewrite blank all the time.
JAMES:
Yeah, blank right? There are some things that I would love to see promoted to Ruby itself. But I think if I could have any two methods, I would take camelcase and underscore, because I use them for some kind of loading [inaudible] somewhere.
AVDI:
That’s a good one.
JOSH:
So speaking of autoloading, the word is that it’s deprecated.
JAMES:
Yeah.
JOSH:
Or at least people are advising against using it.
JAMES:
Let’s talk about the problem with autoloading. Actually, it’s a problem with require in general, right? Require is basically not thread safe. But because requiring typically happens as a program is loading up, it’s generally not a problem for it to not be thread safe. The problem with autoloading is it moves require to indeterminate times in the middle of your program execution. Then you take the not thread safe problem and put it [inaudible].
KATRINA:
So there are problems with redefining constants, with removing constants is particularly difficult. It’s a nasty, nasty tricky little problem.
JAMES:
It seems cool in that, oh this is great if they never use this system, that it never has to be loaded from the disk. But the truth is, if you’re not Rails, your load time is just probably not the bottleneck. Loading up all that
AVDI:
And you probably also don’t need to reload. To me, that’s the big win of Rails style autoloading, is reloading without loading up the whole thing again. Most projects probably don’t have to do that.
JAMES:
If you need real code reloadings, [inaudible].
AVDI:
Yeah. Ruby’s just not very good at code reloading, period. Honestly, if your code is taking too long to load, well then I actually don’t have a good answer. [Laughter]
JOSH:
Think of something funny and say it later.
[Laughter]
KATRINA:
In Sinatra, my problem isn’t that the project takes too long to load. It loads really, really quickly, but I keep forgetting to reload it. So I’m checking, I’m reloading a page not seeing my behavior running, running some command against an endpoint that I just defined and it’s not there. It always takes me a few seconds to realize that, oh I forgot to restart.
AVDI:
That’s right, yeah. In Sinatra projects, that happens to me too.
JAMES:
There are ways around that. Sinatra has --
AVDI:
Yeah, but every six months, they roll out a new recommended way to reload stuff in Sinatra.
KATRINA:
This is true.
JAMES:
I use shotgun and it has worked for as long as I can remember.
AVDI:
But it’s deprecated now. Well not deprecated. That’s not the one I’ve seen.
KATRINA:
It works if you’re on a Mac or Linux. On Windows, it does not. So on Windows, you need to get the Sinatra contrib gem and use the reload.
AVDI:
But neither of those are the ones that they’re talking about now.
JAMES:
I’ve had trouble with reloader in the past. I’ve had something where it will reload and something that won’t reload.
KATRINA:
Regardless of whether or not I have reloading on, I just restart my app occasionally all the time anyway.
JAMES:
Exactly. So that’s a good point. Rails offers a lot of niceties. And things like active support that gives us nice easy language, or things like if [inaudible] that make development environments simpler. It’s hard to know how much of that you want in some project.
KATRINA:
Okay. Here’s something that Rails does that I don’t like and that I really like when I’m outside of Rails. Rails loads everything everywhere. When I have my own app, if I have a Sinatra app, I have the app part that is the views and the view helpers and the actual endpoints that are defined. Then I have my application logic, which I put under lib and then the name of the app. Then I have the command-line app part, which usually includes all of the domain logic like the actual application logic, but the application logic doesn’t need the command-line app logic. The application logic also doesn’t need the Sinatra app. So within lib, when I’m in my own projects, I feel like I’m free to require fewer things in all the little parts and just be very specific about which dependencies I’m coupling together.
AVDI:
I’ll bet your tests run really fast.
KATRINA:
Yes.
JAMES:
It’s great. If you’re writing a rake task to just throw some entries in a database, you don’t need the Sinatra stuff for that. You’re just requiring the application logic and go.
KATRINA:
Oh, I have another thing about requiring. In one of my gems recently, somebody refactored my command-line part of the gem to put the require statement at the top of the file instead of inside of each method definition. I actually rejected that pull request.
JAMES:
You like them in the method definitions?
KATRINA:
In the command-line application, because if I just say my command, it shows me all of the commands, the documentation, all of the commands that you can actually do, without loading my application.
JAMES:
That is a plus on the require statements inside. It has minuses and it’s been discussed heavily. In fact, [Max] has talked about it in the past. One is that, again, you’ve moved non thread safe code to the middle of the application. That’s a problem. Two is that it looks like it’s scoped, like in a lexical scope, but it’s not.
KATRINA:
I hadn’t thought of that.
JAMES:
Once that require fires, it’s required. It’s not lexically scoped.
KATRINA:
That said, this is a command-line application. It’s going to load and either just figure out what methods are available and show that and then exit, or it’s going to load, figure out which method to call, load that, run it, and exit.
JAMES:
I think you probably have a legitimate use there. I don’t think it’s a problem. But that is one of the reasons it’s fallen out debate.
DAVID:
That’s totally crazy. That’s the one place where you’re in bizarro world, so it’s okay.
KATRINA:
It always frustrates me in a Rails application when I say rake –T and it takes 18 seconds to give me [inaudible]. [Laughter]
JOSH:
So the rake tasks handle that by there are some tasks that are dependent on the environment and load the whole application, and then there are some that don’t.
KATRINA:
It still takes forever to actually show me which tasks are [lined up].
JOSH:
Oh, you know why? It has to load all of the tasks from all the gems.
JAMES:
Right.
JOSH:
Boo hoo. Okay, so what else?
JAMES:
Lots of talk about require. What’s another
JOSH:
Folder structure.
CHUCK:
Let’s go bring that up.
DAVID:
Can I talk about CLI app structure?
JAMES:
Sure.
DAVID:
Okay, so this is something that I personally have pain with and love at the same time. Let me throw this out and then you guys can tell me I’m wrong. When I build a command-line interface app, I have a bin folder, I have a lib folder, bin only contains drivers. All they do is reach over into lib. require_relative ‘../lib’ I have a config folder sometimes. Then I have a test or a spec folder. I have a gem file that contains minitest or rspec or trollop or [inaudible]. I have a rake file that all it does is launch the test suite and I have a [inaudible]. Pains that I have with this are that I end up putting models into the lib directory and lib/model/something feels weird to me, but app/models. But I’ve seem to disfavor having a /app at my root, at least at first. Tell me how I’m wrong and how you’re better.
KATRINA:
In my gems I have lib under the gem name and then I have [inaudible]. If I have a command-line app, I have lib, I have the name of the app, and I have all of my models inside of that folder.
JAMES:
What Katrina is describing right now is actually the recommended according to rubygems, I believe, in that ideally we should be name spacing our stuff that we’re putting in lib. The structure is lib, you have some top-level file that you require in there just like Josh described, but then the rest of your code lives under some namespace. That folder in there is the name of your project. That’s the namespace.
DAVID:
That’s essentially your app folder.
JAMES:
I don’t know if I would go so far as to call that [inaudible].
KATRINA:
When I’m doing a Sinatra app, I have lib and inside of lib I have app, which is my Sinatra app. So I have app.rb and then I have a directory named app which has all of the views, all of the public, the JavaScripts and everything, and if I break out my API endpoints occur, getting the names of my resources, that’s also within the app directory. Now next to that, I have the name of my application, which is going to be all of my domain logic, all of my actual application logic. If the application name is rogues, it’s rogues.rb and a directory named rogues where all of that is underneath it. If I also have a command-line app here, I’m going to have cli.rb right inside of lib, which is where I
define the endpoints, but that is just a wrapper for domain logic usually, unless there’s any very specific stuff, in which case I have a cli directory which contains all of the specific stuff to the cli.
JOSH:
Sort of like a side-door into things.
KATRINA:
Yeah.
JAMES:
Interesting point of view. One of the reasons for the standard directories, especially the ones Dave mentioned, is ruby gems is aware of certain directories by default. There was also an old mechanism of installing. It’s pretty much [gone] these days but you used to be able to stick a setup.rb file in your directory and you could do a manual install from that. It too was aware of these key directories. Bin, lib, doc, tests, spec, there’s a couple more I think. Data. It’s aware of data directory. Anyway, it’s these standard directories. If you go outside of those and you need to go to ruby gem or something, you will probably need to specify in the gem spec that you have some kind of special thing. I do similar to Katrina except that I usually put the, in the case of models which you mentioned, I would probably have my project, lib/my_project/ then I’m not afraid to make subdirectories in there. I often do a models directory there and I’ll put my models inside some module in there.
DAVID:
So are there any [peer] directories now for lib? Because it sounds like everything I would keep outside of lib is inside your lib.
KATRINA:
As long as it’s part of my application. I do keep config outside of lib. I keep database outside of lib.
If I have sqlite3 or PStore or anything like that, I’ll leave that outside of lib.
JOSH:
What if you’re generating executable binaries?
KATRINA:
Generating executable binaries.
DAVID:
When I go [inaudible], I have runnable scripts that don’t end in .rb
KATRINA:
They’re in my bin directory.
JAMES:
To be clear, anything in there, in my opinion, should be a require statement, a method call, or something
DAVID:
So you need to require ‘cli.rb’ and then run cli and here’s ARGF.
KATRINA:
Yeah, it’s my_module/my_class.run.
CHUCK:
Then I find this interesting because I tend, like Dave is saying, I tend to put all that stuff outside of the lib folder unless it’s going to be shared to somebody else via Bundler or has a rack endpoint for another rack app or things like that, then I’ll put it all under lib. But otherwise, if it’s the top-level app and it owns the space, then I’ll put app/models, app/views, in Sinatra and just have it all sit out there. My main app is mainapp.rb and that’s out in the main folder. The reason is because I don’t need that extra level of organization to go lib/my_app/whatever. I don’t need that level because this is the app. But if it’s a container that says here’s this app that you’re going to use as part of your app, then that’s where I tend to go more along the lines of what you do. But it is interesting that everybody invents it outside of Rails, that folder structure.
JAMES:
How do you decide about dependencies? Something like active support could give you a whole bunch of tools if you want to bring it over, but I’m usually hesitant to do that until I need a very significant portion of it. I am aware that actives upport allows you to require the individual pieces these days. I don’t have to take the whole thing. But how do you make those decisions?
DAVID:
My rule of thumb is three methods. Then I just require active support. Unless the bin file has to be performant.
KATRINA:
I try to require the specific part of active support that I’m looking for. But it could be so hard to figure out which exact part it is. Sometimes, it’s date calculations, but it’s actually [num] because [inaudible] knows that it should deal with the dates. Sometimes it’s the other way around. I find it very difficult to figure out what I actually need.
JAMES:
Right. Any other thoughts about other than that? I guess I’m a little more resistant when I’m making a non-Rails project. When I’m in Rails, it’s like, [inaudible] whatever. I need it, I need it.
KATRINA:
We’ve already binged the
JAMES:
Right. Once everything’s in there already, I throw it in there. But in my own projects, I find myself asking do I really need this or can I [inaudible] one little method that does exactly what I need that’s not too big. Then I hope I’m good about when I’m, okay this is getting ridiculous, [inaudible]. You know, whatever.
DAVID:
I find a continuum where I start writing what’s going to end up being a cli app but at the time, I think I’m writing [inaudible] script. So it’s 50 lines long, it’s in one file. Everything’s right there. Then it grows, so it has to evolve. The lib directory will appear. Then it must grow more before the models will appear and I’ll reorganize files. Then it must grow more. It’s totally okay to have these things grow over time. You don’t have to start with, probably [what we call] the Rails folder structure or [inaudible] rogues file folder structure, if you’re starting with some [little script].
CHUCK:
Do you guys use Bundler for your non-Rails app?
DAVID:
Oh, sure.
KATRINA:
Yeah.
JAMES:
Yes, absolutely.
DAVID:
Always.
JAMES:
Actually, Bundler has some really nice [inaudible] for non-Rails applications.
JOSH:
Like the console.
JAMES:
Okay, the console is great, yes.
DAVID:
Bundler has a console?
JAMES:
Yes.
JOSH:
It’ll console your app.
JAMES:
It will [route] your bundle and set everything up and basically require and then dump you into an irb session.
DAVID:
I’ll be back in 20 minutes. [Laughter]
JAMES:
It has that. Another feature that’s really handy if you’re building a gem, you’re already going to resolve your dependencies through gems. Remember that gemspecs can now differentiate between things that have normal dependencies and things that are development dependencies. You set all that up in your gemspec, you go into Bundler and call the gemspec method and Bundler will build the bundle according to what your gemspec specifies.
JOSH:
Yes. I like how it DRYs that up.
CHUCK:
You guys just improved my life so much. I can’t even tell you.
JAMES:
It has some nice features we’re using in apps and I definitely agree that you should use Bundler outside of Rails.
JOSH:
That’s like the reverse of what it does when you’re building a gem.
JAMES:
That’s correct.
JOSH:
If you’re defining a gem, you can say, oh here’s my gem file, just pull everything out of the gemspec.
JAMES:
Yes.
JOSH:
And you’re saying create my gemspec based on all the stuff you
JAMES:
No, no, no. I was referring to the one you said first.
JOSH:
Oh, okay.
JAMES:
Where it pulls everything out of gemspec. It can be really handy for making sure that the things we’re doing [inaudible] on the command line hashing after gemspecs.
DAVID:
Okay. I have a callback to something Avdi said about having that environment file, because of requires. I had this [sacred cow] that says if you’re building an application, you can do anything you want with the load path, but if you’re building a library, thou shalt not touch it because you’re going to screw somebody else’s.
JAMES:
That’s a pretty big policy. Because things like gems and Bundler and chruby and stuff like that, they all work by manipulating the load path or various paths. If you are also throwing directories in the path, then you can run afoul of that system.
CHUCK:
Because you wind up loading lib/foo.rb instead of gem/foo.
JOSH:
We talked about this I think a fair amount on the gems episode.
JAMES:
Yes, that’s right. I think that’s a pretty good rule. If you are constructing something to be a gem, the load path is probably best left untouched.
JOSH:
Right. Okay. So how about testing?
JAMES:
Testing, alright.
DAVID:
Testing. Yes, do it.
JOSH:
Cool. How?
[Laughter]
DAVID:
So you wanted conversation?
JOSH:
So totally spoiled by Rails. Rails comes with a bunch of these rake tasks for running tests that make my life so much easier. I don’t have to think about this stuff.
CHUCK:
I can rspec and minitest.
JOSH:
Yeah, I can just drop in rspec-rails or minitest-spec-rails and everything just goes. But if I don’t have that structure, I actually have to figure out for myself how to set all that stuff up and make it work right.
DAVID:
I confess that I was very late to come to minitest. I love rspec. Minitest-spec does not give me enough of rspec to make me like it. It’s different enough to make me hate it. After reading Sandi Metz’s book and I saw how easy it was to just have minitest be there, especially in Ruby 2.0, that was enough to seduce me over to the minitest side.
JOSH:
Okay, but then there’s how do you invoke it? How do you
DAVID:
[inaudible] minitest and [inaudible] it.
JAMES:
Here’s what I do. When I first start the gem, whatever the first lib file is, I make the corresponding, typically these days I just go with rspec. I like the [inaudible] in rspec. I knew it would [inaudible] and the mocking is good. But I will make the corresponding rspec file in the beginning. I’ll just require [inaudible] that one lib file and do it. The first time I find myself wanting any piece of test configuration, I [inaudible] spechelper.rb and I put spechelper in the directory and I switch my requires to just hit spechelper instead and then I require [inaudible] test that way. Other than that, I haven’t really had to do much more about that. The only thing is that if it’s a big project, I eventually reach the point where I have that 90 test files and subdirectories and it’s like, okay I actually have to come up with some structure like Rails has of models integration and pass stuff like that so that it’s not just one massive thing.
KATRINA:
I do a couple of things. The first thing I do is that I create a default rake task that will run all of the tests.
JAMES:
Yes. That’s a great thing to do.
KATRINA:
I do it first because I know that I’ll forget then I’ll be in the [inaudible]. I just know it about myself.
DAVID:
Rake. Rake. Rake. Dang it! Rake. Oh!
KATRINA:
So I write that. But the other thing that I do is usually I’ll just be running one test file at a time so it’s ruby and then path of the test file. But as the whole thing grows, I don’t have one single test helper. I have one that is the minimal that just requires the library, then I have one that will require rack test and anything that I need for APIs or just my API test, and then I have one that requires all of the database stuff but doesn’t require rack-test to come do an integration test against the application, but not be in API, et cetera. So I usually end up with four or five different helpers.
JAMES:
I think that is very wise. If I was smart enough to do that, I would probably use that as my demarcation for the subdirectories underneath the tester spec. Every time I need another one of those files, that’s a big enough context switch there. I probably should destroy my subdirectory.
CHUCK:
I know she explained that and I’m sitting here thinking, okay, how do I get into gSchool again?
JAMES:
Yes. I was thinking that’s very wise.
DAVID:
To Josh’s question of how to set it up, I’m a little embarrassed to admit this but I [inaudible] a rake file that sets up my test suite.
JAMES:
Is that [inaudible]?
DAVID:
Yeah.
JAMES:
These days, I tend to, I’ve gotten out of doing Ruby in the [inaudible]. I just use rspec’s launcher for everything now. it’s because if I want to then I can just add a switch to, what you can do to [inaudible] tests of course, you can do it by adding switches, but I can add switches to pick out a specific example that I [inaudible] or something like that. Actually, in my own projects, I’ve gotten away from using [inaudible] because I want that command-line control rspec, so I just use rspec.
DAVID:
I’ve stopped using the default rake pass because Chuck kept dropping the database. [Laughter]
CHUCK:
I’m never going to live that down.
JOSH:
That’s a good default action.
DAVID:
That’s not surprising. That doesn’t violate the principle of least, oh crap. [Laughter]
JAMES:
Run rake David [inaudible].
CHUCK:
It is not my fault they set the test database and the development database to the same database.
JOSH:
Okay, so speaking of that. That’s actually a pretty significant thing
JAMES:
It is?
JOSH:
In your testing setup.
DAVID:
Oh, yeah.
JOSH:
I had to deal with that too recently of, okay we have a development database. How do we test this stuff without wiping out the development database? We ended up doing the same sort of thing of having a foo-development and a foo-test database. Then the question was what do we do about keeping things in sync in terms of the schemas? Then it’s like, well do I go down the whole path of extracting all this stuff from Rails to every time I run tests I’m going to make sure I dump the schema and load it into the test environment?
KATRINA:
That’s actually really [a surprise]. I spent a lot of time writing Sinatra ActiveRecord back-ends. I was very surprised that a lot of the migration stuff is baked into Rails, not baked into ActiveRecord. You can’t just require the ActiveRecord rake tasks and get all of the migrations and everything. It’s directly [inaudible].
JOSH:
Yeah, it’s in a different gem.
JAMES:
There’s a gem called Sinatra-activerecord.
KATRINA:
Yeah, I wasn’t very happy with it.
JAMES:
It’s not great. But it gives you a little bit of it.
KATRINA:
It lets you generate a migration that has a particular name and that’s pretty much it.
JAMES:
Yeah, and migrate the database, yeah.
DAVID:
I desperately now want to extract all the cli and devops stuff out of Rails into a separate gem.
JAMES:
In a way, that would be great.
CHUCK:
One thing I think is funny is all of the things that we talked about just over the last couple of minutes, how many of you have built a Sinatra app and find yourself re-implementing Rails?
KATRINA:
I have not done that.
JAMES:
That’s one of the chief complaints, is that I did actually see [inaudible] but he runs into Sinatra apps that just rebuild massive parts of Rails. I don’t feel like I do that. I do feel like I try to take a lot of Rails infrastructure, or at least [inaudible]-inspired, for sure. But once I get into the app itself, my [inaudible].
DAVID:
I had a client that Chuck also had that Rails was a dirty word and there were politics involved. So we built a Sinatra app and it ended up including every gem from Rails except for the rails gem. [inaudible]. We rewrote those ourselves. The second app that we built for them, we basically said, remember how painful this is? And we just said, here’s a Rails app. And they went, oh, I guess we were wrong.
JAMES:
I actually saw a tutorial on the internet a while back that was like, I’m going to show you how to write web apps and we’re going to use Sinatra because it’s significantly easier than Rails. Then they started a Sinatra app, imported 75% of Rails and wrote their paths in exactly Rails style. I was like, this is not any easier.
CHUCK:
You’ve got everything with ActionController. That’s the point of the tool.
KATRINA:
I started working on a tutorial last week, right? I wrote the first draft of a tutorial last week that is for Sinatra. It basically started out with the one-page five lines of code that is a whole app and let them run it.
JAMES:
[inaudible] ‘Hello world!’
KATRINA:
Exactly. Then extract out the little runner into a Rackup file. Then as it grows out, extract out the lib. It’s very different. It’s still very tiny. But you do get the impression that. Like Steve Klabnik was doing this with his students in New York, and they were like, “Why do I have to do all of these things? What’s all of these directories and everything?” Of course, the next thing is like rails new and they’re like, “Oh, wow! That is amazing.” [Laughter]
DAVID:
Let me hurt you so that you will appreciate the medicine that comes next.
JAMES:
That’s right.
CHUCK:
But at the same time, sometimes you wind up building an app and you’re using nearly nothing that Rails gives you and you’re way better off going with Sinatra or something else. It’s interesting to the tradeoffs that are there. We need to talk about the tradeoffs since we got here.
JAMES:
That’s one of the great things I find, like I’m pretty UNIX savvy, but if I’m writing a Rails application or something, I will be very reluctant to reach for those tools because it’ll tie me to specific platforms and stuff like that. If I’m implementing some command-line application where I know where it’s going to run or a scripting tool, especially, I have no qualms about just using [inaudible], forking processes or [inaudible].
JOSH:
So what’s the weirdest app setup that you’ve had to deal with? Raspberry Pi?
JAMES:
Raspberry Pi runs [inaudible]. It used to take, I read pretty early on, that it used to take about four hours to build [inaudible] Ruby in Raspberry Pi. It would use [inaudible] binaries then you get one really fast, because of the binary for it. That’s that. But yeah, I haven’t noticed. It takes forever to fire up and it doesn’t run [inaudible], but it’s just Ruby.
KATRINA:
I think the strangest thing I worked on and it didn’t go very well just because I didn’t know what I was doing, there was an ETL type of thing, just data types. We were using JOSH: What’s ETL?
KATRINA:
Extract, transform, load. So you have data in one format, you do stuff to it and you end up with data in a different format.
JOSH:
That’s what I was just talking about. [Laughter]
KATRINA:
We had ZeroMQ that was doing all sorts of things. We were getting data flowing far too quickly for my taste. I couldn’t figure out how to get things to flow at a reasonable rate. Everything [inaudible] or something piling up here or empty here. I had no idea what was going on. Then we introduced Cassandra into the mix, which just confused me further.
[Laughter]
KATRINA:
I think I worked on it for 10 days. It was an experiment. I don’t remember that I actually ended up doing anything with it. I don’t know. And there’s threading and daemonizing involved, which was just, broke my brain.
JAMES:
All the challenges there.
KATRINA:
All the challenges.
JOSH:
So daemonized applications are I think a pretty significant class of applications. I remember hearing you do a podcast a couple of years ago where you talked about the work that you did figuring out how to daemonize stuff for when you were working with [inaudible].
JAMES:
Yeah, some of the versions of Scout.
JOSH:
Yeah, that’s right.
JAMES:
We built our own agent for Scout. Scout in the beginning is basically just a cron job, a very simple script that [inaudible]. I think they may have actually gone back to that at some point for a couple [inaudible]. But there was a version where we had a complex agent that basically ran tests as a daemon. Yeah, there are a lot of things to consider. First of all,
JOSH:
Is that a whole episode for us to do?
JAMES:
No, I mean I [inaudible] real quick. You have to daemonize the application, which means certain things. You have to detach from the client terminally or redirect streams, things like that. There’s a set of processes you have to do. Generally, you want to create a PID file somewhere so that PID of your process has been written down.
JOSH:
PID = process id .
JAMES:
Process id, right. That’s how you’re able to send messages to this thing that you can no longer talk to or communicate with. The creation of a PID file, if I told you the proper way to create a PID file, it would take most of an episode, because it’s incredibly complicated. In order to dump that to a file, to know that it’s okay for you at this point to be dumping that to a file. It’s quite an involved process. Then things like [inaudible] files and stuff, which you have to get setup before you can [inaudible] so that you can complain if you can’t do that or things like that. There’s a very involved process of how you’re going to run. Then in something like Ruby, I think you want to give heavy consideration if you’re going to write a very long-running process. The worst thing you can do is just write your normal script and run it in Ruby, in my opinion. What you instead want to do, you want to write a very dumb small shell at the top which basically is running some kind of loop and when it needs to do something, it needs to fork off a process, do a whole bunch of work and exit that process so that that would gc and clean up all that memory and all that. Exit is the ultimate gc. [inaudible].
JOSH:
So you like the UNIX forking model.
JAMES:
Absolutely. Yeah. I think Ruby is built for that 100%.
AVDI:
[inaudible] MRI.
JAMES:
MRI. That’s true.
DAVID:
Does MRI still bear out the old legacy of it’s kind of leaky long-term?
JAMES:
That’s been long debated. Does Ruby leak memory? My answer is no, I don’t think so. I don’t believe that MRI leaks memory. Here’s what happens. One, when MRI requests new memory from heap, it will not return. If at some point, you do a massive operation, [inaudible] a bunch of data and push that memory way up, that number will never go back down, even though you’re not doing that big work anymore. That makes it look like memory gets bigger. Two, just normal usage of blocks you have to be careful with. Every block is a closure. It grabs all of the surrounding state. If you have some long-lived resources in there and that block is saved in some way that never goes out of scope, that stuff never gets gc’d ever. You have to be careful if you use blocks like that incorrectly, you can make resources hang out for a long time. That looks like memory leaks.
DAVID:
I also made the mistake once of having an app that got strings from the user and converted them to symbols, which can’t ever be gc’d because they have the same object id forever.
JAMES:
That’s right. You should, creating [inaudible] should never be dynamically creating symbols in Ruby, because that’s right, they don’t get gc’d.
JOSH:
Symbolized keys.
[Laughter]
KATRINA:
So one of the things that we mentioned on an earlier show was
JOSH:
I want to pause that for just a second.
KATRINA:
Okay.
JOSH:
That’s that when you’re talking about blocks as memory leaks, this is the number one reason why if I’m in Rails, I will never do one of those class-level macros like validate or [before-filter] or whatever where I specify a lambda within the class context. I always will choose the form where I specify a symbol which is the name of the method which you call.
JAMES:
You have to think about blocks. Every time you issue a block, it’s capturing the surrounding context.
JOSH:
Yeah. Okay.
KATRINA:
So for daemonizing processes, this became a little bit better in Ruby 1.9 I believe and Tim Pease, one of the GitHubbers in [Boulder], has written a gem called servolux.
JAMES:
That’s right, servolux is awesome. Awesome, yes. It’s very good. The thing Katrina’s referring to in Ruby 1.9 is Process.daemon(). It’s core in Ruby. You have it without any require statements and it will do the proper daemonized processes, these 10 very cryptic statements, and it will run through that process. It’ll do the right thing.
CHUCK:
One weirdo thing that I did is, I think it was at RailsConf and we were all there. I never did finish the app, but it was a command-line app that you could put in the id for a tweet and it would pull the content of the tweet in and eval it effectively in Ruby.
JOSH:
Are you insane? [Laughter]
CHUCK:
Yeah, I was late for that. So the case was, how do I make this so that it’s not going to crash somebody’s system when they get the wrong tweet? There are gems out there that do it, but it is a really tricky problem to solve.
KATRINA:
Sandboxing is [inaudible]
CHUCK:
No, not at all.
JAMES:
Sandboxing is very complicated. Ruby has a safe mechanism. I couldn’t in good conscience recommend it.
CHUCK:
It really doesn’t do anything for you.
JAMES:
Well, the problem with safe is it’s basically a set of massive hacks all through the interpreter that disable and enable various things at [inaudible] level. So if you believe they got that 100% right, you’re good to go.
KATRINA:
Go for it.
JOSH:
Do you think all the test cases are worked out for that?
JAMES:
Exactly. Let me tell you that it is very common for bugs to be filed against safe [inaudible].
CHUCK:
But [mdlabs] has a gem and there were one or two other gems that has sanitized the vm so that you can’t do, at least the obvious things. So you require it and then run it all in that context. But it’s still hard to make sure that it doesn’t do silly things. Anyway, you usually don’t run this [inaudible] with Rails, but evaluating user input is scary.
JOSH:
So I have a completely off the wall topic that everyone may gong for this episode, which is selfcontained executables.
JAMES:
Self-contained executables. That is actually a really interesting topic.
AVDI:
I forcibly not gong that. Tell me more.
JAMES:
Yeah.
CHUCK:
I’m not sure what you mean.
DAVID:
You mean like a my_program.exe on Windows?
JOSH:
Yeah.
KATRINA:
So a desktop application.
JOSH:
Yeah. Or whatever.app. I think that MacRuby on the Mac lets you do something like that.
CHUCK:
Yes it does.
AVDI:
[Shoes] does that too.
JAMES:
Yes, [Shoes]. This is a great example. Also RubyMotion will now build OS 10 apps not just iOS apps, I believe.
AVDI:
Wow.
CHUCK:
But it does it still do that through static compilation? So it doesn’t actually run a Ruby interpreter as part of the process, it actually just runs on the, what do you call it?
JOSH:
On the native Objective-C platform?
CHUCK:
Yeah. The runtime. The Objective-C runtime.
JAMES:
I don’t know.
JOSH:
I don’t really know anything about how these work on any of the systems except as a little bit of hearsay. I haven’t ever built one. It seems like a really cool thing to do.
JAMES:
What is that framework LimeChat is written in? Do you know what I mean? LimeChat is an IRC client on Mac OS X.
JOSH:
Yeah, I remember that.
JAMES:
It’s written in
JOSH:
RubyCocoa.
JAMES:
Is it RubyCocoa?
KATRINA:
Is that the precursor to
JOSH:
MacRuby? RubyCocoa?
KATRINA:
Ruby [inaudible]?
JAMES:
Yeah, I think it is actually, kind of, part of what became [inaudible].
JOSH:
Maybe that’s a future episode topic. That could be kind of fun.
JAMES:
There are some interesting tricks to doing apps themselves, like if you are just going to [have an app installed]. And [we’ve seen that before like] [inaudible] and stuff. But one thing people often do, which is interesting, is build a Sinatra app and use bin as the server, because it actually takes a really small config to fire up Sinatra under thin and if you’re under thin, you basically have a [vet] machine underneath it, which means you’ve got evented programming if you need it. There are some cool tricks there. There’s a good blog post about that. I’ll see if I can [inaudible] it up.
JOSH:
Cool. Okay.
KATRINA:
One of the things we haven’t talked about is logging.
JAMES:
Logging. It’s a good [inaudible].
KATRINA:
Possibly the biggest pain ever. [Laughter]
CHUCK:
rails.logger.info [Laughter]
JAMES:
It is [inaudible].
KATRINA:
I haven’t figured out how to do it cleanly, where to put my log files, where to define the logging. Do I use sys log? Do I not use sys log? How do I thread logging through my application? I don’t know.
CHUCK:
I have a really good answer for you. It depends.
KATRINA:
Yeah, thanks.
AVDI:
I have another good answer for you. Listen to the episode with
CHUCK:
Tim Pease.
AVDI:
That was with Tim, wasn’t it?
JAMES:
Tim Pease. It was Tim. Boy, we should have had Tim on for this episode. [Laughter]
JOSH:
Tim, where are you?
KATRINA:
I think I did listen to it and I still don’t get it. [Laughter]
CHUCK:
We were really general there.
AVDI:
There’s a lot to say about that.
CHUCK:
Yeah, we didn’t go into best practices at all for that. But it really does depend on what you’re logging and what you’re [inaudible].
JAMES:
One of the best things I ever heard about logging that really changed the way I think about it is to remember that logging is a feature and needs to be treated as such. It’s often just expected that as part of knocking out stories in Pivotal or whatever, we did that feature plus logging. But that’s not true. It’s that deciding all of these things you’ve just mentioned, that takes time and energy to decide where to put files at. The thing about logging is how are they going to be rotated out so it not fill this hard drive, things like that. That is a feature like any other and should be a separate part of development where you figure out [inaudible].
KATRINA:
Yeah, and how do I format my logs, because I’m likely to want to script things that consume the logs.
JAMES:
Right.
AVDI:
And how do you distribute the logger throughout your application. In Rails, you usually have a logger somewhere nearby in the controller or whatever.
JAMES:
I’m [inaudible] on this and say that I often just throw some kind of globally available constant that I can set up in a config file and everybody can get to, on logger. I sometimes feel a little bit bad about doing it. Oh no, I’m using a global variable. But at the same time, logging is one of those [cross-cutting] concerns that I try to only apply it in the minimal level of places. I try to get it to the right spots where the main thing is what does the logging, so I don’t have logging code at every single class. But also, it’s like Chuck just said, when I’m in Rails and I want to debug something, I can do logger.info or whatever. I really like having that when I’m working with an application or something. So I’m probably a bad person.
DAVID:
Well, maybe. But after reading POODR, I still want my logger globally, but I also want it decoupled. So I end up building at almost the highest of the application, I build a dependency-injector that injects a logger. So I can build it up an inject it and later on, like when I test, if I want to pack it out, I
can.
KATRINA:
I have to admit, most of the time I just don’t log anything. I use puts. [Laughter]
CHUCK:
Again, it depends on how much persistence you want and how much flexibility you want around it. If you need it to persist, then you’re going to put it to some system that writes it to disk. If you want flexibility, you may put it into a database system or something where you would query against...
KATRINA:
Elastic search would be brilliant for that. Just format all of the stuff as json, stick it in elastic search, you can fill that database up for a long time and it just won’t go slow.
CHUCK:
The other thing I’ve seen done is you basically break it into an object or a document, you put it in MongoDB and then you use [map/reduce] because it has automatic map/reduce triggers you can call and you write in the JavaScript. So then it just breaks it up into [concerns] and does a lot of that really nicely.
KATRINA:
If you have time series, you might want to use Cassandra because that also has a really nice [inaudible].
CHUCK:
And distributed storage, which is also nice.
JAMES:
Don’t underestimate how far you can go on puts, right?
KATRINA:
I know, right?
JAMES:
You go in there and you’re like, oh, this is terrible, I’m [inaudible]. But think about it. Even when you give it to somebody else, it’s like, well they can choose to redirect it to a file somewhere or they could choose to send it to dev null if they want to see it. Don’t underestimate how far that simple approach works.
CHUCK:
I’ve seen that too, where it’s piped to a system [inaudible].
DAVID:
It’s funny how you guys want to picks puts and then immediately go to really important things. The next thing I always want after puts is decoration, like timestamps and error level and organization and that sort of thing. Because that’s just how I write code.
JAMES:
If you want that, you just take [inaudible] and go the longer [inaudible].
CHUCK:
But the other thing is, you have the log levels so you can turn those on and off and then your logging is specific to your environment, which is also very handy.
JOSH:
So new topic?
KATRINA:
New topic.
JOSH:
I think we have time for maybe one more topic.
CHUCK:
Probably.
JOSH:
Threading.
AVDI:
Yes.
JOSH:
Threading is something we usually don’t deal with when we’re doing Rails apps, because it’s all request/response and we just don’t do threading.
CHUCK:
Yeah, our webserver will usually handle that if it handles that.
KATRINA:
Most of them do these days.
JAMES:
That’s true.
JOSH:
Most of them do. But when you’re doing non-Rails apps, threading is a really useful tool to reach for a number of problems, like these ETL applications. I just did something where I wanted to be able to write to the second database while I’m reading the next chunk to process from the first database. It’s great to do that in parallel and using threading. But I think that for a lot of people who got their start doing Rails, threading is really scary in Ruby.
KATRINA:
There’s not a lot of documentation or books that will tell you how to think about it in a clean way.
DAVID:
But there are. But they’re all in C++. [Laughter]
KATRINA:
I didn’t even know that.
AVDI:
Or Java.
KATRINA:
Or Java. But I didn’t even know where to ask, when I was starting to write daemons. I didn’t know what questions to ask. I tried to ask around. The only thing that I found was the servolux gem from Tim Pease and even that assumes that you know a lot.
CHUCK:
If you’ve listened to the show also, with Jesse Storimer, Working With UNIX Processes, which is [inaudible].
JAMES:
Yeah. Absolutely. That would probably be one of the first things that would be great to say, is read everything Jesse Storimer’s ever written, because he does really break a lot of this down.
CHUCK:
And it’s all Ruby-centric, which is what Dave [inaudible].
JOSH:
But don’t use ruby.org [Laughter]
JOSH:
You’ll want to jump off a cliff if you try and do that. So many of the docs there are just wrong or lies or incomplete or at the wrong version or whatever.
JAMES:
Probably the best topic to get into when you’re getting into threading with Ruby is make this deal with yourself: that you’ll only use threads by creating a queue then launching a thread then pushing things into the queue that the thread pulls out of the queue.
AVDI:
I was about to say the exact same thing.
JAMES:
If you make that deal with yourself, it’s actually hard to screw up.
AVDI:
Yeah. It actually is.
KATRINA:
Also, what is the thing that you put on the path? Thread.raise_on_exception?
JAMES:
Thread.abort_on_exception = true
KATRINA:
Yes. Do that.
JAMES:
Yes. By default, when a thread dies in Ruby, absolutely nothing happens as long as it’s not the main thread. So your thread dies then your program runs for some indeterminate amount of time. Everything just goes horribly wrong. Or if you do abort_on_exception, the second an exception is raised in any thread, it goes all the way to the top [inaudible].
AVDI:
So abort_on_exception is Ruby’s way of saying, please somebody, find my body.
JAMES:
Yes, right, exactly.
[Laughter]
KATRINA:
In particular, if you’re working with tests, this is a very good thing. JOSH: Oh yeah.
KATRINA:
If you’re writing tests and you don’t have abort_on_exception turned, you’re
AVDI:
Yeah, it’s a good thing to set up in your test file.
KATRINA:
You’re [inaudible] and just won’t understand why.
JAMES:
But seriously, just play with Ruby’s queue. You have to require ‘thread’ to get Ruby’s queue, but it’s a thread safe queue. So as you push items in, or you have an input and part of when you go to pop. If you go to pop and there’s nothing in the queue, your thread goes to sleep.
JOSH:
So it makes it like the semaphores I used to use in Smalltalk, which is my favorite way to do threading.
JAMES:
Exactly. Then as soon as something is pushed into that queue, your thread will wake up, pull that item out, and do whatever it’s going to do.
JOSH:
It’s so much nicer than [inaudible].
JAMES:
Yes.
CHUCK:
Yeah.
JAMES:
Do thread queue is the most important thing. There’s also a size queue in that same class. You can make a queue where [inaudible] size and it will let you put so many items in and then it’ll start throwing exceptions. If something does go horribly wrong and your consumer goes away and your producer won’t just blow up the memory or whatever.
DAVID:
Is there any way to do a size with timeout?
JAMES:
Can you do timeouts on a queue? You could wrap it in a timeout and time it out that way. There’s also the queue. You can change the default behavior where instead of putting the thread to sleep, it will raise a thread error if there’s nothing for you to grab. You could catch that error and then you can go on.
AVDI:
You could do a non-blocking mode.
JAMES:
Yeah, non-blocking mode basically, then you can go on and do other things and try for it again later.
AVDI:
Incidentally, Ruby Tapas subscribers will be getting some episodes on threads and queues pretty soon.
JAMES:
Woohoo!
JOSH:
Are you just using our podcast to promote your work?
AVDI:
Yes, it’s [scandalous]. [Laughter] AVDI:
Shameless.
JOSH:
It was cute the first three or four times.
JAMES:
By the way, that was ten times funnier when I got to watch Josh turn to Avdi and say that to his face. [Laughter]
JOSH:
It’s all in the eyebrows.
JAMES:
Yeah, it is. [Laughter]
JOSH:
Okay, so are we done? Time for picks?
CHUCK:
I think so.
JAMES:
I don’t think I have anything else. Anybody else have anything else?
DAVID:
No, this was a fun episode.
JAMES:
You know what? I would add something else. I think it’s a lot of fun to write things outside of Rails. You should do it. You should try it. It is fun. We’ve talked a lot about the problems and the complications.
JOSH:
I know what we didn’t talk about at all, which is UI --
JAMES:
UI. That’s a good question. Alright, what do you have on UI?
JOSH:
I don’t have anything. [Laughter]
DAVID:
A whole other episode.
JOSH:
Yeah. I think that’s a whole other episode. We’ve got Shoes. We’ve got Curses.
JAMES:
Lots of tools. I saw [MVCLI], a new library shown off at LoneStar that was Rails-y [inaudible] engine. It’s used to do ROM, which is Rackspace command-line I believe. Lots of choices out there, [I’m going to promise you].
JOSH:
Yeah, that’s a good episode topic.
DAVID:
I came from GUI programming land and wanted it in Ruby, so I’ve tried everything and they all stink. We can spend an hour talking about that.
KATRINA:
Shoes is being rewritten to stink less.
JAMES:
That’s right.
DAVID:
Excellent.
JOSH:
What, they put a little foot powder? [Laughter]
JOSH:
Okay, on that note.
CHUCK:
I just want to back up what James said as far as writing regular Ruby programs that aren’t part of Rails. Part of the reason that I want to say it is not just so that you can go and whatever, but seriously, even if you get paid only to do Rails in your full-time job, you will pick up things about Ruby that still apply to the way that Rails works. It is a major level up. You’ll find that you use Rails more effectively too. So we’re not just talking pie in the sky stuff that other people do because they’re awesome. We’re talking about stuff that will very literally pay off for you very quickly.
JOSH:
Yes. It’s like everybody complains about how people who start doing web development with Rails don’t really know anything about SQL or relational databases because ActiveRecord hides it all. Rails hides a whole bunch about how Ruby works and we just talked about it for one hour and a half.
CHUCK:
I ever hear anyone say [pororo] again, I’m going to stab them in the eye. [Laughter]
JOSH:
I’ll make sure to only do it when I’m in California and you’re in Utah.
JAMES:
Alright. Should we do picks?
CHUCK:
Yeah, let’s do picks. James, what are your picks?
JAMES:
I have two picks. A technical tip pick. I used the jsonpath library recently. I had this API I’m working with and they send me big dumps of nested data. If you want to go through that in a safe manner and see if this key is there, then if it is, grab this key, see if that gave you an array. If it is, grab that array and all the way down. I got tired of writing that code so I grabbed the jsonpath gem, which is like xpath for json and you can just give these little expressions. This then this then this then this and [it’ll drill under] the structure for you and get those things out of them. That was pretty handy for a project I’ve been working on. Then while The Rogues have been on retreat, we’ve been doing various team building exercises and [inaudible] one that almost killed David Brady. [Chuckles]
JAMES:
But the one that didn’t kill David Brady…
DAVID:
It nearly got me killed.
JAMES:
Yeah. The one David Brady actually won before we tried to cover the next day is that we all played a board game one night and he won RoboRally. If you’ve never played RoboRally, you should really play it. I [wanted] it almost 20 years ago. It’s been around forever in some form. You’re programming your robots to drive around this maniacal factory floor that’s trying to kill you. Plus everybody else and their robot is trying to kill you and hilarity ensues. I think that’s pretty much all we have to say about that. Those are my picks.
CHUCK:
That was fun. Avdi, what are your picks?
AVDI:
Well, The LoneStar conference gave Katrina a bottle of Bulleit 10 year bourbon for keynoting and I’ve recommended the regular Bulleit on here before, but I just finally go to try the 10 year. Thank you, Katrina. And I recommend it. It’s great. It’s distinctive compared to some other bourbons I’ve tried lately. It’s very dry. Actually, the first time I took a sip, I took a quick glance back at the bottle to see if it was in fact [inaudible]. So it’s a little bit different. But yeah, good stuff. They also gave me, I should acknowledge, they gave me a bottle of Blanton’s, but I’ve already picked that, so I’m not going to pop that out today.
CHUCK:
Alright. Josh, what are your picks?
JOSH:
Well, when I fly, you’re not allowed to have your electronics on at certain times in the flight, so I always like to take a dead tree copy of a book with me. Recently, I got a gift from a friend. He gave me a copy of John Scalzi’s Redshirts, which I’m really enjoying and I’m about two-thirds of the way through it. It’s hilarious. This is basically a fan fiction about Star Trek: The Original Series. We all know that redshirts are the security members wearing red shirts who beam down to the planet and get killed before the first commercial break to show how perilous the world is. This is a story about what if they figured that out and how they respond to that. It’s a real mind warp. So I’m having fun with that. Then, since I’m on the subject of funny Star Trek fan fiction, there’s a book that I read a number of years ago by John M. Ford called ‘How Much for Just the Planet?’ That’s basically, you know how every show has a musical episode? Buffy had a musical episode. I think Xena had a musical episode. This is the Star Trek Original Series musical episode. I lost time of how many times captain Kirk was dumped down a laundry chute. [Laughter]
JOSH:
So that one has been out of print in the dead tree copy for a while, but there’s now a kindle version on Amazon and you can find used copies as well. That’s it for me.
CHUCK:
Katrina, what are your picks?
KATRINA:
I have two picks today. One is The Open Source Report Card. I don’t know if it’s been picked already, but it’s pretty cool. It connects to your GitHub, so it’s got all the data about when you commit, the issues and the pull requests, and all of that, the times of that. I looked at Steve Klabnik’s open source report card. He’s in the top one percent in terms of productivity among Ruby developers.
CHUCK:
We’re in the 99% right?
KATRINA:
I’m not sure. I didn’t check all of us. I should. It has funny things. It will tell you that you’re an early day or a late night committer or that you work best late in the week or whatever. It will tell you who you might be similar to. It says, among other things, that I most certainly have a relationship with JumpstartLab, though my language is much more foul. [Laughter]
CHUCK:
Good to know.
KATRINA:
Good to know. It’s fun. So Open Source Report Card. The other pick is a website I came across quite a while ago. It’s a data visualization of the underground in New York. It’s called MTA.ME. The thing is, as the little trains leave and go on their paths, when they cross each other it’s a visualization with sound of a string instrument. The links of the tracks are as strings being plucked.
It’s whimsical and it’s poetic and beautiful and a bit odd. It’s just strangely beautiful. MTA.ME.
JOSH:
What do you think of the word sensualization? Because it’s not just a visualization, there’s an auditory component.
KATRINA:
Yeah, I like it.
JOSH:
I tried that word out last week and I liked it.
CHUCK:
Awesome. Dave, what are your picks?
DAVID:
My picks are, the first one is the serve gem, and I’m going to open this up for rebuttal. I love the serve gem, because basically, you go into the [inaudible], you hit serve and now on port 4000 or 8080, you can open a browser and it [inaudible], it will render markdown. If you’ve got [Thin] installed, it’ll use it. If you’ve got WEBrick, it’ll use that. It just makes [standing up] a webserver right here really, really easy. I threw out a personal project that I’ve been working on to use that. I love that. I’m open to rebuttal if there’s something newer, because it’s stable. It’s quite a bit old.
JAMES:
In my [inaudible] at OSRC, I pulled a [seldom known] feature out of a standard library. There’s a library and it ships with Ruby called un and it’s called that so that when you do –r you can just put the un after and it’s –run basically. This library’s purpose for existence is to bring [UNIX-y] write commands to Windows. It imports [FileUtils] and then at the top-level scope, you have access to commands like cp or things like that, common [UNIX-isms]. But, as part of that, one of its features is get you an http [inaudible] or something like that, that [stands] up a webserver. A lot of the times you see python one-liners. This is how you start a python server in one line. It turns out you can do the exact same thing in Ruby.
DAVID:
Will it handle [mime types] and all the good stuff?
JAMES:
That’s a good question. [It] uses [WebWork] under the hood and I don’t know what it does and doesn’t handle. I assume it’s [inaudible].
DAVID:
I’ll have to play with it. Serve is very high-featured, very, very cool stuff.
JAMES:
I think it’s cool. I love being able to use [inaudible]. If serve does raw, I think it’s good.
CHUCK:
So basically what you’re doing is ruby –r un.
JAMES:
You don’t have to have the space. The space is optional.
JOSH:
It’s like the ubygems version or rubygems?
JAMES:
That’s right. There’s also a ubygems file so that you can just do –rubygems without the space. But yeah, check out un. Un has a help. I think you can do ruby –run –e help or something like that and it will tell you the commands.
CHUCK:
Does it work on Windows? Do you know?
JAMES:
Yeah, that’s the whole point of the un library, is it works on Windows.
DAVID:
My second thing is ‘Effective JavaScript: 68 Specific Ways to Harness the Power of JavaScript’, so it’s the eight specific ways to harness the power of JavaScript by David Herman. The reason I love this book so much is that in the first chapter, he basically goes through, here’s all the stuff in JavaScript that is basically crap, and you can avoid it. Then the next section is, here’s all the stuff in JavaScript that’s basically crap but you cannot avoid it so you must learn this minefield. Then he goes on to say, here’s all the wonderful stuff. One of the minefields is don’t try to bring on OO framework onto a prototype inherited language. All these frameworks that try to jam OO eventually have impedance mismatches and this is why. Fantastic book, really highly recommend it.
CHUCK:
Yeah, that’ll make your life easier. If you want to know more about it, there was a JavaScript Jabber episode on it in January.
DAVID:
Excellent.
CHUCK:
So go listen. David is very knowledgeable about JavaScript and the ecosystem there. I’ve got a couple of picks. I’ve been working for a client doing push notification stuff. If you need to build that into your app, then I’m going to pick the gs-apns gem. Apns just stands for Apple push notification service. It’s based on an older gem, but it’s the most maintained and had the best documentation. Honestly, it’s not that hard to stand up, but it’s really awesome. A few other picks, I’m sure these have been picked on the show before. But we are sitting around a table with a Blue Yeti Microphone set to omnidirectional with the gain turned about a third of the way up and it seems to be working pretty well. We’re recording it into my MacBook using Adobe Audition. So those are my other two picks. If there’s nothing else, then keep an eye out for the Ruby Best Practice Patterns book that we are putting together and we’ll catch you all next week.
DAVID:
And if you go for a walking off, then take four friends and some water. [Chuckles]
EVERYBODY:
Bye.