The Rogues talk about APIs.

 

So, I went to a wedding last week, my sister in law got married and it was awesome. She came down the aisle to the Imperial March.

No way.

Yeah, she did. That was awesome. And then they had one of their friends do a non-traditional ceremony and including quotes from The Princess Bride like “Wuv… twue wuv…”

[laughs] That’s awesome!

Yup. Great wedding.

Hey everybody and welcome to episode 73 of the Ruby Rogues Podcast. This week on our panel we have James Edward Gray.

Who can’t find mute button. Hi everybody!

We also have Josh Susser.

James, I believe it is right next to the missile launch button on your console.

Oh!

I need a new keyboard that has a missile launch buttons on it. Avdi Grimm.

Hi I'm Avdi. Head Chef at rubytapas.com.

David Brady.

This podcast requires version 12.2 or higher of David Brady and will remain compatible through version 14.

And I'm Charles Max Wood from devchat.tv. This week, we are going to be talking about APIs and it’s probably going to be different from what you are imagining. But before we do that, we want to do the Best of Parley and James is going to take us away with that.

Yes, I'm being punished for missing recording a week. Actually on Best of Parley, I did a thread a little bit, a while back, it wasn’t this week, but it was I'm preparing this new talk for Aloha Ruby Conf in Hawaii and I asked for some input on the list and there was a great discussion on that. But, that is not my pick because it actually spun this other talk that did start this week on everybody asking… I was asking for kind of “Stupid Ruby Tricks” and now, people are asking for “Stupid Git Tricks”. There’s only been like three messages in this thread so far and I have learned like 65 things about Git that I didn’t know before. So, really awesome thread on Parlay for just picking up a lot of cool tricks with Git and version control.

Awesome.

So James, somebody forked your thread?

Yup. Guess so. [laughter]

That was the sound of four mute buttons being frantically searched for. [laughter]

I forked a thread and that’s how we got the fantasy thread that you brought up a couple of weeks ago.

That’s right. That was great.

But I didn’t hi-jacked the thread, I just created a new thread that was the old thread.

So, instead of an email list, it’s on an email tree.

Ooh. All email lists should actually be trees.

I actually need to check out that email thread, James. I saw it in my mailbox this morning and I’ve been too busy rogueing.

You need to read the Git one. It’s like, I'm serious, three messages I learned like the 10 million things about Git. It’s awesome.

That’s going to help me with my Git thread. Okay, cool. Sweet. Oh and then we have… Chuck are we also going to talk about the Ruby Nuby stuff?

Yes, absolutely. We should have the page up within the next day or so. I’m sorry I have just been overwhelmed and haven’t gotten to it.

But the crucial thing is, we’ve picked an end date for the competition.

Yeah and it’s Halloween.

Yes.

Halloween?

Yeah, so you have the month of October to do your video submission. I guess we should offer extra credit for someone who does it in a Halloween costume?

Absolutely. Yes. Full points.

Yeah.

Especially if the Halloween costume is Ruby related.

[laughs] Oh yeah, very good.

And if you make it on the show, then we really want you to come on the show and do it while drinking blood from a virgin’s neck.

Oh man! I was just hoping for somebody dress up like a “recursive function”. [laughter]

Wait, how do you dress like that?

and Mountain Lion OS X shall lie down with the Lambda. [laughter]

Okay, so APIs.

I’ve heard of those things.

Yeah, why don’t we give the definitions since somebody had to give me one before we came on the show.

Okay, I’ll do the definition. Thanks for the setup James. So, an API is an “Application Programming Interface” often contrasted with an SPI, which is a “System Programming Interface”. APIs are the public interface piece of software to clients who are going to make use that software. This SPI (System Programming Interfaces) are usually like private ways of accessing things that aren’t for general public consumption. So, we are not talking about this today. APIs.

Oh, we probably are.

No, we are not. It’s private. Nobody can get into it.

Oh. Ouch! But in Ruby, you can always do well in private.

I actually wrote a Gem called “Exposed Privates”. [laughter]

Yeah, but we can’t discuss it on the episode or we will lose our family friendly rating. [laughter]

Actually, it wasn’t a gem, it was an RSpec plugin. But yeah, anyway.

So we are not talking about the web APIs this time. We’re talking about real programming APIs.

Yeah, well I think the web ends up being a subset of that or something.

Yeah. It has largely different best practices and stuff.

Well, that is because the mechanisms for reaching a web API are different from a programmatic API where, in a lot of cases, it’s just a library that you call or a class that you reference.

That’s right.

Yeah. It’s deceptively easy to screw those APIs up.

So give us an example of a screwed up API, Chuck.

Well, in the pre-show, I think we all kind of brought this one up and then we are like “Yeah!” “Oh man!” it’s Net::HTTP.

Yes. Amen.

Yeah, it’s just not pretty.

Let’s talk about why.

Well, we can do an entire episode where we invite Eric Hodel on and have him talk about how bad Net::HTTP is. If he is the one who talks about how good it is… wait. [laughter]

Yeah, I don’t think that works out well, but I like Avdi’s point. Let’s talk about why it’s bad because it really is bad.

Okay, so do we all have the Ruby doc site up in our browser?

I don’t even need the Ruby doc site. I can tell you why it’s bad. Actually if you wanna know why it’s bad, this is absolutely hilarious. There is a PeepCode Play by Play videos, where they dom with the experts and Yehuda’s got one of them. At one point, he is doing something in Ruby and he goes into Net::HTTP and he is all, “Yeah something like this.” And he literally fiddles with it for like 10 minutes and cannot figure out the right interface to make the call that he wants. And that’s exactly why it’s bad. It’s that, every single way that you use Net::HTTP, it’s a different interface with slightly different semantics like, is it going to be encoding this for me or am I supposed to be doing that? And none of that is ever clear and interface still is different. So, every time you guess, you are wrong. No matter what.

Yes it’s really out to get you.

One other thing that makes me crazy about it is, I mean, you have some of the method names and class names are pretty self-explanatory like .get it’s making a get request and so you kind of get an idea that’s what it is doing. But a lot of these other ones, I can never remember which method to call. If you are doing something different from just the standard .get, then you have to create a different object to do something else, (which is kind of what James was saying) but I mean ultimately, half of these functions are things that you would never guess the name of, to do what you want to do. And for me a good API is something that I can get into and I can almost guest what the name is because it’s named after what it does.

Right. So, it’s like to give Chuck’s example, its .get if you want to get URL but, if you wanna post some data somewhere, its .post_form right?

Right.

Right. So consistency is an important quality of a good API and Net::HTTP is not consistent. [laughs] It’s nothing if not inconsistent. One of the problems I’ve had… I haven’t used it in a couple of years, but I remember I was trying to do something like header manipulation or set some special header on a response or on a request and it’s like the features that you used to set something up, they only apply in the most specific case and then if you wanna do something that steps outside of that covered case, it’s like the API gives up and you have to do terrible things to be able to use it at all. Does that sound familiar?

Yes. You know, I wanna make this request. Now, I wanna make this request using basic_auth. Oh, okay the way you made it before totally doesn’t apply anymore. Now you have to grab all these special objects, construct them, set your basic auth thing and do the request manually, basically.

Like, okay, that is weird.

So, in addition to consistent, we kind of like composable.

Right exactly.

So, my favorite problem with the Net::HTTP API is it’s exception API. This is something that I think not enough API designers think about, is that exceptions are very much part of your API. And more broadly, error reporting is part of your API. The Net::HTTP library can raise I think many, many, many different exceptions; a number of which defines itself. But the biggest problem is that, it can raise at least 3 different families of exception, meaning exceptions that don’t really have a common base class apart from exception. And so, I’ve seen a lot of projects where the code started out handling one kind of exception and then the application crashed. And then they realize that, “Oh, we also need to handle an I/O error” and the application crashed. And you know and they realize, “Oh, we also need to handle a system error” or whatever it’s called.

Or throw it, yeah.

The errors that are root of all the interrupts and stuff like that.

Errno. I think it’s errno.

Yeah. That’s one of the frustrations that I’ve had with it is you have to rescue at least three distinct families of exception if you wanna avoid the blanket, rescue everything exception which is generally good to avoid.

I think I have the complete list of exceptions here that you have to rescue, because I went into that problem all the time and had to do it. And you are right it’s three different families. There is the time out error, that they can throw. Then there’s a couple of them in errno like EINVAL and ECONNRESET. No, sorry it’s four. There is also EOFError it can throw and defiler. And then there are several exceptions that belong to Net::HTTP like Net::HTTP bad responses.

Yeah and this bleeds over in to some of the attempts to build a nicer API on top of Net::HTTP because at least the last time I used it, I seem to recall the rest-client could raise the same set of exceptions that may have changed since the last time I used it.

That is a good question. I don’t know. That’s a good question. JOSH: Okay, do we have other terrible things to say about Net::HTTP?

Well, just to hit the exception thing one more time, if you wanna fix that problem well, Avdi showed you how to do that in his book. It’s very good.

Everybody wins.

That’s right.

The thing that I am noticing, (I'm looking over the API listing again) that it’s just the way that you pass in parameters very so wildly, it is sort of infuriating. [laughs] There’s all these cases that you create a URI, using the URI library. And there are some cases where you can just pass it in the URI as an object to make a request and others where you have to pass in distinct pieces of it in parallel. Like here is the host, here is the port. [laughs]

Yeah, this is almost exactly the Yehuda video I'm talking about. You should watch because he is literally playing with it for like 10 minutes and I think he actually gives up and switches to a different library. [laughter]

We have all been there.

A good contrast to that would be any of the Rails methods that that takes a route, like redirect to or link to. That there are like three or more different ways you can pass a route in and they all work in all contexts. You can have your hash of options or you can use a URL helper, you can use a string.

In any context, they all work.

And the reason they all work is they all pass through the exact same method right? The URL for which is Rails general. So yeah, that’s exactly right. That is a great point.

So what I wanna ask is, do any of you pre-emptively give up on Net::HTTP and use something else that wraps the API?

Yup.

Yup.

Yup.

And what is your weapon of choice?

So, Avdi and I had a fight over this before the show and he punched me in the nose and I have a nosebleed so he win. [laughter]

That is the --- on my record. [laughs]

So what do you use Avdi?

So lately I have been using Faraday every time that I want to do something with HTTP. Faraday is a wrapper around other HTTP libraries. So, it wraps Net::HTTP, but it also wraps some of the things that are alternatives to Net::HTTP like Typhoeus and Patron and some others. And it gives you a kind of a common API to them, so you can swap out your backend. And it has some nice features like having a fairly sane exception API and a nice sort of call back oriented API for handling the responses. You can use like a regular return API but you can also use a callback API.

So, you mentioned Typhoeus. That’s another one that I used and I generally liked both the interface and it gives you some nice features like being able to do concurrent calls out to other end points out on the web.

Yeah. And the things that Typhoeus demonstrates, (I use it as an example of good APIs a lot) it demonstrates a couple of things. Number one, it demonstrates the fact that you can build a synchronous API on top of an asynchronous callback API, but you cannot start with a synchronous API and then build an asynchronous one top. So, Typhoeus very sensibly started with an asynchronous API where you basically pass an on complete or an on error handler into any of your requests. But then, on top of that, as sugar, it layers a very simple synchronous… when the method returns the request is ready kind of API that you might expect from a simple library like rest mime or something. So, you can have, as you need to, you can go from a very, very simple call get and get a response back to having more of a reversal don’t call me, I’ll call you kind of flow. The other thing I like about it is that it actually has a no raise API, which means that it doesn’t raise exceptions, period. Instead, it provides all the information you need to know if the request was successful or not. And you can query the response object to find out if the request even went out or if it hit a DNS problem or if it made contact, but it got a 500 Error on the other end or whatever. And, you can decide whether you wanna raise an exception based on that. You can put that on error handler if you want or you can just interrogate the response. But, it leaves that decision of is this worth an exception? up to you. And considering that, the alternative Net::HTTP actually has an exception for 404, which is not an error in many cases. I prefer the opposite extreme of let the user decide what’s exception worthy.

Yeah. So, James is back. James, what were you trying to say before you disappeared?

Sorry, I lost my internet connection. I was saying that… (maybe you guys covered while I was gone) but Faraday has middlewares in it, so you can swap out back ends like Avdi said, but also you can insert middlewares between; basically where you made the call on and backend gets a hold of it. So, if you wanna do something like parse JSON in the response, you can just put a middleware that anytime a response comes in, it parses the JSON.

Or adding caching is another good example.

Right, exactly.

And another one that I really like is httparty. And for me, the big win is that it has that consistent interface. And the other thing is that, I don’t have to sit there and decide whether or not I wanna call Net::HTTP get or create Net::HTTP get.new ‘whatever object’ . It’s all consistent and you can actually mix it into a class if you have you own class and it makes sense to add that API to the class.

That’s what kind of bugs me about httparty. The way I see everybody use it is the mixed in version and it’s like, I don’t think it makes sense in a lot of cases. It does have an interface that is not that way, I just don’t see people use it like that. My personal favorite in the past has been rest-client, which I think has a beautiful API. It probably does have the exception problem Avdi mentioned. I can’t remember if that was ever fixed or not. But I loved the API for it. It’s very consistent and just wraps Net::HTTP behind a really same API.

Right.

There were a couple of things Avdi said. James, I think you are off the call at the time, but he was talking about Typhoeus and what worked and what didn’t work. I wanna just revisit one of those things for highlighting. So, one Avdi you were talking about asynchrony versus synchrony in the calls and said that the base Typhoeus model is asynchronous because you can create a synchronous abstraction layer on top of that, but not the other way around. So, is that something that we wanna talk about in terms of when you’re creating an API, do you wanna build it for composability or not? That is, I guess a design decision when you are making the API.

Yeah. That is a really good point actually. Like, I was struck by that when we read GOOS and discussed that recently. Maybe it’s just the example they chose the primary example where they were dealing with like messaging servers and stuff like that. But, I was struck with how much they use call backs everywhere. “Here, let me just hand you this and call me when you are done” kind of thing. And that is almost like object oriented programming at its best, right? In that, you are not actually procedurally running through things.

It’s very “tell, don’t ask”. It tends to lend itself, like, if you are having trouble with the whole ---testing philosophy, I think it clarifies some of that, because you spend less time with methods where you are worried about both what they do and what they return. It becomes more about if you have imperative methods that do something, period. One of the things they do might be to call you back, but you are not testing both the outcome and the result value.

It really forces you to decouple your code more though too, right? Because you can’t count on the order. If you queue up two call backs, you can’t say you know you are going to get this one back first. So you purposely write your code, so that you don’t rely on each other to happen first.

Well it’s important to differentiate between just a callback API and an explicitly asynchronous call back oriented API. Because you can have a callback API where you specify… you pass in multiple call backs. Here’s what you do on success, here’s what you do on time out, here’s what to do on error and that can still pause your thread during the whole call and then call one or none of those call backs and then return control view. So, that’s still synchronous, but it’s a callback oriented API. But the nice thing about that is that it’s a much easier transition from that in the world of asynchrony, than it would be if you had built with the assumption of I'm going to get a return and then work on that.

So a good example of a synchronous callback API is REXML or --- both have a screen parser where you can basically set call backs. Call this when I hit a tag; call this one when I hit some texts or whatever. And they will parse potentially a very long XML document that maybe you can’t afford to keep all in memory. And, it will just hit your call backs on events, but that is synchronous because you get the call backs; the tag will come first and then the texts inside the tag or whatever.

So you can just make some stack and manage where you are in the stack.

James, does FasterCSV do something similar?

No, it doesn’t. Well, you can use FasterCSV to parse really large documents. In that you can request a line at a time. But I don’t have like a callback parser for it. It’s an interesting thought, but I haven’t thought about that.

But you can each over the records right?

Yes.

That is effectively a call… maybe the most simple form, but it is effectively a callback.

Yeah, kind of a Ruby convention right? We should talk a little bit about FasterCSV because a lot of people don’t like how I designed the API. So we should talk about that.

Can I ask what are you talk about the public API or the way that you built the library as a whole?

The public API. That is what they don’t like.

Is it mainly the people are thinking table oriented and you are thinking file oriented?

Yes, that is kind of part of it. So, the one being is like when David Brady said, it should be kind of tabular. And FasterCSV does have a helper method called table which will basically parks the whole thing in one big table and then you treat it like a table. So, I feel like we do kind of hit that. The reason that I didn’t go with just that approach is exactly the reason that whenever I'm dealing with CSV, it’s usually enormous and I don’t wanna suck the entire thing in to memory. And I think doing that and kind of tabular format is a little strange, in that, it’s very difficult not to suck the whole thing in to memory. So the way I designed FasterCSV is that basically like an I/O object. It’s very similar to Ruby’s I/O, forwards a lot of those methods except for the ones I need. And the difference is that, as you are reading in line of CSV, CSV’s concept of a line, which may actually be multiple lines if you have a coded field with some embedded new lines in it and stuff. So, it basically behaves like an I/O object. And really even wraps Ruby’s I/O objects. You can pass an I/O into it and it will wrap that and read CSVs as it comes out. So, it’s kind of like a decorator really. You can do it with all kinds of I/Os which comes with very handy. A trick I use it a lot is to pass in a something I opened with zlib, so you can read a zipped CSV file directly through FasterCSV because zlib can read from zip and you can pass that in and CSV reads from it. But, a lot of people don’t like that I/O-ness of it.

What is the objection to that?

In my case it was just ignorance. I came in and I looked and I saw each and each line and I thought, “Oh this is innumerable” and so I was manipulating away and I wanted to rewind back to the beginning of the enumeration. And I got the next line in the file, didn’t do an I/O rewind and that threw me. I was so certain that James had written a bug and so I wrote this huge long bug report and a unit test to test it and James just laughed and wrote back to me and said, “This is why you are wrong.”

So by “some people”, you meant David?

[laughs] No, it’s actually come up a lot. I think the primary objection is that they feel like it doesn’t do just one thing, because it some I/O involvement in it. To be fair, I think CSV is kind of a tricky thing. It’s actually a pretty poor data format in my opinion because technically, you can break just about any CSV parser on the planet. All you have to do is start the document with a quote and then throw something like 10 gigs of data after it, because there is literally no way to know how far it goes or whatever. So you can pick some kind of cut off or something where you stop parsing and just give up or whatever. But then that also means you may fill the parse some valid CSV at the cut off +1 or something. It’s kind of a tricky data format and so a lot of people object to the fact that it’s tied so closely to I/O. I think that they feel it’s not a good separation of concerns. My opinion is that, it’s difficult to have a good CSV parser without some concept of like the stream its reading from. But in retrospect, if I did have to design it again, I probably would go to something like a callback API, I think if I had a choice or at least make that an option and then give the other API on top of that.

So, James I have a question. You say, if you had to do it over, you would do it differently. How much of the design of the FasterCSV API was sort of burst like --- full formed or how much was extracted from you built it up over time and it just sort of evolved in to that?

That is a really good question. So the parser at the core of CSV, for a long time, was pretty much two regular expressions. And it’s that people were complaining over and over again about the speed of the old CSV library and I'm like, “Well you can actually parse CSV with a regular expression.” I threw one out on the Ruby Core mailing list and people wrote back and were like, “Well that is pretty close, but it misses all these ugly edge cases.” So I took all those ugly edge cases and threw them and test. And I tweaked the regex a little and add one more other little regex to handle it, another edge case and by the end it parsed everything people threw at it. And we went with that for a long time and then eventually, (I'm sorry I forget the name but it’s in the change log if you wanna go look). Somebody came back and rewrote the parser not be regex based which is good, because there is some scenarios where because of the way it had to parse fields, if you fed it certain kind of data, it could get exponential and then your parse just took forever. So, his parser is much better about handling scenarios like that. And it was basically rewritten. But the core of FasterCSV is actually really small. And then on top of it, it’s like you say over time, we have layered on helpers like the table one that I mentioned earlier and several others I have had a completely CSV compatible interface at one time but ironically, when we chose to throw that out when we brought it in the standard library. But, yeah, a lot of it built up over time. But the core parser is actually quite small and not very complicated.

The place I was trying to go with that was, one of the common wisdom pieces in the Rails community is that, when you are going to do a gem or plugin or something you wanna extract it from an application, so that it has some grounding in reality. And there are experience behind it so it’s not just something that somebody dreamed up and never really tried out.

That is a really good point. I have another library called gatling gun, which my wraps SendGrid's Newsletter API. I did that when I was building subinterest. So, I was actually using it at the time and as I was poking around with it, I developed a library that I needed to do that.

Do you see a big difference in how this turn out when you do that?

Yes, I designed it much better. [laughs] [laughter]

We know.

I can point to another library that I did the other way around; I designed the library first and then used it in application. The whole time I’m using it, I’m like, wow this really stupid, brain dead and painful whereas gatling_gun is designed really well, I think because I was using it on an application.

Okay.

So, I wanna talk about the design quality that I've perceived as being a good one in building APIs and I'm curious what your take on it is. And that is the quality of layering.

Yes!

[laughs] So like, some of the best APIs I’ve used, they had for instance, a really abstract top level object API, where all the concepts in it are represented as nice objects. Let’s take as an example something that wraps like a C-level API. At the top level, you have something that is all objects and very clean, some simple top level methods. And then at the bottom level you have a really, really thin wrapper over the C library. So, like the methods have exact same name as the C calls. They have the same return values just mapped to Ruby. If there is some sort of special external return like, you have to map the error value to a table somewhere that isn’t wrapped, it’s just handle it manually. Sometimes there’s one or more layers between there. I should have started out with the flip side of this. The flip side of this is when you’ve got a beautiful high-level API and you realize you wanna change one thing about what it does, like you wanna pass an extra header or something. You look at the code and you realized that each of those beautiful high level API methods is basically implemented as one great big lump, which goes down through multiple layers of abstraction to the sort of the core interface. And you realized that if you wanna change one thing about it, you’re going to have to basically copy that entire lump and either make your own or monkey patch in the change that you want. And there is no way to like tunnel in the new values that you wanna pass down or anything. So, some of the better library I’ve seen had this layered architecture, where you when you decide that you need something to little different with it, you can pick a layer lower down and sort of compose something out of the bits at that layer with your own stuff in there.

There is another big advantage to that approach. For example, I’ve seen a case where they were wrapping like web API and they have the lower layer, like you said, that basically mirrored the calls exactly and the parameters exactly and stuff, which is awesome because then the documentation for the API works for that layer.

Yeah. And that is a great point because if nothing else… for example, gotten around to writing documentation for the higher level stuff and you are just not sure how to do something. But if nothing else, you can go to that docs for the original API, the C-level API and just use the Ruby code the same way.

So one problem that I see here is that, there’s the possibility that you are going to be changing the lower level APIs at some point. So, what kind of a contract I guess do you----

I guess that is part of my point is that, this is explicitly making the choice. I mean, normally in OO design we would think about, “Okay I'm only going to make you a contract at this highest level of abstraction. You can use this high level API with confidence but anything underneath that is my icky secrets”.

A good example of this is Ruby Gems where they started making parts of their API private and changing it and there were compatibility issues.

Right. And this is explicitly making the choice to establish like two or three points of abstraction along the continuum that will actually be stable and publicly exposed. And the second level builds on the first level, the same way third party code will build on the first level. And the third level builds on the second level the same way third party code would use it. So, it is making a choice to commit to having multiple levels of public API instead of just one top level public API.

Would a good approach to this though be maybe to create a lower level API Gem or library and then have another higher level library of API that consumes the lower levels, so that you are basically maintaining those APIs in separate libraries or areas and then you know where the contracts are because it’s different at each level.

Rails does that with the different large scale components. So you have ActiveRecord and Active Support and Action Pack. So, when you get big enough, it’s good to do that. But that’s mainly so that people don’t have to use all of Rails all at once. No one really releases a version of Active Support without really releasing a new version of Rails.

There is another good advantage to this approach we are discussing right now; especially I love it when the top layer lets you grab a hold of its handle to the lower layer.

Yes!

So, you can just like inject something in there and I actually used that one time. It was wrapping a web API. I can’t remember which one, but the web API version had changed and they have added some new feature, but this library hadn’t been updated yet but under the hood, I could just grab a hold of that lower handle. And I think it actually use method missing, so I was able to actually call a

method that didn’t exist and it will just worked because it was new thing in that API. But even if you don’t use that method missing, as long as you get people a generic means to construct the request or something, then they might even be able to take advantage of a new feature on like day one, without you having to update the code.

So I have a great example of that, which is very similar to yours, which is that faraday, at the top level, you give it just these method calls like “get” or “post” and in the background, it is constructing a request object, which normally you don’t have to worry about that request object. But, if you do wanna worry about it or if you wanna, say, add an extra header or something, you can pass a block to that post or get, and the block will yield the request object that faraday is in the middle of preparing. It will yield that to you, once it’s done most of the preparation and then you can call arbitrary methods to add a header or whatever to that request. And then finish your block, you are done with it and then faraday takes whatever the request with whatever modifications you made to it and proceeds along. So, you get to deal with that lower level for just as long as you want and then you are done.

There is something we are all familiar with I think, in Rails programming is that you have these hooks in controllers that get in there before and after actions. So, you get the before filter and the around filter, those kind of things. And that’s a similar kind of thing, where your exposing an interface to get down into a lower level of what is going on in the system. Because, normally it does, like, throw an index method in your controller and the action runs and you get the view and it gets rendered and you are done. But with the filters, you are now breaking in to a lower level of the abstraction and you can insert behavior like that. That is another example.

What I love about the Ruby concept of programming is that by coming from C++ and Java, you are always fighting the leaky abstraction problem, which is where you try to build these top level objects, but it turns out that the middle layers got to keep a handle to some resource that will leak memory if you don’t free it. So, it ends up leaking up. And Net::HTTP is an example of something that leaks, right? The exceptions leak out weirdly. And we are actually approaching this from the other directions which is, we like APIs that find a spot on the wall where it is leaking, drill a hole and install a faucet because that’s where we need to reach in and actually like, leak in reverse, we need to like drill down in and get in to those. I think that is really cool.

Just to be totally fair to the people like Eric Hodel that do work on Net::HTTP (though he didn’t design it, I don’t think), I believe Net:: HTTP was going for that layer of abstraction. There are those objects and you can construct a request manually and stuff. The problem is, with each layer the consistency is so bad that I can’t tell where one layers end and where the other begins.

So, we’ve talked about this on a couple shows about consistency of abstraction level. We were talking about that with Katrina Owens when she was on recently. When Kent Beck was on, we talked about how we want all of the operations that happen within a single method, should happen with the same level of abstraction. So you shouldn’t have q.push a, q.push b, q.push c and then, you know, q rearrange all your contents. So, you wanna keep things happening at the same level. And Katrina even looked at, like, what is the shape of your method? Does everything look uniform? You can find problems with your methods if you just see that. So, APIs the good quality is that they can give you this layer access, so that you can have your client code using the API actually operate at a consistent level of abstraction. And then, if there is something that you need to do that is at a lower level, you can put that in some other method and have that operate at a different level of the API.

I think I’m actually a little bit backwards. I think a lot of people start with the top level calls and we talked about that a lot but, again with our GOOS episode, they tell you why you designed that way. And so, you only build what needs built. But, for me, it’s actually easier, especially when I'm doing API, to work at the bottom up. And I do exactly, like what Avdi said, and mirror that bottom layer exactly as I want, which almost never the Ruby interface I want. Because with Ruby I want pretty objects and blocks and stuff like that and things like a C-API or a web API, they are not going to have those as much like we use them. And so, I‘ll build that lower level layer and then I’ll build objects the way I want on top of that. So, I think it’s important to remember that the interface you end up exposing doesn’t have to be exactly what you are using under the hood. Maybe you can even expose multiple levels, again like Avdi said, you can expose like, here’s my direct copy and here is the pretty stuff I built on top of it.

I think a good example of a bad example of that is Ruby/Tk. They just copied the C-library and the documentation says, “Eh, go to look up the C reference.” And that’s it.

Right. [laughs]

Wow.

So, one thing that we can cover just by referring to another episode is versioning.

Oh yes.

So yeah, go listen to Episode 37: Versioning and Releases.

So, since we are talking about layers, you got your low layer, which actually mirrors exactly as it is. In Ruby, we tend to have a pretty object layer on top of that, hopefully well designed. Then on top of that, would be like a DSL right? Something like RSpec’s DSL for describing various test cases and using the “should” syntax to set up actual assertions. So, what do we think about that? Is that still an API?

Yeah.

Okay. So, say what you are going to say because I have a good rant on this one, but I’ll let you go first.

So, my thinking is some of the DSLs really feel like DSLs. They don’t really feel like programing. The RSpec DSL to me, feels a lot more programming because it still feels like I'm defining a block and doing method calls and things like that, very programmatic things. So, I think if you are calling it a DSL and it’s at a continuum between like pure DSL and pure API, its further down toward API. Some of the others are a little bit more DSL-ish and I'm trying to think of some maybe Hamel or something but that is not a good example either.

Cucumber?

Yeah, Cucumber is much more…

So one of the canonical examples of a domain specific language is SQL (Structured Query Language). That’s a domain specific language. It’s meant for doing relational queries. You can’t do generic true and complete computation in it. And people talk about Rails as being DSL for building web applications and I think that’s pushing it. Rails is an API for doing that.

Yes, it’s a set of APIs that makes things easier.

And back in 2008 or so, I saw a bunch of people doing talks at various places, talking about how to do DSLs in Ruby. And basically, they were just talking about a poorly done API… [laughter] …and it’s like, “Oh! If you just do a call to a library and if you do it all with instant C -- blocks, so that self is implicit then, Oh, hey, it’s a DSL”. It’s like “No. It’s not. It’s just an API that doesn’t work well.” [laughs] I'm a big fan of thinking of APIs as some sort of “linguistic construction”.

Yes.

I learned this like way back when one of my mentors in OOP said, “Defining the API to a class is a linguistic process”. He meant DSL, but this predated that term, so he was talking about you are building DSL for interacting with some piece of your application. So, I think it’s great to think about API construction in terms of… Oh, if I were thinking about this like a DSL, if there was some language for describing all these things, that’s great. But mostly, what I see in DSLs in Ruby is that, they remove a significant portion of the feature set of the Ruby language and offered nothing as an advantage.

Okay, that is a good point. But, let’s go back to the RSpec DSL because I don’t feel like it does that. RSpec DSL, I would argue, does a pretty good job there because if you are going to use something like say, test unit or straight up mini test then first of all, you have the underscore problem. Like, if you are typing along testing you are going to wear out your underscore key just defining the method name.

Sure.

And RSpec gets rid of that. The other thing that is nice about RSpec’s syntax, in my opinion, they describe blocks and stuff are actual classes and stuff. You can just go ahead and define methods in there if you need to or mix in some module or whatever.

So, I think a lot of that stuff exists in the continuum between DSL and API. I usually think of that as syntactic sugar in some way.

it’s interesting that RSpec is on the continuum far enough up that people actually complain that it’s too comprehensive, that it’s littering the root object with too many stuff, too many methods.

Let’s get David on the show and we’ll have this conversation with him.

Yeah, we probably need to have that conversation at some point. But, to be fair, I really like things like the describe blocks and content blocks and stuff like that. And the “should” syntax, they go a little too far in some areas and I think they pretty much know it because the recent version of RSpec is kind of taking a step back and trying to figure out something else.

I’ll just +1 that.

I wanna jump in here and basically, you know, with the DSL, I like the example of SQL being a DSL because it’s focused on solving different problems. You are not actually coding up the solution to the problem, you are giving the user, maybe even a non-programmer user some mechanism for communicating with your program in another way or expressing that in another way. And RSpec really doesn’t do that because you still have the full effect of Ruby in place and it’s design for programmers to use it.

Right. And I think the big difference is how you think about things, not what you are typing and that, good DSL gives you sort of on a whole new layer of abstraction to think about things. So, SQL, you are like “Oh, great! I have this concept of a join.” And that’s incredibly powerful abstraction for dealing with tables of data.

Right.

Yeah, but not true and complete. I mean, you don’t use Postgres or what? [laughter]

I wrote an adventure game in PoSQL to prove that it was true and complete.

I actually believe that it is actually true and complete these days. I may be wrong.

Maybe this is a canonical sample. Okay so are we done?

Actually, one more thought Josh. You mentioned that it’s a linguistic concept and that made me kind of realize that, yeah, so objects are all about sending messages to them. The API, you can think of the collection of messages as the API. Like, a collection of messages are the things that I can send to the API, but what is the collection of messages spread out over time? It’s a conversation.

Oh yeah.

And so when I look at the API of an object, I really am thinking, “What are the kinds of conversations I can have with this class or this object? With Net::HTTP it’s like, “I have no idea how to start a conversation I can complete here”.

That’s one of my favorite parts going back to GOOS again. They talked about how the main section, where you first create the objects and hook them up, they call that the match making section. So, you like introduce this object to this object and then they can talk to each other.

Alright, well I think we are pretty much at our time limit, so we’ll go ahead and get into the picks. Avdi, do you wanna start us off?

Sure. I'm going to do a little shameless self-promotion here and say that after many delays and going much longer than I expected to, my subscription screencast offering Ruby Tapas is now available. You can find it at rubytapas.com. The basic idea is that, it’s very short screencasts; multiple times a week focused on the Ruby language and standard libraries and basically intermediate to advanced topics in that area, language and library stuff that you might not know about. Also, idioms and stuff that ways of using the network well and idioms and patterns and some OO design tips and stuff like that. And like I said, you can go to rubytapas.com and check it out. I got one sample video up and there will be more. And apart from that, let’s see… I really thought I picked this before, but I don’t see it in the list. Since I’ve started selling books and now that I’m still on subscriptions, I’ve been using the DPD service for doing that sales. They have been handling my storefront. It’s a great service. They have been fantastic to work with. Every time I needed a new feature, they work with me again rolled out. So, I highly recommend them if you want to sell some kind of digital product. And finally, everyone knows I like to do drink picks and it is the time of year to pick Apple Cider! No, I'm not talking about hard cider; I'm talking about stuff that comes out of apples when you squeezed them. It’s getting towards the autumn season here on the East Coast of the US and my favorite time of the year and its Apple Cider season. And man, if you can get your hands in some Apple Cider, that is one of the best liquids I know of to drink. [laughs]

Avdi, this is actually the thing that I am going to miss most about living in Pennsylvania, this Apple Cider.

Yeah. If you find a good farm that makes it well and doesn’t mess with it too much, you got it made.

Heat it up, spice it. Good stuff.

Make apple butter.

My wife just made Pear butter last week, very good. Josh what are your picks?

Okay. I don’t think I have done this before. I'm picking an answer on Quora. This popped up recently. Somebody wrote this amazing analogy of describing database scaling problems in using analogy of library full of books. So the question was, “Why is it hard to scale a database, in layman's terms?” and Yishan Wong gave a really great answer. It’s gotten a lot of up votes. If you’ve not gone on Quora, this is maybe a good time to check it out. The other thing is Automattic, the company that runs WordPress and stuff like that; they have privacy policy that is great to copy. They put together a privacy policy and they released it under a Creative Commons license. The nice thing about it is that, in such clear terms that it doesn’t require a bunch in the way of explanation of what the privacy policy actually means. You can just read the privacy policy and that makes sense. So they put all these efforts in to it. They decided, “Oh, we don’t want everyone else to have to do a lot of work to figure out how to say this stuff in a way that makes sense. So, here share it.” And I have noticed it popping up a few places and we liked it, so we are using it on our stuff too. Kudos to them for being good citizens and sharing that. So I haven’t done an entertainment pick in a little while, I think. There is this cool series that I started watching on Netflix streaming a while back, which is a retelling of the Sherlock Holmes series just called Sherlock and it’s a modern production from the BBC. On Netflix, it only looks like its three episodes but each episode is like an hour and a half long or something like that. So, they have taken some shorter episodes and combined them into short stories. It’s cute because it’s all modern and it’s not all the same as the old stories and they have updated it for modern politics and things like that, things like text messages. But it’s pretty cute and got some good fan reaction. Anyway, it’s up. So I started watching that and I have been enjoying it a lot.

I think the actors from Sherlock are some of the main actors in The Hobbit movies.

Yeah, I think that’s true. I think there’s definitely overlap there.

Watson is Bilbo.

Watson is what?

Is Bilbo.

Yeah.

As a massive Sherlock Holmes geek, I endorse this pick. I’ve watched a few of them and I feel like they do preserve a lot of the spirit of the character. And they don’t make Watson a buffoon, which is nice because Watson was not a buffoon.

Yeah.

Alright. James what are your picks?

I’ve got a couple. Continuing on the service oriented architecture that kind of been on lately. There’s a good talk from Mountain West Ruby Conf this year about Service Oriented Architecture and he calls it Real Talk, which I may or may not be familiar with. But, that’s actually the good part of the talk is it’s a pro service oriented architecture talk where he first spends half of his talk telling you why you shouldn’t do it. [laughter] It’s really nice because he goes through and he says, you know, “Well, yeah, you need more RAM, blah, blah, blah”. He talks about a lot of things like that. Again he is pro. He goes and talks through the positive sides of it after this but I feel like he’s very realistic about it. You know, know what you are getting yourself into. And he disagrees, I would say, with the book that we are currently reading, Service Oriented Design with Ruby and Rails. So, you get to hear some contrasting viewpoints in there on a couple of points and maybe we’ll get to discuss those when we do that episode. It’s not very long; it’s by BJ Clark and its pretty good stuff, so you should check that out. And then this one because I'm going to miss it this year and I’m basically really sad and I need to make sure everyone else knows about it. The RailsRumble is coming up, which is where you take a week end to build a Rails application from scratch. You’re in a team or whatever. This is a really great event and they host (every year, I think) and there are prices. A lot of apps that people know where originally RailsRumble projects. So, it’s a great chance to like spring board something you have been playing with or wanna get off the ground. Scout App if you know that one, started this way. I did it with Ryan Bates one year, we built Go versus Go.

Friday Hug was RailsRumble project.

That’s right, Friday Hug. Yeah, so there are lots of great things that come out of it and people throw them in there and then some of them actually go big. So, it’s a cool thing to do if you have the time. This year it’s going to begin on Saturday, October 13th and you get 48 hours to build a Rails application from scratch. So, if you are not going to be in Hawaii that weekend like I am, you should seriously consider giving it a shot. Pick good team together and it’s a great chance to hone your skills and see what you can accomplish. And it’s a lot of fun. That’s it.

Awesome. Dave what are your picks?

alright. So real quick, I wanna pick the MG Editor. So, like, Mary-Graham or Mary-Golf, it’s just a command brew install MG. This is a very tiny Emacs-like editor. The advantage of it is it’s got all of the features and stuff of Emacs turned off, which means all of the startup time is also turned off. So, if you are like me where you doing a lot of work in Emacs and then you go to, like, push codes somewhere and you are not using magit mode to use Git from inside Emacs like me, it will open up VI to do your editing. And I get around VI just fine, but I prefer to do with Emacs. So, with MG gives me that ability. So, just a cool little Emacs-like editor. My second pick is, I'm actually going to pick a book I haven’t read yet, but I have downloaded it for my Kindle and if iOS 6 version of Kindle wasn’t crashing continuously, I would have actually started reading it today. It’s Practical Object-Oriented Design in Ruby: An Agile Primer by Sandi Metz. I met Sandi Metz at Rails Conf this year. She is absolutely brilliant. She got some good insight into this stuff and three or four people have recommended this book to me very, very highly, including Brandon Hays who tweeted that if it is not our next book pick, he will personally set me on fire. [laughter]

Which is a shame because I was going to make it our next Book Club pick, but now I don’t that I can.

Yeah exactly. So out of self-preservation, I would like to suggest that this be our next book pick. Out of the other four Rogue’s sense of morbid curiosity, our next book, would probably be mastering Visual Basic 4…

I haven’t thought of that. Good idea.

…just so that I can do the... what is it? Robert Frost? I lit myself on fire and people came just to watch me burn.

Yeah, let me add to that Sandi Metz pick real quick. While I was on vacation, I hooked up with a junior Rubyist. He is friend of mine and he is just getting into Ruby and learning a bunch of it and he was just gushing over that book. He said he is only about 75 pages into it but he was working through it real slow really trying to get all the code and stuff. And he is an old C++ programmer and he said he couldn’t believe how many “AHA!” moments he was having just reading through this book. So, I’ve heard nothing great things.

Awesome. Hasn’t that pick been picked so many times now?

Yeah, it’s like every week for the last month. [laughs]

Has anyone not picked it yet? Chuck, have you picked it yet?

I haven’t picked it yet.

Next week it’s your turn.

If you want, you can pick the video of her talk from GoGaRuCo.

Oh yeah.

But it’s not the same. Alright I guess it’s my turn then. I had an adventure this weekend end with my mac pro, it would just randomly freeze. Like all the program would just freeze up, like every single one. So, I can move the mouse around, I could move the windows around, I can click on things, but nothing would happen. And it was really frustrating. So, I wind up taking it in to… there’s a little Mac , it’s not a Mac store but its chain of stores here in Utah that takes care of this stuff. And it turns out that if you plug a UPS (Universal Power Supply Battery Back Up), if you plug that in to your Mac Pro and you hook up the USB, then it tells you that the battery backup is plugged in, but it also enables all of the power saving options in your computer. And one of those options that’s turned on by default is the hard drive, putting the hard drive to sleep. Well, it turns out that the firmware in my drive will allow it to go to sleep, but doesn’t respond when it tells it to wake back up.

Oh no.

So, it would freeze and sleep is what finally figured out the correlation was. So, my first pick is updating the firmware on your hard drive [laughs] and being aware of whether or not your hard drive supports that option because it was just a lot of pain. And also end up reinstalling the operating system trying to make it work because I didn’t realize that that’s what was going on. So my other pick is Time Machine because it made it really easy to just to bring everything back to where it was. Those are my picks. My last pick is something that I think we have mentioned a few times on the show and that is, my new VA and she’s just been awesome. And all of the stuff that I have been waiting to get done for the last several months, I just handed off to her and she did in a much more reasonable amount of time than I was getting it done before. And it’s been so, so super nice to be able to just say, “Hey, I need this done.” and not worry about it and just get a response back saying, “Okay, it’s done. And go look at it and just know that okay, yeah that’s done right. Just do it again that way next time”. I can’t even tell you how much stress it takes off to be able to do that. So, if you wanna find her she is @therubyrep and I don’t know if she has a website. Does she have a website Avdi?

It’s devreps.com. Twitter is therubyrep, I believe. Yeah, @therubyrep on Twitter and rubyrep.com.

So, you are welcome to hire her, but when she gets too busy, I will come find you and kill you.

As will I.

[laughs] Anyway, I just wanna---

Ruby Rogues may stop releasing regularly again, which would be the really tragic consequence, right?

But totally hire her.

Yeah absolutely.

Anyway, so those are my picks. I’ll put links to those on the show notes. And we’ll wrap this up. And just few reminders, you can sign up for Ruby Parley on rubyrogues.com. You can also sign up for the Nuby episodes that we are going to put together and audition for the show. And we will have that website up really soon and it should be pretty obvious where to go get to that. And other than that, we will catch you all next week!

Thanks to David Brady for being our special guest this week.

Eyyyyyyyyy.

He's almost pretty enough to pull it off too.