001 RR Testing Practices and Tools
The Rogues talk about Ruby testing practices and tools.
Show Notes
The Rogues talk about Ruby testing practices and tools.
Transcript
CHUCK:
Hey everybody and welcome to the first Ruby Rogues podcast. Today’s panel is made up of James Edward Gray, Aaron Patterson, Peter Cooper, David Brady, and I'm Charles Max Wood. If you don't know these guys, I'm going to give you brief introduction; James has been in the Ruby community for a long time. He was in charge with the Ruby Quiz. He also wrote Faster CSV and he helps out in the community in lot of ways. He also is one of the organizers of the Red Dirt Ruby Conference.
Welcome, James.
JAMES:
Thank you.
CHUCK:
We also have Peter Cooper. You can find his stuff at peterc.org and he's on Twitter as @peterc as well. He's the force behind Ruby Inside, Ruby Flow, and Ruby Weekly. He also wrote the ‘Beginning Ruby’ book for Apress. And he just moved in to a new office.
PETER:
I did.
CHUCK:
You like it?
PETER:
Yeah, it’s pretty good. I guess I just want to suffix everything you said there, by I like to think of myself as Ruby’s version of Michael Arrington, but just not as successful -- and not as charming, obviously.
CHUCK:
If one of us is Ruby’s version of Leo Leoporte, are we going to have a falling out here?
PETER:
Oooh, nice.
DAVE:
None of us gets to be Leo Leoporte.
CHUCK:
Okay. And that was David Brady. David is a developer that I’ve actually met in person. In fact, I think I've met all of you except for Peter. Anyway, David is here local to me. He wrote the Tourbus library, which is a great tool for testing and load testing your application. And he gets involved in a lot of things as well; I think one of my favorite things is his blog heartmindcode.com and he's actually done a couple of pod calls with several people, including some guy names James Edward Gray.
DAVE:
Yeah.
CHUCK:
Great stuff.
DAVE:
It was a disaster just like all the other calls, so I'm consistent.
CHUCK:
[Chuckles] Yeah. And then last but not least, we have Aaron Paterson. He works for AT&T interactive as an open source programmer. He's the only person on the Ruby core team and the Rails core team. And he worked with Mike Dalessio as the --- developers and you can find him on Twitter @tenderlove. He's also our resident, what is it mycology expert?
AARON:
Yes. [Chuckles]
CHUCK:
He's our mushroom guy.
AARON:
[Laughs]
JAMES:
I'm so glad we found one of those.
CHUCK:
I know. We've been needing some mushroom help on this podcast.
DAVE:
I need to talk to you later, Aaron. I’ve got this thing on my foot. I'll talk to you later.
AARON:
[Chuckles] I don't know about those kind of mushrooms, but I'll see what I can do to help. DAVE: Okay, so semi-mycology. All right.
AARON:
Yes.
CHUCK:
We got to be careful about what kind of fungus we talk about here.
DAVE:
And that’s practical mycology. [Laughter]
CHUCK:
All right, and I'm Charles Max Wood. I do screencasts and podcasts on various topics; most of them related to Ruby or technology in general. My main place for putting up content is teachmetocode.com. So, anyway, that's us.
JAMES:
Yay!
CHUCK:
All right. So let’s go ahead and get into the topic on hand. It kind of started out as a discussion over Test::Unit versus Rspec, but I think people are really just curious about what you do use and don't use for testing, and why you do or don't use them. So, let’s just kind of everybody chime in and talk about what you are using and why. Let’s start with Peter.
PETER:
Yeah. I guess because a lot of the stuff I have to do is a lot of reporting on stuff. I do some client projects and things like that, but I have to kind of try everything, because I have to write about it at some level or another. So, I definitely not going to be the more opinionated people here today, sort of saying, “Oh, I only use Test::Unit. I only use Rspec,” and kind of really defend that, just because I need to try so many different things, just so that I can get people a reasonable opinion on what I
think about each one. So I actually have used all… the main choices I have pretty much used all of the different technologies for like one project here or there. If I have to pick a personal choice though, I think, logically, I prefer Test::Unit, well actually all the reasons that DHH put forth, it’s more straightforward -- all that kind of stuff. But then, Rspec, I’ve used that as well. I use both, but not obviously on the same project because that will kind of suck.
CHUCK:
Right. What about you Dave? What are you using?
DAVE:
I'm a little bit like Peter; I'm kind of like a testing gourmand and I use whatever the guy before me on the project set up (because I'm lazy and I don't like to set up something new.) I'm going to come down to the opposite side of DHH just for the sake of controversy. I respect DHH and I like Test::Unit -- I have nothing against it -- but I love Cucumber, I love Rspec. And I have written unit tests with… oh, somebody just hung up. That’s awesome. [Laughter] I’ve written unit testing Cucumber and it is every bit as tortuous and wrongheaded as you might think, but it’s really illustrative to kind of see to break in and actually write a unit test, to see this is, “Oh, this is how Cucumber works.” And there is this extra layer of regexing and manipulating stuff this way, but there is not just a syntactic sugar, but kind of an eloquence that comes out of your testing, that I am desperate to sit with somebody that is really good at Test::Unit to try and create that same kind of eloquence, because I just can’t do it in Test::Unit and I can in Rspec -- and I definitely can in Cucumber. And so I feel my tests are much more readable. I feel much more confident to what I
am giving in the end user with those tests. So that's what I use. I'm going to insert a question for everybody else and we can bring it back around Peter if we want as well which is that, I committed a heinous, heinous, horrible crime and sin (and I think James is probably going to on this,) but tourbus has no tests and the reason why is it’s a testing tool and I'm not really sure how you test a testing tool. It’s very meta. The way I shipped it, so that you could test it is I shipped an application with it. If you go into tourbus in the source code into examples contact app, there's a little Sinatra app that you can run, and then you can run tourbus and see that it tests this app and everything works well. But I have no idea how to automate that. Probably, I can do it with Cucumber where I could actually go at a process and start the server and have something else test it, but I'm not sure how to test that. So, that’s my thing. And I like the higher level, I like the eloquent stuff even though there is a little more cognitive load that some more things you have to remember that are there. But I find that it lets you say, in one or two sentences, what you mean, (unlike the way I speak to people, because I’ve been monologing now for three minutes.)
CHUCK:
[Chuckles]
DAVE:
But the advantage to it is yeah, it lets you be very, very elegant, very eloquent. And yeah, very interested to hear other people find out how you would test a web testing tool?
CHUCK:
I think we are going to come back to that. I wanna get everyone’s opinion on this and then we can come back to that, because I think it’s an interesting thing to touch on. James, what do you use to test?
JAMES:
I usually use Test::Unit, just because it’s in Ruby and I'm pretty lazy, so you know, as far as getting other things. I use Rspec quite a bit mostly because I work on projects that help people set it up with Rspec, so I use that quite a bit. I do prefer Test::Unit over Rspec, which I think is pretty rare; I know a lot of people prefer Rspec. I like Test::Unit better because one time I was teaching someone how to do tests and I show them Test::Unit, I showed them Rspec and some different features in them and I can’t believe how bad it went when I switched to Rspec. Maybe that I did really bad job, I can accept that, but what surprised me about the whole should syntax with it’s magic and your should dot equal equal and things like that, or things like ‘should b_include’, you know, instead of ‘include?’ and just calling the method. I basically fell off a cliff when I started trying to explain that to a programmer, that was just getting probably intermediate level knowledge and it was funny how much easier it was to explain Test::Unit where I was like, “You just call these methods. You wrote these methods here, you call this method here and you check if the result of that method is what you think it was going to be,” whereas when I was explaining the same thing in Rspec, it was like, “Okay, you now use these three magical constructs to invoke that method that you just wrote. I know it doesn't look the same, but is the same.” And you know, I personally find that very difficult to explain to other programmers. But at the same time, I do like things about Rspec but obviously, I prefer being able to define context and nest them just put simple strings in there instead of wearing out my underscore key, when I name a long method name in Test::Unit. So, there are things I like about Rspec, but the should syntax is one of the things I definitely don't. And so, I typically just use what’s with Ruby. Although in 1.9, Ruby does come with Minitest, so I use that more now than anything else, which is kind of a nice little advancement on Test::Unit, I think.
PETER:
+1.
CHUCK:
Interesting. And you Aaron, what are you using?
AARON:
I use the test framework that provides me with maximum trolling ability. [Laughs] And so, whenever I enter into project, if they are say using Rspec, I do my best to switch them completely to Test::Unit or vice versa.
CHUCK:
[Chuckles]
DAVE:
What do you have installed on the production server? That's where I'm going to test.
AARON:
Yes. [Chuckles] No, I'm just kidding. I have to mention too that where I am, it’s close enough to lunch time now, that I am having a --- for lunch to enhance this podcast. CHUCK: Oh man, you’re lucky I just had lunch or I’d be jealous.
JAMES:
We may have to start with, “What are you drinking?” segment. [Laughter]
AARON:
Okay, so I typically use Test::Unit… or actually I use Minitest, I don't use Test::Unit. And I don't know, I don't use Rspec because I find it to be too wordy and I don't know, I just don't like it. It’s not for me. I used to be a Java programmer, so I enjoy xunit… Oh, my god, did I say that? [Laughter]
JAMES:
Please remove him from the call.
AARON:
I was never, ever a Java programmer. I’ve never done it in my life.
CHUCK:
[Chuckles]
DAVE:
I don't understand the religion that you just said there because xunit is Test::Unit; they are identical.
AARON:
Yeah. Well, I don't use Test::Unit; I use Minitest, which in 1.9, Test::Unit is actually subclass of Minitest, so it’s almost exactly the same. The only difference is really are you get different assertions and actually one of the reasons I love Minitest is it does test driven randomization, so your test will run in a random order, which means that you can… I'm sure you’ve been in a situation where your application, like one particular test fails in isolation, but when you run with --- it works, right? And that’s typically because one test like setup some particular environment that another test depended on, you didn’t know that when you writing the test. Since Minitest runs the test in a random order, you can’t make one test depend another. So you'll see and error case.
DAVE:
I have a stupid question; is now the time for it or should I wait for people to go through their things?
CHUCK:
No, go ahead. I think I'm the only person who hasn’t chimed in yet, so go ahead.
DAVE:
Okay, so my stupid question is Minitest would actually drive me completely up the wall if I had one random ordering that triggered the bug and none of the other ones do. So here’s my stupid question: when my tests crash, can I tell Minitest, “Run that again, but do it in the same order.”
AARON:
Yes, you can. When you run tests with Minitest, you'll see a debug output that tells you the seed that was used. And you can just apply the seed when you run the test.
JAMES:
Kind of a -- to some of the things Aaron just said, having learned the little Minitest lately, there's some other neat things about it; like it goes through and all the positive methods are equal and things like that but the negatives are refute and then equal or things like that. And what's nice about that whether or not you like a word or not, it doesn’t really matter, but what's nice about it is its consistent, whereas with Test::Unit, I'm always trying to guess the name of the opposite method, and I'm sometimes wrong about it, whereas Minitest is more consistent. And Ryan also added a feature to it while he was at Ruby Conf I think called benchmark, is that right? Minitest benchmark.
AARON:
You can write benchmarks in Minitest -- which is pretty nice.
JAMES:
Right. But there's also what that new feature? It’s got like assert linear, and so you can…
AARON:
Basically you can do fits, you say assert linear and you provide it with a function and it asserts that. As the number of iterations grow, your performance should also stay linear. So let’s say somebody implements a function, they change the function to be n square or something like that, then this test will fail.
CHUCK:
Wow. That is really cool.
JAMES:
It is cool.
CHUCK:
We've been talking for 10 minutes and I'm almost convinced to switch over to Minitest.
JAMES:
It comes with Ruby 1.9.
AARON:
The other cool thing too is if you like the spec syntax, there is Minitest spec which provides most of the functionality that Rspec provides, so you get describe blocks and it blocks and nested describes and all that stuff too, so you should check that out.
CHUCK:
All right. Those definitely sound like things to look into. I'm going to chime in real quick and mention what I’ve been using. I’ve actually been using Rspec, but I started out using Test::Unit; people started talking about Rspec and it was before I really got to know Test::Unit. It’s funny if you look back on my blog far enough, you'll find an article that says, “I love the Rspec. DLS is basically what it came down to”. And there’s a comment on there from David Chelimsky that actually says, “Yeah, but Test::Unit has a cool DSL too.” And I just didn’t know about it at the time, but that’s why I switched to Rspec. And the whole debate over Rspec versus Test::Unit got me thinking why am I using one tool over the other. And I don’t have a strong reason for using Rspec over Test::Unit or Minitest, but I did give up my Cucumber habit. And that’s primarily because I just started using Capybara in my tests directly, and I found that it actually gave me like a 2-3 times speed up over writing the Cucumber tests, and then figuring out how to get the regular expression to give me what I wanted.
DAVE:
Interesting.
CHUCK:
I think Cucumber was useful when I had stakeholders that cared what my specs look like, but I
don’t see that a whole lot with my clients. Most of them just want a website that just looks nice and does just what they want. Anyway, that was one thing that I thought was interesting whether or not to use Cucumber. As far as Test::Unit versus Rspec, I found that one is a sweet flavor and one is a salty flavor -- and I like one more than the other sometimes, so haven’t had a strong feeling for that.
But it sounds like Minitest has some cool features there.
AARON:
So, I hate Cucumber. [Chuckles]
JAMES:
Me too. Yay!
AARON:
[Chuckles] I actually wanna talk about why I hate Cucumber.
CHUCK:
I was going to ask that.
DAVE:
Please. This call is going to get badly for me, I can tell.
CHUCK:
[Laughs] Go ahead, Aaron.
AARON:
It’s not that I really hate it so much. I think the idea is cool. It’s nice that you can write it out on kind of English senses like it does this, it does that, etcetera, etcetera. And you can show that to your stake holders and what not and I didn’t really have too much of an opinion about Cucumber until I had somebody on my team writing Cucumber specs. He was a QA guy and wrote some Cucumber specs. And everything was going fine until it didn’t. So he had a hard time. He couldn’t figure out why some particular Cucumber spec was failing, and to debug that, just killed me. He couldn’t figure out who to do it. It took me hours. I spent hours trying to debug the Cucumber failure, and that’s what really killed me.
JAMES:
I always find it to be like one of the most ironic things in computing that people say things like, “Never, ever try to parse a piece of HTML with regular expressions,” and then they turn around and write Cucumber specs. [Laughter] Which is basically like the identical thing.
AARON:
[Chuckles] I don’t wanna say just bad things about it; what I did like about it is that we actually had QA guys who could just write out stories about stuff, that was actually awesome. My real main gripe about it was debugging the specs when they broke.
DAVE:
And that’s the thing that I champion and I wanna stand in my desk and pound my chest and then pee on anybody that disagrees after I say it is that we talk about being Agile about having onsite customer, about having stakeholders involved, and the arguments against Cucumber will say, “You know, it’s a fantastic tool for getting stakeholders to the table, BUT…” and then they spend the next 5 minutes completely dismissing the value of that. And I'm like, “No, no, no, you don’t understand. That is so valuable.” And I agree with DDH that he says he sees a lot of programmers using it and they are not showing their stakeholders. And that’s dumb. That really is dumb. And if you are showing your stakeholders, if you’ve got a marketing guy writing specs or if you’ve got the CEO, if you are in a startup who he is a salesman and all of a sudden he realizes, “I can write my own spec for how this thing works,” the light goes on and it’s just an amazing developmental tool. That said, yeah, it is regexs all the way down, and that part of that implementation is kind of sucky.
CHUCK:
So I have to ask you then, Dave, since you’ve have been using Cucumber, have you had a stakeholder actually get involved and start writing Cucumber specs?
DAVE:
Yeah, Mike wrote them on Public Engines.
CHUCK:
Oh, okay.
JAMES:
So, do you find that that went fairly well or did he regularly get off the tracks where the…
DAVE:
All the time. The Cucumber’s primary used case…. Oh, this is going to sound bited. [Laughter] Cucumber’s primary used case in my experience -- speaking as a guy who likes Cucumber -- is that, it is not that it gives you a document showing how the server is supposed to work, but it really throws the rubber on the road at all the misunderstandings between you and the stakeholders, in the way that things should work. So Cucumber is very, very painful to get set up and to get working, and that’s a feature. That is absolutely is a feature, because the pain is coming out of the misunderstandings between you and your customer. It’s not accidental pain. It’s not accidental complexity. You are designing it wrong and you are implementing it wrong. Your domain model stopped at the API and did not get carried internally and that’s why you are having a hard time writing these specs to make them cooperate.
CHUCK:
Wow.
AARON:
Yeah. But to me, it sounds like a crutch. And it’s really a crutch that’s showing that you have a broken communication between you and your stakeholder.
DAVE:
Yeah. I am going to tentatively agree with you. I think it’s a crutch, and I think the leg is broken and so a crutch is a good idea, but I do agree with you that if there was anything better, that the field is open, yeah. I will concede that.
AARON:
I just feel like if I’ve spent more time improving communication between me and my stakeholder, and less time debugging Cucumber specs, I would gain more.
DAVE:
I completely agree with you. And if you don’t use Cucumber, you won’t do the other thing. [Laughter] That’s the danger.
AARON:
I respectfully disagree.
DAVE:
Okay, okay. I haven’t actually met you. This is actually the first meeting between Aaron Patterson and myself. This is danger zone time for both of us, but hopefully we are both wearing clothes. [Laughter] But you very well may be right. Your reputation precedes you. The vast majority of programmers I tend to find that they go to their laziness as their first tool, and Cucumber forces you out of that. And so, I'm making a bad case for Cucumber. I'm almost making the case that it’s good because it sucks, and that’s not the case that I want to make.
AARON:
[Laughs]
DAVE:
It plows head long to a pain point that needs to be addressed. And most developers don’t address it; and you probably do and good developers do. You are absolutely right.
AARON:
One thing that I did in one of my jobs is I would get the business people that actually come pair program with me and we didn’t have…
JAMES:
That’s an awesome idea.
AARON:
We didn’t have any Cucumber specs, I would just walk them throughout Test::Unit once and I’d be like, “Okay, let’s talk about what this test does.” And we would read it together and then it was actually really fun because she found bugs in my implementation and I was like, “Yes, okay.”
JAMES:
I’d feel a lot like what Aaron is saying there that basically as the programmer, I know we are famous for our horrible social skills, but one of my jobs is to translate between their ideas and the program that makes that happen.
DAVE:
Yeah.
JAMES:
That’s what I try to do when I'm working with my customers.
AARON:
I think one of the whims of the Cucumber though is that it facilitates that communication. It kind of breaks down the barrier that, so we can pair program with a business person, right?
DAVE:
Yeah. That’s the better way of putting my statement, that it breaks down that barrier to the communication and the first thing you realize you have to do is have an argument, because you totally misunderstood each other. Absolutely true.
CHUCK:
So basically, what you are saying is that the secondary feature of Cucumber is that you can write tests with it?
DAVE:
Tertiary, yeah.
CHUCK:
[Chuckles]
JAMES:
And the first feature is that it allows you to communicate in a very simple, adventure game kind of way.
DAVE:
It brings business people to the table. That you show them a Cucumber spec and they go, “Oh that’s a nice document,” And you say, “No, this is executable code.” And they say, “Holy crap!” And they get it and they are like, “I wanna type. I wanna type.” And you just won the business use case right there. But yeah, there may be better tools.
AARON:
Until you have to debug it and then you start [inaudible] [Laughter]
CHUCK:
All right. Well, I'm going to turn the discussion a little bit. I mean, we are certainly getting into stakeholder involvement and we really, I think the purpose was to talk about testing tools and I think this is all valuable, but I really want to get into David’s question about how do you test a tool that’s meant to test?
JAMES:
Not to stop that question again, but can I mention one more tool that I thought of, that I forgot to mention?
CHUCK:
Go ahead.
JAMES:
If you wanna have a fun afternoon some time, go play with Riot. It’s a pretty neat little testing framework and I’ve used it on a couple projects now. The reason I think it’s cool is it’s kind of a little bit of a lot of different testing frameworks, but one of the things that’s kind of neat about is it is built as being very fast and stuff -- which it is and that doesn’t really matter to me that much -- but it kind of turns some of the typical testing concepts around a little bit; so like the closest thing it has to a assertions are basically this block of code and when that block of code returns true, it passes and when it returns false, it fails. And what’s neat about that is it rarely makes it impossible to do multiple assertions in the same tests because you only get one return value -- true or false. Another neat thing about it is you can kind of set things up and then you can just run things on this object that you’ve set up. So anyway, what I think is neat about it is how much it’s changed how I think about testing in just a short time playing with it. So if you wanna have a fun, mind expanding experience, sit down and try to test a project with Riot; it’s pretty neat.
AARON:
But what if you need to do two assertions?
JAMES:
Then you would write another test.
CHUCK:
[Chuckles] [Overlapping talks]
CHUCK:
Then you use your return value and then you do assertion one and assertion two.
JAMES:
That’s right.
DAVE:
This is a completely side topic, but one of the things I'm getting into recently is this notion about East-facing code that James Ladd and Sean Allen are doing, and getting into the way like Smalltalkers think about code. And what I'm finding is that the way most of us do object-oriented programing is extremely stateful and extremely procedural. And I'm getting people from the Smalltalk community saying, “Dude, you don’t know how to do OO.” I said, “You need to back that up because you are saying that nobody knows how to do OO, except for like 2% of the people out there.” And I haven’t got a definitive answer on that, but I'm willing to listen to what James just said on account of it’s falling into that same weird groove that these Smalltalk OO guys are freaking me out with right now. So it’s something interesting to me.
JAMES:
I've worked with a new guy in the past a bit like you’re talking about, using RestNode getters and setters and stuff like that. And I've tried that in the past year and every time I try it, I built software that way for a while and it just ends up just feeling a lot less practical to me, and I think I’ve always been switching back to my ways of putting getters and setters and…
DAVE:
--- ways. That’s the right word. That’s the word they used too.
CHUCK:
[Chuckles]
DAVE:
I'm definitely playing with it though, so it’s interesting.
JAMES:
Okay, sorry. No more tool side track. Back to David’s question.
CHUCK:
It’s all good. So yeah, I mean I think what we’re talking about here is it’s not just testing testing frameworks, but its testing like large frameworks that are meant to be able to do a lot of different things. I mean, who knows what you are going to be testing with that testing framework, right? It’s like testing Rails; I mean, where do you start? But you know, so I'm kind of interested in how do you test the tool that has that wide swath of what it should be capable of doing?
AARON:
I´ll start. [Chuckles] I don’t know. Typically, what I do is, this is going to sound really lame, but what it do is just break the problem down into smaller problems and test those smaller problems. I mean, that’s the really the only way you can deal with it, right? Look at the larger problem you have, break it down into something smaller and test that particular thing. If you are having problems thinking about how to test this whole big thing, it means your thing is too big. You need to break it up, decouple some things.
DAVE:
You heard it here on Ruby Rogues. My thing is too big.
AARON:
Yeah. My thing is way too big. [Laughter]
DAVE:
Actually with tourbus, I started doing that. I started breaking down and breaking it down and before I got it small enough that I can get it under test reliably, I was so close to the implementation detail, that the tests were just insanely brittle. And so it really was kind of a tricky problem. It really made me wanna look to a tool like Cucumber, where I could say given a running Rails app or given a Sinatra server on port 3000, and then you could actually have full process acceptance level, test planning at the highest level. But yeah, as soon as you step down smaller than that, all your tests are very implementation specific at that point.
CHUCK:
I think it’s interesting though that we’ve had Aaron chime in and say go way low level, way low level and then Dave come in and say, “Yea. I really like to go high level.” I think there’s a good mix there.
AARON:
The thing is, you don’t just stay low level, that’s the problem. You do it, you test small bits and then you need to move up from that and make sure those work well together. Back to the Cucumber thing, whenever we had a Cucumber test fail, to me that indicated that we had fuller coverage in our smaller scale tasks, right? So what I´ll do is just spend my hours crying, figuring out what the Cucumber tested and then fix that in our units functionals and integrations, and then move on with my life.
JAMES:
Real programmers admit that crying is involved.
CHUCK:
[Chuckles]
AARON:
Yes.
DAVE:
Always.
JAMES:
But there are things that are extremely difficult to test. For example, I was working on a project a while back that was a game on a system and it forked off processes to do it where it is going to communicate it with signals between the processes and stuff like that. And testing that kind of stuff is so hard, like forking off a process in the middle of your Test::Unit run and then getting that process to do the right thing, but meanwhile, kill it so it doesn’t finish running tests or things like that. I found that just trying to test that, I was writing way more code that I was to actually do the application. So I guess that in going to kind of throw it one step further. Is there things it’s okay not to test?
CHUCK:
That’s an interesting question. One thing that comes to mind when you ask that question is Rails gives you unit tests, functional tests and integrations tests and I always write as many unit tests as I feel like I need and then I test just about everything else with an integration test. I don’t actually write the functional tests because unit testing skinny controllers is kind of what Dave was talking about where you were just testing implementations. Did it make this fall, did it do this, and did it set this variables. It seems kind of pointless to me. And ultimately, I wanna know what it does with real data.
DAVE:
I do the same thing by the way, Chuck. I test every odd numbered level, I’m not kidding. I test it unit level, but I don’t test my views. I test it at the integration level, but I don’t test my functionals.
JAMES:
So yeah, I totally agree. I never test Rails views. I think that’s insane. It means that whenever the designer goes in there and makes a change, I have a bunch of broken test that I just get to go changed and they did nothing for me, you know?
DAVE:
You know what that is though, James? I think that might be cultural. Because when Rails 1 came out, it was still acceptable to have hideously fat views, and so it might have made sense back then to test your partials and to test your views.
CHUCK:
I wanna chime in here a little bit too, because I mean we are talking about not testing views and not testing controllers, but ultimately, we are, through our integration test. So, is there anything that you it’s okay to not have any coverage on in any way? I mean, things that are too complicated or certain types of problems or certain types of implementations that you just don’t wanna test?
JAMES:
I´ll answer that. I think the answer to that question is yes. And I think programmers have a hard time admitting that sometimes. But yeah I think tests is about risk trade off. So you’re saying , “I’m going to write these tests because I think it will help me write better code.” But in the cases where that’s not true, I think its okay to make the trade off. So for example, in that complex scenario where I’m forking processes and communicating with signals and stuff and things like order that the events happen in matters, for me to write tests on that is incredibly complicated. And so I have to decide, “Is it worth it for me to build that incredibly complicated infrastructure?” And maybe there are scenarios where it is, but I think there are scenarios where it isn’t.
CHUCK:
That makes sense.
DAVE:
My rule of thumb is I basically consider how much time I’m going to spend testing, versus how many times the tests are going to save me down the road. How many of these tests are migrations, other than by inspection. And in one case where I was writing a set of migrations that were meant to be run continuously to integrate a production server, Chuck you worked on this, this was at Public Engines…
CHUCK:
I remember it.
DAVE:
We were continuously migrating… crap is coming in to the old server and we had to keep the data server current with it, so we were migrating hourly. And so I tested the hell out of those migrations. I mean, I had 10-20 megabyte fixtures that would set up just evil databases and retarded databases to try and migrate because they occurred every two hours and in the real world. But when I wrote a migration to rename a column in the database that just the developers needed to run, I thought, “I can spend 20 minutes testing this or I can spend five minutes debugging this on everyone’s work station combined on the development team, not worth it. I´ll just throw it out there, if it breaks, I´ll debug it.”
CHUCK:
Yeah. And that makes a lot of sense. I’m going to have to kind of wind this down. I really wanna stick to our time limit. I don’t wanna this to go over an hour, and were already at 40 minutes on this recording. So I’m actually going to bring us into our next section and this is the panel picks. The idea behind the picks it really could be anything; if you found a cool tool that you use for coding, that’s awesome. If you have a nice ergonomic chair that you wanna tell us about, that’s cool. If there’s some music that gets you in the groove. I mean, whatever it is, if you’ve found some books or some iPad app or anything like that, whatever it is that kind of gets you excited, that’s what we wanna hear about. And just for a couple of minutes, tell us what it is and you know, tell us why it’s cool. So let’s go ahead and start with James and then we'll go around the panel.
JAMES:
So, whatever cool thing I’m into right now?
CHUCK:
Yeah. Just anything. If you’ve got some any cool coding tool, then that’s cool. But it doesn’t have to be tech related. It can be anything.
JAMES:
It can be anything. Oh, geez. Let’s see. That’s a good question I was totally unprepared for. So I’ve been playing a lot with tmux, which is a lot like Screen if you are familiar with that. And I’m finally getting into a point where I have some pretty cool tmux skills, and so it’s one of those things where I’ve seen it over and over again and every time I spend a little time with it I'm going to learn it a little. It seems a little intriguing but I've actually recently taken the time to sit down and learn it well enough. And I find that the further I go in it, the cooler I’m getting -- or maybe it’s just the cooler I think I’m getting or something. But anyways, I do things now like I fire up tmux sections and then I
split the window a bunch of times and I´ll have my Rails server going on one and script console going on one and a bunch of thing. And then I´ll end up switching to something else, so I just detach form that session and I can create another session and do that or attach to different ones. Or I´ll end up weaving all together, so I´ll just detach and leave it in the background and when I come back, I reattach to it or move windows around or kind of like split windows out into their own panes and things like that. So if my focus changes, I can shift things around in my terminal. I don’t know. It makes me feel like I’m in more control of all these data that I’m keeping track of as I'm messing with an app. So it’s making me think about things in new ways and I’m enjoying that.
AARON:
Oh, I have to plug tmux too. It’s awesome. I work remotely, and so I need to do like remote pair programing and we use tmux for that and it’s just great.
CHUCK:
I wanna chime in here. If I hadn’t worked at a company where they were using screen, I wouldn’t know what you guys are talking about. Do you wanna just briefly tell people what tmux is or what screen is, so they’d get a better idea?
JAMES:
Sure. It’s a terminal window manager -- for lack of a better way to describe it. So in your terminal, basically, if you run screen, that comes in Mac OS X. Tmux you have to install it. If you use homebrew, you just brew install tmux. But it’s a terminal window manager, so it’s kind of weird when you are first getting into it because you are like, run screen in it, it might look like it refreshes your terminal and nothing else happened. So its almost like you just put a layer between you and your terminal -- which is actually basically what it does.
DAVE:
Exactly what it did, yeah.
JAMES:
Right. And then you can do things like tabs in screen, which are basically these virtual windows that you can switch back and forth with easy keyboard shortcuts. And the keyboard shortcuts were all kind of designed around one key, so they are useful… it doesn’t take too long to get the hang of them. And then you can do things like split the screen horizontally and vertically. And like I said, as you can get more advanced you can actually move screens around so you can take this pane you’re doing over here for a little bit, but then you figure out, “Oh I’m going to be working in the console all day finding all these problems,” so then you can split that pane out into its own window and let it take up its own space and kind of promote it. I also have shortcuts where I can hit a keyboard shortcut and type in a term and it will search the man pages for it, and pop that up in a separate window and then when I’m done, I quit out of that man page like you normally would, but because I put that in a separate window and that was the running process in it, as I quit that, it’s just done and I go back to my normal windows where I was. And I have other programs like that like top if I’m looking at processes or SSH if I need to connect into a server, things like that. So you can kind of put them on these temporary window context and then when you leave on that, it’s just over and you don’t have the underlying shell still sitting there, waiting to hear from you. And then the other thing that’s amazing about is being able to detach. So like if you go on to a server and you are going to do some long running job, I just go on to the server, fire up tmux, launch the right process, detach and disconnect from the server. Then whenever I wanna check in on it, I just reconnect and reattach to the session and it’s like my screen is right where it would be. So like if that job is printing output as it goes, it’s there. I see the output and where it’s at and things like that.
So it basically just gives you a really amazing level of control over the processes you are interacting with.
CHUCK:
That is really, really cool.
DAVE:
The real value of that last bit of being able to detach really shines if you’re on a flaky connection, where detaching might be something that happens to you, rather than something you choose to do.
JAMES:
Exactly. It’s basically like you never get screwed. If your connection goes, you just wait till its back, then you log in, you reattach and you are good.
DAVE:
Yeah.
CHUCK:
That’s awesome. All right. I’m going to chime in next with my pick. And this isn’t a coding thing, but it’s something that I actually bought and tried out this week and it worked awesome. Yesterday, I had an interview over Skype with Tom Preston Warner from github.com, he’s one of the cofounders over there. And I bought this program. I actually recommended it to Dave, Sight Unseen and said you might wanna try this for his pod calls, it’s called Ecamm Call Recorder and it is awesome. It is so cool. And so I recorded the whole interview with the call recorder, and then turned around and opened it up and it put me on the left and him on the right, and so you could see us both talking and moving. It was really cool. So I just wanna kind of throw that out there. Something if you’re looking at recording Skype calls, not even for a podcast, but just for something else. If you like having that video recording of whatever people are doing in the room, it’s really, really cool. So let’s go ahead and ask Aaron. Aron, do you have something cool you wanna share with us?
AARON:
Mindcraft.
JAMES:
[Chuckles] That’s awesome.
AARON:
I mean Minecraft. CHUCK: Is that a game?
AARON:
Yes. [Chuckles] have you not heard of this?
CHUCK:
I think I’ve heard of it but I’ve never played it.
AARON:
Oh yeah, don’t play it because you can’t stop. [Chuckles]
JAMES:
Give us the spiel Aaron. Give us the twenty minute... tell us about your addiction.
AARON:
It’s basically like everything that I used to love about Legos, but now that I’m an adult, I don’t wanna clean stuff up, so I just do it on my computer. I don’t have to clean anything up. [Laughs]
CHUCK:
I could just see Aaron say, “Hi, I’m Aaron and I’ve been using Minecraft for two years…”
DAVE:
[Chuckles] Hi, Aaron!
AARON:
[Chuckles]
CHUCK:
So what is it like Legos? Is there more to it than that?
AARON:
It’s basically like Legos, except there are other stuff that happens in the world. Sometimes, there's bad guys, but I’m a very… I’m a very scared cat, so I shut off bad guys. [Chuckles]
DAVE:
I am with you. I am with you. I turn off the baddies too.
AARON:
[Chuckles] I just play without the bad guys because I can just build stuff. Currently, I have a cactus farm. I’d been farming cactus. And I don’t know. Its super fun.
CHUCK:
Farmville for Lego maniacs?
AARON:
Yeah, basically. But there’s no achievements. So the only thing you really play for is for the sake of building things.
JAMES:
And then you bring your friends in and show them what you created?
AARON:
Exactly. It’s kind of like the newer version of building train sets at home. [Laughter] And you don’t need a garage for it or anything.
CHUCK:
So is it a desktop game or an online game?
AARON:
It’s desktop game written in Java, actually. And I have heard that you can hack it with JRuby. You can mod it with JRuby supposedly, so that was awesome. But I haven’t tried it yet because I am too busy farming… cactus. [Laughs]
CHUCK:
[Laughs]
JAMES:
Not mushrooms?
AARON:
And mushrooms, too. But you can’t farm the mushrooms. At least I haven’t looked up how to do that. You can find mushrooms, but you can’t farm them.
CHUCK:
Yeah, but he can identify every single one.
JAMES:
That’s true.
AARON:
I can. That’s true. I feel like I should recommend something other than a game.
CHUCK:
No, that’s great.
DAVE:
Yeah, I was going to recommend Portal 2, but thanks for screwing that one up for me.
CHUCK:
[Chuckles]
AARON:
I didn’t know it. So I didn’t know what Portal 2 was until Eric showed me some videos of it, and now I really want it. I guess my PlayStation doesn’t work now or something.
JAMES:
It works on the Mac too.
CHUCK:
Yeah. Just don’t try and play it over the PlayStation network.
DAVE:
Oooh.
JAMES:
Ouch.
AARON:
Yeah, to be honest, I’m not really a computer game player, like I got Minecraft and that is the only computer game I’ve played as far as I can remember, except for when I was a Linux user, this game called Crack Attack. That was super addictive. Has any of you guys played…. what is that Mario game with the pills.
CHUCK:
Doctor Mario?
DAVE:
Doctor Mario.
JAMES:
Yeah.
AARON:
Yeah. It’s basically Doctor Mario, except an OpenGL implementation of it. And I was super addicted to that game, so. [Chuckles]
CHUCK:
Interesting. All right. Well, let’s go ahead give Dave a chance. Are you talking about Portal 2 or something else?
DAVE:
No. We already have one game, so I’ve got two things that are bouncing around on my cranium. One of them is Portal 2, which is just freaking amazing. Just go to YouTube and look for Portal 2 official and just like find the official trailers for it. They're crazy. They are hilarious. So the other that is bouncing around my head right now is… I’m always doing crazy, --- field stuff and right now, I’m going through a second reading of Learned Optimism by Martin Seligman. It’s a book optimism and about kind of the stories that you tell yourself when things happen. It’s just astonishing how easy it is when something goes wrong to say you are bad person or this always happens or this happens in every area of my life. And yet people who are optimistic will just kind of say, “Oh, that just happens once,” or “it’s just in this one area,” or “It was just this thing I did. It wasn’t part of who I am.” And what’s really, really interesting is that people who are inherently optimistic will do the minimization whenever something bad happens, but when something good happens, they’ll say, “Oh, I’m awesome and this happens everywhere in my life and it happens all the time.” And pessimist do exactly the opposite. And it is everything you need to manufacture a depressive state in your body and in your mind. And what this has to do with coding, it’s very pervasive. It either applies to everything or it doesn’t have to do anything with it at all -- depending on your view, I
guess. But I am simultaneously very optimistic and very pessimistic; when things go wrong, I blame myself. When things go right, I give myself credit. But when those two things fight with each other, they short circuit themselves. And so, just a fantastic book that I’m reading right now. It’s called Learned Optimism by Martin Seligman. And if you are prone to negativity or prone to feeling bad for any long period of time, you definitely need this book because he will show you literally how to hack your brain to stop being so negative and stop feeling miserable. So that’s what is interesting and new for me.
CHUCK:
Nice. It sounds like an interesting book. Both you and Brendan Hayes recommended it to me and I was highly disappointed to find that there’s no Kindle version for it, but I’ll have to get it up anyway.
DAVID:
Be more optimistic.
CHUCK:
[Laughs]
DAVID:
As long as you are not telling yourself, I couldn’t get the book because I’m a bad person.
CHUCK:
No. I’m a bad person, so I couldn’t get the book.
DAVE:
Yeah, yeah. CHUCK: All right. Peter, do you have a pick for us?
PETER:
I’m going to do a few things, but really, really shallow. So quick-fire list. I’m reading one of those sort of pimp RailsCasts by Ryan Bates. Excellent stuff. Always love all of those, always learn something new even its really simple what he’s showing you how to use it, it’s just great to see what he does. Just love that. So digestible, so good. Another thing I really like is Spotify. I know you probably can’t get this in US; you kind of got Pandora or Radio and a few things like that, its basically a system where you can pay like $10/month and I´ll get access to like a back load of like hundreds of thousands of albums, I can play them whenever I like in like an iTunes style client. I haven’t bought music since, so that has been great. Saved me tons of money and I can listen to… like if I wanna listen to the whole genesis album, I can just click on it and then bam, listen to the whole thing. So that’s come in really handy for like discovering new bands and new music to cope to and stuff like that. And then I have two books; the first one is Eloquent Ruby by Russ Olsen. Which is awesome. It’s just like 30 chapters, about 7-8 pages each and it just picks on like a single like how to build DSL using method missing or something to do with testing or just picks one thing, digs into it, shows you an example and goes from there. Really enjoyed that and had lots of positive feedbacks on that people from Twitter and that. And the one that perhaps you may or may not have heard of is a book called the Linux Programming Interface by Michael Kerrisk. It came out a few months ago. NoStarch published it. Its beautiful book. I think it’s about 1200 pages, really heavy. But he kind of does the same thing as Eloquent Ruby, but for the whole kind of systems programing side of Linux. So I think there’s something about 70 or 80 chapters in there and they are also like 10-20 pages each. And you know, you literally have a page about how process launch in UNIX, how massage queue works, you know, how a socket works. It’s kind of like a really small how-to in every single element of the behind the scene things that are going on in Linux -- and UNIX more generally. And I’ve just really enjoyed picking through that chapter by chapter; learning the ins and outs of each part of Unix. And even if I don’t use it, it’s good to know what does happen when you create a daemon process and all that kind of stuff. So I really enjoyed that. So like a $100 and I actually did put down the money for this. I didn’t get a free sample or something and just loving it so far. Those are my picks.
CHUCK:
I was going to say, I think you just spent about $150 of my money.
JAMES:
That’s what I think. I’m sitting here writing it all down like, “Holy cow, that sounds amazing!”
CHUCK:
Yeah. I’ve actually been writing down anything is linkable that has been mentioned for the last half of the podcast. And so hopefully, we get most of this linked up in the show notes. And yeah, so lots of good stuff. We need to wrap this up, because I really do want to keep this to an hour and so we got about 4 minutes left. I wanna thank everybody for coming on to the show. And next week, the plan is to talk about some of the Ruby VMs out there, and the kind of direction that Ruby can take in the future. And this is something that was recommended by Aaron. I’m really curious to see where that goes and what everyone’s experience is with some of that stuff. So, anyway.
PETER:
We'll do our homework.
CHUCK:
Yeah. So check in next week and then we'll get that up. I’m hoping to have the site up this afternoon and get the feeds submitted to iTunes, so that we can get out there and help people listen into this. As soon as it’s up, it’s up. I´ll let you guys know, so you can tell people where to go to listen and download at the very least.
JAMES:
Awesome.
DAVE:
Cool.
001 RR Testing Practices and Tools
0:00
Playback Speed: