Sam Saffron and Robin Ward discuss the technical details of building a large open source application use Ruby on Rails and Ember.js.

Special Guests: Robin Ward and Sam Saffron.

 

DAVID:

“Your startup disk is full. This also means you are finished.”

[Hosting and bandwidth provided by the Blue Box Group. Check them out at BlueBox.net.]

[This episode is sponsored by JetBrains, makers of RubyMine. If you like having an IDE that provides great inline debugging tools, built-in version control, and intelligent code insight and refactorings, check out RubyMine by going to JetBrains.com/Ruby.]

[This podcast is sponsored by New Relic. To track and optimize your application performance, go to RubyRogues.com/NewRelic.]

CHUCK:

Hey everybody and welcome to Episode 117 of the Ruby Rogues Podcast. This week on our panel, we have James Edward Gray.

JAMES:

Hello everybody.

CHUCK:

David Brady.

DAVID:

For topical use only.

CHUCK:

I’m Charles Max Wood from DevChat.TV. And this week, we have two special guests. We have Robin Ward.

ROBIN:

Hello.

CHUCK:

And Sam, is it Saffron?

SAM:

Saffron, yeah.

CHUCK:

Saffron. Okay.

SAM:

Howdy.

CHUCK:

Since you guys are new to the show, do you want to introduce yourselves really quickly?

ROBIN:

Sure. You want to go first, Sam?

SAM:

Sure.

ROBIN:

I love to volunteer other people.

SAM:

[Chuckles] Excellent. I’m Sam Saffron. I work for Discourse. I’m a co-founder. Love the show.

Excited to be here.

DAVID:

Cool.

CHUCK:

And Robin?

ROBIN:

I guess I’ll go ahead. I’m another co-founder of Discourse. Ruby developer, I guess a lot of JavaScript these days. I’m delighted to be here and geek out.

CHUCK:

Wait, did you say JavaScript?

ROBIN:

I do a fair amount of JavaScript, yeah.

CHUCK:

Are you on the right show? [Laughter]

ROBIN:

I like Ruby too. Look.

CHUCK:

Can I bribe you to come on the right show? [Laughter]

JAMES:

Hey, these people actually listen to the show. That’s cool.

ROBIN:

I wonder how many developers you have come on who don’t know any JavaScript, because if you’re doing any kind of web app, you probably are at least using a little.

DAVID:

Actually, that would be hilarious if JavaScript Jabber actually had just a run of developers who knew no JavaScript. [Chuckles]

CHUCK:

Oh, geez.

SAM:

Talking JavaScript, are you guys fans of Turbolinks?

JAMES:

[Chuckles]

SAM:

Let’s bring it into some controversial territory to start off.

JAMES:

Yeah. Let’s just get it here right off the bat. I didn’t know you guys were going to start slinging mud. [Laughter]

DAVID:

I tweeted this week that ‘I hate Turbolinks is the new I hate the asset pipeline’.

JAMES:

Yeah, pretty much. I loved how it is not supposed to change anything. I moved probably the dumbest simplest app over to Rails 4.0 recently and Turbolinks broke the hell out of it. [Laughter]

CHUCK:

Oh, really?

JAMES:

Yeah.

ROBIN:

You know though, people did hate the asset pipeline when it was new, but I think they all liked the idea of it. It just didn’t work as advertised. I think Turbolinks is a little different in that many people hate the idea of it.

DAVID:

I think it’s a little more like the CoffeeScript call where people hated the call that DHH made of, “We’re going this way,” and a lot of people are like, “You know, the asset pipeline, everybody uses that needs some form of that.” But CoffeeScript, no I just want to use JavaScript. How do I turn this off? I wonder if Turbolinks is more in that direction. CHUCK: Yeah.

SAM:

The issue is conceptually, it has issues. The state gets all mucked up. You want to be able to just click on a link and not do those CSS requests and JavaScript requests and just move to that new page, bring the html in. But the tricky thing is that libraries like Angular or Ember, they maintain state for you and take care of all that, but there’s no real clean way of saying, “I want to start clean slate on this new page.” Just rewind me to here. I don’t know, like Haskell transactional memory. If it had something like that, you know, where you just rewind back to where you were and start again, that would be great, but there’s nothing like that.

JAMES:

Yeah. But let’s think of it this way. How many people fire up your Ruby server and just let the same server run for six months and just keep using that and don’t ever restart that? Probably not.

Probably at some point, you want to refresh or something.

CHUCK:

The thing is, for me, is that it seems like it’s somewhere in between going whole hog into Ember or Angular or something and just doing full page refreshes. And I don’t know, I start getting toward the complexity where I’m starting to get into that gray area and I just go all the way to a framework. I have a hard time just seeing where I’m going to use it and get major big results out of it.

ROBIN:

Yeah.

SAM:

Yeah. It’s tricky for me as a library maintainer. I maintain rack-mini-profiler. You get reports. “Your library doesn’t work with Turbolinks,” and “Whose fault is this?” [Chuckles] Is it my library? [Chuckles]

DAVID:

No, Turbolinks doesn’t work with my library. That’s the correct way to say that.

ROBIN:

That’s why I object to making it a default. Some people will just say, “Well if you don’t like it, just turn it off.” But because it’s on by default, all these other projects are affected.

JAMES:

Right. Exactly.

DAVID:

I genuinely think it’s a bit on the part of Rails. They’ve realized that people are moving toward this JavaScript MVC on top of Rails MVC. More power to them. They’re going to try and tame the wild wooly west that is the JavaScript ecosystem. I don’t think they have a chance in heck of succeeding at it. But that’s what Angular and Ember and Backbone and all these guys have tried to do. We might settle on some frameworks down the road. But yeah, we’re not quite there yet.

CHUCK:

Yeah. The thing is that those frameworks, they give you a decent-sized toolbox and with Turbolinks it feels like they just gave you a hammer.

DAVID:

Right. A hammer that won’t fit in any toolbox, because it has a three-foot long handle. [Chuckles]

ROBIN:

I think if you build a site in a very particular way, it works well. If that’s what you’re looking for.

DAVID:

Yeah.

CHUCK:

That’s what I’m trying to say. If you’re going to cobble the whole thing together with nails, the Turbolinks is a good thing, because it’s a hammer. But if you need screws or you have some widget that you need to add in, it doesn’t quite do the job.

SAM:

Actually, the funny thing about Turbolinks is that it’s super awesome for a non-JavaScript site. If you’re using no JavaScript anywhere, then it’s the perfect solution, because it gives you all of this for free.

JAMES:

Why didn’t we put it in Jekyll? Oh my gosh. [Laughter]

CHUCK:

Since we’re talking a little bit about the frameworks and stuff, I’m curious. Because you guys with Discourse, you went with Ember. You guys fully adopted it. There really isn’t a whole lot going on that isn’t served up through the JavaScript framework.

ROBIN:

Yeah, that’s true. I’ve said we have an API, virtually 100% API covered from day one, and we know it’s battle-tested because we use it. There are a few functions like, I think resetting your password and some of the open OAuth stuff, it’s still done via regular server-side rendering and requests like that.

SAM:

And indexing is interesting as well. If you disable JavaScript, you can actually see stuff on Discourse and that’s because we render twice. We’ll render a basic format of the site just so people that don’t have JavaScript or disable JavaScript or crawling websites could still get the content.

CHUCK:

I remember talking to somebody with jQuery UI and they call that progressive enhancement. Is that the approach you’re taking?

ROBIN:

I would say no, not at all.

SAM:

No. It’s not progressive enhancement. So many features are missing there that you’re not really getting the full functionality when in that mode. It’s more, at least display the content. That would be the strategy there. The progressive enhancement thing is a lot of times they’ll come and they’ll make the site completely work without JavaScript and then layer the JavaScript beautifulness on top of it and make it work wonderfully. But it doesn’t really work well, that strategy, with the JavaScript MVC framework.

DAVID:

Is progressive enhancement the new kid’s way of saying degrading gracefully?

CHUCK:

Mostly.

ROBIN:

I think it totally is. The problem is I think if you stick to progressive enhancement it limits what you can do. For example, we really believe that infinite scrolling is a good thing. That’s the kind of thing you can’t progressively enhance. There are many kinds of applications that you just would never be able to build in a browser, even though you have this awesome toolkit to do it, because it wouldn’t degrade to that properly.

CHUCK:

Yeah. Do you feel like you do sacrifice some accessibility?

ROBIN:

Well if you mean for people with screen readers and stuff like that?

CHUCK:

Yeah.

ROBIN:

I don’t think so. You can turn on, I don’t remember the key shortcuts in Chrome, but you can actually turn on screen reading in any browser. Screen readers these days are really powerful. They can read most content that’s in a DOM, especially in a text-heavy site like Discourse. It’s pretty easy to just go through it. In the future, I actually want to add more of, I think there’s a standard called ARIA. I don’t know if I’m pronouncing it right. The idea is you extend your html tags with certain attributes to make it even easier for screen readers to jump around and I really want to get on board with that. But yeah, I think it’s false that a JavaScript-heavy site is totally inaccessible these days.

CHUCK:

Yeah, that makes sense.

SAM:

It’s security-minded people, we won’t browse the web with JavaScript because we just don’t trust [those]. [Laughter]

DAVID:

Honestly, I run NotScript in Chrome and I’m a big fan. Yeah, there are just so many security problems with JavaScript and it just hurts my heart to see so many people just saying, “Well JavaScript is the way the web runs now.” That’s like saying, “IE6 is the way the web the runs now.

Just deal with it.” No, I’m going to whitelist some sites instead.

ROBIN:

Don’t you find that just about everything is broken if you disable JavaScript by default?

DAVID:

Yep. [Laughter]

DAVID:

The first time I visit a site, I have to whitelist tons of stuff. What’s really fun is when you whitelist a site and then it still doesn’t load and then you check the whitelist and there are 17 new sites that have all been brought in because of the JavaScript and you enable those and then it still won’t load and there are 17 more sites that have to be whitelisted. Those are the ones where if I know the author of the site, they get a screenshot from me saying, “Seriously?” [Laughter]

SAM:

I guess, yeah. With JSONP and all of that, a lot of sites build now on multiple domains serving the JavaScript. It makes it very, very complicated to say what you allow and what you don’t. On the flipside, there’s the cross-site request forgery that goes on that people can just trick you to go to a site and own all of your accounts everywhere because there are vulnerabilities out there. We just fixed one this week. It’s crazy, the amount of vulnerability that’s out there with cross-site request forgery. I’ve seen some of the craziest things. You would not imagine they were an issue and they suddenly are an issue. I think a friend from a previous company blogged about one where they use JavaScript proxy objects to suck out a CSRF token and all sorts of crazy, crazy things like that.

DAVID:

Yeah. My father-in-law installed something that installed a toolbar and he called to say, “What do I do now?” I’m like, “Well, you can run your antivirus and whatnot.” Then he said, “Oh no, I think it’s fine. Gmail just prompted me to change my password and I think that I can get it.” I said, “I’m on my way over right now.” [Laughter]

DAVID:

We got his accounts back.

CHUCK:

Oh wow.

JAMES:

So you went all in on the Ember decision. What made you choose Ember and in particular over the other choices? There are lots of frontend choices. Why Ember?

ROBIN:

That was something in the beginning. I’ve been following a lot of the frontend frameworks for a while. I think one of the main reasons is because my previous gig before Discourse, we worked on a really heavy JavaScript app that had all these awesome visual flourishes and stuff. It was a really rich jQuery app, but the code just became totally unmanageable. I ended up with these files that were super long. We tried to do the best things like namespacing things and putting them in the appropriate place, but it was just so disorganized. After I got off that project, I was left with this huge desire to organize code. Various frameworks were attractive to me. I looked at a few of them. I looked at Backbone and Ember and Angular and stuff like that. I don’t know. I liked the way shared certain values with Rails, the convention over configuration and stuff like that. A lot of it is I think actually closer to an aesthetic choice. But for a Rails developer, I thought Ember was a lot more familiar. It just was an easier fit. I guess Yehuda being a core developer on Ember and taking a lot of the stuff that he learned on Rails is the reason for that.

JAMES:

That’s a good point.

CHUCK:

Are you guys using Ember Data or Ember Model? I don’t remember which one’s which.

DAVID:

Yeah, are you using the one that isn’t finished yet? [Chuckles]

SAM:

They’re all not finished yet. [You’re forgetting] none of these are finished.

ROBIN:

Yeah. Ember isn’t actually 1.0 yet. It’s prerelease 6. Angular is 1.0. But no, we’re not using Ember Data or Ember Model. We just use jQuery/Ajax. We have a few helper functions. We actually have something called Discourse Ajax, but it really just wraps jQuery’s Ajax stuff in promises.

SAM:

And does CSRF tokens now. That’s [inaudible].

ROBIN:

It does.

JAMES:

Did you find it difficult to work with Ember with it not being 1.0 yet?

ROBIN:

There were times, if I’m going to be honest, that it was difficult. In the beginning, when we started, there was no router in Ember. I hand-rolled my own, which is not necessarily the greatest way of doing things, but it worked. Then they came out with the router and then went through a serious revision. I actually had to swap out the router twice in our project and that was a fair amount of work. Now that the router is stabilized, I think it’s in pretty good shape. There haven’t been any major changes that have broken it in a serious way for many months now. It’s in good shape.

JAMES:

Maybe another even more controversial question and maybe a little more on topic for us is, why Ruby? That’s not where your past experience lies, right?

SAM:

No. My past experience is a bit complicated. I worked in .NET shops for years and years and then took a break from .NET and started doing Ruby back when it was Rails 2.0, did that for a year or two. Then I moved to Stack Exchange where I worked on C# code again and ASP .NET MVC and all that stuff with Jeff. Jeff started Discourse with Robin who is actually very, very experienced in Rails. He never really took the break that I took. Then there was his big blog about why Ruby. I guess the thing is that I love Ruby. I’ve always loved Ruby. Robin, I think as well is in that boat.

ROBIN:

Absolutely.

SAM:

So there is that. If you want to have a big open source project, you don’t have too many options [at the moment] on the table of what to do. You could do PHP but you’re never going to find people to develop on it. Everybody likes bagging PHP. I’m sure there are people who can code tidy PHP and whatnot, but it’s just there was no chance I’d work on a project that was PHP.

DAVID:

Yay. [Laughter]

SAM:

The language is beautiful in Ruby. A lot of the decisions there are just fantastic. I love it when I’m editing Ruby code. So there was that. We did experiment, I guess, with the idea of maybe doing Node.js and then you do JavaScript everywhere. But all of the frameworks where flaky and young and there wasn’t a real equivalent of Rails for Node.js at the time. I don’t think there is even an equivalent of Rails for Node.js now. So you’re building on something that is a fairly big [caucus] of knowledge with a whole bunch of gems that can help you out. What other options really are there?

You’d be able to do Perl, maybe. There’s still Python. Those are about it.

DAVID:

That’s cool.

ROBIN:

If we’re going to be honest, Jeff, he still really likes C#. He likes the language that he’s the most familiar with. But he knew from day one that that wasn’t going to fly for creating an open source project. The ecosystem around .NET open source is just nowhere near as good as it is for Ruby or Python.

SAM:

And the [inaudible] served on Linux boxes, right?

ROBIN:

That’s true.

SAM:

We needed technology that is friendly to Linux. To have all of those Linux-isms built into the language properly, like forking and whatnot. If you’re dealing with .NET, all of that’s second-class. It’s built for Windows. Those concepts aren’t there, signals, forking, et cetera. Mono is not nearly as pervasive as Ruby is on the Linux desktop.

JAMES:

You touched on this but you mentioned that there is this vibrant community. Since Discourse has gone open source, are you seeing a lot of Rails developers contribute and help you make it better?

SAM:

Oh, definitely. The vast majority of our contribution is actually in the Rails department, on the Rails code. We’d like to get more JavaScript contribution, but it’s taking a while. There are developers that have been doing Rails now for years and years and very few who have been doing Ember for more than a few months. So it’s a lot harder to find the Ember developers to contribute as opposed to the Rails ones. Also, the Ruby developers are very, very passionate. Stuff has to be a certain way and tests need to be written a certain way and patterns need to be used in a certain way. I love bringing opinion and cleaning up all of the code. We’ll except pull requests that definitely improve on stuff. Avdi was talking about you replaced mac with a pluck. Those kinds of things where the code gets simpler and easier to understand, easier to maintain, we love those pull requests. We also love feature requests. We get a little bit of everything. We’ve had to push back on some style-only changes. Sometimes people will come in and see a few hash rockets somewhere and say, “No, we want to use Ruby 1.9.3 syntax and we’re a Ruby 1.9.3 project,” and we’re pushing back a little bit on style. We used to accept style changes everywhere, but had a few regrettable pull requests that we accepted. The whitespace one is the one that bothered me most, because there are all these little hearts in our commit logs now. When you browse it on GitHub, you see these little hearts in all sorts of places where somebody decided to do that. It really doesn’t add much to the code. I’ll clean up. At the moment I use vim. That’s my editor of choice. I’ve got Trailer Trash plugin and it always shows me when there are trailing spaces, so I’ll clean them as I’m going through the files. But I won’t go and do a full sweep through the codebase.

That’s pointless.

DAVID:

Do you accept style changes if there is also a material change to the code?

SAM:

Yeah. As soon as there’s a material change to the code, we accept that, for sure.

DAVID:

So there’s the radar level to sneak under, guys. [Laughter]

JAMES:

Fixed this typo and removed 3,000 trailing whitespace [inaudible]. [Laughter]

SAM:

Oh, yes. That’ll work.

ROBIN:

I would accept that.

JAMES:

Oh, you would accept that? Awesome. You mentioned there were some regrettable pull requests. What did you bring in that you wish you had thought twice about?

ROBIN:

We haven’t had to revert very much, but I think the pull request Sam was talking about was one that just changed whitespace in dozens of files, or maybe hundreds of files and they put an emoticon in the commit message. Because of that, now when you look at GitHub, you know how whenever a directory, if you look at a folder or even a file that’s had a commit underneath it, it always has the commit message there. So there’s a heart and this heart bothers us because it’s constantly laughing at us in the face every time we look at our project directory, because it’s just all over the place for all these files and it catches your eye. That’s what he was talking about.

JAMES:

That’s hilarious.

SAM:

That’s about it. I wouldn’t say I’ve got a big list of pull requests that I regret. In general, vast majority of pull requests we accept and we don’t regret. It’s a great opportunity as well as a contributor to learn a bit. I got an email a few months ago about somebody was asking can you mentor me, I want a mentor. Then a few months later, I’m looking at the pull request and you are doing a little bit of mentoring when you’re accepting pull requests. You should do this a little bit better. This is maybe a better idea. This is the reasoning. You can learn a lot as somebody new to open source or new to contributing to open source just by submitting these pull requests and us stepping in and saying yeah, this is why we do it this way or this is why we can’t accept it, if there’s something going on there.

ROBIN:

I actually learn from people who submit code to us, because they say, “Oh why didn’t you do it this way.” I’m like, “Because I didn’t know you could do it that way.” [Laughter]

DAVID:

Every exchange I’ve ever had with James on [inaudible].

JAMES:

I know. [Laughter]

JAMES:

Yeah, right. That’s every exchange I have with everybody else. It’s funny. People always think, “Oh my gosh, you’re an idiot.” It’s like, “Yeah, you’re right. I’m an idiot. Thanks for teaching me something.” [Laughter]

SAM:

Yeah. You have to leave your ego. If you’re running a big open source project, you better chuck your ego somewhere else and forget about it.

ROBIN:

Yeah, absolutely.

SAM:

That’s, I think, the death of a lot of open source projects, is a big dictator with [bunches] of ego and he’ll never accept anything unless it was his idea or his style or whatnot.

ROBIN:

If you come from a corporate community or a close source thing, you’re not necessarily used to sharing your code with tons of people. In a way, it’s like baring, you’re going out there naked or something. It’s like, “Look. This is me. This is me. I have faults. There’s a mole there on my leg. I know it’s there, but the software works and we’re going to try and make it better.”

CHUCK:

Can I ask really quickly about performance? I know that that was something that came up.

DAVID:

Actually, before you go there Chuck, can I ask another community question?

CHUCK:

Sure.

DAVID:

I’m looking at the Discourse page on GitHub right now. I think it’s very telling. You guys have 176 contributors right now. You only have eight open pull requests and there are four of you working on this?

SAM:

Yeah.

ROBIN:

Yeah.

DAVID:

So you have 1,654 forks of Discourse. Is it common for people to fork the repo in order to standup their own copy of it? Or is there just that much dissent?

ROBIN:

I think most of those forks are people who are working on pull requests and submit them to us. Sam actually can speak to this in a minute, but there a few people out there who forked Discourse because they wanted to do their own login provider or something like that. But as far as I know, no one’s been, “I’m forking this project because I just have a fundamental disagreement with it.” If they have, I haven’t heard about it.

SAM:

Yeah, same. I haven’t heard of that. That is not something that happens. I think the majority of them are just people that are just, “Click the fork button and see what happens.”

DAVID:

Yeah. I forked it today just so I can go after one of those files that code climate didn’t like. I hope it’ll turn into a pull request. But I’ll be coding for a while. I’ve got to learn your codebase first.

SAM:

Yeah. Talking learning the codebase, one thing that I guess hurt us early on, we were in a huge rush to get this out and we introduced specs a little bit later than you normally would, just because there was this massive rush to do prototype after prototype and get something out there and look at it. We didn’t have, really, the tooling around, and then we introduced specs. Now we have a reasonable spec suite. I’d say we’ve got 2,700 specs for Discourse. A lot of them are testing multiple things. It’s a really well-tested codebase. But to get to the state where we are today, it took a bunch of changes as well to the whole attitude toward testing, which for me was very interesting. At some point testing, for me, became really, really painful. We used Guard to run our tests. For some reason, for me, the workflow just never worked. I’d forget to run Guard and then I’d run Guard and then I’d be waiting three minutes for Guard to finish running. Guard would finish running and then it could run the spec that I had to run. I’d just be waiting constantly for the spec suite. It’s not there was some sort of big technical reason except that Guard didn’t really work for my workflow well. A few months ago, I came up with a way of, we created a test runner. If you runrake autospecit’ll run the specs. It’s very, very smart. It’ll abort very long test runs. So if it starts running all of the specs and then you just start editing a spec file and save it, it knows that it’s time to interrupt the spec suite, run this one spec for you, then it’ll keep running these specs until they pass as well. If you fail a spec by saving it and then go and start editing around the project to try and fix that spec, it’ll detect that and run that one failing spec, not a hundred of them. There’s a matter of fidelity, that it caught that fidelity of running the spec. But once that was in place, it’s like for me now I can’t even think about writing something without the specs because it’s just so much easier. It’s so much easier just to come in and write these three specs and to boot up the website and go to the pages and look and see if it’ll work. And you’re leaving something behind, as well, which is awesome.

ROBIN:

A lot of people who I’ve heard, I guess maybe who haven’t fully bought into testing or that experience with it, think that it’s slower than hitting save and refreshing a page. But I think what a lot of people don’t understand is that it can actually often be much, much faster than ever going to a browser and hitting refresh, even if your page refreshes fast. It’s like, “Why not do it?”

SAM:

It’s the big test runs that are killing people, I think. If you’ve got a test run taking three minutes to run, and you’re constantly waiting for those three minutes, that’s where this all falls apart.

ROBIN:

Three minutes is nothing, [based] on the places that I’ve worked too. I’ve seen 20-30 minute runs.

CHUCK:

The other thing is that it tests everything. If you’re dealing with something that’s deep in the code, you can’t test every place that that touches with a single refresh. There’s that aspect as well.

ROBIN:

True, absolutely.

SAM:

Yeah. It was interesting. GitHub did something interesting with their test suite recently. I think Iman worked on it. They heavily parallelized it. That is a test suite that took 25 minutes to run. Now they’re running it on a 64 core machine. They can get it to run in under a minute, which was very interesting.

DAVID:

Nice.

SAM:

So there are a lot of very cool things going on in the community around testing as well. You know where to look. I guess for the listeners around, I know Guard are looking to add APIs to enable the style of testing like autospec does. There are actual missing bits of the API to allow for these interruptions and whatnot during a run. That’s something to keep an eye on. If anybody’s looking to contribute to Guard, I’m sure they’re looking for people to help with Guard 2.0, which is going to come out at some point.

ROBIN:

Unfortunately, autospec is a little tied to our project right now, isn’t it?

SAM:

Yeah. And it’s going to become a little bit tighter as well if we do the JavaScript tests.

ROBIN:

Yeah. Right now, we do QUnit either on the command line with the full suite or in a web browser, which is how I often do it, which again is not the best.

DAVID:

Can you actually do a full code coverage or whatever code coverage level you have of your JavaScript, can you do that headless?

SAM:

Yeah.

ROBIN:

Yeah.

DAVID:

That’s awesome. I’m going to spend some days just playing with your project just to learn how you did that. That is awesome.

ROBIN:

Actually, when you run the QUnit, a lot of the QUnit tests are unit tests or [slight like controller] tests and stuff like that. But we have a bunch of integration tests. The cool thing about that is if you run it in a browser, in the bottom corner, you can actually see the website running in a minimized, zoomed out version and you can see it clicking on things and stuff like that while it runs it. That runs totally headless as well. It’s part of Jenkins build process. We can’t deploy unless that works.

SAM:

And we use, I guess it’s driven by PhantomJS, which is an awesome project that everybody should know about. CHUCK: Phantom is so cool. Can I ask my performance question now?

SAM:

Sure.

CHUCK:

I read the one thing about the GC malloc limit that did a whole bunch of stuff. I get the idea there. I guess what my question is, is you’re pushing a lot of data around in order to make these forms work. Just on the Parley list, we have several hundred people on there. How is it that you make all of this work and respond so quickly? Are there some tricks that you’re using?

SAM:

Well we use Ruby [inaudible].

ROBIN:

That’s a huge question.

CHUCK:

Everybody knows the common, like caching and eager loading and things like that.

SAM:

We use Redis. Redis is our cache and we build on it. We’ve done a lot of work around improving Rails performance and ActiveModels serialize their performance as well to make sure that it works well for us, because that’s our frontend. Our frontend is little JSON payloads and we need to make sure that we generate them really, really fast. We’re pretty much on Ruby 2.0. We moved to Ruby

2.0 so we’ve been really early adopters of technology at Discourse. Now we’re running the latest patch level of Ruby 2.0, which actually I’d recommend now. It was harder to recommend patch level 0 because there were some seg faults that happened, which were nasty. But now that it’s gone through a few releases, I think Ruby 2.0 is awesome and there’s no reason not to use it.

JAMES:

I agree. It’s very good, but you do need to be on the cutting edge release. I ran into a bug with keyword arguments just the other day and I was only one version behind. [Chuckles]

DAVID:

Wow.

ROBIN:

For the record, we actually still run on 1.9.3.

SAM:

Yeah.

ROBIN:

So we don’t use 2.0-specific syntax yet.

SAM:

We want Discourse to be adopted in lots and lots of places so we can’t really place the barrier too high. I’m not saying we go down and allow 1.8.7, but 1.9.3 is reasonable. It’s a supported release.

JAMES:

Yeah, I think it’s reasonable. The core team’s end-of-lifed 1.8.7 so I don’t think there’s anything to be gained by going there.

ROBIN:

We certainly recommend 2.0 for production, because it’s very fast, especially with the GC tuning.

SAM:

Yeah. And a lot of the way that we got to the speed that we did on server-side was extensive profiling with rack-mini-profiler. I’ve spent a lot of time looking at flame graphs. I’ve even monkeypatched bits of Rails in some spots to make sure that it works well for us. A couple of really interesting examples are we deliver a ton of assets in dev. So there’d be 200 assets, 200 little JS and CSS files that you need to deliver to the browser very, very quickly. In general, the way that that works on a Rails project is that it has to walk the middleware stack, reach the router and then it’ll deliver that file. And walking the middleware stack is something that can take 5-10 milliseconds in dev. Multiply that by the 200/300 assets and then a refresh of the page on Discourse in dev used to take four and a half seconds or five seconds. That is a killer as a developer. Why do I have to wait five seconds just to refresh the page? So I wrote a little bit of middleware that just sits in the front of the middleware stack and it checks to see if the JavaScript or CSS file didn’t change. It’ll just tell it, “Look it didn’t change and it doesn’t have to walk the whole stack.” I’ve got an open [inaudible] request will Rails to decide if we want to accept that kind of middleware stack into Rails proper for dev mode. I think it’s reasonable now that all of these massive client-side libraries are coming about. People want to deliver a lot of assets really quickly in dev. Otherwise, you’re stuck doing the standard thing that happens in a lot of teams is that they reach this problem and they go, “Okay we’ll give up on having files and we’ll just have one big file that we’ll deliver back from Rails even in dev,” so they concatenate in dev as well.

ROBIN:

There’s the flag, which I think is really oddly named debug. I think it means debug_assets and it really means, “Do you want to concatenate your assets?” Most frontend MVC developers that I know always turn that on because even though Rails is one of the best platforms, even though it’s not by default like Turbolinks is the default, it’s still one of the best platforms for doing these web API JSON applications. But it’s a really non-sensible default when you have hundreds of files. The asset pipeline makes it easy to have so many files, but then it makes it really slow in development mode.

SAM:

There’s a long-term vision in the asset pipeline of moving to source maps and source maps will solve this problem as well. I’m just not convinced that we’re going to see source maps on the desktop like in Rails releases for a while now. Because an example would be that sprockets in Rails 3 is sitting in a year-old version. They’ve got a version locked for a year old version of sprockets and they’re not upgrading because there was a release of Rails 3 that was completely bust due to an upgrade of the asset pipeline. They’re like, “We’re afraid now. We don’t want to upgrade it anymore.” So I don’t see us moving out of this world of very, very slow frontend thing unless we put something like that in place. So that was one example.

DAVID:

I have a question for you. You’re in this wonderful position where we talk about, it’s a horrible position actually, but I’ll lead up to that. When you build a project, you find yourself doing a lot of dev ops. It’s my job to deploy this. I built the thing. I have to make it go. You are building a thing that other people have to deploy. You are like the puppeteer of other people’s dev ops. So you’ve got this monkey patch in Rails and maybe Rails is going to adopt it at some point, but now you’ve got this mixed user base. Some of them are not running that version of Rails. Some of them will be.

SAM:

No, no. We ship a lock file. There’s one Discourse out there and it’s locked to that. We’re not going to just ship a gem file and let people run amok.

DAVID:

Okay.

SAM:

That does open an issue that extensibility-wise, there’s something that’s been open in Bundler for very, very long, the ability to have what is called a secondary gem file. Occasionally, say you pull our project and you want to just use Pry to edit it, but you don’t want to have to modify the gem file and gem file lock because then you’re going to be fighting merging upstream and whatnot for just your little Pry gem that you wanted to bring in. For Discourse, it’s much, much worse, because we want to build a platform that people can extend. Some extensions are going to want to bring in gems and we need to check that that gem doesn’t clash with our gems. We need to build it its own little lock file and whatnot. That’s going to be very interesting when we reach that time where we just have to extend Bundler and make it work with our plugins. That is something that has been open for many, many years. Technically, Bundler can allow for it. It’s got all of the ways of resolving these dependencies. It’s just that untangling it and getting it to work is going to be tricky.

DAVID:

Yeah. Okay, so you’ve end-run the evil question that I had, but you addressed the core question which is, how do you deal with it right now? No, we just locked to the version. There’s no way to have a too new version of Discourse with a too old version of Rails. But yeah, there is this problem that’s coming that you’re putting out a framework and it’s got to be compatible with other frameworks, because it depends on them.

SAM:

Yeah. The way we deal with that is that we’re constantly deploying. If we upgrade our dependencies, we’ll be deployed within a few hours and we will check that in production and [watch our] logs and we’ll make sure that everything is good. We’ve got a test suite to protect us. We’re also doing forward-looking work. For example, if you look at our gem file, it’s very, very interesting.

You’ll see a bunch of patches to gems. It’s a fascinating little thing to look at.

DAVID:

I have it open right now.

SAM:

Inspired by some work that New Relic did, I think when they were doing an upgrade from Rails 2 to Rails 3. They had this multi-boot thing where you could boot up your project either in Rails 2 or Rails 3. That allowed people to start working on the Rails 3 upgrade without interfering with the Rails 2 work that was going on. We’ve got the same going on with Rails 3 and 4. We’ve got a few people that are contributing and working on getting Rails 4 working on Discourse. But that’s not blocking us and we don’t need to have a fork. We don’t need to merge that fork all the time. You can just work on the Rails 4 stuff with the official Discourse repository and submit Rails 4 fixes now, which is great.

CHUCK:

I want to hark back a second to a platform that people can extend. Have you built any of that? Is it extensible right now other than maybe through Rails engines?

ROBIN:

Since day one, we’ve had a series of example plugins in the vendor directory, like you said, as engines that plug in. The emoticon support, for example, that’s the one on every Discourse site, but it’s actually vendored in and you can turn it off if you want. It knows how to plug into the JavaScript and the server-side for the rendering pipeline as well. But in the future, we want to make it so that people can selectively say, “I want Discourse install the gems. Then also I want these extensions to Discourse that do these various things to customize it to my experience,” and that’s what’s really tricky.

SAM:

At the moment, I’m working on some extension to the plugin system. I think the big issue with plugins is that they have to be super simple, because if it’s really complicated to extend your system, then nobody’s going to do it. I think that requiring people create a gem and then deal with the merges of their lock file is just a bit too much, because it’s scary to have to do that. Rails engines are awesome, but they’re also a complicated construct. Often, people don’t want to have to think about that. They just want to say, “Just give me this JS file, give me this CSS file, and be done with it. Just put that in a little file,” and have their extension work. A good example of that is WordPress, the way they do extensions. You edit one file and you’re up and running. I’m working on some prototypes at the moment of that style of extensibility. So you can just edit or [inaudible] rb file and suddenly have that extend Discourse.

ROBIN:

Some people want Discourse itself to be an engine, but I don’t see that happening anytime soon. It’s such a big application. I can’t imagine just plugging it in to part of another site. While we control everything like that, we don’t have to worry necessarily about fitting it into other versions of Rails that don’t run the whole site as one container.

CHUCK:

That makes a lot of sense. You can go ahead and deploy it under Passenger or what have you as a subdomain or a sub-URI and just make it work that way.

SAM:

Yeah, the subdomain stuff is tricky. It’s been very, very hard for us to get subdomain stuff working perfectly. We try, but it’s one of those features that unless you’re using all the time, you’re just going to miss all sorts of spots, and nobody uses it. It worked for a while. I don’t know that

ROBIN:

Oh, you’re talking about subfolders, right Sam?

SAM:

Yeah. Deploying Discourse under mysite.com/forum I guess would be an example.

CHUCK:

I just went through that and I actually had a sys admin guy help me out with that. He’s one of my picks. But yeah, it’s painful.

SAM:

Yeah, it’s not easy. It’s so easy to create a little subdomain anyway, like discuss.domain.com. I’m sure that effort is really warranted in a lot of cases.

ROBIN:

This is, I guess, one problem with having a site or an application that has so many options, is that there’s the set that we like and that we use all the time. We dogfood those. But there are some settings that other people use that we don’t really and it’s hard for us to make sure those are constantly working, beyond unit tests and the obvious things like that.

CHUCK:

Well we’ve been dancing pretty close to this. I found a page that says that you guys do multisite stuff. You can effectively standup an application and then say, “Yeah, I’m running Discourse but I want four forums running on the same application,” and it’ll just work.

SAM:

Yes.

ROBIN:

Yes.

CHUCK:

I thought that was pretty awesome. I’m a little bit curious though. How much extra work was it to get something like that hooked up?

ROBIN:

I’d say it was a lot, wasn’t it Sam?

SAM:

I don’t know. The thing is, if you do it early enough then it’s not such a huge problem.

CHUCK:

Well now we know who did all the work on that, right? [Laughter]

SAM:

There’s one gem that does it called Apartment, which is reasonably popular, but I found it had a bug with Sidekiq. We need Discourse to work multithreaded, so not only do we want to be able to serve different sites from the same process, we want to be able to [spin] threads and run each thread as a different site which is very, very tricky. There was actually a bug in Rails 3 that I got patched in Rails 4 that I monkey-patched in Discourse to get that to work. The vast majority of the difficulty was actually debugging through to figure out where are the issues, where are the wholes, where to inject the middleware to make all of this magic happen. But the actual code, if you look in the vendor directory, there’s a multisite gem that does that that is reusable. There’s not that much code. It’s pretty straightforward, what it does. But then everything needs to be aware of this concept of a multisite. We’ve got a messaging bus that delivers information back to the clients and delivers information between all of our processes. For example, if you get a new notification in Discourse, it’ll just pop up magically for you. The API is trivial. You just go, “Tell Bob that there’s a new message,” and then it just shows up. It gets a callback in JavaScript to say that that happened. All of that needs to be aware of multisite as well. You don’t want it leak information into the wrong site.

ROBIN:

That’s what was hard, right?

SAM:

Yeah.

ROBIN:

The first version was up and running pretty fast. It was more like we found that it was leaking over time. But I think now, it’s really solid, isn’t it?

SAM:

Yeah, it is pretty solid.

ROBIN:

I haven’t seen anything leak in ages. I think we’ve nailed down all the areas.

CHUCK:

Another thing. I’m hearing you talk about multithreading. That’s not something that we’ve really, we’re just starting to get into this with Rails. Did you find that making Rails thread-safe or making some of the processes that you run thread-safe was tricky at all?

SAM:

Only for the multisite stuff. That was the only part that I saw that it was tricky. We’re very careful not to have too much global state sitting around that’s we’re going to be fighting about.

ROBIN:

We’ve been running Sidekiq for a really long time, even well before we launched. I think that that forced us to make sure that any of our components, at least that we were using in that respect, worked in a thread environment.

SAM:

Yeah.

CHUCK:

Which Ruby are you running? Are you running MRI or are you running something else?

SAM:

We’re running MRI. MRI [inaudible].

JAMES:

Do you use...

CHUCK:

Celluloid?

JAMES:

No, I was going to say do you use a threaded server like Puma I think it is? Is that [inaudible]?

SAM:

Well, there’s a long history to this. We launched on Thin and we’re using Thin at the moment. The reason is because this messaging bus has long polling, meaning that you can open a connection back to the server and wait for it to come back with the results and just hang there. We could have now 10,000 sessions out there connected to Discourse waiting to be notified about stuff. To get that to work, back at the time, the only option was Thin. Thin and Rainbows.

JAMES:

That very interesting.

SAM:

Thin and Rainbows. But Rainbows, I don’t know if it’s gotten much popularity. But since, Rack 1.5 came out and it introduced something called Rack hijack and that allows you to hijack a socket. I’ve built the messaging bus now has the ability to work on direct hijack mode as well. That is supported across a lot of servers. Puma will support Rack hijack. Passenger supports it. Unicorn supports it. Now, there are two interesting caveats here. One is that Unicorn is a very, very, very, very compliant webserver. Rails decided to lock the Rack version to 1.4 up in Rails. 1.5 won’t boot. Unicorn will actually check that when it boots up and disable Rack hijack if they’re claiming to be a Rack 1.4 app. On the other hand, Puma won’t do that. They’re a lot more forgiving there. Puma will work with Rack hijack. Passenger unfortunately has a bug. As we’re recording, it has a bug. I don’t know how long it’ll be there, because [inaudible] working on it, where it supports Rack hijack but it doesn’t work. Even though you hijack a socket, it still holds up one of the spots in the load balancer, so you just get stuck in [turn]. But with Rack hijack, I’ve tested locally and I can easily have on my dev box 5,000 connections sleep for two seconds and serve everything back within five seconds.

JAMES:

That’s awesome.

CHUCK:

That’s amazing.

JAMES:

Yeah. That’s very cool.

CHUCK:

I want to hear a little bit more about this message bus, because it sounds complicated. [Chuckles]

SAM:

The tricky thing about the message bus is having a delivery that’s reliable and having something that’ll scale. The design of the Discourse messaging bus is it’s a polling message bus. It stores no state. Whenever each client comes, it says, “Look I’m interested in these channels.” Every request where it’s asking the server for information, it will always tell it all the channels it’s interested in. The server doesn’t need to hold state for that client beyond that one request. It can just throw everything away. It can share a [bunch of state]. That design scales really well. It uses Redis to ensure that all of the messages that are delivered are delivered in order. There is an issue, I guess, that a lot is unreliable on the web. You can do all of these tests and everything’s pristine, but once you start introducing clients and somebody will ask for something and then halfway through never get the message back. You have to handle those kinds of situations. This message bus has a global id that’s constantly increasing and you can always catch up to any messages that you missed, meaning that if you close the laptop and then open the laptop an hour later, it’ll still be able to catch up.

DAVID:

That’s cool.

ROBIN:

We don’t currently do that though, do we Sam?

SAM:

No, we do that. We do that.

ROBIN:

So if I get a new message on the frontend and the number pops up then I miss it, it’ll come later?

SAM:

Yeah. It’ll come later. Whenever

ROBIN:

Oh cool. Neat.

SAM:

Yeah.

ROBIN:

See, I didn’t even know that.

SAM:

So that’s built on Redis and it’s built on Rack as well and it’s an MIT gem. Anybody can use this in a Sinatra app if they want or in a Rails app, which is pretty cool.

JAMES:

What is that gem called Sam?

SAM:

It’s called message_bus.

JAMES:

Oh, there you go.

CHUCK:

I’m surprised that wasn’t already taken.

SAM:

Yeah, me too. [Chuckles]

SAM:

The other one, if people are looking and shopping around for these kinds of solutions, what’s the other one?

ROBIN:

Faye.

SAM:

Faye, yeah. Faye is the other famous one for doing this kind of stuff. We just choose a slightly different approach. It also has, I think, a Redis backend. I don’t think it has what we call a reliable channel where it can recover and reorder and make sure that everything comes there. There’s a very interesting test that I have inside the message_bus that forks 30 processes and starts hammering the bus with messages from everywhere and makes sure that they’ll come back. Those kinds of things, I don’t think Faye is worrying about solving at the moment.

DAVID:

We talked about this in the pre-call, but I noticed that you guys have a Code Climate badge on your front page, on your readme. I just want to tell for the readers that they have a crappy GPA. These guys are awful. [Chuckles]

DAVID:

You’ve got a B- GPA, barely, by the floating point epsilon, 2.67 is the minimum for a B- GPA. To be fair, 70% of the files have a straight up 4.0 A, but you’ve got files that are flunking. You’ve got files that are C’s and D’s. I am fascinated by this. I’m very excited by this because this is the first Code Climate report card I’ve seen that hasn’t been a boring look at me, I’ve got a straight 4.0 all the way across the board. I wanted to ask you guys, how much do you use Code Climate? How does it help you? Does it help other people? How do you explain this, young man?

ROBIN:

Do you think most people only put them up if they have a straight A? Do you think it’s that kind of thing? [Laughter]

ROBIN:

I don’t know.

JAMES:

Probably.

ROBIN:

I think it’s, “We rock.” Well I want to be clear, we didn’t actually add that. That was a pull request day one. Someone added it.

DAVID:

Awesome.

[Laughter]

ROBIN:

We accepted it. I’m like, “Hey, this is cool. It shows people where to look.” I actually think it’s helped contributions a fair bit. We’ve got a lot of them which are like, “I don’t know your project, but I just opened Code Climate and I found an F and I’m like hey, I can do better. Then here’s what I did. I broke it up into these functions. All your tests still pass.” We’ve got a lot of requests like that and it’s been really good.

DAVID:

Yeah. I do notice that. I forked the project just so that I could jump in on one of the big red F files.

ROBIN:

And we love that.

DAVID:

It’s almost like a, “Hey, jump in here,” kind of keyword.

ROBIN:

It’s awesome.

SAM:

Yeah, I try to spend a bit of time refactoring a lot of the mess I created, obviously. I want it cleaned up. I want to be careful about not over-decomposing stuff, because occasionally Code Climate can encourage that. Create 70 functions where one slightly longer function would have still made sense and now you’ve got something so decomposed that you have no idea what’s going on.

DAVID:

We’re not ever going to reuse any of the parts of this, so let’s not make 70 different reusable parts out of it. Yeah.

SAM:

It’s a tricky balancing act. But yeah, some point you’ve gone beyond what you should be doing in one function and it should be broken down. A lot of Discourse has been Jeff driving, “We need these features. We need these features. We need these features.” And a lot of times, when you’re dealing with pipeline stuff or topic stuff, the natural thing to do is just to look at where this bit of the pipeline is and just introduce some new functionality in there. A lot of times, you just omit creating a function, which you probably should have. So it’s good that Code Climate is there keeping us a little bit more honest. It’s definitely helping a lot break this down and get it to something better.

ROBIN:

There’s definitely parts of the code that I’m not happy with, but it’s a pretty big codebase. The worst thing you can do is assume that we want it to be that way or that we’re proud of it or something like that. No. We’re totally happy to have requests any time to improve that. No, we know that it needs to be fixed and worked on.

DAVID:

Well, let me put a positive spin on that and actually put words in your mouth. This is why I find your report card so interesting. Tell me some feature or some tradeoff that you have made that you’re happy with. I’m not going to ask you to say you’re happy that you’ve got an F or that you’ve got a D on a file, but is there something that you had a choice between polishing this rivet from a B to an A or from a D to a C but instead you were able to use that time to ship something else? Is there an example of that that you’ve found as you’ve developed?

SAM:

Well, that’s daily. We’ve got a whole bunch of those [inaudible] [Laughter]

SAM:

If all we were doing right was working from F to A, then that’s all we’d be doing. I’d imagine that there’s months of work to take this, to A everything. It’s every day we’re making that decision that we have to build new things. We refactor as we can. We improve as we can. I guess the goal is whenever you go into a bit of code, try to leave it a little bit better than when you got in there. I think that’s a great strategy to take as a developer. We’re constantly learning, which is awesome. There are some things that I find pretty much unfixable. For example, we have a little DSL that we defined for site setting so each site can have a bunch of settings that it has. There’s a DSL there. The DSL is just not, Code Climate doesn’t like the DSL. But the DSL is perfectly readable when you look at the file and say, “This makes sense. There’s nothing really that I’d like to change here.”

JAMES:

Sam, I want to push back on that a little bit. You told me that in the pre-call so I’ve been staring at that file for a while. To me, that feels like stuff that could be in some kind of configuration file or something like that. Do you not think?

SAM:

Yeah. But the configuration file can be a .rb file, right? There’s nothing really stopping that. Like why should it have to --

JAMES:

Yeah, sure. I think it could. But since it’s just key-value or something like that, I think there are maybe even better choices. I think.

DAVID:

You know if we move it to YAML, Code Climate won’t grade it. [Laughter]

DAVID:

The solution is, hide the evidence.

[Laughter]

JAMES:

In a way, though I think that’s right. Anyway, it’s a great challenge for the listeners. Go take a look. I’ll put a link in the show notes to this. The model is SiteSetting and go take a look at it. See if somebody out there has a great idea of cleaning it up. I think there are maybe some untapped options there. While we can joke that it’s hiding it in YAML, I would say it’s maybe moving configuration data out of the code, which is actually a viable thing to do, I think.

SAM:

It can live in the config directory as opposed to living in your models, which is fair enough.

JAMES:

Right.

DAVID:

Sure.

ROBIN:

I think I would be happy to accept a pull request like that, assuming it worked properly.

SAM:

Yeah, assuming all the functionality remains. As long as they don’t tie us in a corner where we can’t do things because of that.

ROBIN:

It’s amazing, I think, how little we’re actually sold on that we’d be, “No, we wouldn’t change.” Most of the time, if something’s an improvement, we’re happy to take it in.

JAMES:

Yeah, I was trying to get you guys to fight to the death. I’m really disappointed with how quickly you [inaudible].

DAVID:

No. Yeah. [Laughter]

ROBIN:

Look, I know some parts are bad. It’s just room for improvement. We do our best. Like Sam said, I don’t think it’s usually a conscious thing like, “Oh my god, I have to get this feature out. There’s no time to polish it.” It’s like these things add up over time and there are many reasons why you might create some code that is not ideal. If you just spend all day polishing it, you wouldn’t ship anything.

DAVID:

As junior developers coming in who want to get into open source, this is a great place. Code Climate will tell them. I don’t mean to be just pimping Code Climate. Wait, isn’t Code Climate one of our sponsors?

CHUCK:

Yes.

DAVID:

I’m totally pimping... [Laughter]

DAVID:

You guys need to be using it. It’s fantastic. But no, the point is that this is a great place for junior developers to come in and it’s like a jump in here tag. Here is where you can start. Please help us fix this. Do you guys, as the senior devs and the guys who know all this stuff, do you ever sit back and go, you know I just need some Zen personal time? I’m just going to pull up Code Climate and make something better.

SAM:

I do that occasionally, especially messes that I’ve created.

JAMES:

Yeah, I think maybe the one thing I would say, listening to you talk, I definitely agree that you can’t spend all your time improving and not shipping stuff. But on the flip side, I really feel like we sometimes undervalue the gain from improving something like that. When we have to go back and maintain it every time, every time you have to dip into that stuff and you think, “Ah, I don’t have time to fix this part right now,” you’re paying a price.

SAM:

I think that’s a great point. That’s a time when you should be cleaning it up. I think that’s the best kind of approach to take, is that you’re going, “Ah, again I have to deal with this huge function. I don’t understand what it’s doing.” Just clean it at that point. The flipside is going and cleaning up a file that you’re just not necessarily going to look at that much or isn’t that important. It’s not really part of your focus at the moment. You’re not thinking about solving these problems at the moment.

I think a lot is about the timing of when to do it and when to clean those things up.

DAVID:

Sure.

ROBIN:

I find personally, from a time management point of view, that it’s much easier when you have to go into that file to add a new feature to also clean it up while you’re in there. You’re already dedicating your brain power to re-understand how that class works to add the new feature. So while you’re in there, you might as well tidy. I find that I personally rarely will just say, “Oh, I’m going to do an F today and take it to an A.” But when I know I have to go in there, that’s when I’ll do it.

DAVID:

That’s part of my workflow, too.

SAM:

And we have some big challenges with big models. That’s something that we’re struggling with a bit. We’ve looked at separating bits of responsibility and functionality out of some of our big models like the User model or the Topic model.

ROBIN:

The Post too.

SAM:

But there’s a lot of logic in there and it’s very hard for us to figure out what is the right way of untangling this, both so Code Climate will be happy, but more importantly, so we can maintain it and stuff is in the right spot. That’s a place we need help.

CHUCK:

That’s where a lot of the good refactoring comes in and a lot of the things that Code Climate is trying to push you toward, is recognizing where the edges are, where the boundaries are, where the seams between different areas of functionality go. If you just go in and blindly refactor it so that you’d get a lower complexity score, you may break it up at the wrong place.

SAM:

Yeah.

ROBIN:

Yeah.

JAMES:

I think that’s true. I’ve been also glancing through the Topic model and it’s not too surprising that it’s a god object, because surprise, Discourse revolves around Topics. I think there’s probably some places to clean up there. One thing I’ve seen in there multiple times is methods that have the same prefix, like notify, notify, notify, notify as a prefix. It’s almost always a hint of an object that’s trying to break out.

ROBIN:

Yeah, for sure.

CHUCK:

So we’ve already been talking for about an hour and fifteen minutes. So this is probably going to be one of our longer episodes. But I have a couple more questions. One of them is what are the things that you’ve learned while working on Discourse? What are the big lessons for you guys over the last six or eight months or whatever?

ROBIN:

Wow. [Chuckles]

SAM:

The community is awesome. I’ve learned that I love working in open source, really. A lot of what I’ve learned is about the community and the way it reacts. Also, the Ember community has been great. But just the way that it’s hammered into you as an enterprise dev that you can’t make changes and to be in Discourse has just been such a great opportunity to be able to go and submit a patch here and a patch there to other projects and have a more active role in the community, as opposed to just floating and sucking on all of these projects and expecting them to work the way I want them. For me, that’s been a big thing, learning about the community.

JAMES:

I listened to a talk from Netflix. It was a while back. They put this idea in my head of one reason to throw your code out there is to affect the discourse, and I don’t mean your app, I mean the discussion, that by throwing it out there, this is a large open source Rails application that uses the heck out of Ember. You’re changing the way people will do Rails applications just by throwing this out there. That’s a cool thing. You can show people and say, “You know what? Ember works for this kind of stuff. Here’s how you use it.”

SAM:

And Rails scales. Rails can scale. [Chuckles]

ROBIN:

Oh, we already knew that. I feel really lucky to work on open source, too. I basically have the same answer as Sam. Just today, I was waiting for the elevator in my building and I’m like, “I work on an open source project full-time. How many people are lucky enough to be in that position?” I recognize how fortunate I am and how awesome it is. Just yesterday, I released a little micro-library for benchmarking Ember view performance under the MIT license. What’s great is I didn’t have to ask. I just know that that’s okay. I could just throw it out there. I can spend time working on it because it contributes to our project. It’s just been a really amazing experience.

DAVID:

Jealous.

ROBIN:

[Chuckles] The other thing I wanted to add was that it is interesting that we are, I believe, we’re probably the largest open source Ember app out there, possibly one of the largest Ember apps in production out there, period, which is pretty cool. But there’s also not too many, maybe you guys know more, but I haven’t seen too many examples of large Rails apps, too. We’ve got to be one of

...

JAMES:

No, there’s...

ROBIN:

Sorry?

JAMES:

There’s not. I agree with you.

ROBIN:

We’ve got to be one of the largest Rails apps too, even if we’re not using ERB for most of our views when we’re doing JSON, we’re still in a pretty awesome position that we’re one of these examples. Even though our Code Climate isn’t the greatest, we’re working on it. I think it’s a great opportunity to have an open source idiomatic Rails app out there that people can learn from.

DAVID:

When you say one of the larger Rails apps, do you mean of all time? Or do you mean...

ROBIN:

No, no, no.

JAMES:

Open source.

CHUCK:

Open source.

DAVID:

Open source. Okay.

JAMES:

Yes, it’s actually

ROBIN:

Ember, we might be one of the biggest. Sorry.

JAMES:

Go ahead.

ROBIN:

I was going to say, for Ember we might actually be one of the biggest Ember apps out there. But for Rails, no. There’s no way. You look at GitHub and all these other gigantic sites, no. But open source...

DAVID:

Yeah. Groupon.

ROBIN:

Yeah. But for open source, I believe that it’s a good chance we’re one of the biggest.

DAVID:

You guys are sitting close to about 10,000 lines of code. You’re in that ballpark. Your model directory has 7,800 lines of code in it. People coming from .NET, if it’s not 100,000 lines of code, it’s not a real app. But in Ruby, where a single line of code can say a lot, that’s a hefty, heavyweight app.

ROBIN:

It’s pretty big.

SAM:

Yeah.

ROBIN:

I think that might be one of the reasons why our Code Climate isn’t that good, to circle back.

SAM:

It’s actually fascinating because in rack-mini-profiler we have a .NET port and we have a Google Go port and everything. I was looking at the breakdown on GitHub. Ruby, nothing. Actually, the Ruby port of mini-profiler has the most functionality of all of them. It does more than every single one of the other mini-profiler things out there. For example, flame graphs is something that’s Ruby only, which is absolutely awesome that people should check out.

DAVID:

But it’s unrepresented statistically because it’s eight lines of code? [Chuckles]

SAM:

Yeah. It’s Ruby. You can just do it in seconds.

CHUCK:

Now, is that due to, we’re talking about the expressiveness of the language, but there are also some power features in the language. Does that affect it, too, some of the capabilities a language has with metaprogramming and other things?

SAM:

There’s that. There’s also that you’re building on very tight libraries that give you a bunch of functionality. Rack is awesome in the power that it gives you. A lot of times, in lots of frameworks, you’re just stuck doing what Rack does for free, manually.

DAVID:

I see the phrase monkey patch a lot in your gem file. I’m just thinking. How many thousands of lines of code would this be in Java or .NET if you had to rewrite the whole class hierarchy just to add a feature?

SAM:

Oh, yeah.

ROBIN:

Oh, yeah.

SAM:

Massive. That’s the other thing. The GitHub representative of Discourse is like, it’s mostly JavaScript and there’s a segment of Ruby.

ROBIN:

That’s wrong.

SAM:

But it’s not counting all the Ruby gems. It’s also counting all of the libraries like EmberJS and jQuery and whatnot and they all add up. Switch it over and get the real numbers and it’ll actually be much more leaning toward the Ruby [side of it].

ROBIN:

I think it’s close to 50/50. The way it’s displayed right now is 75 JavaScript, which is absolutely not true.

CHUCK:

I have two more questions. I’m going to ask them both at the same time and you can take it where you want, because they’re related. The first question is what is the hardest part of working on Discourse? The second question is what’s that part of the code where you cringe when you have to go work on it?

ROBIN:

That’s a good question. Let me think about that for a sec.

SAM:

For me, it’s actually occasionally I struggle with Ember. It’s just natural. It’s a young framework that is still going through a lot of change. The optimistic part of me is that I can see a way out of a lot of these things. A lot of my issues that I have with Ember are being resolved and being worked out.

For example, it has this concept of metamorphs where it injects a whole bunch of stuff into the DOM that is not going to be there in a few versions of Ember. The way that you set and get things, you have to do .get and .set all the time, which really annoys the bejeezus out of me. That is going to go away eventually, as well, as JavaScript progresses. So for me, that’s been a bit of a struggle as coming online with Ember and learning all of the bits and pieces of it. I’m sure that a lot of the community are, it’s way easier now than it used to be, say a year ago, but there’s still plenty of, I guess, room to grow and to improve things there.

ROBIN:

For the record, on the team I’m the one who adores Ember. It’s like it’s my, well it’s not my baby, but it’s the thing that I really hold dear. This is one area where Sam and I butt heads a little bit. The thing for me, the code path that I don’t like, is actually the rendering code path which is one of the most complicated ones we have. When I say rendering, I mean rendering a post. When you’re typing in a post, it does markdown, it does BBCode.

SAM:

Oh yeah, that’s very difficult.

ROBIN:

It can insert emoticons. It has tons of rules. It’s all fairly well-tested. Actually, I’d say it’s one of the best-tested parts. But it’s just layers of before you render, you do this, after you render, you do this, and if there’s a quote you, you do this. It’s become very difficult. Every time I have to jump into that text rendering pipeline, I’m a little worried.

SAM:

We have one on the server and one on the client. It gets complicated.

ROBIN:

That’s one of the interesting parts, is that it actually is mostly the same code path. We have it in JavaScript because we do a live preview as you type. But on the server-side we actually run it within, what’s it called?

SAM:

Ruby Racer.

ROBIN:

The Ruby Racer, to actually run the same rendering to make sure that people aren’t [spoofing].

SAM:

That’s something I learned at Stack Exchange, that you’re in for a world of pain if you’re not using the same markdown parser on both sides.

ROBIN:

Speaking of markdown parsers, there actually is light at the end of the tunnel for this rendering thing, because there seems to be a much better JavaScript markdown rendering library that we’re going to consider swapping over to that will hopefully alleviate some of these concerns. I know it can’t do all of them, but it should help.

CHUCK:

Have you guys contributed as much to the JavaScript and Ember community as you have to the Ruby and Rails community?

ROBIN:

I don’t know. That’s hard to say. I’m certainly active in the Ember community. I released that view thing the other day. We’ve done a lot of things like profiling performance and creating the app. But I don’t know that I’ve contributed as much to Ember as Sam has to, say, Rails. I don’t think that’s true.

SAM:

I think you need to look at contribution as not just code. Robin blogs very, very aggressively about all of these things. I think that’s just as legit a contribution as changing code inside Ember.

ROBIN:

Yeah, that’s true.

SAM:

And he’s done tons of that. There’s not a week that goes by that there’s no new, “This is how you do this in Ember. This is how you do that.”

ROBIN:

Yeah, I blog a lot about it. I run, in Toronto, the EmberJS, I help run the meet-up night and stuff like that. I’ll talk about it whenever anyone ever wants me to talk. I’m active that way. But in terms of actual contributions, I haven’t done as many. But I have helped people debug things and come up with JS Bin reports and stuff like that. But it’s not exactly the same.

CHUCK:

Alright. Well any other questions, you guys?

DAVID:

No.

JAMES:

No. I’m good, thanks.

SAM:

Where’s Avdi? Where’s Josh? [Chuckles]

JAMES:

They had the night off.

CHUCK:

Yeah. [Laughter]

CHUCK:

To the listener, we’re recording about 10 hours out of sync from when we usually do.

SAM:

Yeah, it’s my fault. Australia’s too far.

JAMES:

It is very far.

CHUCK:

Well we get people from Europe. It’s just too far in the wrong direction. [Chuckles]

CHUCK:

But yeah, we’re really happy to have you on the show

DAVID:

Absolutely.

CHUCK:

And really appreciate all the work that you’re putting in both in the Ruby and JavaScript communities as well as in Discourse itself. For those of you who don’t know, we just switched the Parley mailing list over to Discourse. It’s a whole lot easier to manage. I think the discussions are really much better facilitated.

JAMES:

With categories and everything. We’re super excited about it.

SAM:

Awesome.

CHUCK:

Yeah, it’s wonderful.

ROBIN:

Please give us all your feedbacks so we can continue to improve it.

SAM:

And pull requests. As many as you want.

ROBIN:

Yeah, also pull requests.

JAMES:

It turns out we have this mailing list of a bunch of Rails hackers. [Chuckles]

CHUCK:

And I know that you guys have been getting some feedbacks. That makes me feel good, too.

SAM:

There was one little question that they had. I was just looking through. We can quickly address some of the little questions that people had. Why did we drop IE and not support that? Discourse is an IE 9 and up project. That was Jeff’s call and I completely agree with it. It’s just the intention was to be a modern application. You just have to leave some of these things behind. IE 8 has just a whole bunch of features that are missing internally that just make a whole bunch of things in JavaScript a lot more complicated. Even IE 9 doesn’t have pushState properly. So you can’t muck with the URLs. You’re stuck using those hash URLs, which are a lot uglier and less correct. That was the big reason that we dropped all the IEs. I guess that was one question there.

CHUCK:

When you’re talking about dropping old IEs, does that mean that you’re not testing it in the old IEs?

SAM:

It won’t work. It just doesn’t work in IE 8.

CHUCK:

It just won’t work?

SAM:

If you try and launch Discourse in IE 8 and below, you’ll get a message saying please upgrade to a modern browser. IE 8 is not supported.

ROBIN:

In fact, I haven’t committed it yet, but I have a pull request upgrade to the latest jQuery, 2.0.3 and I think it doesn’t support IE 8 anymore.

DAVID:

Wow.

CHUCK:

Yes. That’s correct. 2.0 does not support IE 8 and lower.

ROBIN:

So I think, assuming I can commit that safely, I hope the warning message still works. That’s a good note. I have to make sure that I don’t break that.

CHUCK:

It was interesting that I think the jQuery maintainers basically said that when they dropped the old IE support, all of a sudden there was more code to support Firefox I want to say, than IE in jQuery. They dumped a ton of code that was in there to compensate for it, just a crazy amount. It’s very interesting to hear you talk about it. Anyway, we’re getting toward the end of our time here. We should probably get into the picks. David, do you want to start us off?

DAVID:

Sure. I’d be happy to. I keep a list of things that I want to pick and this is a very special pick for me. People listening to the previous episode know that we recorded it at the retreat. I just want to go on record saying that my picks for today are Dana Gray and Mandy Moore. @dgrayinok on Twitter and @therubyrep on Twitter. Ladies, you saved us. The retreat could not have been possible, would not have worked anywhere near as well as it had. Dana literally went shopping. Well, both of them went shopping for us. They cooked for us. It was fantastic to be able to have our minds clear and to be able to be focused and not have to worry about all the logistics of surviving at the retreat. It would not have been possible without Dana and Moore. So I just want to take airtime on our show, on our podcast, to all of our listeners to say that James’ wife Dana and Mandy who left her family at home for a week to come hang out with a bunch of nerdy hackers, thank you so much. You guys were absolutely fantastic at the retreat. The reason the retreat was so wonderful for all of us is literally because of you two. So thank you Dana, thank you Mandy. I really appreciate it.

JAMES:

And the one time David almost died, neither of them were present. So that pretty much tells you [Laughter]

CHUCK:

I actually had some picks, but I just want to pile on with Dave. It was a terrific experience. They really did make it happen. I can’t really add much more than that. But thanks, big thanks. James, do you have some picks for us?

JAMES:

I do. I’ve got a couple. We use a Campfire setup at work, talking on Campfire. We have a couple of different rooms and stuff. For a while, I used the website and then for a while I pulled it out into a Fluid app and that was okay. But I actually wanted a little bit more, like the ability to switch rooms with keyboard shortcuts and stuff like that. I asked on Twitter and several people turned me on to Flint app. It’s in the app store. It’s a little pricey, but for low hassle. It’s gorgeous. It has what I said, keyboard shortcuts to switch between rooms. It has lots of customizable notifications and stuff like that. I find that really helpful for Campfire. Then while we were at the Rogues retreat, we made awesome use of Josh’s Jambox as we did multiple calls with Kent Beck and our publisher to talk about the book. He had this awesome Jambox. Then right when I got back, I found myself needing almost the exact same thing for a local event, a portable speaker/mic thing. I went and I didn’t get a Jambox, but I found a JBL Flip, which is real similar in what it does. It gets good reviews and I’ve used it once now locally and it’s just exactly what I needed. If you need a portable speaker/mic, speakerphone Bluetooth thing for your computer, you probably can’t go wrong with either the Jambox or the JBL Flip. Both are cool stuff. That’s it.

CHUCK:

Nice. Now I have one more thank you I have to put out there. That is Brian Stevens. He is on Twitter. You can find him on the web at DataPorters.com. He’s @bdstevens on Twitter. Anyway, what happened was I was trying to move things around because we wanted the Parley Discourse to be at parley.rubyrogues.com so I had to move the signup page, because that’s where it was hosted. I was having no end of trouble with nginx trying to get it to work. He actually looked at my nginx config and in about two minutes sent me back, “Here’s what you need.” It worked just great. I also want to say a thank you to that just because it’s related to Discourse and how we got this all setup and stuff. So thanks Brian for helping me out. Let’s see. Sam, what are your picks?

SAM:

I’ll actually pick something we talked about. We talked about moving our markdown to this new markdown library. It’s called marked. It’s a proper lexer/parser for markdown that can be used. I see it has a lot of potential and I’m thinking of using that at Discourse so I’m going to plug that as one of my picks. The other one that is, I notice Adobe gets a lot of bashing a lot of the time, so I’m going to plug an Adobe product [Chuckles] because why not. No, I use Adobe Photoshop Lightroom a lot for my photography. I think it’s an actually awesome product and one of the best ways of managing large amounts of photos. I think it’s definitely worth looking at if you’ve got a large collection of digital photos.

DAVID:

Cool.

CHUCK:

Nice.

ROBIN:

My turn?

CHUCK:

Yup. Go ahead, Robin.

ROBIN:

I just have one pick. I guess it’s not exactly tech-related, but there’s a website that I really like called Feminist Frequency, which is a series of videos that are totally free. They basically examine representations of women in pop culture and stuff like that from a feminist perspective but also break it down into words and concepts that are really easy to understand. In particular, they have a new series that you guys might have heard of called Tropes vs. Women, which analyzes actually video game tropes, like the damsel in distress and stuff like that, from a feminist perspective. I just love those videos, so I thought I’d give that a shout out.

CHUCK:

Sounds interesting.

DAVID:

Cool.

JAMES:

I listened to the video game one. It’s great.

ROBIN:

Yeah. Anyway, she has tons of them on all sorts of, I love pop culture and it’s really interesting when she gets down into it.

CHUCK:

Cool. Alright. Well, we’ll go ahead and wrap up the show. Thanks for coming again, guys.

ROBIN:

No problem.

SAM:

It’s a pleasure.

DAVID:

Yeah, this is awesome.

SAM:

It’s great.

CHUCK:

      Alright.     Well    if     you     want     to     contribute     to     Discourse,   you        can        find     it      at

GitHub.com/discourse/discourse.

SAM:

And we have a topic on Meta that outlines ideas of stuff you can do, so we’ll link to that from the show notes.

DAVID:

Do you have a Discourse set up for people to discourse about Discourse?

CHUCK:

Yeah, that’s Meta.

SAM:

We sure do.

JAMES:

Yeah. It’s called Meta discourse.

CHUCK: meta.discourse.org for that.

DAVID:

Cool.

ROBIN:

And we also have try, which is if you just want to see how the site works and our various features, you can go there and post whatever crap you want. It gets reset every 24 hours or so.

DAVID:

Cool.

JAMES:

Some of our listeners have asked, the Parley Discourse runs on the same Discourse everybody else uses. We’re not using a special fork. If you want to make Parley better, all you’ve got to do is go make Discourse better.

SAM:

That’s right.

CHUCK:

Yup.

SAM:

And it’s in fact using the same processors that are serving out Meta Discourse and all the rest of them.

ROBIN:

You can be pretty sure it’s on master.

CHUCK:

One other question I want to ask, and this is something that I’m trying to get in the habit of, are you guys going to be at any conferences or events that people can find you at?

SAM:

Yes. We’re both going to be at GoGaRuCo. I’ll be speaking with Jeff about performance, in particular about measuring Ruby and all sorts of tricks you can use.

ROBIN:

And I’ll just be [inaudible].

SAM:

Yeah. All the Discourse team will be there, so that’ll be exciting.

CHUCK:

Nice. Well Josh will be gratified that he got me thinking about asking that question now. [Chuckles]

JAMES:

That’s awesome. I’m going to be there, so I’m going to come say hi.

SAM:

Awesome.

ROBIN:

Please do.

CHUCK:

Alright, well let’s wrap up the show then. Thanks for coming again and we will catch you all next week.