LAURIE:
The inevitable appearance of Darth Vader will always happen.
[Hosting and bandwidth provided by the Blue Box Group. Check them out at Bluebox.net.] [This episode is sponsored by Component One, makers of Wijmo. If you need stunning UI elements or awesome graphs and charts, then go to Wijmo.com and check them out.] [This episode is sponsored by Peer60 Incorporated. Peer60 Incorporated knows that the best JavaScript developers hone their skills by listening JavaScript
Jabber podcast. If you’re looking for a frontend or full-stack development opportunity, helping Fortune 100 companies understand their customers better, email jobs@peer60.com.] [Do you wish you could be part of the discussion on JavaScript Jabber? Do you have a burning question for one of our guests? Now you can join the action at our membership forum. You can sign up at JavaScriptJabber.com/jabber and there you can join discussions with the regular panelists and our guests.]
CHUCK: Hey everybody and welcome to episode 99 of the JavaScript Jabber Show. This week on our panel, we have Aaron Frost.
AARON: Hello.
CHUCK: Jamison Dance.
JAMISON: Hey friends.
CHUCK: AJ O’Neal.
AJ: Yo, coming at you live from the place where Google Fiber is. Provo.
CHUCK: Merrick Christensen.
MERRICK: Hey guys.
CHUCK: I’m Charles Max Wood from DevChat.TV. We also have a few guests with us. We have Isaac Schlueter.
ISAAC: Hello.
CHUCK: Laurie Voss.
LAURIE: Hi.
CHUCK: And Rod Boothby.
ROD: Hello.
CHUCK: So, the reason we have this crowd here is that there is some rumor going around that npm incorporated.
ISAAC: That is correct. We did incorporate. We’re a legit company now.
CHUCK: Yeah. And you raised a bunch of venture funding, venture money.
ISAAC: A little bit, yeah. Not a huge round, as big startup venture funding rounds go, but enough to do some really interesting stuff over the next year or two.
MERRICK: Cool.
CHUCK: I’ve heard a lot of people saying all kinds of things about what this means for npm and Node. Do you guys want to just briefly talk about what you plan to do with the money or the company and the direction that you’re heading in now?
ISAAC: Sure. I’ll speak to that. The main thing we get with this money, the main thing that we’re spending it on, is infrastructure for the npm registry and website and assorted services, and also hiring people to work on those things full-time and make them better. Also, our long-term goal is not just to keep raising money, but to actually produce revenue. And so, we’re doing that by creating new products and services for companies that need them, companies that have been literally demanding and begging that we figure out some way to take their money in order to give them additional services on top of npm.
MERRICK: Like what kind of services?
AARON: Yeah, you guys have a documented list of those services or can you talk about them?
ISAAC: I’m not a big fan of vaporware. It’s gotten me in trouble in the past to make promises about what we would deliver before we actually have a thing that we can deliver. But that being said, the first item that we’re focused on is figuring out a way to do hosted private modules and a way to make it so that you can have a high degree of convenience in hosting private modules without having to host a whole replica of the entire npm registry, and also manage who has access to those modules, who can see them, who can publish to them, and so on, and just making that whole story a lot nicer especially for enterprise customers that have a lot to gain by using npm inside their company in a
much deeper way.
CHUCK: Now, would they self-host those modules or would they be hosted on npm and only have certain people have access to you?
ISAAC: No, we’re going to continue to host them. And I think that actually in the long run, that’s a very valuable thing to be able to do. We’ve heard from a lot of companies that they really just want to use the exact same workflow that they have today where their developers can publish things and other people can install those things on the other side of the company without having to get checked by some secret cabal of people or whatever. And they like npm. Npm is being used inside of a lot of companies. But then, they can’t use it for their private stuff and it gets really hacky and inconvenient to have multiple different systems for different sorts of things. So, we’re going to try and make that really streamlined and easy for them. And like I said, I don’t want to go too far down the path of saying how that’s going to work because it doesn’t exist yet. So, it’ll probably change before it does. And I don’t want people to get too attached to any implementation details before it’s been implemented.
MERRICK: That’s cool though. That’s an awesome feature.
JAMISON: It is. I just wanted to say that’s a pain point that we’ve had as Node users. We’ve had to dance around a lot to get around the fact that we can deploy via npm installing unless we have private modules that we don’t want public on npm. So, that’s a cool story.
LAURIE: One of the things that we’ve discovered from literally every company that we’ve spoken to about what is it that you’d want out of us is that they all go, “Yes, that is exactly what we wanted, that thing. Let us pay for it yesterday.” As a company selling a service, it’s a really great thing to hear that all of your customers have always wanted the thing that you’re about to build.
JAMISON: It’s not a bad problem to have.
MERRICK: Is that then the monetization strategy? Keep regular npm free and open, and private modules you pay for a number of modules? I know you don’t want to talk about the details exactly. But I wondered how do they plan on monetizing npm?
ISAAC: So, I think there’s a common saying in, I don’t know who said this first and I certainly am not the one who did, but if you’re not the customer, you’re the product. I think that a lot of times, in reference to something like Facebook or Twitter, it’s pretty clear what you’re getting. They’re showing you ads and you’re showing up to look at those ads. So, you’re like the product and the advertiser is the customer. In the case of npm, or something else where it’s a community kind of activity, it’s not always bad to be the product, right? You could say the same thing about GitHub. You could say GitHub, if you’re using just public open source code, by showing up at GitHub, you’re making GitHub relevant and you’re increasing their importance in the tech ecosystem. And I think that is an example of being the product, but it’s also as the product, you’re getting a lot of good out of it. When it comes to a situation where you want somebody to host something that is private, or act as your agent in some kind of delicate way, like holding onto some secret stuff that you don’t want the world to see, in that case you probably don’t want to be the product. In that case, you probably want to pay for it. And that sets up a whole different kind of relationship, because now they have a fiduciary duty to treat you at least somewhat responsibly. You’re going to think about it as some kind of a contract. It’ll be a little bit more binding. And so, the way that we’re approaching it is basically the free stuff will stay free because honestly, it’s best for us if it’s free. In the long run, it just doesn’t make sense from a business point of view to start charging for stuff that’s currently free. All that’s going to do is make people go elsewhere. But if we’re going to be hosting secret stuff, private code or things that you use in your website that you don’t want the world to see, in that case yeah, you should pay for that. As a person who hosts, who runs websites and stuff, I wouldn’t trust someone to host that for me unless I was paying for it.
CHUCK: Yeah, I really also like the model where, like you said, people are using npm as part of the workflow. It’s stuff that they use. It’s stuff that they need for their applications. They don’t have to do anything differently other than they go in. They pay to be able to make some stuff private. But for their developers, the rest of it is just the way they’ve always done it, which is really, really slick.
ISAAC: Yeah.
JAMISON: I wanted to step in and just ask really quick about the history of npm up until now, how it started off and it went through a bunch of transitions before ending up in your guys’ hands in the form of a company. Do you want to talk about that a little bit?
ISAAC: Sure. So basically, I wrote npm shortly before coming to Joyent. And I’ve been on JS Jabber a few times now and told the history of that part of things. While running the Node project for the last couple of years, it became increasingly clear that the thing that was holding Node back more than anything else was the fact that some of these enterprise and business friendly features didn’t exist in npm. So, big companies who were trying to use Node as part of their infrastructure had a really hard time and wasted a lot of developer cycles either reinventing the wheel or just dealing with the kind of friction that happens when you have multiple different ways of managing code. So, while at Joyent actually, Rod and I got to talking about what the future of npm would look like, what the future of Node would look like. And it was a bit of time before we formally decided, “Hey, let’s do a company. Let’s do a startup here.” And yeah, so it came together extremely fast. A lot of people I
know had told me “Get prepared. Get ready. You’re going to spend six to nine months talking to investors and they’re all going to jerk you around.” And I was prepared for that. I was like, “Okay, well you got to do what you got to do.” It came together extremely fast, I think. The guys at True basically, I think we weren’t even actually incorporated yet, but they were like, “Okay, well how soon can we talk terms?” That’s also good problems to have, right? There’s a lot to be said for their kind of insight into how this is going to impact the tech scene and then also how it’s in its right, a pretty viable business.
ROD: I’m convinced that it was all Laurie that got the pace of it going. Because when the three of us sat down for the first time, Laurie said, “Yeah, okay. So, I’d like you guys to be ready in six weeks.” And somehow, six weeks later, we managed to get funding, closed it, and started up, have office space, everything. We’ve been very, very lucky. And the team at True has been amazing. VC’s, some of them are pretty cynical. The folks at True are like a family. We hadn’t even closed the deal before they were offering us space in their office while they’re busy doing a thing like raising their fourth round, feeding us a lot of chocolate, helping us with every single aspect of the business in terms of thinking about how we scale up or how we can save money by outsourcing the finance function and who to work with and all that kind of stuff. Very mundane, but it’s made it really easy to focus on the most important thing, which is the community and the core open source project.
JAMISON: That’s super exciting. It’s awesome. So, I know there’s been some pushback or questions about forming a company around the package registry and npm in general and stuff. And I think a lot of that comes from the example of other communities like Ruby or Python where they have non-profits. And it’s this benevolent, free, it’s like a resource that’s provided for you but there’s not a company around it and there’s not any paid services around it. Can you talk about why you chose to go this route instead of set up the npm Foundation similar to what RubyGems did or something like that?
ISAAC: So, setting up a not-for-profit foundation is absolutely something that we’re considering. There’s a few pros and some cons to going that route. Doing a foundation is still on the list of possibilities. Basically, in the short term, it’s a little bit faster to get a company set up. And I actually wanted to. I saw an actual business opportunity here and wanted to go and pursue it. So, we were going to do that anyway. As far as having a foundation that owns the open source code, that’s still possible. But in order to do that, we would have to set up those legal entities and transfer ownership of those things and also figure out how to fund that. You still have a lot of politics involved in something like that. And you still have a lot of interested players and investors and interested parties who are going to try and nudge things in their direction. In fact, in some ways, it can be more political because you can’t just say, “Well, revenue says we do this.” And so, by doing the company first, we actually keep that option open. But we can pursue the immediate needs of the community right now, which is actually delivering solutions for npm within the enterprise.
CHUCK: Something that’s slightly related to this as far as a foundation or a company and who owns what and who controls what, what about the code licensing? Because npm was open source, (I’m assuming it’s still open source. I don’t know why I said ‘was’), a lot of people have contributed to it. How do you get them to grant a license to npm, Inc. or whomever in order to open that up so that you can use it commercially?
ISAAC: Well actually, the npm license already allows anybody to use npm commercially. The npm client itself is open sourced under the Artistic License 2.0 and what that says is basically you can do anything. I’m not a lawyer. Don’t treat this as legal advice. But basically what that says is you can use npm in almost any way you want, very similar to MIT or BSD. The additional restriction is you can’t change it and still call it npm without sending those changes upstream. So, if you said, “I’m going to change npm so that it installs from PyPI and uses all of PyPI’s things,” or whatever, you’d have to give it a new name. And that’s actually good, because otherwise it’s going to be really confusing and stuff. And the other thing is you can’t charge a license fee for npm itself. And as far as the multiple contributors, they still own their stake in npm. They copyright statement right now changed. The big difference is the copyright statement changed from Isaac Schlueter and npm contributors to npm, Inc. and npm contributors. The vast majority of npm was written by me and a relatively small number of other people and we still have a really good relationship with all of them. This company is built on open source. We live and breathe open source and we’re going to keep keeping things open. Even a lot of our paid projects, we’re looking at, “Okay basically, what can we get away with open sourcing?” And is there a way that we can have our whole infrastructure still be built on open source pieces and just have the actual private data private? So, it’s not really a conflict, as far as I’m concerned.
CHUCK: That makes sense. What about the repository of node modules?
ISAAC: We will have to do some things to make things a little bit more regimented and rigorous and get away from some of the loosey-goosey way that things have been done in the past, which you can get away with when you’re an individual but not so much when you’re a company. In particular, I’m talking about stuff like Terms of Use and Terms of Service stuff, which we’re in the process of making all that rationalized. Historically, basically the npm registry is owned by me. But all of the things that are in it are owned by their respective publishers. And I’ve reserved the right to delete or remove or modify things as necessary in order to keep it going good. So basically, we need to get a more formalized Terms of Service that shows that.
AJ: So, one thing that I thought about a couple of years ago and almost acted on but didn’t, obviously, was there was an app store for a
decade before Apple came out with it. Linux has had repositories and had clickable download stuff interfaces for a very long time. Apple took that same idea, attached a dollar per item, and boom. It exploded into a huge thing. It was also on the phones, which was a completely new market. But Apple took an idea that everybody, well that was in the Linux community, was already using and they put it on their Mac desktop and they put it on the phone, and it exploded. And I thought, well we have all these communities of software that’s mostly free. But maybe there’s an opportunity for something where to have a license to this module, you pay a dollar, which I think is good for a lot of people. Because it doesn’t mean necessarily that somebody’s going to get super rich, but it does help fund that hobbyist development. What do you think about that kind of model?
ISAAC: There have been folks who have asked us to implement that. We haven’t figured out if or how it would be good to do that. But it’s definitely on our list of things to investigate and figure out and see if there’s a good way to do it. The biggest problem and the reason why, another reason why I wanted to be the one to be at the head of this, this npm, Inc. company is that bringing money into the equation can change things in really dramatic and subtle ways. And the minute you start saying, “Well this arg parser costs a dollar and this other one doesn’t,” well now you have an incentive for the author of the paid one to go and trash talk the free one or vice versa. And people act differently when there’s a dollar sign attached to it. That’s not to say that it can’t be done in a good way. It’s just to say that there are some very real hazards. So, we’re approaching that very carefully.
CHUCK: I want to talk about some of the other concerns that I usually here when this kind of thing happens. I
spend most of my time programming Ruby and I remember when Heroku hired Matz, Yukihiro Matsumoto who actually continues to push forward the development of Ruby. Or there have been several other libraries that have been brought in-house by various companies that want to sponsor them or things like that. And the concern always comes out to be, “Well, then they’re not going to be as interested in the community,” or, “They’re not going to be as interested in keeping things as open and transparent as they have been,” and things like that. And it seems like you’re approaching it from the right direction talking to you now. But do you worry that if you have an investor that has some influence or some other entity that buys additional stock in npm, Inc. that things might go sideways one way or the other?
ISAAC: I think if there’s an investor who’s going to buy something and then destroy it, that’s not a very good investor and we would be fairly unwise to get involved with them. We did a lot of diligence with True before deciding to accept their money. And also, we retained a lot of control over the company. I can’t go into details of exactly how our cap table is structured, but I’m not so worried about us getting taken over. We had a very valuable thing, they recognized that, and they've invested accordingly. And I think that they really understand that the value is coming from the open source community. They’ve had investments in other open source things that they’ve managed very wisely. And ultimately, it’s easy to say that VC’s are bad, and certainly a lot of them are. [Chuckles] It’s a little overly simplistic, because VC’s are people. And some of those people understand how open source works and some of them don’t. And it’s the same with anything else. You could say, “Oh, Sally broke my heart and therefore women are the worst,” but it’s not really a reasonable conclusion to draw. There are lots of people with a lot of different approaches to investment. I’m absolutely doing the CEO thing and not answering your question. [Laughter]
CHUCK: Yeah, I’m trying to ask all of the hard questions and address some of the concerns people have about some of this stuff.
ROD: One of the things that are indicative of how much we’ve thought about this is the kind of people that we’ve brought in. So, when we started down this route, actually the guy we called was Toni Schneider who was with Matt Mullenweg at WordPress for a long, long time and is probably the mellowest guy to ever be a VC. He put us in touch with Puneet. And Matt Mullenweg himself is an angel investor in this effort. And we went to people who have done open source, who believe in open source, who spent a lot of time around it, and understand the value of it and understand that you have to balance things and grow them while still taking care of the community. It’s not always perfect. But hopefully that indication of years of, particularly Isaac’s effort, to dedicated effort to open source and then all the people that we’re bringing in is an indication that we want to push this in a way that does the right thing.
MERRICK: Right. Isaacs, when you started building npm, did you ever have an idea that this someday might be a business that you would lead?
ISAAC: Not really, not originally anyway. I wrote npm because we needed a thing that does what npm does. I wrote a thing that would be a
package manager for Node programs because it was a big pain in the ass to install Node programs that people were talking about on the mailing list and I couldn’t actually use it in my programs. So, it just seemed like we needed a thing that did that. And over time, it’s grown into this very big and relevant community force. And I think there’s a lot of really interesting places that we can take it.
MERRICK: Sure.
ISAAC: I always thought I would do a startup about something else. [Laughs]
MERRICK: Sure.
ISAAC: This wasn’t very high on my list that I want to write a package manager company. That didn’t even make sense. Some would say it probably still doesn’t. But it’s something that’s become source of value. And there are additional ways that we can add value. And some of those ways will make us a few bucks. So yeah, if I can keep doing this as my job for a while, that’d be great.
MERRICK: Very cool. And the other question I had was Node is starting to see a lot of play in the browser space. People are starting to refer to them as Node packaged modules instead of Node modules that are packaged by Node as well. And I’m wondering if you see npm expanding beyond just packaging up Node code around.
ISAAC: I think that using npm and the Node style module pattern in client-side JavaScript is absolutely catching on and it’s a really, really interesting thing. I would like to hand the mic over to Laurie in a minute. But basically, I just want to point out that npm actually stands for No Prescribed Meaning. So yeah, it’s Node Packaged…
JAMISON: [Chuckles] I thought I was Node Package [inaudible] [Chuckles]
ISAAC: It is. No Problem, Meatbag. So, it’s whatever you need it to be.
ROD: Norwegian Polka Music. [Chuckles]
CHUCK: Watch out. AJ might start humming some. [Chuckles]
MERRICK: I want to hear Laurie’s take on this web browser.
LAURIE: So, the reason I immediately wanted to chime in on that stuff is because I’m ridiculously passionate about web stuff. And when Isaac was first talking about Node to me back in 2008 when we used to work together at Yahoo, the first thing I said was, “Well, can I use it to make webpages?” And he was like, “Well, not really.” There’s this thing called EJS but it doesn’t really work and there’s this ExpressJS thing. And the story just wasn’t there. And I was like, “Alright, call me when it actually makes webpages.” And then four years later, it makes webpages. And he called me and was like, “Hey, we should do this thing.” And I was like, “Okay, yes,” because I’ve been looking at the web space for my entire professional career. And what really gets me going is when I’m like, “Ah, this would make it easier to make the web bigger.” And that’s all I really ever want to do. So, I think Browserify in particular is a gem of the Node ecosystem that not enough people understand and not enough people use. And there’s a bunch of other stuff in that space. There are Sass compilers and the way that you can use Grunt to get your asset pipelines in order and stuff like that. Node’s not there as far as web development is concerned yet, but it’s nearly there. And one of the big things that I personally want to do as part of npm is make Node the way that you build web pages.
MERRICK: Very cool.
LAURIE: That’s really what I want out of it.
MERRICK: That’s awesome. It sounds absolutely terrific.
JAMISON: Aaron, I know you had a question you wanted to ask a while ago.
AARON: Yeah. So, I know you guys can’t talk about all the new stuff and the possible stuff coming in your newer versions. And that sucks because I was super excited to hear all you guys’ stuff. But maybe you could talk about some of the existing things that are going to change or some of the existing things that are going to go away. I think that that might be something interesting for me and maybe for the listeners as well.
ISAAC: Some of the current things, this actually isn’t an npm, Inc. thing. It’s just an npm thing that has been a thorn in many people’s sides for quite a long time, which is that the ability to published a changed version of a package, like I have version package at 1.2.3 and I can un-publish that version and then publish it again with different contents at the same version number. It’s not really something that was ever supposed to work. It just did and we didn’t lock it down. So, people started taking advantage of it and depending on that behavior. And then others were occasionally extremely frustrated by it because they would get new bugs even though they didn’t change the version number. They’d have two different instances of their program running and getting error numbers with different line numbers and really confused. [Chuckles]
ISAAC: It’s like, “We have the same version installed. Why is it different?” I saw this at Joyent a lot when I was working there. I just never got around to preventing that. Though recently we did actually roll out a change that makes it so that you can never reuse a version number of a given package. And people have generally reacted with either total elation or total outrage at this, or some combination. [Laughter]
JAMISON: I know I heard some wailing and [inaudible].
MERRICK: Yeah, I definitely saw some raging as well.
ISAAC: Yeah.
CHUCK: Yeah, but it only takes one time where you build an application and then you deploy it to another machine, say production, and it doesn’t work.
ISAAC: Right. It’s one thing if you just, “Oh, that module’s gone.” Okay, they un-published it. It got deleted. Maybe there was something leaked that shouldn’t have been in there, or some horrible security bug, whatever. Maybe it’s good that it broke. When it works but is different, that’s actually much more hazardous. So, that is no longer an option. That’s no longer a thing you can do.
JAMISON: And I think the pain point of that is basically some laziness on the part of publishers. It is less convenient. I can see their objections to it. But like you said, the upside is that hopefully you won’t break production in incredibly difficult to diagnose ways.
ISAAC: Right, right. You’ll break production in very easy to diagnose ways.
JAMISON: Yup, which is so much better, and better than what I do now.
CHUCK: Well then, you eliminated a whole ton of, “Well, it works on my machine.”
ISAAC: Yeah. Yeah, exactly. So, the other thing that’s changing or has already changed and is going to keep changing a little bit is we have a more advanced and carefully thought out infrastructure at this point. We have a single write master with multiple read slaves. We have monitoring setup on all different things. Laurie can actually talk more about this because I’m in the lucky position now where I don’t actually know how everything works. [Chuckles]
LAURIE: Yeah, I guess if there’s one thing that I really, really want to go away, it’s downtime.
JAMISON: I was going to ask about that.
LAURIE: And we [laughs] we’ve sort of been, npm’s been infamous for being randomly down a lot, which is something that at my last company I got a lot of experience in keeping things up all the time. And that’s definitely something that we’ve already made huge progress on. We’ve hired some really, really great people who know DevOps insight. What they’re focused on right now, they’re sitting three feet away from me listening to me saying this, is that they’re focused on making sure that we can’t prevent it ever going down but we should know within two minutes what went wrong. And we should have a plan to make sure that that gets replaced as soon as possible and never goes wrong again. And we are making enormous progress on that. We were previously talking about, “Oh, it didn’t work on Sunday.” We’d be talking in terms of a day when it didn’t work very well. And now, we’re talking about, “Oh well, we had seven minutes of downtime. Here’s why that happened.” So, we intend to keep focusing on that. And as Isaac was alluding to, part of the way that’s happened is that we radically changed, well not too radically. We made a major change to the way that our backend database is working which was previously we had one giant CouchDB that contained all of the package metadata and it contained all of the packages themselves. All of the binaries were in this single giant database. And I don’t know how many of you have spent a lot of time with databases, but one of the big things that everybody who’s spent a lot of time with databases knows that you should never put your binaries in the database. It’s a terrible idea. It always goes wrong.
CHUCK: [Laughs]
MERRICK: Is that even true of CouchDB then, huh?
LAURIE: I have never met a database in 15 years of which it is not true. And it’s definitely not true of CouchDB. You definitely shouldn’t do that because you are taking this thing which is meant to sort and organize data and you’re giving it binary data which it can neither sort nor organize. It can’t do anything with that data other than get really fat. So, we moved the binaries out of Couch. So now, we have a much, much, much smaller Couch, which is just the metadata. And we’re encouraging people to replicate that and let us take care of the binaries, or let them mirror their binaries separately as binaries without having to mirror and entire database full of binaries. And that would work really well if it had just started that way from nothing yesterday. But unfortunately, obviously we didn’t use to do it that way. So, we have some people who depend on the way that it used to work. And so, we have jumped through a fair number of hoops to make sure that it remained backwards compatible and there is still a copy of a Couch that has all of the binaries in it. And you can still replicate from that the way that they used to. And it still works as well as it used to, which is to say not very well. [Laughter]
MERRICK: So, where do you put the binaries now? Are they just on some sort of CDN?
LAURIE: They are at the moment sitting on Manta, which is a super awesome S3 plus Hadoop hybrid thing that lets you store binaries and also operate on binaries in a distributed way, which is one of the coolest things that Joyent has every come up with. We are in the process of putting that in more than one place, because as good as Manta is, there’s only one of it. And it’s not up 100% of the time. So, we are putting some engineering effort right now as we speak, again into taking those binaries and putting them in multiple places. Everything that comes out of the registry right now is sitting behind our CDN or other geodistributed CDN cache thing called Fastly, who have awesomely donated all of their services to npm the open source project. So, very little of the binaries gets actually served from us. Very little of that stuff is coming from our servers directly. It’s mostly being distributed by the CDN. But the ultimate source of it is Manta and soon a couple other binary stores that we’re going to be setting up.
MERRICK: Sure. I guess also, removing your version publishing helps you not have to invalidate your CDNs, your Fastly I should say.
LAURIE: Yeah. We purge the cache pretty actively anyway. But not having a binary that can randomly change from one binary to another without changing name certainly simplifies that.
MERRICK: Very cool.
CHUCK: So, one thing that I just wanted to point out, it seems like some of the things that I keep hearing from other open source providers, people who write open source or manage open source projects, is that they really need people to have more time to work on things. And that’s one of the things that I think is really exciting about you guys incorporating and having a round of seed funding, is just that it sets things up so that you guys can go fulltime on this stuff and solve a lot of the problems that we have and give us better features. And it’s not just coming from a
place of you love the community and you want to give this stuff to us, which having talked to you guys, it comes across that you do. But it’s also in your best interest as the company grows and provides these other features and products, that you do that. And so that’s the thing that when I read it, I got really excited about, was that we have this opportunity in the community for you guys to take that money and basically spend a lot of it on us.
ISAAC: [Laughs] Yes. That is the intended goal. We are spending the money on npm and the open source community. In the process, we’re also trying to figure out ways that we can be long-term sustainable and sell services that, to be honest, people really want to pay for in such a way that that will continue to feedback and help the ecosystem keep growing. The open source community is what makes npm worth investing and worth buying from, worth being a part of. And so, that’s our well that all our value is coming out of. And the last thing that we want to do is anything that’s going to make that well flow less productively. And you know, there are things that we’re not prioritizing right now just because we have to put out existing fires and deal with uptime and also deliver products that people want to pay for. But I think in the long-term, we’ll have a lot of opportunity for doing things that are just strictly making the open source community better. There are features we can add in terms of security and reliability and just other goodies that we could do to make the life on an npm user a lot better. And having people who are working on that fulltime is really the way that that happens. As nice as open source and community charity and goodwill are, when it’s not your job it’s not the thing you’re doing. It’s something that you’re spending a weekend on here and there. And it’s not really fair to expect that people are going to contribute that much while also holding down a fulltime job. Also, we bought a lot of stickers. I bought 10,000 stickers from Sticker Mule. [Laughter]
CHUCK: Oh, buddy.
ISAAC: If you see me, I’ll give you some.
JAMISON: You just throw them into the wind like they’re flower petals?
ISAAC: Yup.
JAMISON: That’s awesome.
CHUCK: At New Media Expo, there was a guy that gave a keynote that actually threw money out, off the stage.
JAMISON: [Laughs] That is so tacky.
CHUCK: It was. It was really tacky.
JAMISON: That’s the tackiest thing I’ve ever heard.
CHUCK: And then half of the presentations the next day, the people introducing the speakers were throwing money out. And then as soon as they got on stage…
JAMISON: Oh my gosh.
CHUCK: They were running to get it back up. But it was still pretty funny. But yeah, that would be awesome. Do you know when or where you’re going to be that we can get these stickers? Because I know that that’s the only reason people go to conferences.
ISAAC: We’re going to be at JSConf and also at NodeConf. And also, we’re going to be at 200 Frank Ogawa Plaza very soon, hopefully, if the lease doesn’t fall through. [Laughter]
ROD: No pressure.
LAURIE: Yeah.
ISAAC: Yeah, what we’re actually looking at, we were actually talking about this earlier today because we’re getting close to moving into this new space in downtown Oakland. And we’re thinking about a way that we can very directly support the Node community and the Oakland JavaScript community by having some additional space. Like if you want to drop by and hack with us or something, we will be able to have a little bit of room and hopefully it won’t get out of hand.
CHUCK: That is cool.
JAMISON: That sounds awesome. I have a change of subject. I know for a while Nodejitsu very generously donated time and resources to maintaining the npm package registry. And then npm, Inc. started and took it over. Do you want to talk about the transition that happened there?
ISAAC: That’s a very accurate description, yeah. Actually it was IrisCouch before it was Nodejitsu.
JAMISON: Okay.
ISAAC: And then Nodejitsu acquired IrisCouch and with it acquired the custodianship of the npm registry. And they found themselves in a position where the way that the npm registry was structured, with having a single big Couch that was doing, not big Couch, a single large CouchDB, that was serving all of the puts and all of the gets, was becoming very, very difficult to maintain as it does. And the approach that worked really well at the start of 2013 no longer worked by the end of 2013. And they were getting into these situations where it was going down and just impossible to come up. The database was getting too many writes to ever be able to [inaudible]. The bandwidth and hosting bills for Nodejitsu were getting pretty outrageous. So, it got to a point where basically Nodejitsu had to reach out for help. At the time, I hadn’t actually… We were still investigating whether or not it made sense to make a company around this, whether it was even something that we could build a business model around. And I’m not a fan of the traditional .com bubble approach where you just raise a bunch of money and then figure out what you’re going to do. I didn’t want to start the fundraising or the VC funding thing until we had some kind of idea of what a business plan would look like.
JAMISON: You’re not going to pivot npm into a tower defense game is what you’re saying? [Laughter]
ISAAC: I said I wasn’t going to talk about vaporware, but I’d make an exception for that. [Laughter]
ISAAC: Anyway, we were basically getting crushed by this exponential wave of people coming into npm and trying to use it and really bring excited about it. And so, the scale npm fundraising drive, which actually we did on this podcast, we came and talked about it on here, that basically got us out from under being crushed by this wave of people who were adopting npm. But if you sit down and do the math, even a pretty junior software developer or DevOps person in San Francisco or in New York, in an American city, is going to cost a fair bit of money. And so, if you want to have five, six people fulltime on maintaining the npm registry and also pay for hosting and bandwidth and all of this other stuff, 300 grand is not going to actually get you that far. It was enough to get us to where Nodejitsu wouldn’t have to make the choice to either shut down the npm registry or nothing. [Chuckles] Basically they had to find some money or turn it off. With that in mind, we did the scale npm fundraising drive. Charlie Robbins ran that whole show. It was a Nodejitsu operation. I worked on figuring out ways to, now that we’ve kept the lights on for a few months, let’s figure out ways to keep the lights on for eternity. How do we actually have a system that will scale? Not just do the next hundred thousand packages, but the next million packages, ten million, hundred million packages. How do we actually take npm into the stars and keep going with it? And the way to do that was actually by not just having it all get thrown in a single CouchDB. The way to approach that is to put a CDN in front, not keep the tarballs as attachments in the database, and do a lot of these other things that took very careful steps. And also, take a lot of people being on call and people owning and maintaining each part of this. So, the way to do that is to build a company with a dedicated focus on it. So, that’s why I pursued funding. And like I said, we raised a lot more than 300 grand. But it’s still, two and a half million is a big number, right? If that was in your personal bank account, you’d be like, “Hoohoo! I’m rich.” But for a company, companies are pretty expensive. If you have a bunch of employees, say you have 10 employees, that’s only going to last you a year or two at the most. And so, you really need to figure out a way to turn that two million into 20 million in revenue in a reasonable amount of time in order to take it to the next step and to continue growing. I think there was a lot of perception that we probably could have managed a lot better. Just for starters, I think it would have been a lot better to be more clear, which I guess in retrospect, 20/20 hindsight, the npm logo’s all over the scale npm site and it’s not very obvious that it was a Nodejitsu thing, that the goal was to keep Nodejitsu able to keep the npm registry up for longer. And also, there’s this weird perception where it’s like, “You asked us for money,” and then, “You must have already been in the process of raising funding. Because as we all know, raising funding takes six to nine months. So, you were probably almost done by the time that happened.” Well, as it turns out, we hadn’t actually started yet. True Ventures just came through really, really fast. So, that much in the public perception I think is a little bit unfortunate. There’s not much we can do, I think, at this point to correct that stuff except to move forward and continue to earn people’s trust by acting good. Companies are inherently untrustworthy. And I have exactly the same feeling about most companies. I have a much easier time trusting a person than a corporation. But I think that’s just incumbent on us to do the right thing and to continue doing the right thing indefinitely so that we can earn back some of that trust that we spent in that whole process. To be honest, I think it would have been worse if Nodejitsu had just bottomed out and had to turn the registry off for a few months while we were figuring out our funding situation. We picked the least bad of a couple of even worse options.
JAMISON: Yeah, turning the registry off, that sounds pretty bad. [Chuckles] That clarifies stuff a lot. That’s really helpful.
ISAAC: Turning the registry off is effectively what was happening in October and November because there were so many outages and downtime. We’re basically still recovering from that public perception of npm is a thing you can’t depend on. So, there was really no great option. The better option would have been to go back in time a year and create npm, Inc. then. If we had done this at the start of 2013, it would have been a little bit harder to get funding probably, because we wouldn’t have been as big or as popular, but still probably could have avoided a lot of that rocky time. That being said, there’s a saying that Jason Smith told me that there’s, “The best time to plant a tree is 30 years ago. And the second best time to plant a tree is right now.” So, this is the second best time to create npm, Inc., was at the start of 2014. So, that’s when we did it.
JAMISON: So, this is like the climax of the conflict in the movie that’s going to be made about this in 20 years.
ISAAC: [Laughs]
JAMISON: I don’t know, this is the low point. And then the happy montage is about to start. [Laughter]
CHUCK: Boy, I hope they could get a good voice actor for me. [Laughter]
JAMISON: Chuck will be featured prominently, I’m sure.
CHUCK: Yeah, right. [Laughter]
JAMISON: Cool. No, that’s exactly what I wanted to know. That’s super helpful. Thanks for clearing that up.
CHUCK: One other thing that I guess I have to ask, you’ve been a little bit shy about talking about the future, but what are the things that we can expect to see in the short-term that you’re working on right now with npm?
ISAAC: Well, like I said, private modules. As far as big features go, that’s going to be the one to keep an eye out for. When you pick apart what that means and how we’re going to go about implementing that, there are some other feature directions we can take it in once we have those parts. Some other interesting things, a surprisingly common question when I was running the Node project that I saw a lot from newcomers was, “Okay. I downloaded Node and I installed it. And I did npm install and I did all these modules. And I even made a module. And that was all really easy. And no I have a little server program that’s going. So, now what? Where do I
put it? How do I make this go on the webs?” And so, the fastest way that we can do that is just expose some of those goods and services that are out there and have a section of our website where you can look at resources and stuff. Another thing that’s going to come back very soon, which I know Rod has been really, really just eager to see, is download counts. There was a little bit of conspiracy theories about who we had taken away download counts and then we were going to start charging for them and that’s a whole [inaudible]
JAMISON: [Laughs] Yeah. That was funny actually.
ISAAC: Thanks. No, we are not going to be charging for download counts. Those are coming back. The reason they went away is because we were keeping time series data in a CouchDB. And just FYI, don’t ever do that. It’s a terrible idea. Time series data’s hard to get right. And CouchDB’s the wrong tool for the job. Everybody told me that when I first whipped it up and it was like, “Well yeah, but this kind of works. Why not?” And here’s my not, [chuckles] because after while it falls over and it takes forever to get any kind of data out of it. Laurie, do you want to talk a
little bit about how downloads counts are working?
LAURIE: Well, there’s not a ton of complication to it. We have logs from our CDN. It was previously from the server directly and now it comes from Fastly. We have an event every time somebody downloads things and we are throwing those into Manta, which like I said earlier, it has Hadoop-like functionality where you can just run shell programs on the data in place and then get it to map reduce stuff back up to you. And it turns out that map reduce is a really great way of counting stuff up. It’s basically the simplest thing that you can do with map reduce is count stuff up. So, we have a Manta job that counts up all of the day’s downloads and throws them into what is, whisper it, a MySQL database. I’m so sorry everyone. [Laughter]
LAURIE: Because it’s so small. It’s five million rows. It doesn’t matter what you put it in when it’s five million rows as long as it’s not Couch. [Chuckles]
LAURIE: And so, I am literally in the background as I am talking on this podcast spinning up that has the five million rows on it and spinning up the Node process that will serve them out. And about an hour ago, I open sourced the code that runs the web service. So, they’ll be back very, very soon. I promise. And it’s just the most drop-dead simple web service you can ever think of. It’s just a tiny little one-method Hapi service. And that will be a public API that anybody can call, if you desperately need to know your download counts every single day. You can just curl it now.
JAMISON: That’s cool that it’ll be public.
MERRICK: That’s cool.
ISAAC: We’ve also been considering that there’s a lot of other stats that we could be tracking. We may have other things to add to that. Like I said, the top priority is bringing back stuff that had either fallen over or we couldn’t keep maintaining for whatever reason. And we’re almost back to feature parity of where we were at the start of last year.
JAMISON: I just wanted to ask about the transition in stepping down from leading Node to npm, Inc. That’s a huge deal. It just totally slipped my mind as we were talking only about npm, Inc. But how did that work? Did you just decide that it was too much work to do that fulltime as well as npm, Inc.’s stuff?
ISAAC: They’re both like three fulltime jobs. So, I think yeah, there was no way. The reason I hadn’t really done much with npm in the last couple of years is because I’ve been running the Node project. And Node had gotten to a point where really, what it needed was more of what TJ Fontaine was already bringing to the project. He started on Node core and at Joyent I think it’s been a year and a half, a year or so, or maybe two years now. And he brings a lot of very high degree of rigor and consistency to the Node project. And he’s not also trying to manage a package manager ecosystem. So, he’s got a better view of some of the low-level stuff that was happening in Node. I think my skillset in Node was around figuring out what the right API surface should look like for the JavaScript stuff. And I think I did an okay job with that. TJ is a better leader for where Node is right now. And it was very clear that npm was the thing that needed the most attention. So, it seemed inappropriate actually for me to keep running the Node project. And effectively, TJ had been doing that job for quite some time already. So, I think stepping down was, it felt very obvious. It felt more like we were just publicizing something that had already happened by the time it did become public.
JAMISON: That’s cool.
MERRICK: Cool.
CHUCK: Alright, well I guess we’re at the end of our discussion here. So, let’s go ahead and do some picks. AJ, do you want to start us off with picks?
AJ: Yeah. Since we mentioned the proverb of the best time to plant a tree is 20 years ago, there’s actually a pretty cool article that I’ll link to that’s a better your life new year’s resolution type of thing the church of Jesus Christ put out. And also, I will link you to RandomUser.me which is this wonderful, lovely API where you can get… They have 140 different picture profiles available. And then probably quite a bit more than that in terms of the usernames and email addresses that are all fake. But it’s just a cool site if you need to grab a bunch of fake user data to template somewhere, you can get them with cool pictures and with email addresses that look legit except for the example.com and that kind of thing. And I thought I had something else, but I don’t remember right now. So, I’ll go with that.
CHUCK: Alright, Merrick what are your picks?
MERRICK: So, I have three picks. My first pick is an album that came out two weeks ago called Ledges by Noah Gundersen. It’s just absolutely wonderful singer/songwriter music. And with that, another album that came out this week by William Fitzsimmons, Lions. And again, more awesome acoustic music. And the last pick is actually one that’s sort of a nostalgic ode which is the first time that I talked to Isaacs, his pick on JS Jabber was the book ‘Nonviolent Communication’ by Marshall B. Rosenberg. And I actually read the book after that. And I thought the book was absolutely terrific, one of the best books I’ve read to date. I even wrote Isaacs probably a slightly uncomfortable email for him. [Laughter]
MERRICK: But the book was just terrific. So, I wanted to also make that part of my picks.
AARON: Tell Merrick not to call me. [Laughter]
MERRICK: Is that Merrick kid going to be on the call? Because I don’t know if I want to do it.
AJ: I second the motion too or third it, or fifth it, or whatever. I listened to it on audiotape. It was really good. I really enjoyed it and I wish that that was part of public education, was a class where you just listened to that book or something.
CHUCK: Yeah, I can totally see it. Merrick has a collection of restraining orders from JavaScript developers. [Laughter]
CHUCK: Alright. Aaron, what are your picks?
AARON: I just have one pick, because I’m not very prepared this week. So, there’s a new album by a group called KONGOS. They have a new album called Lunatic and it’s pretty wicked awesome. So, I’ve been coding to it this week and really enjoying it.
CHUCK: Jamison.
JAMISON: I only have two. The first one is MountainWest JS. I’m going to keep saying it until it’s over. And I’ll probably forget when the podcast gets published, so you’ll probably hear it one more time after the conference is over, but JavaScript conference in Salt Lake City, Utah, March 17th to 18th. Tickets are still on sale and I would love to see you there. I’m going to be there. I think Chuck’s going to be there, a couple of other people are going to be there.
CHUCK: Yeah. Jamison’s speaking.
JAMISON: Yup, unfortunately. But the other thing is it’s this GitHub repo for an HTML5 game called nothing-to-hide. I guess it doesn’t matter what it’s written in because it’s a good game. It’s not a game that you look at and think, “This is good for an HTML5 game.” It’s a legit good game. And it’s about reverse surveillance. So, the point of the game is there are all these cameras all around the levels and you have to stay within sight of the cameras because this crazy city state is going to shoot you if it can’t surveil you at all times. It’s pretty interesting. But I also thought it was cool that they put 100% of their game just up on GitHub as they’re developing it right now. So, those are my picks.
CHUCK: Awesome. I’ve got a couple of picks here. One is Monster Legends. I’ve been playing it on the iPad, but I guess you could play it on Facebook as well. It’s a fun game. You breed monsters and feed them and you get them big. And then you go and you battle with other people’s monsters. And there’s also a track of battles that you can do to get experience and stuff. And it’s a lot of fun. I’ve been really enjoying it. So, I’m going to pick that. I’ve also been getting into the project management software Redmine. And I found a plugin that I’m really enjoying. It’s called ekanban. And so, I’ll put a link to that in the show notes as well. It did take a little bit of doing to get it set up just because the instructions aren’t great. But I’m really enjoying that. And then I also read a couple of books this weekend. The first one is ‘Tribes’ by Seth Godin. And I really enjoyed that. Just a terrific book. And I’m trying to remember what the other one was. So, apparently it didn’t stick with me. [Chuckles] But anyway, those are my picks. Isaac, what are your picks?
ISAAC: I have two picks which are not really related. The first one is CodeScouts.org. Code Scouts is interesting. It’s a not-for-profit program for empowering people who are disenfranchised people in tech and getting them into tech. it’s primarily and educational thing. They do focus on women in tech, but they’re actually not genderfocused as much as values-focused. So, they have really cool programs and are actively trying to make our technology communities better. And the other, also in the vein of making our technology communities better, I want to plug npmjs.com/jobs. Oh my god, I’m cheating. I’m so cheating. [Chuckles]
ISAAC: This is nuts. It’s supposed to be something else. But no, we are actually hiring. And if you liked what you heard on this podcast, you should come work with us. We’re trying to create a company that is not so inhumane.
JAMISON: I’ve read this jobs page and I got to say, it reads really nice. I really like the focus on people as opposed to “wicked awesome skill and so much money,” and I don’t know. It seems like it’s focused on the right things.
CHUCK: Does it say ninja on it?
JAMISON: [Chuckles] It does not.
ISAAC: No, no. We don’t need anybody to be stealthy, or commit murder, or steal things. No ninjas.
JAMISON: Or smash dorm rooms.
ISAAC: Yeah. No ninjas needed.
AJ: Y’alls doing it wrong.
ISAAC: We do need responsible software developer types. So, if that’s more your thing, give us a call.
CHUCK: Somehow, I wound up going to HYPERLINK "https://npmjs.com/jobs" https://npmjs.com/jobs and it was warning me about the certificate.
ISAAC: Yeah. Our .com site is not on SSL because there’s nothing private.
CHUCK: Okay. Laurie, what are your picks?
LAURIE: So, my picks are all things that I’ve been using this week and thought, “Oh man. That’s great. I should tell somebody about it.” One I just tweeted about was, and in fact I already mentioned it, was Fastly, our CDN. I’ve worked at a lot of places where we’ve repeated skirted around the, “Oh we need to geodistribute this thing someday and it would be really great if people in China can get it as fast as people in US [east] get it.” And you never get around to doing that because it’s such a huge pain to implement. And Fastly do it and have never gone down on us and are super responsive when we have problems. And they’ve just been super fun to work with, so that’s great. It’s load balancing and CDN-ing done absolutely right. The other one that I’ve only been using for half a week now is Slack, which I think is still in private beta or maybe might be invite only or something. But it’s basically, when we started the company, we all started talking to each other over email and then IM. And then we were like, “Oh we have to go to IRC. Why is there nothing better than IRC?” And we complained about the fact that there’s nothing better than IRC. And people are like, “There is. It’s called Slack.” And it really is. It’s great. It’s got a good desktop client and a mobile client and a web page. And it lets you keep in touch with all of your messages and see the unread stuff. And oh man, it’s been such a pain point for such a long time. I’m so happy to talk about it. And the last one is Hapi.js, which I also mentioned, which was the framework that I built the download counts in. Like everybody else who’s been paying attention to Node, I’m very familiar with Express. But having been a web developer for a very long time, I’ve looked at Express and gone, “There’s a lot of stuff that I would like this thing to be doing that it doesn’t really do. And I can plug it in, but I wish it was just built-in.” And Hapi has been built by some people who clearly have had similar thoughts and have built in a lot of the stuff that I think should be built in and left out a bunch of the stuff that I didn’t think should be there. And in general, it’s just been making me happy as a framework. So, I thought I’d shout out to that.
JAMISON: Ha! Makes you happy.
LAURIE: Exactly.
JAMISON: So good.
CHUCK: Alright. Rod, what are your picks?
ROD: Three picks. The first, because I’m a financially-minded geek COO, is a team called Accretive Solutions that True put us in touch with. These guys are amazing. They are actually just like the Palo Alto rug store in that they’ve been involved with everything. They set up the books and did everything for a company called The Facebook and then Instagram. [Chuckles]
ROD: What they do is they roll in. They make sure that everything is electronic. The company doesn’t have any physical paper records. They set everything up from day one so that it’s all in an easily shareable folder which means that when you need to do future financing or anything else along those lines, all audits, et cetera, is all taken care of. And because they’ve done it for dozens and dozens of these companies, they know exactly how to structure it from day one so that you radically reduce your cost. All of the stuff is the best practices and it means that you can then focus on your core business. So, Accretive Solutions, they’re all over the place, but really, really like the team that we’ve come across. The second pick is Oakland. We won by deciding to be based in Oakland. It’s 12 minutes for us to go from the 12th Street, Bart Street to downtown San Francisco. And yet we pay I think about a third in rent. There are some great Vietnamese sandwiches out here. [Laughter]
ROD: It’s a really, really nice environment to live in. The final pick is something that Laurie came up with, which is the real definition of what npm is. And that’s Nice People Matter. And we’re going to have a great big sign of it on our wall. And I think that’s the coolest part of what we’re doing so far.
CHUCK: Awesome.
JAMISON: That’s cool.
CHUCK: Nice People Matter, Incorporated.
ROD: [Laughs] Yeah, I’m not sure it has the same ring with the incorporated afterwards. [Laughter]
JAMISON: It sounds like some Soylent Green type thing with that. It’s made of people. [Chuckles]
CHUCK: Alright. Well, thanks for coming guys. Really appreciate you being so open and just talking about this stuff. Because I know a lot of people are concerned about it and a lot of people are excited about it. And so, now hopefully we have a lot of the answers that they’re looking for.
ISAAC: Awesome. Thanks for having us.
CHUCK: Yeah, thank you. We’ll talk to you next week.
ISAAC: Okay.
AJ: Huzzah!