02:43 - Oren Rubin Introduction

05:43 - Testing

18:18 - Page Object(s)

27:10 - Protractor & Selenium

32:06 - Checking UI (Screenshots)

37:04 - End-to-end > Full Coverage?

40:03 - When should you start testing?

42:21 - Cucumber

45:39 - Debugging

Picks

Paul Ford: 10 Timeframes (Jamison)
Kishi Bashi - “In Fantasia” (Jamison)
Matt Zabriskie (Jamison)
http-backend-proxy (Aimee)
repl.it (Aimee)
React.js Training with Michael Jackson and Ryan Florence (Joe)
React Rally (Joe)
AngularConnect (Joe)
ng-conf (Joe)
Ruby Remote Conf Videos (Chuck)
Angular Remote Conf (Chuck)
15 Minute Podcast Listener chat with Charles Wood (Chuck)
Dave Haeffner: Elemental Selenium (Oren)
CSS Secrets by Lea Verou (Oren)
Cloudinary (Oren)

Special Guest: Oren Rubin.

 

It sounds like you're on a walkie talkie.

Does it? Come in, Jamison.

Hey everybody and welcome to episode 177 of the JavaScript Jabber Show. This week on our panel we have Jamison Dance.

Hello, friends.

Aimee Knight.

Hello.

Joe Eames.

Hey, everybody.

Dave Smith.

Ahoy!

I'm Charles Max Wood from DevChat.TV and AngularRemoteConf.com. We have a special guest this week and that's Oren Rubin.

Hi there.

Do you want to introduce yourself real quick?

Sure. I've been in tech for almost 20 years doing machine learning, distributed computing, compilers, a lot of web development, and test [formation] for the last few years.

Very cool.

So, once you solved all the easy stuff with distributed computing you came to the real hard problems which are frontend JavaScript, right? [Chuckles]

Yeah. And I actually agree. I totally believe that, that it's much harder, yeah.

Oh, I was being stupid. You actually think it's harder?

I actually feel that it's at least as hard as. When I started doing web eight years ago, things were a mess. Things aren't that perfect right now. But you do have Angular, Ember, React. You have a lot of frameworks here that keep everything more than easy. But you didn't have that at the beginning. And I think the first thing I did with web was actually, I did an extension for Firefox. [Inaudible] extended Greasemonkey. And debugging, you couldn't debug easily. You're not [inaudible] for that. But even today, I really think that web development, because it's asynchronous most of the time, it's not that simple. It's really doing architect work. And it's not as simple as people think. Even CSS is not as simple as people think. “Oh, that's for stupid…” No, it's really hard.

I feel better about myself now.

Yeah, me too. [Laughter]

I thought you just had too much of Selenium. [Chuckles]

Well, it is interesting, right? I mean, you're talking about different… how many different languages do you have to know in order to build for the web? You have to be fluent or semi-fluent at least in HTML and CSS, if you want to count HTML a language which it kind of is. It's a markup language so it's not Turing-complete. But then you've got possibly JSON or XML if you're dealing with APIs. You've got your JavaScript on the frontend and then maybe JavaScript on the backend or some other backend language to build all the rest of that stuff out. And then it gets more complex from there if you add other things to your stack. So, I can see where you're coming from there.

Well actually, it's not just a difference in the language. I did [inaudible]. I love compilers. I learned more than a few dozen. But what I really saw, I worked at Wix. It's a big company that does a lot of web. And I've noticed that people are coming, great people, amazing people that come from the backend to the frontend side really struggle much harder learning JavaScript and doing frontend. And those coming from Flash, they hit it off in a second. They needed just a day or two just to find out what's different. But it's more, this was more like a mental change for the backend guys. It was really easy for the Flash frontend guys.

I'm sorry I distracted us. [Chuckles]

No, it's all good. What we have on the docket today is UI validation. And it sounds like that's UI testing, making sure that the UI looks and behaves the way that we expect it to.

First of all, I want to probably start… you did a lot of shows probably about why testing is good. So, people usually talk about unit testing. And we all know that testing is good. And you have regression tests and continuous deployment is amazing. Every change you make instantly goes up and you deploy a new version. But the question we should probably start with is when to use endto-end, real end-to-end, real browsers. When do I just use unit tests? And even that is, the answer is always “It depends,” as always. But over the years I've noticed that I was a big, big fan of unit testing and still is, because it improves your code quality by…

Can we just stop and get a really brief definition of unit testing and end-to-end testing?

Oh, that's great. [Chuckles] So, let's explain probably the entire jargon of the testing world. So, every application is actually build out of components and units. And each unit is composed out of smaller units. And they all have inputs. They're all models. Like you have, you put stuff in and you get stuff out. Every component is. So, unit testing usually when you talk about unit testing, it's about the granularity of your testing. It means the lowest, the smallest granularity that you can do. That usually is either class, if you have a class which has a few methods. That's usually the granularity.

When you're talking about UI testing, the question is, you can do that for controllers which they don't… by best practices, they don't access the DOM. So, you can just call, invoke methods and check the results. Actually, you check that the state of your object changed. So, that's one very small unit. Usually when you're talking about UI the smallest unit is actually a component, because there's [inaudible] thing as just an object. Usually it comes up with, if we're talking about an Angular component, either an Ember component or with Angular, you have directives. And you're talking about, it's probably a template as well that's involved. Logic as well. And usually it's CSS as well. By definition it's harder to find just a unit test here, because CSS as we know it is global. If you say, even if you have your CSS for just one component and it has a rule that says all the [divs] should be red, color red, then it affects the entire application. Of course, until we have Shadow DOM which is far, far from us. So, even unit testing are not unit testing as we think.

And we can check on integration between a few units and where they're talking to each other and have, probably they're exposing a different API. And they talk between themselves. This is integration test. And the end-to-end testing means that you're checking the entire stack. That usually means both frontend you have, someone clicks a directive, a component. That calls the controller. It goes and calls a service. It sends an HTTP request to the server. It talks to the database and responds, keep coming back to the UI. And this entire flow, this is called end-to-end testing.

Acceptance test means that not just your end-to-end. It usually means everything that is required for you to deliver a new product. Usually it's not just functional testing. Functional testing, you check the application and you make sure… if you have a calculator if you check 'one plus one' and then you want to make sure the answer is two. But you can have a different type of validation, not just functional, that it says two. You can have UI validation. It means it's in the correct place and it looks, all the buttons are aligned and you have the right font. You have memory leaks. If you do 'one plus plus' again and again and again and again, you want to make sure you're not… you don't get an out of memory exception. And of course, there's performance testing, that you want to make sure that the answer, you get the answer number two fast enough. So, there's a few when you're saying acceptance test, you mean a bunch of them.

And one more thing that we should know, you can really [inaudible], the formal word is [inaudible] testing. It means, what are the minimum amount of tests that I want to do before I want to deliver? Or at least say, “I don't see any fires so it's probably okay.” And you really want to do that because there's a big difference between a commit of a user when he changes a code. He wants to not just commit and push the code to the global repository. There's a difference between that, that you want to do it real quick. You want to check everything's okay and then I'm pushing my code. But before I release the new version, I want to test everything or do acceptance testing and have everything working perfect before I ship my product. And that includes a lot of commits from so many developers in my organization. So, that's basically the terms.

Thanks. That helps a lot.

So, there's a lot of unit testing going on for JavaScript. Jasmine and Mocha are usually the best, most popular [inaudible]. Most of them are pretty similar, pretty straightforward. You do a lot of mocks because if you want to check one single unit, you probably want to mock everything else. The mocking isn't a simple thing to do. It's simple to do,, but the data for initializing your data, that's usually the hard part. But usually unit testing is easy, because you get started… first of all it's in the same language. If you want to do unit testing in JavaScript you write in JavaScript.

Usually when you start doing end-to-end testing, at the beginning… for example if you take Selenium which is an open source project for writing end-to-end testing, the majority of the documentation was the Java binding. Most people started with that. So, you couldn't ask a frontend guy, please write the test as well. You had to have a team of QA automation. That's a different language. That's Java. No. How can I…? I know JavaScript. That's Java. It's a completely different thing. So, that was the first barrier for frontend developers writing end-to-end testing.

And I guess the second thing that's different from unit testing and end-to-end testing, it takes a long time. The setup isn't that easy. You need to open a browser. If you want to write in parallel, you have to run a write, run everything concurrent and have 50 browsers. You probably can't do that on the same machine. [Chuckles] It's not going to work that smoothly. [Not just] as unit testing. But I think the hardest part of writing end-to-end tests and the most challenging for moving from unit tests to end-to-end tests is starting to [do] everything asynchronously.

So, with unit tests, you can [inaudible] things. You're mocking even the time. You're used to running everything synchronously. You call, you invoke method, and you make sure something was invoked. But end-to-end testing mimicking the real user, starting from the browser. And real users, if you work with real databases, real servers, that takes a while. So, you want to do a simple login. Then you have to start. The big question is, “Oh, wait a second. How long do I wait? How do I know that I'm ready? How do I know that the page is ready?” So, that's [chuckles] that's the first thing that people usually fall when moving from unit test to end-to-end testing, is how do I know I'm done? When can I start testing the second thing? And it's funny because the answer usually lies in organizing your code better. Are you guys familiar with the page object design pattern?

I am. And I actually have a quick question about something you just said.

Sure.

So, you mentioned about actually hitting your backend. I'm curious, your thoughts on actually mocking that out to speed up end-to-end tests.

Oh, that's a great question. As we said, it depends on the granularity. Because if we're checking, if you have unit tests we said, you know exactly what went wrong. This unit, it doesn't work. And probably it has 20 methods. We'll find out what went wrong. And end-to-end means if you're talking in the higher level, then you get great coverage, amazing coverage. Think about the simple cases. You sending, just login to Gmail, send an email to yourself and open it up, you'd probably do a coverage of 30% of Gmail's code. And not in a very, something in a small test. So, the coverage is great, but when something goes wrong you don't know exactly who to blame because there are so many units involved. You don't know which specific unit is involved and which one is to blame.

So, your question is, how about we split it in two? How come we can't mock the backend and it probably will speed up things enormously? And the answer is of course you can. There weren't any, a lot of out-of-the-box solutions. Every time you mock, you have to set up what's the result in all the cases.

Yeah. It's a pain. I'll have to use is as one of my picks at the end. But we use something in Angular that does that so that we don't actually have to hit our APIs in our end-to-end tests.

Yeah. So, here [inaudible]…

But if it's an end-to-end test and you mock out one end…

Is it really an end-to-end test? [Laughter]

Or is it like end-to-middle? [Laughter]

[Inaudible] of that question, too. It's like the front end-to-end. [Laughter]

Frontend to other side of the frontend?

[Laughs] Yes. [Laughter]

So, that's why I say it depends, because if you want to just simply commit your code and push it, not deploy it to the new version, if I want to make sure that I didn't do so much harm because I only changed the UI… I know that I changed only CSS or some JavaScript part, how come. now I didn't change the API between the client and the server, how come I have to check everything? Before we deploy, we do want to check everything. You want to make sure that we don't have any bugs. But here right now, when I only changed something in the frontend? So, it's usually, you change your test because this is the smoke testing probably, as I mentioned. That would fit that very well.

Yeah.

But there's one more thing that there's a difference between unit testing and end-to-end is, because end-to-end takes so much time, just opening the browser takes a few seconds, and the scenario takes a long time, usually there's a different methodologies on how to do end-to-end. So, what happens is in unit test, usually you, because the setup is so quick you can check one little thing, one method, and validate it and that's it. And you have different tests that check for example different methods.

So, because the setup is so expensive for end-to-end testing, if it took you 10 seconds to open a browser and load your application you probably want to have more than just a little step. What happens is that you have a complete user story in every test for end-to-end. You go from checking small stuff to checking a full use story. And that works as well because what you do actually when you check end-to-end, you actually check the business logic. You're checking different things. You do want to check a lot of things because you're checking a full scenario that some user might take, not just a small part. You want to make sure that not just he goes and put the book in the cart, but he can actually make the payment as well. That's a full user story.

I guess that leads us probably to the, how do you write end-to-end testing? And the first rule of writing end-to-end testing is writing page objects. So, I'll explain a page object in a sec. So, a page object means that everything you touch, don't touch the implementation. Always have another layer of abstraction which is the business logic. Never work against, for example if we're using Selenium never work against the Selenium API. I remember when I just started, I read a [inaudible] that says if you're using the Selenium API in your tests, you're doing it wrong. I think it was one of the, one who wrote Selenium who said that. And he was right.

I remember a case when I just got started and we did a lot of tweaks. We did a lot of, there wasn't any Ember and Angular back then, so we wrote our own MVC just like everyone. And we started, we want to be in continuous deployment and we started writing tests. And eventually we see that it didn't work out. So, we want to split things up and to get things done faster. So, what we did was we write the API for the entire business logic. That took us like a weekend. And then you tell all of the QA guys, most of them were [inaudible] super technical but you can tell them, “Here's a bunch of classes and here's the interfaces and the methods that you have. And now you can write your tests just like almost in English. Write your test and I'll talk to the dev guys and they'll do the implementation.”

So, we split the tests. Someone wrote the test, and even before the implementation was ready and someone else who knew the application by heart because we built it, started building the implementation, the 'how'. So, eventually two weeks later, we had so much done because we could split the teamwork. So, that was the first time I've noticed that page objects are really helpful.

And the second time was because I noticed that a lot of people don't know exactly, how do I wait? When do I wait 'til the page is ready? So, page objects helps you with that because if you have a class called 'account' which has a method called 'login', if you're writing your page objects correctly, the login, let's assume you're going to a gallery page, the login returns the gallery page when it's ready. Then when you're writing a test, you're looking at the test, it seems like almost synchronously because you're saying, login and then you get a gallery. And then it says 'gallery dot go to next page'. You hide all the implementation of waiting for the gallery page to be ready inside something else.

So, page objects aren't your account object or your gallery object. Your page objects are things that actually work on the UI itself?

Yeah. Usually, I guess page object isn't the correct word. That's the phrase. But [inaudible]. It's more like component object.

Oh, okay.

Because if you have a gallery, and it's a component. It doesn't have to be a full-blown page. You can have five galleries in the same page, right?

And just to be clear, this is the thing you write to test your page or your component. It's not the component itself, right?

Yeah.

It's just the API to drive it.

Yeah, exactly. It's the driver. Every component that you have in your UI, [inaudible] Angular, has reversed engineered component for the testing which is in your testing.

I want to ask about this a little. But it sounds like you're advocating for 100% coverage in end-toend testing. And in my experience, end-to-end tests have been very brittle. They're easy to break when changes that don't break the functionality happen. And so, they can end up taking a lot longer to maintain. And then the more coverage you have, the more time you spend maintaining them. Can you talk about that balance?

Can I throw and example out really quickly? So, let's say you have a button that's inside of a div. And you decide that you're going to move it outside the div, or remove the div. And now your reference to that button has changed, where in reality the form still submits and everything else still works.

So, in your test, you're talking about the user experience. And the user experience is of course you want to do a login or submit some form. It's probably add the new [inaudible]. He's talking in the business logic level. What page objects also gives us is that you split the locators. What is the locators? The hard part about what we did in [reverse] engineering is that, let's just say you created a gallery in Angular and now you want to have the gallery driver in your test which has three methods: go to next image, go to previous image, and show full screen. So, this is the interface that you're exposing to the users. And the nice thing is that you separate now between finding the gallery and inside the gallery, how do I click the next button, the previous button, and the show more.

So, the problem usually starts when you talk about end-to-end testing about flaky tests. The first thing is what I mentioned, asynchronously. The second thing is the locators. How do I find that specific button and click it? If you want to fix stuff very, very quickly or almost never, is writing your page object right so that they have the locators for only that component. You're splitting the locators. A few years ago, everyone had one page, big page, comprising the locators for everything in their application. And nowadays you probably see that they have small locators in every component. It's just like software engineering intended things to be. Naming stuff is hard, but then let's put everything exactly where it needs to be.

So, every component is in charge of just that component. And of course you want to reuse that component again and again, because you have the gallery in the first page as well, in the second page as well. And you're reusing that exactly as you reused your Angular component. Now you're reusing your drivers. And it's probably when it's going to break, you know very fast what breaks. Was it finding the gallery? Did I change something, that element? How do I find it? And inside of it, how do I move the next and previous? So, I guess the… almost [chuckles] It's funny that page objects, writing page objects correctly is probably an answer for most of the questions in end-toend.

But yes, end-to-end are fragile. And usually are fragile because they're supposed to be, kind of like aspect-oriented programming, just like CSS a bit is. Because if one guy or gal, sorry, is writing the test and another is writing the application and then don't talk to each other, they're just assuming, “Oh, let me find. How do I find that button?” Well, it has some CSS class. Well, wait a second. CSS class are used for styling. Now you're going to change the styling because you want to make the gallery look completely different. Why should it change my testing? So, a lot of companies started adding the role debugging symbols to the component. So, that means that if you change something in your, you want to change your… add some class, remove class, CSS classing I mean, it won't affect your end-to-end testing.

At Wix we did that in the framework level. Because a gallery for example could have, Wix was, still is, a build your own WYSIWYG. So, what we did there is that you had at the MVC level the controller, it says, it was more like Angular directive, it says, “I'm supposed to have a next, a previous, and a show more buttons. I don't know who they are because I'm the logic. I don't know anything about the UI. But if you give me a UI, a template, it must have those three parts.” And you can check those statically. And the nice thing is that if you enforce those statically the UI never broke. Because you knew that every UI component that you have always had those three parts, previous, back, and those debugging symbols.

So, this is changing topic slightly. But I think I watched one of your talks or was looking around maybe on the company that you worked for, on the website. And I think I read a blog post about using Protractor outside of Angular to do end-to-end testing. I was curious, if you wanted to talk about that.

Protractor. Well, I just thought that Selenium is the DevFactor testing tool. It's a W3 kind of proposal. So, the nice thing about Selenium is if you use Selenium most of the products out there use Selenium in some way. And Protractor is one of those that uses Selenium and gives it a few more, few extra tips. Like you can find an element not just by an ID or a class, but also by different data-binding attributes. Regarding the asynchronous, it helps just a bit with Angular. So, Angular has dirty checking. So sometimes, it takes time between you change something until the UI is getting the updates. And it's not just Angular. There's a similar thing as well in Ember. But what Protractor did is talking to Angular and saying, “Okay, let me know when you're done refreshing the UI and then I'll start checking.” So, it doesn't help us a lot for doing the logging and removing from one page to another. But it does help us a bit with Angular's taking a big time to update the UI.

And in fact, I think the fact that it's in JavaScript really helps a lot of developers. There's more and more agile teams that even the developers writing the tests themselves, and that's super, super awesome thing to do. And even the fact that it's JavaScript, even though there's now JavaScript Selenium, everything that you have in JavaScript, if you're using the same language then you can do both. And that's a great thing for an agile team.

Have you ever had a chance to play with something like Zombie where it's not driving an actual browser but it's driving a mocked up browser using JSDom? Can you maybe contrast just the general approach of driving a real browser versus a more mocked out browser?

Well, there's, I was talking [inaudible] Slimer and Phantom. So, there's a few…

So, Phantom's the headless browser. But there's actually something…

Yeah.

That's even more lightweight called Zombie.js where they just mock out all of the browser APIs in JavaScript and then they use JSDom to render stuff to a DOM. But there's no layout. It just renders it to a DOM that's a string. And then you can assert and test stuff against that. But so, it's not a real browser that you can see. But it's still kind of a browser-ish.

[Chuckles] Well, the big question is, and this is part of UI validations as well, is can it do screenshots? What I mean by that is, when you're talking about UI it's funny. Until last year, nobody ever talked about how everything looks in the UI, how the pixel looked like. And it's funny because it's so important. You had… I've seen so much bugs for example that the CSS, somebody giving the width instead of a 1024, give it 124. So, it was so small, the window, you can't… it wasn't operational. But all the tests passed. Because if you're checking the DOM, everything that the user sees, you didn't check that. So, there's a lot of those kinds of errors. They're very frequent. You have zed index which makes one component above the other. There's the sizing.

There's, [inaudible] had the flying pony where they wanted to release something just for the administrator. And they released it for everyone. And the testing didn't catch it because if you're checking the DOM, a specific DOM element for some text, how do you know there's nothing added to the DOM next to it that is above it? For example in their case, that was a flying pony, which is a nice bug. There's a lot of things that if you take a screenshot, there's a lot of benefits for it. And the biggest benefits, you get a thousand assertions, a thousand validations, in one screenshot. First of all you check CSS. That means the sizing, the fonts, the layouts. And you check so many functional testing because you check all the text on screen. So, I guess the question is, why not take a screenshot?

Well you can't in an automated manner though, right? Because somebody [inaudible] eyeball it.

Yeah, how do you automatically diff them?

Usually, you can switch, I'd separate between two things: regression testing and production testing. So, if you're checking, [Google's] checking the production, the result will be different every time. But we can check, we can take screenshots only if you have some magic to check the layout. How big is the font? But if you're talking about regression testing, then you probably can set, just as in unit testing, you set the same data to begin with. And if you have the same stimuli, the same action that clicks and the same HTTP request, then you should end up in something that is very similar to a pixel by pixel result. Well, I've stated that. But there's a reason nobody, almost never use that.

It's funny because if you render it twice, it's funny that you probably won't get a same pixel by pixel result. And that's the duty of anti-aliasing. There's display adapters which aren't deterministic and you get some false positives. And you get things like text that shift by pixels, even sub-pixel shifting. So, there's a lot of UI that keeps changing, even if you render, if you refresh the same page twice. When you look at it, you see everything, it looks the same. But if you just check pixel by pixel it's not. So, I guess there's going to be popping a lot of companies more and more that will check more like a human does and not just as pixel by pixel. Because it sure helps a lot.

And by the way, if you're talking about UI validation, taking images, you don't have to check the entire application. You can do a mini unit testing, like component for just the gallery. And take the screenshots of the gallery in different states. And just for that component. It doesn't mean that you have to take the entire screen. It's very easy to crop, after a screenshot. Every browser supported. It's very easy to find out where your element is and just crop the image for that element. So, checking UI is, I think it's very important.

And more and more, by the way there's a big trend in the last two years. I'm talking about Angular and Ember and React. And there's a big trend. People are moving, a lot of business logic, critical business logic is moving from the backend to the frontend. I don't know if no one noticed that. But they used to be like, five percent of your…

Nope, we hadn't noticed. [Chuckles]

So, five percent of your code used to be the frontend. And you never tested that. Oh, we'll do it manually. And if you guys know, there's the reverse ice cream. That's the testing pyramid.

No, I haven't heard of that. What is it?

It means, like the triangle, and it means that in the bottom you have a lot of unit tests and a few less integration tests. And the more you go up for the end-to-end testing, the top, usually people say, okay, there's very few end-to-end testing. People usually do it manually. And that's because until now most of the code as in the backend. Now, it's people fighting. Ruby on Rails or Django, JDB frameworks. But now the code is, the code has moved on to the frontend and you can't just manually test it because it's not feasible. You have so many devices, so many different resolutions. We didn't talk about responsive design yet. If you want to check, take the screenshot. Your website looks completely different in your application, if the width is 1000 pixels, if it's 300 pixels.

One reason for that pyramid or upside down ice cream cone that I've seen is that the end-to-end tests aren't just more brittle or harder to do, and a lot of that is getting easier. But the other thing is if you're running something like Selenium it takes a long time. And I want to get immediate feedback from my tests. And so, that's why I tend to not write as many end-to-end tests. And I'll test my frontend with many more unit tests and other things that run faster and do some of the mocking out of the backend for the critical pieces. And then you know, work my way down to just having end-to-end tests for the absolute essential stuff.

And that's right. Because they are slow. And as hardware gets cheaper, people will start running concurrent. And then if you get a chance, do everything end-to-end. And if it wasn't that brittle then you'll probably want to do that.

So, you're saying that if the speed were the same, you would prefer full end-to-end test coverage over full unit test coverage?

[Chuckles] So, let me clarify. It depends what the state of your company. If you're a startup and you want to move fast and the most important thing you want to have is full coverage, you have to have full coverage to release a product if you're using continuous deployment.

Sure.

You have to have full coverage. Unit testing, getting everything with full coverage, it's not that easy. So, especially the frontend. There's a lot of CSS and there's a lot of HTML and JavaScript which are in components and they aren't being testing. So, of course I prefer first everything to be fully tested. And I can do that very simply with end-to-end testing. And hopefully one day it will be easy to write end-to-end testing. It still isn't because [inaudible] tools aren't… really suck. But as a company grows, you probably want to have more and more unit testing, especially where your critical business logic is. So, that depends whether it's in the backend or frontend. It doesn't really matter. You want to have more unit tests on that specific region.

Okay, so you're suggesting that end-to-end tests are more pragmatic and immediately valuable to your business and unit tests are a longer term, maybe help keep your development speed up as your project gets more complex. Is that what your saying?

Yeah, they help [keep you sane], yeah.

Sure. Interesting.

Because I had… I even tried [chuckles] for a year to have a hundred percent unit coverage, and that was hard by the way, reaching the hundred percent. And it didn't [worth my while] because I did have bugs, integration bugs. So, the software wasn't flawless, even though I had a hundred percent coverage. Even if you write a hundred percent coverage, by the way the coverage isn't the only thing that you want to check. So, it's only one thing that you want to check. Usually, if you have a very… this is your business logic code, then you should probably… I'll give an example how you want to test that. So, if you have a loop, you want to test the same code in three different attempts. Once that it goes by the loop and passes and doesn't go in, second test would be that you only enter once, and the third time is that you enter the loop at least once. So, those different tests. The first one is enough. The second one is enough to have full coverage. But it doesn't mean that you're testing everything at the maximum level. So, I guess it all depends.

We've been talking a lot about coverage. But do you think that when you're writing tests, maybe it's better to focus on actually writing the test first and then maybe going back and check the coverage after? Because I feel like maybe when you're only focusing on coverage, you lose track of actually what you're trying to test. And that way by just actually thinking about it, you're testing what you really need to.

When you talk to people and ask them, how do you start testing? What do you start testing? People usually say, “Oh, I check with analytics and then I found out what the critical things, the paths that users do.” Usually those that are converted. And you start with those. So, you want to have check which paths are critical for me and start testing that. So, even when you want to write the test, it's not about having a hundred percent coverage. It's about trying to focus on the most used paths and have them covered, the critical paths. So as I said, having a hundred percent unit coverage has [inaudible]. The only nice thing about it, being a hundred percent covered, was that if you add a few more lines of code, in a second you drop below a hundred percent and then you said, “Wait a second. I probably didn't test that.” So, that was the only advantage I saw in a hundred percent coverage.

So basically, end-to-end is the, this is the acceptance test. Usually it means that only if everything works as a user sees it, only then do I deploy. Unit tests, as [inaudible] here, it's for us. It makes our code better. It's not just about regression testing. Your code becomes much, much better when people use unit tests. I guess the basic thing is that you think more about what you write. And a second reason is you think more about reuse, because you're reusing a component again, right now when you're writing the unit test. So, you're already starting to use the same component twice. So, you're thinking more about the API. And then you'll probably use the same component again and again in the future. And it's almost ready. You say, “Oh, I have a great API. Let's reuse it.”

And there's one more thing that we didn't talk about which is… you guys know about Cucumber or any other [inaudible] language?

Yeah. I'm a Ruby guy, so I've done plenty with Cucumber.

Maybe talk a little bit about what it is though, because I think there are a lot of people here that haven't used Cucumber, or a lot of listeners.

Yeah. So, Cucumber what it does is it provides you a way of writing your tests or specs in natural language or English basically, for me. The way that it does this… so, Cucumber was originally written in Ruby. I don't know if there are ports to other languages or not. But the way that it works is it has a regular expression that it uses to extract the important parts out of the sentence. So, you'll structure something like, “Given there is an image on the page with URL blah, blah, blah, Google logo or whatever. And given that I'm signed in,” is going to be your next sentence, “When I click on the logo, then,” and that's another line. And the last line is going to be something like, “Then I should see Google.com or I should be on Google.com.” And so, what it does is in the background it translates all of that into… and you have to write all these steps, but it translates all of that into something nice like the code that actually runs those. So, it'll make sure, it'll test that the logo's on the page and it'll sign you in when you say, “Given that I'm signed in.” And then when I do this, “when I click on the logo,” it'll actually click the logo in your test. And then it'll verify that you got the result. In other words, that your browser is now at http://google.com.

Well, I have to say that over the years my thought about this kind of development has changed. Because I used to hate it. Why? Because after you write code for 20 years you can read code faster than you can read English. So, and you have always had better tooling that helps your [inaudible] completion and stuff like that. So, I always preferred writing code. But it's not just about you. You have other people in the team in your organization that probably don't know much about code. If you can use, if the product manager can help you write and help you right there, that's a big plus.

But I guess more than that, they force you to do the layer abstraction like the page object that I told you. They force you to separate between the business logic, the English one, and the code inside. So, that's the nice thing that after learning about page objects, they're forcing you to write in page objects. They don't force you to do it on every component. But at least at the minimum level they do force you to do some separation and some abstraction. And so, your tests are business logic based. And that's the great part of them. If you have QAs that don't know how to code, that helps a

lot. Or if you have product managers.

But a disadvantage is of course debugging. I always claim that I do DOP, debug-oriented programming. I always care about what happens later on, when you have to debug. Most of the time you debug your code more than writing code. And I strongly believe that. So for me, the most important thing to do in every language or framework is always how easy it is for me to debug.

That's it. That's the most important thing.

Yeah, I found that it works really nicely, that approach, because it is English, works really nicely if you have non-code people like you said reading your specs. But other than that, I found that having the extra layer between me and my tests is more hassle and more stuff for me to maintain than its worth. Anyway, I know that some of us have to hop off pretty quick here. So, I'm going to go ahead and push us into picks.

Before we get to picks, let's hear from our silver sponsors.

Jamison, do you want to start us off with picks?

The first is an article by Paul Ford. Paul Ford is the guy that wrote that giant article about code in Bloomberg. And he writes a lot of stuff about technology. I just really like his writing style. This one is a speech he gave to a graduating group of I think they're UX designers or UI designers or something. And he talks about time and the invention of the concept of time and clocks and schedules and things like that, which made me think about how measuring time affects my life.

And then he ends by talking about how as a group these people who design interfaces will be using their time to design them but they'll also be using a way larger amount of other people's time. So, if some of them end up working at Google and add a button to Gmail, that's going to consume, I don't know, millions of man hours. And just the responsibility that we have as people who build things that other people use, to make sure that those are good experiences because it'll affect a lot of people over the amount of time that it's used. It just made me think about the responsibility of building things, which was cool.

My next pick, it's a song called 'In Fantasia' by Kishi Bashi. I don't know how to say this band's name. It just, I don't know. Every once in a while I'd be listening to a song at work and then I'll just get distracted and get the feels. This song gave me the feels today. Just a pretty song.

And then one more pick. I'm going to pick Matt Zabriskie. He and I co-organized React Rally, a conference about React that just happened this last week. And he has been amazing to work with. He works super hard. And he does great code and he does great life. So, if you know him or have a chance to know him, you should get to know him. Those are my picks.

Alright. Aimee, do you have some picks?

Yup, I have two. So, the first one, we were talking about it. I'm going to pick it. If you do any Angular code and it sounds appealing that you may want to mock out your APIs and your end-to-end tests I'm going to put a link to this library that I've used before called http-backend-proxy. And you just put in the actual response that you want to get back. And you can use that obviously instead of hitting your actual API.

And then the second pick is a website called repl.it. And it's a basic environment for a bunch of different programming languages. So, you can just program in these languages in your browser if you want, rather than actually having to set up the full environment. I'm sure there's limitations to it, but it seems like a good starting place if you just want to play around. So, those are my two picks for today.

Awesome. Joe, what are your picks?

Alright. Well, I'd like to start off by talking about some training that I went. I want to pick this training after React Rally. I went and took the two-day React course that, the company I think is actually called React Training. But it's Ryan Florence and Michael Jackson who do the training. I went there. It was a two-day training. It was completely awesome. I was totally overwhelmed by how good those guys are at training. Their curriculum was great. They did a great job explaining it. I just couldn't speak any more highly about it. I already know React somewhat well but those guys were, they just did a great job of explaining React and the why of React and teaching. I really enjoyed and was impressed with their training. So, I'd like to pick them and their training company.

Also at React Rally, Dave Smith gave a talk at the end. And it was brilliance by Jamison and Matt Zabriskie's part to put him at the end. He gave this talk on how React literally waters his lung. And so, the videos are probably going to be up by the time this is published. So, you should head over to ReactRally.com and watch that. That, if nothing else, if you watch no other talk this year, you should watch this talk. I was laughing my head off and entertained and educated as well. [Chuckles] It was just an amazing talk.

I'm excited to see this.

Yeah. Props off to you, Dave. Or, my hat's off to you, Dave.

Then I also want to remind everybody that AngularConnect is coming up here in just over a month or about a month and so, over in the UK. I'll be there speaking about not Angular. I'm going to be talking about empathy, which will be fun for me to talk about something that isn't technical. But I'm sure it'll be an awesome conference. I'm excited to go to that.

And then my last thing is an announcement that tickets are now, the ticket lottery is not available for ng-conf. So, you can head over and register for a ticket for the lottery. And we will be, rather than doing the click-fest we had last year where we sold all the 700 tickets in 10 seconds, we're actually going to do a lottery to try to make it a little bit more fair for those who aren't in the right timezones or for people who can't get away from work at the moment when tickets go o sale. So, you can go over, pick up a free ticket to the lottery, and then there'll be lottery drawings. The first one starts on September 22nd. And there'll be others after that to fill in the tickets that don't get sold. So, I'm super excited for ng-conf 2016. I'm so excited to announce all the awesome things we're going to be doing. So, that's it.

Alright. I've got a couple of things. The first one is, I know this is a JavaScript podcast. But the Ruby Remote Conf videos are now up. You can go watch those. There are some that are pretty good that are not necessarily Ruby-focused. And there are a couple in there that even though they are Ruby-focused, have a message that kind of transcends Ruby. A couple of my favorites were Dave Thomas gave a talk. Dave Thomas is from Pragmatic Programmers. Dave Thomas gave a talk on Elixir, the programming language. So, go check that out. It's a different paradigm and some cool stuff there. And then Peter Cooper who you may know from JavaScript Weekly, he actually gave a talk as well talking about how he uses Ruby to just solve everyday problems. And I think that's something that's applicable to anybody. So, I'll put links to both of those in the show notes.

While we're talking about conferences, Angular Remote Conf is in a few weeks. So, if you want to come out and get a bunch of great information about Angular without actually having to travel, then come check it out. We're going to have about 12 hours of talks. You can get it all for about $200. And you can use the coupon code JABBER. That will get you 20% off.

And the last thing I'm going to talk about is, I've had just an awesome opportunity to talk to people who are on the other end of this particular recording, and that is you listening to this. And I've gotten some great feedback on the shows. And I really, really appreciate getting to know all of the people who enjoy what we do. If you want to have one of those calls, you can schedule a 15 minute call with me. Go to JavaScriptJabber.com/15minutes, that's 15 minutes. And if you do that, then it'll take you to Calendly.

You might have to skip ahead a few weeks because I'm booked out three or four weeks at this point. But go in there and get a time. Because I want to know who you are. I want to know where you listen, which episodes you've enjoyed, what we can cover in the show that will make a difference. And overall, just find out who the people are on the other end of this thing are. And it's not just so that I can, you know, for my own personal gratification. It really is because I enjoy meeting people and I really do need the feedback that I'm getting. And it's not just about the shows. So anyway, for what it's worth, go check those out.

Awesome. I just scheduled myself for a slot.

Oh, dear.

I should have put a blacklist. No, I'm just kidding. [Laughter]

Oren, what are your picks?

Oh. Well, I'll start with a great Selenium blog to get started. 'Elemental Selenium' by Dave Haeffner. If anyone wants to start using Selenium right now that are listening, it's a great place to get started. Very simple examples. How do I do this? How do I do that? So, you start with the basics and moving on to the very complex stuff.

Second pick. That would be Lea Verou. She has a new book on 'CSS Secrets'. Really amazing stuff. Nobody every talks a lot about CSS. And it's one of my personal favorites, doing cool stuff with CSS. So, I have to pick her.

And there's… last will probably be, ah, Cloudinary. Cloudinary is a product I started using which every company I used to work in, we did the images in-house, dealing with images, uplink to CDNs and having transformation, cropping them and resizing them. And there's a SaaS solution that does that. So, every time you [inaudible], every company struggles, someone does this SaaS solution which I love.

And for the latest, that's Docker. Docker is amazing. For everything that is doing Selenium, if you want to have 20 browsers up in five seconds, now there's Docker. So, helps a lot. So, I'm in love with Docker.

Alright. Well, thanks for coming, Oren.

You're welcome. I had a great time.

Yup. Us, too. Alright, well we'll wrap up the show. If people want to follow up with you, figure out what you're up to, what should they do? Where should they go?

They can contact me directly, probably either Twitter, Facebook, LinkedIn. Or they can check out what we do at Testing.io and just find out how we're trying to help the world making testing simple.

That's the best way I guess.

Are they okay to swing by your house?

Sure. [Laughter]

Well, next month I'll be in San Francisco, so not next week. So, I can stop at their house.

Oh, there you go.

[Laughs] They can send me the address.

Alright. Well, thanks again, Oren. We're going to wrap up the show and we'll catch you all next week.