01:10 - Janie Clayton Introduction

03:02 - iOS and Robotics

08:39 - System Architecture

11:24 - History

14:19 - Robot Design

18:52 - Why is robotics more complicated?

20:09 - Assembly

21:24 - Rewriting Objective-C Apps in Swift

27:25 - Design Patterns

29:21 - Connecting Worlds

30:52 - Realtime

33:44 - Open Source and GPUImage

Picks

ORSSerialPort (Andrew)
Honeycrisp Apples (Jaim)
NSScreencast Episode #188: App Transport Security (Jaim)
Rush Revere and the Brave Pilgrims by Rush Limbaugh (Chuck)
The Magician's Nephew by C. S. Lewis (Chuck)
MONEY Master the Game: 7 Simple Steps to Financial Freedom by Tony Robbins (Chuck)
GPUImage (Janie)
OpenGL ES Pixel Shaders Tutorial (Janie)
Programming Sucks (Janie)

 

That is me dressed up like an anime character standing in a TARDIS.

Ahh, gotcha.

Oh, we’ve all got one of those. [Laughter]

I have several.

Hey everybody and welcome to episode 126 of the iPhreaks Show. This week on our panel we have Jaim Zuber.

Hello, from Minneapolis.

Andrew Madsen.

Hello, from Salt Lake City.

I’m Charles Max Wood from Devchat.tv and this week we have a special guest and that is Janie Clayton.

Yay.

Do you want to introduce yourself?

Sure. My name is Janie Clayton. I have been an iOS developer for the last two years. I have a couple of books out of the iOS 8 SDK, crafting IOPS in Swift through The Pragmatic Programmers that I wrote with Chris Adamson. And just out today I am a contributor to the Swift apprentice by Ray Wenderlich. I run Ray’s Tutorial team. I spent a year working for Brad Larson, the creator of GPUImage and I currently work as a contractor at Black Pixel.

Wow. So can you do that with Ray Wenderlich and Pragmatic Programmers? Don’t you get the Ray Wenderlich guy and the Pragmatic Programmer guy [inaudible]?

Well, I had taught to the editor at the Pragmatic Programmers. I didn’t actively contribute to the book this year; I was a little bit swamped and I didn’t mean to contribute to the Swift book though I had to sign a contract saying they could use the work that I did last year and I had to contact them if I wanted to work on somebody else’s book. And I contacted the editor and I said, “Hi, is this a conflict of interest? Is it okay if I work ok this book?” And I never heard back so I just went ahead and did it anyway. [Chuckles]

No news is good news, right?

Yeah, pretty much. I made them aware of it and I only did one chapter. I did twelve pages but I got my name first on the book as it came first alphabetically so huzzah. [Chuckles]

Huh.

I don’t have that problem. I’m always last. [Chuckles]

Yeah, you come in behind me wouldn’t you?

Yup, I can write everything but I’m still last [inaudible].

But I’m just about right in the middle.

Well, I’m not co-authoring a book with you guys soon.

Well I’ve never co-authored a book so it doesn’t matter [crosstalk] books anymore.

Yeah, there you go.

Anymore. [Laughter]

We were talking before the show and we thought we would talk about robots which is something you think about when you think about iOS. Can you give us a brief overview of how you think about robotics when you're talking about a device like iPhone or an iPad?

Actually, our robotics, we didn’t utilize iOS technology. Wee rewrote them for the Mac so all of the stuff is in Objective-C, Cocoa, Swift, etc. I know that we were talking at some point about trying to eventually impossibly replace our control software on the Mac with an iPad but that wasn’t something that was at the top of our priority list. Do you want me to give you an overview about how I got involved with that job and then why we decided to rewrite all of our control software?

Yes.

Okay. I have been programming for about two years and I got my first programming job back in December 2013 and I was working for a startup in Madison that I’m not going to name because I’d like to be able to say not necessarily nice things about them. I don’t want anybody to know who they actually are but I was the oldest person that were [inaudible] a decade and all the people who were running the company were like 20 years old and none of them knew how to code. It just wasn’t really a good fit and I was there for about two months and I knew I was about to get fired and I started trying to think about what I wanted to be doing with my career and I knew that I wanted to work with somebody like Brad Larson.

Brad’s iTunes U course that he has up on iOS programming was taught at the school that I went to for programming and when I was going to school there I kept hearing all these stories about him, about how he did all these impossible things and how – like Apple told him that something he wanted to do couldn’t be done and he figured out a way of doing it anyway and they offered him a job and he was just like, “No, I have this job where I build and program robots. I’m totally going to – this is my own company; I’m going to stay and run my own company.” And I just really thought he was cool, awesome person that I wanted to be like.

So I reached out to him and told him I was interested in learning GPUImage stuff because I wanted to, at that point, to GPUImage contract work and we wound up working on a contract together. We got to be friends and a little over a year ago, he brought me on to work at his company. Originally, when he brought me to work at the company, the idea was that I will go inn and I would do bug fixes and other things to keep our software maintainable because we’re having a lot of issues with crashes. We had a bug that cost $10,000 worth of damage because there was a nil messaging error. The entire first two months that I was there, while I was working on things Brad kept going, “You know what, this is so much easier to do if we just did this in Swift. Just to be safer, it would be in Swift. It would be easier, the code will be better if it was in Swift,” and then I finally just said to him, “Dude, you clearly wanted to do this in Swift so why don’t we just rewrite the software in

Swift?”

So we spent about ten months rewriting our legacy Objective-C software in Swift and that was what I got to do with him for the last year.

Wow, that’s really cool.

Yeah, it was really cool. [Chuckles]

So what kind of robots were you controlling over there?

We had a robot that has three axes, so an X, Y and a Z. the purpose of the robotics is doing Microscopic precision fluid printing. And what that primarily was used for is originally developed for doing biological research for creating protein microarrays but then in the last five years or so, our primary clientele has been people doing printed electronics and working with things like carbon nanotubes.

Wow. So it’s just like 3D printing on steroids?

No, it’s not 3D printing. The way that the fluid id controlled is using ultrasonic frequencies. This was something that Brad [inaudible] is getting his PhD in Material Science at the University of Wisconsin was that using high frequency audio waves, you can control the amount of fluid that is being dispensed by the printer. I, of course, am interested in audio so when he was telling me that they're sound to control the printing on the robots that got me incredibly excited. But no, it’s not 3D printing per se; it’s doing printed electronics so it’s a little bit like almost an ancient printer except we’re not spraying black massive quantities of fluid. We’re doing very precise small amounts of fluid that can be done to print circuits so you can do printed electronics on flexible surfaces.

Oh, okay. And you make the computers seem to control it?

Yes. One reason that Brad went with the Mac system was because it had a really good robust ability to do good user interface stuff. A lot of our clients are from overseas – in China and in other countries where you don’t necessarily want to have something that’s incredibly complicated, textbased user interface.

We have a lot of buttons and a lot of icons and a lot of things that aren’t associated purely with English in order to try to make this as intuitive as possible.

Am I correct? Because normally if you're creating this kind of system and you want industrial users or researchers or whatever it is, you probably thing, “Well, Windows has a huge market share. I can’t make this Mac only,” but I get the feeling that when SonoPlot sells one of these, they sell a Mac with it and it’s just part of the system.

Yup, that’s correct. Yup, we go and we install the software on the iMac before we send them out and we have to set up some parameters for each of the robotics systems.

It happened a lot more recently. We’ve had times when the control soft computers have been stolen from the lab and we’ve had to go and buy another Mac and send it out there and we’ve had to actually do that to having them go out and buy it themselves because we have a bunch of settings that we need to put on to the software but yeah, that was a consideration that brad made when he decided to do everything in Cocoa. Cocoa was really – it allowed him to do a lot of development very quickly. There’s a real advantage for having a primary platform of just knack instead of trying to make it cross-platform.

We just include a Mac in each of the systems. I think the systems are high-end systems like $52,000 and buying a low-end Mac is a thousand bucks. The gray end scheme of the cost of the system isn’t that much. [Chuckles]

So the key to describe the system architecture – so how is the SonoPlot connected to the iMac?

We have an electronics box. So one of the things that we needed to make sure we could do with Swift when we transferred everything over to Swift was to make sure that we could connect to the serial port and be able to connect to the electronics. So our control software will have a navigation thing where we want to tell the X axis to move over 100 microns. So there’ll be a button where you can set the number of microns you want it to move over and then you click it. And then that signal is translated over into code that get sent to the electronics board which then sends it out to the motors. And then the motors go and do stuff so we’ve got nice little chain between the control software and the Mac, the microcontroller and the electronics box, then the programming on the motors.

I actually wanted to ask you about that. So the background is I have an Objective-C serial port library that’s probably the most popular one; there are like two out there [laughter].

It’s one of the two most popular in the world.

And when Swift first came out I thought it would be kind of fun to rewrite that in Swift but it was really just actually impossible primarily because Swift 1 didn’t really support the passing function pointers or passing Swift functions into APIs that expect the function pointer – a C function pointer. And I remember you and Brad did a blog post called Swift serial port which was a really good blog post and it had nothing to do with serial ports. It was just all [laughter]. But I’m curious to know how you solved that problem. Did you end up writing the serial port part of that code in something else?

With Swift 2.0 it’s – I think you can do it now but before that it was hard.

So that was just a caveat. Brad wrote a lot of the serial port software. He was doing that while I was working on some parts with Objective-C just to make sure this was something that we could do. I’m assuming either we did it in Objective-C or else I’m pretty sure – I know that we needed to use Objective-C to connect to the camera. We have an IIDC compliant camera and that one did require us to have C function pointers but I don’t think that the serial port required us to have C function pointers. I think we were able to do everything that we needed to do.

I know we were able to take everything and just convert everything to UTF-8 strings and at the [inaudible]. I know that I needed to go through and do stuff on the ASCII chart so I know we were able to break everything down into just the numbers and everything is so [inaudible] with the ASCII chart. I’m able to sync commands that way and we didn’t need to use C immutable function pointers.

Interesting. I guess it’s actually a part of the API that requires function pointers is the I/O kit discovery stuff and you may be didn’t really need to use that because that’s more of if you don’t know what devices are hooked up and you need to find them and be notified when new ones are connected.

That was an issue we ran into.

Cool. Well, speaking about this you mentioned that the system already existed. When did Brad’s or SonoPlot’s software start? How old was the code based on what you were working on to start with?

The company began in 2003 and for the first couple of years, Brad had written the software in C++. The decision was made in 2007 to completely rewrite the software to Objective-C and I think that took about six months. At the time that I was working on the code based on about seven or eight years old.

Kind of an old code basin.

Oh yeah.

You know, Objective-C land.

It was an older code base. One of the first things that I needed to deal with while we were there was we had a plugin for interface builder because there were bot parts in the UI that had been created before interface builder existed so I had to figure out some way to break our dependency on this plugin for interface builder and I was like, “Dude, why would you use a plugin for this?” “That’s what all there was back with XCode 3 how many years ago.” [Chuckles]

Yeah, he’s not wrong about that. There is actually a popular third party thing called BW Toolkit. It was an XCode parent interface builder plugged in and I use that and it kind of sucked when XCode 4 came out and it did not work anymore.

Why would – I think that there’d be better tools and the XCode would be a good thing instead of having to work around in order to get them to work. [Chuckles]

You're right. It was cool because you can see your custom UI and interface builder back then. And then they took that out and then they added it back finally in XCode 6 in a much better design way. This new stuff is way better than the old stuff but for a while they just took it out and there’s nothing to replace it. You’re stuck figuring out how to deal with that.

See, I don’t’ remember back then – I remember [inaudible] was really cool when they came up like the IB designable stuff that I didn’t realize that they'd actually have that functionality in their [inaudible]. I think my first XCode I started working with was XCode 4.

Yeah, they had it before in XCode 2 and 3 and probably even – it was not even called XCode before that but I didn’t – I started with 2.0 but it was pain. It was way harder to use than the IB designable stuff. It’s definitely an improvement.

I didn’t want to mention that the IB designable stuff, at least with XCode 6, it only works for iOS but it didn’t work with Mac stuff so we weren’t able to use the IB designable stuff in our code base which was total crap. [Chuckles]

I know it works now because I’m using it in a couple of things I’m working on now but in XCode 7 – in Mac apps.

Yay.

That’s a good thing. I feel like Mac is not getting as much attention as iOS from Apple lately.

Oh, absolutely not. When I would go and I would look at all of the NS whatever documentation that they had available, I think they have last been updated in 2010. I saw a couple of things where it was like, “You’re either doing this in Cocoa or in Carbon.” It was just like, “Oh God, really? You haven’t updated it since then? Why?” [Chuckles]

Yeah, we stick [inaudible] of all the Java, Cocoa documentation. [Laughter]

I want to veer a little bit more to the robotics for a minute. You mentioned that you’d tell it to move one hundred microns which is – if you don’t know, a hundred microns is really tiny. [Laughter] I’m curious, how do you design your robots to be that precise?

Interestingly, we went through when we were doing our prototype system. So our large system on the micro plotter 2 has a resolution of 2.5 microns and I believe that a lot of that had to do with what motors and axes we decided to purchase. So I know that our axis that we have on the primary one are from NSK and that we use animatic smart motors and that we needed to make sure that our motors could go through and actually get the resolution that we wanted to accurately.

So I was talking about we have this prototype system. That was replacing our desktop system that I believe had a resolution of 20 microns and we wanted to have a little bit more precise than that so we tried a couple of different types of motors. We tried some stepper motors but we couldn’t ever get the stepper motors to accurately be able to keep doing ten microns or lower. So I think of some motors that were hybrid stepper and we weren’t able to use just pure stepper motors. We had to get slightly more expensive motors that had a little bit more precision associated with them in order to be able to hit our specs of 10 microns.

And then are there libraries that translate the rotational – what am I thinking – so how far has it to rotate in order to get you –.

All the motors that we have for each of the systems are programmable so it isn’t that there’s an external library that exist somewhere else, it’s that each manufacturer of these programmable motors goes through and has their own software and has their own documentation.

One of the things that Brad got to do that I was really jealous of him being able to do that he told me I shouldn’t be jealous about, he had to go through and work with the motors and be able to figure how to program them and figure out their command sets and be able to figure out their tuning parameters in order to make sure that they were able to go through and behave the way we wanted them to. I thought that all of that was incredibly interesting and I really wanted to work with that. Unfortunately, that was not something I got to work on.

The other question I have with this is a lot of robots or robotics that you wind up working with, they have some kind of controller on the other end? So you just hook the whole thing up to a serial port and you send it a command like ‘move to this XYZ coordinate’ and then the controller on the other hand actually figures out how to move the motors – is that what you're doing or are you giving each motor or each controller on the motor its own instructions?

We’re sending each of the motors their own instructions I believe. I know that we don’t have a lot of programming done with the microcontroller and most other of our competitors have incredibly robust microcontrollers and a lot of the logic is being done electronically and through hardware. We decided to do a lot of our processing through softwares. A lot of the program that we have about where we’re telling the motors to go is being done within the control software on the Mac.

I guess my last question is how many inputs can you give the system? So you have X, Y and Z and up, right and left or whatever, and then you’ve got other things like you’re doing fluid stuff with sound frequency and stuff like that – how many different things do you have to keep track within your program at the same time to get it to do what it’s supposed to do?

So we’ve three axes – we have the X, the Y and the Z. we have the dispenser and then we also have an external camera. We use the camera because we want our customers to be able to go through and get screenshots and videos of all of the work that they are doing. We have people who’ll go and make videos of their projects that they're working on and they’ll include them with their research materials. We will go 100% on their topics at various conferences. We also needed to have the camera available so that the customers can go through and actually be able to line up and move the dispenser to where they want it to be on deck because that’s completely controlled by them. So we have to keep track of all of the axes, the dispenser and the camera.

And then in the software on the Mac, do they put the design in and then the program figures out how to tell your dispenser to move up and down and around to get to the right place to do things?

Yes, we had an external CAD program called SonoDraw that we include with all of our systems where people can go in and they can draw circuits and they can draw various different things. One of the things that I was responsible for working on in the control software was – the SonoDraw goes and creates an XML document and I had to set up the XML parsing within our control software in order to be able to send the robot the commands about how the axes were supposed to be moving.

So I’m going to make it sound like it’s as simple as just telling it, “Move up, down, right and left. Do the dispenser thing with the sound and everything else,” and that’s all there is to it. Why is robotics a little more complicated than that?

It’s a little bit complicated because what you're saying is very true –you're telling it to move left, right, move whatever but you also have to keep track of all the stage of everything moving all together.

I know one of the big things that Brad was interested in with Swift and one of the reasons we decided to go with Swift as opposed to stay with Objective-C was because of the more robust error handling. When they introduced the error handling stuff in Swift 2.0 the more complicated parts of our control software where we have 20 or so different commands that all have to be able to execute without throwing an error in order for things to go through and complete successfully because, again, we had an issue before with nil messaging errors where the dispenser would be rammed into the deck because it would try to go to zero and zero is all the way down.

It is as simple as saying you have to make sure that this does something, this does something, this does something but then you also have to make sure that all of them are doing that safely in order to make sure that you're not causing any damage to the robotics.

It’s very complex to try to make sure that everything goes the way that it’s supposed to in order to avoid creating anything that would cost damage.

And of course there is also the difficulty of actually building the robots, right? Those seem like they're pretty sophisticated to me.

There were actually surprisingly not that difficult to assemble. We tried to get as many parts off the shelf as we could from other places and so we had a couple of machine shops so we go and create the base plates for the robotics. I’ve put one together in about a day; it was really nice to be able to put a robot together because there were just days when I just did not want to sit in front of my computer and code. Being able to just go and put on headphones and get out the screw drivers and the axes and be able to construct the robot with some incredibly therapeutic thing that I kind of miss right now. [Chuckles]

I know how you feel having been a hardware engineer that like to build stuff and now I sit in front of a computer all day.

I feel really bad; I set up an electronics shop in my basement with a soldering iron. I’ve got all of my different stuff and I kept thinking when I have my free time, I was going to go downstairs. I was going to go and I was going to build electronics and I haven’t had free time yet. [Chuckles]

Well, I was going to say with the list of things you told us you do when you introduce yourself I don’t know how you have any free time at all. [Laughter]

I don’t. I recently started from home and I realized, “Oh, you’ve got two more hours a day that you can do stuff,” and I’m like, “Yeah, I’m not actually getting two hours a day. I don’t know what happened to it. It just got swallowed up by all of the other things I’m doing.” [Chuckles]

you started to mention this but a big part of what you were doing was rewriting the Objective-C apps that were already there in Swift. I want to ask you how that went. I’m curious to know – first I’m curious to know what was the overall thinking behind deciding to do that? Because it’s a pretty big job to rewrite your apps. Other than I know Brad was excited about Swift but in terms of real legitimate ‘this is a good idea’ kind of reasons. What was he thinking there?

Again, part of the thinking was that Brad was very interested in being able to have better error handling. We had [inaudible] of the code base was about seven or eight years old and as it grew in complexity, there were different things in there that we couldn’t go through and actually change. Brad created GPUImage and we weren’t about to incorporate GPUImage into our camera software and the software because it was just at a point where if you even looked at it then things would break.

We had an error that would happen that we didn’t have any idea what was actually causing it so we get this passive-aggressive thing in the console saying ‘this is a generic error message but your user deserves better’ and we couldn’t go through and customize the error message because we have actually no idea where the error was being thrown.

We had a lot of spaghetti code, things were tightly coupled. There were a bunch of parts that were fragile and we got to the point where we couldn’t add improvements to the software because they just – it was too difficult to go through and decouple all of the code.

One of the thoughts behind it was being able to go in and actually re-architect the software in a way that made it easier to test and to be able to modify it and add more functionality later. But no, a big reason that we decided to go through and update all of the stuff was the error handling because we, again, when you have errors in your software it’s a pain. People get annoyed when it crashes. [Inaudible] hardware, it causes physical damage; they cost a lot of money.

I didn’t want to mention, I do a conference talk about this software project that we did and I’m going to be presenting it at CocoaConf San Jose in the first week of November. Anybody who’s interested in going and hearing me do a whole big spiel about it.

I wish I was going but not going to be able to swing it this year but maybe I can make it to one of them next year. Well, I don’t want you to spoil your conference talk but that said I’m curious to know how you went about the rewrite. Did you just create a new XCode project and start typing or was this sort of a piece by piece refactoring parts at a time?

We started a new XCode project; Brad had done aside a project working with the error handling and the serial port stuff just to make sure that Swift was a viable thing. We didn’t just randomly, one day say, “Alright, we’re just going to redo everything in Swift.” We actually went through and did some benchmarking and did verify that some of the stuff that we’re a little worried about would actually be able to work in Swift.

I don’t know if you remember this or not but when Swift first came out, all of the grey beards were saying, “Oh, this is a bad idea. This is horrible. It’s not ready for production. You can’t use Swift as a normal app; we’re just going to stick with Objective-C.” So back in those early days, we weren’t entirely certain whether Swift had gotten the point where it would do everything that we wanted it to do or not. Again, Brad put together the serial port and the other stuff and that served as the basis for our project.

He went through and he figured out how we could add functionality to project. The more complicated things were, they can be added later so I believe first things we put in there we put in the serial port and we put in the electronic stuff. I know that dealing with robotics was one of the last things that we put in the code because it was the most complicated part of the code but we broke it down into pieces and we slowly built it out bit by bit. Like I said, I worked on the XML wrapping. I actually had spent two months going through and learning libxml2 and wrapping that in Swift so that was an interesting experience. Then I did stuff with the camera and we just – we had discreet pieces of the functionality that we would port over and we write unit tests around to make sure that everything worked.

Cool. So I’ve heard recently – I can’t remember what company it was but some company was saying that they rewrote their app from Objective-C and Swift and the total number of lines of codes in Swift was way less than in Objective-C. I’m curious if you’ve seen that, too – did the code base get simpler in Swift?

Oh, the code base shrunk dramatically. Back when I was first doing my conference talk, I thought that our code base was only 25% the size of the equivalent Objective-C stuff but recently, when we just shipped this off I believe about a week or two ago and the final software – it wasn’t quite as small but it was still less than 40% of the size of the Objective-C one in terms of lines of code. We also didn’t had the unit test in the Objective-C and we do have unit test in Swift but there’s a dramatic shrinking of the code base.

So which products do the Objective-C are making it verbose?

A lot of the work that we did, we actually went and rewrote things in Swift so we had data sterilization where we’re going and saving things to and from the user defaults. Instead of using the NS coding syntax, we went through and we wrote our own syntax in Swift. So anything that was a Cocoa framework is not something that you can really cut down on a lot in Swift.

I was recently working with some core graphics stuff and the core graphic API is incredibly verbose. There’s a limited amount of code that you can cut when you're going and you're interacting with an Objective-C framework.

If you're going through and writing everything just in Swift, you can get a lot of wins off of that because you're actually writing all of your conditional logic in Swift. But if you're going and referencing something that’s written in something else, you're not going to see the dramatic gains.

Do you have any sense of, after rewriting this app in Swift, do you have any sense from the field whether it cut down on the number of errors and bugs that users see in the app?

Absolutely. We were able to – I believe we’re able to cut out 40% of the bugs that we have just like inverting the Swift because there were things that just wouldn’t even have been – the compiler would’ve been allowed us to do.

A lot of our issues were with nil messaging errors and that can’t happen with Swift. So we’ve seen a huge dramatic increase in the safety of the software and being able to prevent bugs from even occurring.

So I’m curious on what the architecture of the actual application looks like? So you have one [inaudible] buttons and view controllers and you're talking to robots. What are the patterns you're using for the code?

We didn’t actually use a lot of design patterns. We were able to utilize higher order functions in order to create helper functions that were being used throughout the application. We didn’t use a lot of classes; we had a lot of [inaudible] and we implemented a lot of protocols. A lot of the things that they were talking about at this most recent WWDC about the protocol oriented programming are things that we had already started implementing in our code.

One of the reasons that Brad was interested in Swift was because he had been interested in Haskell and was interested in all of the different functional programming paradigms and aspects that existed in Haskell. So he went and learned Haskell and was able to take a lot of the programming patterns from Haskell and applied them to Swift.

Okay, so if you take a typical view controller by moving the robot, try setting the X and Y, let the user do that, do you have access to all the code? Is there any kind of tractions that you're using within the view controller?

Okay, so you're talking about things that are specifically Cocoa like the view controller. We don’t have a lot of view controller stuff in our code. I think 15/20% of the code had to do with user interface stuff. Most of our code had to do with actual logic assisting with the robots and being able to parse documents and controlling the dispenser.

When you're talking about all of the stuff with the view controller, yeah that was stuff that we had to use Cocoa designed patterns around but most of the work that I did on the application was not on view controllers; it was on actual classes that controlled the electronics or controlled the robots or controlled the errors or controlled the dispenser or controlled the camera.

So we had classes associated with each piece of hardware that we were using and that was where we were able to see a lot of gains. We were able to apply a lot of this stuff because we weren’t actually using view controllers or using things that were Cocoa frameworks.

Okay, I’m curious about how you connect the two worlds. We’re trying a lot of embed stuff and [inaudible] low, low libraries but do you also have a user do things that make these [inaudible] happen? I’m curious what the connecting points are so you wouldn’t have a lot of view code; you do simple things but there’s certainly complex processes and I’m just interested in where the [inaudible] are and how you're separating those things out.

Yup. We’ve had very little code that was in the view controllers. A lot of it was – were telling the robot to move somewhere and we would send a command from the button out to our instance of a robot class and we would go and figure a series of complicated maneuvers that would be done within the robot class. So all of our logic, instead of being in a view controller, was in a robot.Swift class.

You mentioned something – I think you said the app was called SonoDraw so that’s the app that people can use to draw their whatever it is that they're making, they're printing, right?

Yup, that’s our CAD program.

But that’s separate from the robot control app, right?

That’s correct.

Yeah, okay. So you probably got a lot more UI code in SonoDraw than you do in your actual – whatever you call it.

Right. SONOGUIDE is the control software. So in SONOGUIDE you can go in and you can upload a SonoDraw file and so the file is just an XML document that goes and gets parsed by the cursor within the SONOGUIDE application.

So there’s the answer to Jaim’s question, the communication between them is the XML file that you get out of the one app and put in to the other. That’s pretty cool.

So is there a real time component to robotics that you're writing for this?

What do you mean by real time?

Real time system – if it doesn’t happen this time, it’s not correct.

Yes. We did have listeners to make sure that if we sent out a message and we didn’t get back in a certain amount of time, we would have a timeout error. We were actually able to go in and customize our errors to the point where if you had an error, we knew exactly what the error was.

So if you're system timed out, we could send back a message to the user saying that the system timed out. Or if the electronics was disconnected or unplugged, we could send back an error about that or if the robot hid an obstruction instead of it just being ‘this failed and we don’t really know why’.

We were able to go through and actually do a lot of conditional logic instead of complex error handling in order to be able to give very specific details back to the user about what their problem was so that they can go in and they could fix it because it’s not really a point of going and continuing to try to send messages to the camera if the camera isn’t plugged in.

True. It sounds like a pile level error – is there any concepts like if this is 20 milliseconds late then that’s an error.

Yeah, we had timeout errors.

Do you get to the point where it’s like a hard real time where it’s guaranteed to happen at a certain thing? Is that even possible with Swift or even dealing with that kind of stuff?

Yeah, you just use timers. You’d set it up where you’d have a callback lock where you send off an asynchronous message and if you don’t get it back within a certain amount of time then you would throw an error.

Okay, so you listen for [inaudible] to complete then it’s throwing an error?

Yeah.

Alright.

Of course now I’m saying all of these and Brad will listen to this and go, “No, no. You're explaining it wrong.” And I’ll be like, “Dude, I don’t have the code in front of me. I don’t remember.” [Chuckles]

Yeah, when you came to a hard real time that’s very specific and I’ve never done any hard real time things but who else have done this audio where it sounds bad. [Laughter] Not worth something like breaks but when you get into robotics where things are moving, if you're off by even a tiny amount you can totally break stuff. And you get to a point where your operating system has to be ready to handle that and [inaudible] how well Swift works that but you're doing – working to [inaudible] OSX, OS 10, callbacks for timing – that makes sense.

There’s only one part of our software that we had to go back to Objective-C for and that was, again, the IIDC camera class. We had a library that we used to go in direct with the IIDC camera so that’s a fire wire protocol compliant camera and that particular library had C function pointers that we could not use in Swift so I had to go through and spend two months going and porting that over to Objective-C in order for us to be able to implement GPUImage but aside from that one thing, everything else we’re doing was able to be translated over properly in Swift.

I wanted to just tap on to that. I think what you're talking about is actually open source, right? The GPUImage, IIDC camera example?

Yes, that’s the project that I did while I was working for Brad. That is my code.

It actually feeds right in to my next question which you kind of just answered but you mentioned earlier that one of the things that made you guys want to redo all these was that Brad created GPUImage and yet couldn’t use it in the existing project and you guys wanted to use it. I wanted to know a little bit about how you use GPUImage and also what you did with that because I know graphics programming is another one of your interests, one of the things you know about.

So back when Brad created this back in 2007, that was before we had AV Foundation. So all of our video code is written in – we used the core video library and it was very complicated, there’s a lot of stuff in there that just worked like magic and if you change something it would break and you wouldn’t really know why.

So when AV foundation and GPUImage and all of these other things came up in the last couple of years, it was incredibly difficult to go through and actually try to add it into the current code base. So what I was responsible to go in and translate over the IIDC camera stuff, one of the tasks that I had was going through and trying to get rid of all the core video stuff that we have in the original code and bring over just what we needed to do.

So the reason that we have GPUImage is because we do filter our images. We do go through and we add a de-noiser and we have a couple of other filters that we apply to our video and eventually we’d like to get to a point where –.

So Brad is very interested in machine vision stuff. It would be really cool if we could go in and we could actually add edge detection and other things associated with machine vison by writing filters for the GPU to be able to add that functionality to a robotics systems. So that’s like a long term thing that we eventually like to be able to add to our system that we don’t currently have and we wouldn’t be able to add if we kept the old code base.

We should point out to those listening that GPUImage is a framework that Brad created but it’s pretty popular now for doing really fast image processing that’s GPU accelerated and it works on iOS and the Mac and it’s like Core Image only it does some things that Core Image can’t do and it’s faster in certain areas and stuff. It’s pretty cool.

If you haven’t checked that out, definitely check it out.

It’s also open source so if you're using Core Image and you're applying a filter, it’s kind of in a black box; you're not able to actually see what the code does. So if you're somebody who’s interested in going and learning graphics, programming or anything having to do with image processing, it’s a really awesome resource for people to be able to go in and actually look to see how something works. So it’s really awesome resource just for the code base in and of itself.

Yeah, that’s an excellent point. That’s a big advantage.

Alright, well let’s go ahead and so some picks. Andrew, do you want to start us off with picks?

Oh yeah, I’ve been kind of lazy thinking about my picks for this episode. I guess I’ll do a self-surfing pick but it’s related to some of the stuff we’ve talked about and I think maybe I’ve picked it before but that is my serial port library. It’s called ORSerialPort and it’s written in Objective-C. it’s been around for a few years but it works very well with Swift, too, and I have Swift examples for how to use it.

I think it’s pretty cool. Makes it really easy to use a serial port from your Mac program. If you’ve got an Arduino and you want to make a little Arduino project and then have a Mac app that controls it and you know how to write Swift or Objective-C, this is pretty easy to get up and running with not a lot of code. And if you’ve ever looked at the underlying serial port APIs on OS 10, they're all piercey and they're not exactly friendly to beginners so check that out. I think that’s my only pick today.

Alright, Jaim what are your picks?

Alright, I’ve got a couple of picks. One, it’s October right about now and that means it’s apple season. One of my favorite apples is Honeycrisps which originated in Minnesota so you can get them pretty fresh here so the stores are full of them. It’s an amazing apple for eating so if you like an apple and you're in an area where you can get them fresh, I definitely recommend getting Honeycrisp apples. Good stuff.

Second pick – I’m a subscriber or NSScreencast by our ex-panelist Ben Schierman and I just had to admit that I haven’t been checking in that much to what he’s been doing but I checked in a week or two ago and found one episode they did on the ATS, the new security framework or protection for iOS. If you run into that and if you go into iOS 9, most of us just turn it off because we didn’t know how to fix up the service stuff ourselves but for diagnosing and helping your server team figure out what’s happening, they you did a great episode on the ATS. So if you're a subscriber and not checking in as often as you are, you might want to check out this episode on ATS and those are my picks.

Alright. I’m going to pick a couple of books. I’ve been reading to my kids at night when I’m home and have time which is usually three or four nights a week. And the books that I’ve been reading have been fun. So the first book, I’m just going to – actually, I’m going to pick three books and I pick two of these on the other shows so whatever.

So the first book – the first two are going to be books that I’ve been reading to my kids. The first one is one that my father-in-law gave to my daughter for Christmas. It’s a historical fiction and they travel back in time and experience historic events. It’s just been really fun and very educational for my kids. The one that we read was Rush Revere and the Brave Pilgrims. I know it’ll probably turn some people off that it was written by Rush Limbaugh but this isn’t a politicized book. It really is just a ‘here’s what happened in history’ and some background and context that the kids understand what’s going on the boat and as they settle the colony. It was really good, a really great read.

The other book that I’ve been reading to my kids is The Magician’s Nephew which is the first book chronologically in the Narnia series. It’s right at their level. Great books, a lot of fun and I’ve enjoyed those books myself so I’m going to pick those.

Master The Game. Most of it is about investing and how to invest to ensure the future lifestyle and income and he’s really done his homework and there are a lot of things in there that I have never really thought about.

I don’t know if I agree with everything in the book but it presents a lot of different options for you so that you can make the decision that works best for were you're at, investing as far as how much risk you can take and where you want to wind up. So I have to recommend that book because it was really great. Those are my picks.

Janie, do you have some picks for us?

Yes. Since I am interested in graphics programming, I wanted to mention a couple of different resources wherein you briefly talked about GPUImage. I highly recommend that to anybody who’s interested in graphics programming because I personally have not taken any nap since I was 18 and that was longer ago than I’d like to admit. So it’s really cool to be able to go in and look at the filter showcase and find a filter that does something that’s really interesting and then be able to go back and look at the code and be able to pick through it and try to figure out what the math was behind it or what all of the different aspects of the shading language actually mean. It’s really a good learning if you're interested in figuring out how to write graphic shaders.

Also, along with if you want to learn how to write graphic shaders there is a wonderful tutorial from Ray Wenderlich’s site that is awesome for entry level graphics processing. Most of the resources that identify what I’m trying to figure out graphics programming are either really math intensive with all the weird creepy Greek symbols that I don’t remember – it’s been a really long time. Or you assume that you already know the math and you know a lot of the other stuff and so it’s incredibly difficult to go through and actually try to learn programming. So that particular tutorial I thought was really awesome because it assumes that you don’t actually know all of these different things. And it explains Perlin noise, it explains a lot of the equations it came from and it explains what they're actually doing. So that’s my recommendation for anybody who’s starting off with graphics programming is to do that tutorial and then after that tutorial, go look at GPUImage.

And then my last pick that I wanted to talk about was an incredibly awesome blog post that I will occasionally read when I feel very bad and it’s called Programming Sucks and it just talks about how all of programming is basically nothing but despair because everything in the world is ten minutes away from completely falling apart and that the only reason that the internet works is because you have hundreds of people like Google who are running around like hamsters in cages, 24/7, frantically patching things and keeping everything from falling down around our ears.

So I mentioned when I started off that I had this job that I didn’t particularly fit in very well in and I kept thinking, “Man, everybody’s code sucks. One day I’ll go and I’ll work for some company where all of the code will be awesome and perfect and wonderful,” and I realized we all feel that way. We all think that everybody else is as good as horrible and that everything is falling down around our ears and that the apocalypse is neigh but it’s okay; we’re all in it together and it’s reassuring to read that and figure out that I’m not the only one that feels this way.

Boy, I’ve had those days, too. [Chuckles]

Someday I’ll work with a real team that writes good code. [Laughter]

Someday.

I don’t know why, I’d be on that team. They’ll let me on the team but I don’t know.

Yeah. Alright well, if people want to get more information about you or follow you somewhere and arm up for a road trip to wherever you live where do they go?

Everything that I have is associated with Red Queen Coder so my Twitter handle is @redqueencoder. My blog is hosted at redqueencoder.com. If you just go online and you Google Red Queen Coder, there’s plenty of places where I go and I spout of all of my various ramblings and people can consume them if they so choose.

How did you come up with the handle?

I use to play chess a lot when I was a kid. I was also really interested in Lewis Carrol. So there was a character in Alice: Through the Looking Glass who’s the Red Queen. It was the red queen from the chess set and Alice saw her running through the woods as fast as she possibly could. She said to her, “Where are you trying to get to by running so fast?” She said, “Darling, I’m not trying to get anywhere. I was just trying to stay in one place. If I wanted to get anywhere I’d have to run twice as fast.”

So I started learning programming and I got into iOS programming. A couple of months after I started, my teacher came in and said to everybody, “Alright, everything you know is now obsolete because iOS 6 came out and everything has changed.” And I realized that I would have to keep relearning things over and over again every single year and I’d had a little bit of despair because I wanted to learn graphics programming and audio programming and I was afraid that I was going to wind up spending a year just learning all of the changes that came from the previous iOS and that I’d have to throw it all out to learn the next iOS. So I felt very much like that Red – like I had to learn as fast as I could just to stay in one place and I had this despair that I’d never be able to learn any of the cool stuff that I really wanted to learn.

That’s a great explanation. I like it. [Laughter] Alright, thank you for coming.

Thank you for having me. This is great.

And we’ll go ahead and wrap up the show. We’ll catch everyone next week.