Panel

Ben Scheirman (twitter github blog NSSreencast)
Rod Schmidt (twitter github infiniteNIL)
Charles Max Wood (twitter github Teach Me To Code Rails Ramp Up)

Discussion
 01:09 - Backend Experience
 01:49 - Connecting to APIs & Synchronization

Windows Azure Mobile Services
iCloud
Wasabi Sync
TICoreDataSync
Buoy Explorer

07:10 - Third-party APIs

OAuth
Instragram
Stripe

11:57 - Parsing

NSXMLParser
NSXMLDocument
Cocoa with Love: Using libxml2 for XML parsing and XPath queries in Cocoa
libxml

18:18 - JSON

RestKit

22:38 - Libraries

AFNetworking

31:02 - Building Backends for iOS Applications

DeliRadio

35:05 - Security

SSL Pinning
Charles
API Keys
Secrets

41:28 - Support Caching

NSURLCache

45:34 - Charles

Picks

objc.io (Rod)
Mackie | Onyx Blackjack 2x2 USB Interface (Ben)
Runscope (Ben)
Daring Fireball: Markdown Syntax Documentation (Chuck)

Next Week
 Debugging
 Transcript

[This show is sponsored by The Pragmatic Studio. The Pragmatic Studio has been teaching iOS development since November of 2008. They have a 4-day hands-on course where you'll learn all the tools, APIs, and techniques to build iOS Apps with confidence and understand how all the pieces work together. They have two courses coming up: the first one is in July, from the 22nd - 25th, in Western Virginia, and you can get early registration up through June 21st; you can also sign up for their August course, and that's August 26th - 29th in Denver, Colorado, and you can get early registration through July 26th. If you want a private course for teams of 5 developers or more, you can also sign up on their website at pragmaticstudio.com.]

CHUCK: Hey everybody and welcome to Episode 13 of the iPhreaks Show! This week on our panel, we have Ben Scheirman.

BEN: Hello from Houston!

CHUCK: We also have Rod Schmidt.

ROD: Hello from Salt Lake!

CHUCK: I'm Charles Max Wood from DevChat.tv. This week, we're going to be talking about "Connecting to Backend APIs and just Backend Systems" in general. I'm curious, how much of this kind of thing have you guys done in the past?

BEN: That's pretty much the central part of any app that we develop. Most apps aren't really self-contained; it's a functionality. A lot of them required data that's accessible somewhere else. Or, even if you generate the data on the device, usually, people want to access that data elsewhere as well. So sometimes, you can consider things like iCloud, but that's more of an Apple-centric solution if you're billing it out for the web or for multiple platforms, and maybe you would consider building your own API and synchronizing with that.

CHUCK: When you're talking about building your own API, I know that there are these syncing services out there that you send data to it and it does something with it, do you know under what circumstances that would be a good idea versus building your own API that does specific things with the data on the backend?

BEN: It really depends on where, how much focus you have, how much time do you have to build something and where your skill set lies. For folks who aren't server-side developers, building an API is actually a tall order, and there are plenty of solutions out there that will do that for you at the cost of, sometimes flexibility, sometimes data portability, and you're sort of at the whim of the interface that they provide for you.

But there's systems like Windows Add/Removal services, which allow you to just focus on your part - the mobile client portion of it. But they have support for saving data and sending push notifications, that sort of thing. There's also parse and there's just a bunch of others out there that can synchronize data. Some of them are focused purely on synchronizing Core Data models. So there's iCloud Core Data, which receives some sort of a lot of negative press,

 

the first one is in July, from the 22nd - 25th, in Western Virginia, and you can get early registration up through June 21st; you can also sign up for their August course, and that's August 26th - 29th in Denver, Colorado, and you can get early registration through July 26th. If you want a private course for teams of 5 developers or more, you can also sign up on their website at pragmaticstudio.com.]

Hey everybody and welcome to Episode 13 of the iPhreaks Show! This week on our panel, we have Ben Scheirman.

Hello from Houston!

We also have Rod Schmidt.

Hello from Salt Lake!

I'm Charles Max Wood from DevChat.tv. This week, we're going to be talking about "Connecting to Backend APIs and just Backend Systems" in general. I'm curious, how much of this kind of thing have you guys done in the past?

That's pretty much the central part of any app that we develop. Most apps aren't really selfcontained; it's a functionality. A lot of them required data that's accessible somewhere else. Or, even if you generate the data on the device, usually, people want to access that data elsewhere as well. So sometimes, you can consider things like iCloud, but that's more of an Apple-centric solution if you're billing it out for the web or for multiple platforms, and maybe you would consider building your own API and synchronizing with that.

When you're talking about building your own API, I know that there are these syncing services out there that you send data to it and it does something with it, do you know under what circumstances that would be a good idea versus building your own API that does specific things with the data on the backend?

It really depends on where, how much focus you have, how much time do you have to build something and where your skill set lies. For folks who aren't server-side developers, building an API is actually a tall order, and there are plenty of solutions out there that will do that for you at the cost of, sometimes flexibility, sometimes data portability, and you're sort of at the whim of the interface that they provide for you. But there's systems like Windows Add/Removal services, which allow you to just focus on your part - the mobile client portion of it. But they have support for saving data and sending push notifications, that sort of thing. There's also parse and there's just a bunch of others out there that can synchronize data. Some of them are focused purely on synchronizing Core Data models. So there's iCloud Core Data, which receives some sort of a lot of negative press, and I don't think anybody is really seriously considering using it until Apple addresses some of those things. And then there's things like Wasabi Sync and TICoreDataSync, I think it's called. Those are also more open solutions where you'd have access to the data outside of your application if you wanted to write your own web app, for instance. But it really depends if the set of data is sort of constrained to your device, if it's like your task list or like a shopping list or maybe your own recipes or something like that, where you're creating the data and that data should be yours on any device that you'd carry versus an app that just consumes some resource of data. For DeliRadio, we have a vast amount of bands and music and show information and things like that. So really the mobile clients can't synchronize all that data; it's really based on your usage. So you search for a location, you're going to get data relevance to that location, and it doesn't really make sense to store that in the database.

I love that sound. [Laughter]

Yeah, that makes sense.

I don't know what to call those, but there's apps that sort of make sense to have a database with all the data and then you want to synchronize that. And then there's other apps that are just sort of consumers of data available in the internet. Like for instance, a friend of mine built an app, it is called "Buoy Explorer", and it's for getting weather conditions from buoys on the coastline. His specific use is for surfing. He wants to know, "Am I going to drive an hour to go to the beach?" if the conditions are not great for surfing. But fishermen and wind surfers and other types of outdoor types might be really interested in that data, so he's able to download the data that you're interested in directly on the device. There's just lots of different, I guess, aspects of the type of data you're dealing with, whether or not it's yours, it's created on the device, perhaps, or you're just consuming some data available on the internet.

Right. With the Wasabi Sync and the TICoreData Sync, are those just systems that you can just push Core Data (I don't know what to call them) models, records to, or is there more take than that?

I'm probably not the best person to talk at length about this. I know there is a NSBrief episode on Wasabi Sync, which is where I heard about it. The idea is this sort of a drop in synchronization techniques so that you would just still interact with Core Data and under the hood it would -- I don't know, for lack of a better word -- serialize all your transactions into HTTP calls, which then would make it to their server and synchronize across other devices, and probably even max as well, not just mobile devices.

Yeah, that makes sense. I've kind of dreamed up a solution that I have no idea if it would work where basically you'd just say, "This is the change that was made," and you give it enough information and then it goes into some document database like CouchDB or MongoDB, and then something else can play it back out the other side.

Yeah, you still have to be careful. Synchronization is a really complex topic.

Yeah.

What would happen is, if you made 5 or 6 changes on your phone and you put your phone in a drawer for a week, and then made sort of similar but different changes on your iPad, that's synchronizing all the time even though you made those after the ones on the phone. So, what do you do on that scenario? There's just lots of things to consider when your devices don't have connectivity all the time. So synchronization is certainly, probably, a side topic to just connecting to APIs. But, it's something that you should consider if you just want your data available on another device; it's definitely something to look into.

Yeah, that makes sense. Let's talk about a bit about third-party APIs because that's kind of a backend that you didn't build or control. I see a lot of apps that hook into things like Facebook, fewer that hook into Twitter.

Uhm-hmm.

Have you guys built stuff against the third-party APIs?

Yeah. A lot of those are APIs that either are clients have built for us or we build for the client. But there's still APIs and they are right, like the mobile client is not necessarily a trusted consumer, so you have to provide all the security that you would provide, and then the other scenario. So you do things like you could implement OAuth in sense for user authentication. Doing that usually entails opening up a WebView suit to some URL on the provider's website. I recently did this for Instagram. You go to a specific URL and the user logs in in Mobile Safari and then there's like callback URL that you'd configure when you set up your application in Instagram's developer portal, and that callback URL on iOS devices can have a custom scheme associated with it. So you just put something other than HTTP in there, something that uniquely identifies your app, and you can configure your application to respond to that scheme. So that's how it knows to open your application back up with the callback rather than just redirecting to a web page. It's part of that redirection in the URL; you'll receive an authorization token or an access token that you can then save and use that to make up integrated request against the API. Typically, you would just store that in the keychain. If you are logged in, you would have access to that or you'd have an access token. If you're not logged in, then you would go through that a lot flow.

That makes sense. Rod, have you built against third-party APIs for stuff?

Yeah, I've built against 37signals/backpack-api and the Evernote API, and also a client API that we built together.

Yup. I find that a poor API really limits what you can do on mobile. Sometimes, when our clients provide APIs, they're not really considering what the used case might be on mobile; they're just providing a general purpose API. That oftentimes either limits us or makes our job a little bit more difficult. For instance, if you wanted to know, let's say, you had like some sort of drilldown thing where you search and you get a bunch of records, and you want to know under each record how many articles are underneath, you might want to show a little badge number with the account. If the data wasn't returned to you ahead of time, either with the account sort of denormalize and stored in that response or with the child attributes or child items right there in the response, then you wouldn't know that without making multiple calls to the API to get back that data. So a lot of times, you have to really consider what is the use of each screen and what data does it want so that you can optimize the API to fulfill that requests without causing you to make unnecessary queries.

That makes sense. What's your experience with the 37signal APIs, Rod?

They generally are pretty good. Sometimes, it's pretty frustrating because you're not in-control of the API and they'll change things out from under you then you have to work around it suddenly. Or, they're not designed for synchs so you got to come up with your own mechanisms to keep track of what's changed and deal with that. Issues like that that you have to deal with; it can be quite frustrating.

So what are some of the better APIs that you guys have worked with third-party APIs?

I really like the Stripe API. I've most use that on Rails. But APIs stand on their own; you can use them in a mobile app as well. I just really like the Stripe API. I use it for collecting payments on NSScreencast, and it's tightly integrated with the NSScreencast web app. So if you had, say, an iOS app that you were selling a physical good, you could use Stripe to take a payment rather than doing an app purchases. It is to be a physical good or service, of course, but you could do that.

Hah!

I think, like a sign of a good APIs, just really, really well-done documentation. It doesn't necessarily guarantee a good API, but it certainly a sign of the better ones that I've tried that it makes it easy for you to sort of visualize, "How do I connect to it? How do I make a request? What are the responses look like?" that sort of thing.

Yeah, good array handling.

Yup!

RESTful API, JSON Returns.

So typically, the APIs you guys working with JSON APIs, have you done much with XML or SOAP or something [inaudible].

[Chuckles] One of the earlier apps I worked on was, they had an XML API that would return all the data upfront for the entire app. [Laughter]

Sometimes, that's beneficial. Like if you're willing to pay the price to say, "Okay, I'm going to download this thing once right now, and then from that point on, I can just use the app," that might be useful if the data doesn't change very often. But in this particular case, the XML format, not only was it just a lot of data (it was like 2 or 3 megs of JSON - I mean of XML), but just the TagSoup, the way that it was formatted, it was really strange in the sort of exposed some wonky normalization they had in their database, and they were not really shielding their API from that complexity. For my perspective, it was way more complicated, but it needed to be. This was years ago before I was -- I don't know if they were any really nice XML parsing libraries aside from NSXMLParser -- but trying to parse a nasty looking XML document with NSXMLParser is kind of a task I never want to really do again [chuckles]. Because it's a StreamParser, you get basically events that like, "Hey! I'm at this tag! Now I'm at this attribute! Now I'm at that attributes on you!"

Oh, wow! [Laughs]

And you kind of have to build your own model of what's going on of the context so that you can extract the data you care about. Now, the benefit of that speed and memory, you don't construct an entire model representation of this XML document in memory, which in this case would have been really bad for 3 or 4 megabytes of which would amount to just a gigantic NSDictionary. So, there are pros and cons to using that type XML parsing technique, but man! It sure wasn't fun.

Yeah.

Backpacks API or API returns XML, on the Mac, you can use NSXMLDocument; on iOS, there's a C-level API that's kind of like that. I end up using, I think it's on Cocoa with Love, there's an article about how to simplify that so that you didn't have to use NSXMLParser and you could do it more like NSXMLDocument, then you would use (what's that called) XML Paths, what do they call that to extract --

XPath?

Yeah, XPath. That's right.

Yeah.

To extract downloaded data.

Yeah. And there's other parser that's out there that you can take advantage of now. This was years ago before there was really a third-party library community in iOS. I'm trying to remember when NSXMLParser came out, I can't actually look it out. But LibXML has been around forever, and that's really what this is on. So let's say NSXMLParser, that came out in 10.3 so it's pretty old. And then on iOS, it came out when -- it was available from the gitco. So that must be just a thin wrapper over to the XML.

Yeah, that's what it was. Here, I'll post the link.

Nice. What about JSON? Well, let me back up a minute.

We didn't talk about SOAP...[Laughs]

[Chuckles] Yeah. And you kind of imply that there were better ways of doing XML than necessarily using, maybe, NSXMLParser. Are there the libraries that you use or those the ones you go with?

NSXMLParser is fine and it actually offers really fast performance, but it's just more complicated than a document based parser. Now that I think about it, I don't think I have had to parse an XML document in really long time, which is kind of nice and refreshing. But --

Yeah, Returns definitely turns JSON if it's not already there.

Is there a class, Rod, for parsing XML? Can you use NSXMLParser outside of like the event-driven style?

I don't think so. You can use NSXMLDocument, but parser is just stream wire and they're event wide.

Okay. So NSXMLDocument, is that on the iOS as well?

I don't think it is yet; I think there's just the Mac thing. But this article, it is a wrap around the live XML and basically, it downloads all document and then you use this to function calls to extract out "What do you want with the XML Path?" and that worked pretty well.

Yeah, we turned an app that used to sort of a -- it was like a flow API; it was for soccer stats. They had just an enormous amount of stats and we couldn't just download all the information because it change regularly rather they kept adding to it, and it was just too much data. But you could drill down into details screens. The way they did that, for whatever reason, was static XML file sitting on an Apache server somewhere. So we just heavily used caching and we go download the file at a given path, and that path would be this team, this player, this season, and then we download that. Once the season closed, that data is static so you can cache it forever. So what we would end up doing is just using the HTTP caching semantics like If-Modified-Since and ETag to keep a local cache of all the content. If you use NSURLCache for that, then you can specify how big you want your discache to be and it will expire things. Once it get reaches that Clang, it will expire things that are older. That worked out really well for the common things like a season stats or game's particular stats; we could just go fetch the file once, parse it, and keep that data cached. And then we would just make another HTTP request for it every time we needed it. But 9 times out of 10, that data would still be in the cache. So the only real problem that we'd have to overcome is parsing it repeatedly, which turned out to not be that big of a deal in this particular app because the files were relatively small.

So should we talk about JSON for a minute?

Yup! I think pretty much these days, you're going to find that every API, pretty much every API, has a JSON variant just because it's a lot easier to mentally consume, it's a lot less data over the wire, plus you have NSJSONSerialization, which gives you really fast serialization of Objects to Dictionaries and Arrays of attributes. So it's just natively supported and really easy to work with.

What about libraries that map the JSON to your Cocoa Objects? I've always wanted to use one of those, but I haven't ever got nor around to it.

I have real one; it's not great at all [laughs]. So I didn't ever open source it because it was really specific to what we were doing. The way it worked is, you would have a based class, which I called (what did I call it) smart JSON object, or something like that. What it would do is it would leap over the keys that it sees in an object. If you had, let's say, a list of cars and the car had make, model, year, and color, maybe. So we would look on the class for an attribute that represented each one of those things. But we would have to take care to sort of match the naming guidelines of JSON versus Objective C. Since ID is a reserved word, I don't want to use that as a property name, so I would end up looking for something called CarID and I would do it like car with a capital "I", Pascal cased. And then if we had something like (I'm trying to think of) like engine type -- let's say that was a property on the JSON dictionary, it would probably be all lower case engine_type. And then the equivalent Objective C property name would be engineT type (with no underscore). So I would just make those assumptions and try to search for a field name or a property name with that same name and use that to set it, and it would do type conversion and all that stuff. But you'd run into some problems about like "What do you do with dates?" they come down as strings. You have to probably create NSDateFormatter and assume a specific date format so that you can probably parse that. So there's some challenges to overcome. Another thing that I didn't solve, or I guess, let me back up a little bit. That automatic detection or mapping at JSON attributes to properties on my Objective C model, obviously, it didn't account for 100% matches so I had to provide a dictionary of keys to value overrides to use. So if something an API was named something really wonky but didn't make any sense in my object, I can change the name and provide a dictionary mapping between the two. And then you have the added challenge of, "What if you wanted to have like sticking with the car model? What if the JSON API returned everything as one big flat object, but you wanted to create a richer object model with like an engine class?" then you might need to provide some sort of mapping for multiple classes to map to the same document. And then vice versa if it was split apart and had multiple children but you wanted to flatten it out and simplify them the Objective C side; you might have to solve that as well. Every solution, I seem to do this. It's really handy, but also really complicated in cases like that, become difficult to understand. The best example I can think of is RestKit. RestKit has a pretty good mapping library.

Sometimes, I've used the protrade, just adding init with dictionary I want model object and then let it consume that dictionary and translate it. It mostly like a protocol like NSCoder, but JSON coder or something that you can implement on your object.

But then on your init with the dictionary, you're just setting keys and values yourself? How does that work?

Right. Yeah, you just look at the dictionary and --

I usually just start out that way until I have something a little bit more complicated. We ended up having just an enormous API that we needed to consume for DeliRadio. Every single screen has a different set of data so we ended up coming up with some conventions, which would alleviate us from writing all that parsing code ourselves.

Uhm-hmm.

Nice. I'm a little curious because a lot of the APIs that people consume are for some of these larger companies; you mentioned Instagram, Facebook, things like that. Are there specific libraries that are built around those APIs that are out there?

Usually you'll find, for the popular ones, I think you'll find a Client Wrapper around it.

Well, on iOS now, there's the Social Framework for talking with Twitter and Facebook kind of a higher level up.

So I guess my question then becomes, are most of these frameworks or client libraries really good? Or, are some of them not so good?

I have a friend who is really opinionated about this, and he really thinks that if your API needs a client wrapper, then it's not a good API. [Chuck laughs]

However, there's still some HTTP plumbing that you might not want to have to deal with. And having an Interface in Objective C where you get code completion and things like that make it a little bit easier to get started and understand what's going on.

Yeah.

I still think that having a thin wrapper is good to get started, but also like should not hinder you from consuming new things in the API. For instance, if a new feature comes out and there's just not a

method for that in your wrapper, then does that mean that everybody uses your wrapper can't use it? Or, did they still use your wrapper to execute request against the API? I think, the Social Frameworks on iOS do a good job of not locking you down to specific features on a given API, but they just rather keep you away to authenticate the user and sign requests outgoing. So basically, when you're going to, say, fetch a user's Twitter followers, you can use the Social Framework to get access to their Twitter account, and it would prompt to them for permission. Once you have permission, you can issue a request to any endpoint on twitter.com, and it will use the OAuth AccessToken and authenticate that request for you. So you don't really have to deal with the authentication in OAuth plumbing, but you do have to construct the URL yourself and all the parameters yourself. I think that's a nice example of obstructing away the complicated things that are just going to be mundane for people doing this over and over again. But still, if Twitter comes out with the new endpoint that exposes some new functionality, you can take advantage of it from day 1.

Yeah, that makes sense. To be honest, I've used some libraries that obstruct away certain aspects of some of these APIs in Ruby. In some ways, they're nice. In other ways, they tend to get in your way. But yeah, then I go down a level to the REST Client Library that I'm using and just make the call myself. In those cases, it stored the Auth Token that I've got so I can just do what I need to do. And I've consumed a lot of APIs across different things from Ruby and Rails. I'm a little curious about what some of the nuances and differences are between a web stack and an app stack like iOS.

I think, the multi-threaded nature or the need to be multi-threaded on User Interface, like a smart User Interface type device like iOS, is much more apparent than it is in, say, a web request in Ruby. Whereas, if you need to talk to an API within a web request, the users are already waiting in their browser so you can block a thread and make that call, and it will just appear slow to the user, but it won't feel broken. In iOS app, if you make a request on a main thread, it can potentially cause your app to crash because your User Interface thread is not responding to events. But the user is certainly going to notice it and it's going to be a bad experience. So from the gitco, you just need to make everything asynchronous. There's lots of handy class methods on NSData, NSString, that allow you to issue a network request and get the contents of some network resource and put it straight into an NSData or in NSString. So if you wanted to get some content, you could do that. I see that a lot with beginners; they just see this one-liner method. It's like, "Oh, I can download google.com's HTML into a string just by saying NSString stringWithContentsOfURL," not knowing that that's going to make a blocking call and talk to the network. And on a slow network connection, that could be a really long time. Even if it's 100 milliseconds, somebody's going to notice because your content is not going to be scalable; things aren't going to animate, things like that. So even though there are those convenience methods, I tend to not ever use them because everything else are done asynchronous. So I lean heavily on AFNetworking just because I like the interface it provides; provides an easy way to just assume that request you're going to make are JSON requests. So you can say, "Create a JSON request operation with the given URL optionally parameters," and then the callback that you're given already has a parsed JSON document for you because you told it you were going to make a JSON request. It's got the same thing for XML. So when I mentioned not having to do any XML parsing, that's the reason why; it's that I've been using AFNetworking for a while that I just say, "Oh, I'm just going to go get XML from this endpoint," and it just gives me back a document I can work with. They have other things like PLIST operation if you want to do fetch a PLIST, or if you wanted to fetch an image. I like these higher level of requests or concepts. It gives you the same asynchronous behavior, it gives you success and then array callback. But in the success callback, the thing you get back is the thing you requested. You don't have to do any additional parsing.

Yeah, I should use AFNetworking more. Usually, I end up using NSURLConnection or just doing a DispatchAsync and calling some other blocking method that does something.

Yeah. The problem with that is you can never cancel that.

Right.

And what would you do about errors? It's like such an attractive one-liner method, but then there's so many edge cases that you have to consider.

Yeah.

And cancelling is actually really important. We do this a lot, like if we're tapping in through screens and each screen needs some sort of information, so we make a little request to go get that data. We're okay with doing that because for the common ones, that data gets cached so those appear instantaneous. But then, the user might hit, like say, the browsing for a band or whatever, they hit a band and they actually hit the wrong one, they hit the back button. You want to be able to completely cancel that request and not waste anymore system time on it so that we can focus on what the user actually wants, which is the next request they're making. So anytime the user hits the back button, we cancel any request that were happening as a result on that ViewController.

And this URLConnection, you can't cancel it, but it's nice to have AFNetworking take care of all that gamification for you.

NSURLConnection didn't really get useful until (when was it) like 4.0 or something? I think one of the reasons why there was a proliferation of networking libraries is because it was so primitive. Like if you wanted to download data and report progress on the download, you literally had to shovel bytes around into an NSMutableData yourself to calculate how much is come down and how much is left. I like having that low-level capability, but 9 times out of 10, I just want to make a request and get the callback I need saying it's done or it failed. So I definitely like the higher level concepts. Now that NSURLConnection has block based behavior that you can say, "Okay, execute this request and give me the response," it still has a bit of unfortunate side effects that NSURLConnection doesn't necessarily assume that it's an HTTP request. So when you get back in NSURLResponse in order to get the response code or the response body, you have to cast to the response to an NSHTTPURLResponse.

Right.

And...I don't know. I've never used it outside of HTTP so, that to me is complexity that I'd rather not deal with every time I make a request.

Right.

One more question, have you guys built backends for iOS applications?

I've collaborated on and helped, worked with the guy doing the backend, that's about it.

I'm doing primarily Rails actually right now as the backend for DeliRadio. Once we got the iOS app where it needed to be, then the web app needed work and as new features come out, we implement that API. So I would say, more than half of the iOS apps we work on, we also have a handling building the API for, and part of that is just a risk thing. One of our engaging with a client, sometimes, they'll give us an API. Or, they'll say, when we're trying to scope out a project, they'll be like, "Oh, there will be an API for this." [Laughter]

I've learned the hard way that sometimes when they say there will be an API for that, they really mean they're going to drag a -- I forget how this works in visual studio -- but, they were like file only web service and they name a WebService1.asmx and they drag like a data connection from old design surface onto their -[Laughter]

And you're literally getting data straight out of their database that -[Crosstalk]

Have you worked with some of the clients I've worked with?

When the service is called WebService1.asmx, you know you are in for a hard time. [Chuck laughs]

And then of course, the iOS doesn't know what a data set is; the .NET people in the audience will laugh at that. Anyway, there's sort of the whole gamut. I've worked with APIs that were heavily SOAP based. I'm not sure if WCF still access where I think it does, but WCF being like the .NET web service layer, there was a time when I had to alphabetize the input parameters in the XML document for the input. Otherwise, it would reject it as like not matching the contract. Stuff like that is just kind of really frustrating. Some sort of REST like API, just sort of a little bit more flexibility pass in params either in the post body or in the query string and it gives you back data; just so much easier to work with.

I have to say I haven't directly built APIs aimed at iOS, but I've built my fair share of JSON APIs that have been used by iOS. Sometimes, it takes a little bit of tweaking so that it lines up better with what the iOS developer is doing. But for the most part, if you've got a well-designed API, they can consume it without too many problems. So as long as it's well-named and secure and all that stuff that we've kind of alluded to, it works out well. And it is an interesting problem to solve, but it's fun.

A lot of these API sort of changes we've made have been heavily influenced just by usage of the iOS app, which mean like general purpose APIs can work for a long time. But if you really need like fast performance, a lot of times, you need something really tailored to what you're doing.

Yeah.

So you have specific data requirements for screens that mean that API has to behave a certain way, and you might not design it from that way from the gitco. If you need data for 5 or 10 items in a list that you don't currently have, you need an easy way of getting that data either with one call that includes that all for you, or with minimal calls so you don't have to do one per row. It was kind of a similar problem with Select N+1 issues and database-backed applications, like you want to make sure and get all the data in NSViews Queries as possible; it's the same thing with the services.

Yeah, it makes sense. I usually don't try and make pitches on the show, but if you do need a

backend built, I am qualified to do that for you. [Laughter]

So give me a call.

You want to talk about security?

Yeah!

I have never written an app for a bank or anything like that, but there's still the notion that like we should secure the API against unauthorized tampering and/or just spying. People want to protect their intellectual property and they don't want unauthorized API clients and that sort of thing. So one technique I've seen used is to sign a request -- first of all, you should probably be using SSL.

I was going to say [laughs] use SSL --

If you use SSL -- I recently did a screencast on SSL Pinning, in which I take an app that was built that uses SSL, and I open up Charles and set up myself as a proxy for that domain as self-signed certificate. The self-signed certificate, you have to trust the, I think there's a setting in most networking libraries whether or not you want to allow request on trusted certificate authorities. But if you trusted, it might be the user's device; they may be the one trying to compromise your API so you can't trust the user's device either. Anyway, they may trust intermediate certificate and therefore you could spy on the content going to in front of the API, but not only can you spy on it, with Charles, you can actually set a breakpoint on the response or the request and you can modify it. So if you lose something like, "Yes, you're logged in and here are your credentials. Are you an admin? True or False?" If that was just a response coming back from the API, then you can set a breakpoint in Charles and change that "Is admin" flag from false to true, and all of a sudden you have more capability in the app than you should have. So that's obviously pretty scary as a selfpinning is a technique where you embed your certificate and it will extract out the public key in that certificate and compare it against the public key for the service you're talking to. If they don't match, then it will reject the request. That's a handy technique and it's pretty low complexity as well; dropping in your certificate and enabling your property, and AFNetworking will allow you to do that. And since you're using public key and now the actual certificate, it will work even though you're certificate gets renewed, that sort of thing.

Uhm-hmm.

So once you've done that, you can have pretty good assurance that you're talking to your server and the response hasn't been tampered with. You might also want to prevent other people from making request as you. One of those things is with the -- you might have like an API key so that some random person on the internet can't just stumble across your endpoint and start making requests. But if you're able to sniff out request coming out of the device, then you could easily get a hold of an API key. One thing you might consider is request signing. With that is, you can be implemented any number of ways, but one way I've had seen it implemented is when you take the URL, the method HTTP like git put hatch or whatever, then all of the parameters to that method alphabetized. The reason you alphabetize them is because the server has to use same operation; it has to be deterministic. So you alphabetize all the params and then you take that and you HMAC-SHA1 encode that with a private secret that you don't share with anybody else.

Right. That private secret is just to sequence your characters that is unique. And the reason that that's important is because then, if I make the same request as you but we have different accounts,

that's secret will make that signing different and so it can verify the identity of --

Right. So you might see all of the requests or all the parameters for requests. But if you change any one of those, that will generate a different signature, and signature is also a parameter of the requests. So the server receives the request, does the exact same operation you just did, and compares the signature you generated with the one that it generated, and if they don't match, it rejects it. That would allow you to intercept to that request and make it again, like somebody else could take the same request and make it. But if they tweak any of the parameters, it would fail. And you could also add an additional time component to this to say, "Okay, this is the date/time of the requests," then the server could change the setting of "Yes, all that was valid. But the time was yesterday so I'm going to reject it."

Yeah, I've seen that pretty frequently where if the date/time doesn't within a certain open band that it'll accept, then it won't accept it.

You still have to worry about the API key is somewhere on the iPhone app. So if you're storing it on disk, then you still have the problem of somebody gets sort of crack open the iPhone and take a look and see what's inside and see if they can find that in the string somewhere. But all these things or steps you can take towards making it a lot more difficult for somebody to reverse engineer your API or make unauthenticated requests or whatever. Like I said, I haven't done this for a bank or anything like that, but I think it's good enough security that's easy to implement to go like 80% of the way there.

Yeah. I've done some PCI Compliance stuff and I've done some HIPAA Compliance stuff. I have to say, they're not a lot of fun, but they're interesting problems to solve sometimes.

Yeah.

The real rob is how far do you have to go in order to make it secure. For the most part, you've covered the major points with SSL, which is if you're familiar with public/private key authentication, that's effectively what SSL is, and just encrypting your keys. You can also encrypt your messages; I've seen people do that where the payload is the postdata, it's just encrypted datas, and then encrypt it and then send it, and then the response is encrypted and sent back. So really, it just depends on where you're at. I think for the most part, a lot of these approaches are overkill. If you have SSL, you have an API key, if you're really concerned, you'd do the signing, I think you're pretty safe.

Yeah. I think that's probably good enough for security. And then I think, the end of thing to stress is just support caching from the gitco. On the iOS side, if you're using NSURLConnection or AFNetworking, you can just enable NSURLCache and everything should just be cached from that point onward. I say it should just be cached; it really gives all of the power of the server to control what the caching parameters are. So NSURLCache allows you to set up a memory capacity and a disk capacity. So you might say, "Okay, I'm cool with 10 or 20 megs of memory," which is a lot actually. Let's say, "I'm okay with 5 megs memory and 20 or 50 megs on disk," and when you make a request, you can set cache policies of whether or not to honor request from the server or cache parameters from the server. It really allow in the server to say, "Okay, this expires on this time." Or rather, the server can just say, "Here's the date of when it was modified," and then the NSURLCache will save the date and that you tag if it's provided one, and then make conditional request for that resource, and when the server has a chance to respond with 3 or 4 not modified if that resource is still fresh.

In my experience, a lot of the Server APIs, they're not doing that though; they don't give me that information. Such shame on the experience [laughs].

Shame on them. I think that's like step number 1: make sure you do this because doing it after you shipped is, well, it's impossible. And if you start saying, "Oh, well, I'm going to cache this thing for 30 seconds," that might put a gigantic strain off your server because if you have 100,000 clients out there and they're all trying to refresh the same page and the data doesn't really changed that often, then maybe a little bit of caching makes sense. But it's kind of a slippery slope and you have to validate these assumptions. Like caching for a little bit makes sense, but what about data that you'd expect to be there soon as you make a change like you get a push notification when you got a framer request. You swipe to unlock your screen and then you look at the framer request's screen and it's cached. If it's cached for 30 seconds, it's going to feel like it's broken like you just totally out of the frame request and I don't see it in the list. So things like that, they're immediate like you post a tweet, you immediately want to see it in your Twitter client. Things like that, caching, it's just a tricky problem to get right. That's why I like letting the server be in control so you can say, "Okay, cache this list for 30 seconds." If that turns out to be a problem, then the server can change the parameter and say, "Okay, that was a bad idea. Let me change it to 10 seconds." But if you shift a client that hard code is caching parameters, then you're in for a much more difficult debugging experience.

[Laughs] I'm laughing because I've made that mistake in the past.

Let me tell you, that's why I'm so bullish on making sure that -- from iOS perspective, you really don't have to do a whole lot. Use something like Charles to validate that you are sending those conditional headers. For the headers, you'd be looking forward the, If-Modified-Since or If-NoneMatch with an ETag. The two can be used interchangeably, they can be used together, but the ETag is generally content-based cache so it's like a hash of the content. So if the content ends up changing, then it'll generate a new ETag. But oftentimes, the server still has to do a database request; they'll actually pull out a record from the database and compare the ETag of the new record with the old one. Sometimes, you could perhaps get away with just using modified date by, if perhaps you had that data, a little bit easier to fetch. But maybe when you need to go fetch the entire record, you could just check a modified date somewhere. So it just depends on the use. But again, servers can be deployed and changed and fixed and all while people have these mobile devices in their hands and they may not update it for 2 months.

I have a question for you about Charles, because I've used Charles in the past, but usually, I'm intercepting stuff from my browser to my web apps and sets primarily what I wind up working on for my clients. When you're talking about doing it from an iOS device, so you're doing it from the emulator? Or, can you actually use Charles to intercept requests to your machine from your iPhone?

Usually, I'm doing it from the simulator because I'm just trying to, in active development, I'm trying to take a look at what the actual data coming out of the devices. But you can easily set up Charles as a proxy and then go on to the network settings of your phone, and say you used a proxy server, type in the IP address and the port number and it will use Charles. Honestly, you can really take a look at other apps on your phone and see what they're doing. A lot of them use SSL, but not all of them reject untrusted intermediate certificate authorities. So it just depends on the networking library they use and what the signs are. And then if you do decide to install the trusted Charles certificate on your phone, then at that point, the app doesn't know that you're talking to Charles; it thinks that you're talking to a trusted server if they're not using SSL Pinning. This is one of the ways that Path got in trouble. People found out that they were sending your entire address book up to their servers, and they did it for a legitimate -- I can't really say a legitimate -- they did it for a reason, and that reason was to support a feature that they wanted to have in there. Whereas, if I

was on Path and I had your contact in my address book, when you join Path, they want to let both of us know that, "Hey, your friend is already on Path." So it say a seemingly useful feature, but the ramifications of that might that your entire address book, it's send up to their server with contacting information for all kinds of people. You'd imagine how dangerous that could be if these people are celebrities or political figures or somebody's got to have Tim Cook's phone number in their address book, things like that. The only way that somebody figured that out is by performing a man in the middle attack to sniff the data.

[Laughs] Interesting.

So it turns out that that same feature could be implemented without sort of violating anybody's privacy, and that's by hashing the content instead of just sending it in unencrypted. So instead of knowing your actual phone number, I just know the hash of your phone number. But if I also receive the hash of your phone number when you sign up for that app, then you can compare the hashes, and it's just the same as if you were comparing values, but you don't actually see the phone number.

Yup, it makes sense. Alright, well, we're just about out of time. Anything else we should cover before we wrap this up.

I don't know. It's just, API design is kind of a tough beast. And once you've consumed a few of them, you'll realize, "Hey, that wasn't really a great idea. This should be better, if I'm pulling up this way."

Yup.

I don't know. It's just kind of a learning process.

Yup. Alright, well, let's go and wrap up the show. Rod, what are your picks?

Alright, I'm going to pick, for my first pick, will be a new Objective C newsletter called "objc", it's www.objc.io. The first issue they had 'Lighter View Controllers' and then the second week were just came out, it's all about 'Concurrent Programming', so there's other good stuff in there.

Awesome!

Yeah, I guess I only had one choice. That's it. [Laughter]

Alright, Ben, what are your picks?

I have two. Earlier, on the precall, I mentioned that this little device I got, it's like new USB Recording Interface, it's called the "Mackie Onyx Blackjack". So far, I'm liking it. It gives me a little more "uhmf" for in terms of gain control, but it also has 2 channels so in case I ever wanted to have, say, a guest recording or whatever, I could do that. Anyway, it looks a lot cooler than my last one so that's obviously super important [chuckles]. But basically, it just lets me plugin my XML right and can get a USB port on the other end. So that's a recent purchase of mine. And then talking about APIs, a lot of times it's useful to sort of monitor APIs and see if what errors are coming back. Sometimes, that can be done in logs or whatever, but I ran across this really cool new startup that a friend of mine is part of called "Runscope". What it'll do is it'll serve as sort of a middle man for an API, so it's complete pass through. You basically say, "Okay, instead of the host for the actual API, I'm going to use this modified host that sends the data to their server and then they send it to the actual API," and then they give you their response. By doing that, you can actually view all of the content and you can copy request and replay them later. There's just all kinds of interesting things you can do; you can graph errors. So this looks like a really, really useful tool for, maybe in development, but I think you can also use a simple action if you wanted to see like what average response times are, what different request of the plug, that sort of thing. So that's it, runscope.com.

There's also an app called "HTTPClient" that lets you interact with any Web API, do it with a Mac client, plugin your request, and play with it.

Yeah. What I like about this is that I don't have to actually do anything. Once I've changed to the host name, it will just pass all the headers and the body and everything over, and then you can replay them later. Actually, that brings me to it. Good tip that I used when building DeliRadio, you can usually inspect parameters or you have like a callback where you can send a requests or log the details request, like I'm about to make a git request to this URL with these parameters. So oftentimes, I wanted to see what their responses looked like, but I didn't want to clutter up the logs by putting them in and just NSLogging all my responses. So in development, my motto actually log a curl statement in NSLog, which allows me to just copy and paste that right in the terminal and see what the data is. And that was really, really invaluable in determining, "Oh, the API is returning this," so we know it's an API issue or we know it's an app issue.

Nice.

So, those are my picks!

Alright. Well, I've got a couple of picks. The first pick that I want to pick is "Markdown". I guess it's not really a markup language; I guess it's kind of a markup, but it's a really simple one.

One would say it's a marked down language [laughs].

Yeah! [Ben laughs]

And you can compile it to HTML and things like that. So it's really handy; it's a nice way to go. I'm going to post the link to the "Daring Fireball Markdown" explanation because I think it's awesome.

That's really all I've got. I used Markdown to write my READMEs on my open source stuff.

Which app? Just the text editor?

Yeah, usually.

There's lots of Markdown apps.

I use Vim for the most of that types of stuff because I'm usually in Vim anyway. But I use an app called "Marked" and I have a Vim Key Binding, which will open up the current file in Marked, and it has different styles so you can simulate the GitHub style. So usually, I'm writing a README in markdown, like you said, but then it'll show me what it looks like as I'm writing it, which is pretty awesome.

I only use Marked when I need to like export it and make it look really nice.

Oh, yeah. Because you can do like a PDF Exporter or HTML Exporter whatever, but I like seeing it as I'm writing it.

Yeah. Alright, well, we'll go ahead and wrap up the show; we'll catch you all next week! Thanks for listening!