Check out Andrew’s Wired In Kickstarter! Go here to check it out and contribute!

02:53 - Warren Moore Introduction

03:58 - Working for Apple => Doing Metal (Transition)

05:15 - 3D Graphics APIs

11:14 - Metal = Objective-C API    

13:11 - The Programmable Pipeline (Shading)

19:40 - Metal

22:07 - Draw Call Batching

25:30 - Testing Metal

26:01 - Who Is Metal For? What Else Can Metal Do? (Besides 3D Graphics)

33:35 - Getting Started with Metal

38:15 - Multithreading

41:34 - Metal Adoption and Future

45:11 - 3D Graphics Programming Resources

 
ANDREW:

Hang on. We're going to wait for my six-year-old to quit screaming.

Hey everybody and welcome to episode 106 of the iPhreaks show. This week on our panel we have Andrew Madsen.

Hi from Salt Lake City. I've been working on a project for the better part of the year with a couple of friends. It's called Wired In. We launched a Kickstarter – well, last week as we record but by the time you hear this, we'll be a couple weeks old. We've been very pleased with the response, but we still need a push to get to our funding goal, so I want to talk about it on the show. Wired In is a sign that lights up. You can change the colors, and you can control it over Bluetooth.

The idea is that when you're sitting at your desk and you're in the zone; you're working; you don't want to be distracted, you can turn the sign on as a signal to tell people “Hey, don't bug me.”

But we've got a lot of other cool ideas in mind, and we've heard from people they are using for all kinds of cool things. Relevant to listeners of the show is that we're going to release an SDK so that you can write Mac and iOS apps that will control the sign. We're actually also going to release the same thing for Windows and Android, but I don't want to talk about that too much.

Go check it out. It's called Wired In. It's on Kickstarter. You can find our website with a link to the Kickstarter at wearewired.in. That's my announcement.

Awesome. Jaim.

Hello. I don't have a Kickstarter to announce [chuckles] but hello from Minneapolis.

Alright. We also have Alondo Brewington.

Greetings from Goldsborough.

I really want one of these signs. I'm Charles Max Wood from Devchat.tv. We also have a special guest this week – Warren Moore.

Well, hello from beautiful San Francisco.

As usual, can we get a short introduction from you?

Sure, absolutely. I started life as a developer – as a web developer – several years ago and got bored of that pretty quickly. Then, I moved into iOS freelancing. Actually Alondo and I are former co-workers. And then, about two years ago, I got hired by Apple on the camera and photos apps team where I was a UI engineer for a little over a year. Then, the most recent development is that I left Apple about a year ago and started writing full-time for metalbyexample.com which is my blog all about real-time 3D graphics on iOS – that's the skinny.

Awesome. I keep trying to think of jokes like if you wrote Garage Band and you used Metal, then that would be a Metal Band [chuckles]. I don't know. I was trying to come up with one for Death Metal but, anyway.

That actually raises a question in my mind, Warren, which is – I'm kind of assuming, maybe I'm wrong – but did you use Metal in the photo stuff you did at Apple? Why did you decide to quit and write about Metal?

That's a great question. I get asked that question a lot. The reason that I left Apple was mostly because of burn-out. I've just wasn't able to do the work there anymore and the fact that I picked up Metal was really more happenstance than anything that I've been doing prior to that. I've been fascinated with 3D graphics since I was in middle school – so for more than half of my life.

I quit probably a couple of weeks after WWDC last year where they announced Metal and where Metal debuted. I started writing probably about eight weeks later when I went to CocoaConf in – I guess it's in Denver. Alondo was there. He was there at the founding of the blog, but the two are really unrelated. I didn't do much 3D graphics programming in my official capacity at Apple.

Okay. Yeah. I wondered about that. It seemed like an odd transition, but it actually makes more sense now.

Yeah. It's something that I've always been passionate about, but I really haven't had the space to explore professionally. So I'm trying to transition into that now.

For people who aren't familiar with Metal, can you give them a bit of an overview of what it does?

Yeah. What I actually wanted to is give a 30,000-foot view of what a graphics API even is; and maybe step back even further and talk about what a graphics API is not. Because it's really difficult to explain what Metal is and why it's so great without putting it in this broader context if that's okay with you.

Right now, you've got all this high-level 3D gaming engines like Unity and Unreal – are tremendously popular. They basically let you plug in contents and write some scripts and, out the other end, pops game. It's obviously not that simple, but these are more or less content-offering tools at this point. They don't necessarily require as much low-level programming as game development used to. You have all of these different components that come together in a game engine. You have a physics engine; you have resource management; you have input; you have audio; you have AI.

3D rendering is only one component of a game engine. There are a lot of different possible lowlevel APIs that you can talk to with your rendering core. When we talk about 3D graphics APIs, we're just talking about that one component that talks to the GPU rather than all of these attendant subsystems like input resource management, UI and so forth. Just to put it – that's the high-level context that I wanted to introduce. Does that all make sense so far?

Yes.

Okay. About 25 years ago, there was an API introduced called OpenGL. Perhaps you've heard of it. It was very much tailored to the hardware of the day. The essential task of a 3D graphics API is to abstract over this type of processor called a graphics processing unit which we call a GPU and feed data to it that it then renders to the screen – basically draws a three-dimensional figure into a two-dimensional image for display on the screen. That's probably one of the most abstract ways to describe what the job of a 3D API is.

OpenGL was introduced in order to provide a high-level API to the graphics hardware of the day. That, basically, would allow you to say, “Okay. Here are the points that comprise the triangles, the make of my scene.” And you would set some configuration about how those triangles should be lit and shaded and shadowed and textured. Then, OpenGL would crunch on those numbers and write out the appropriate pixels into the render buffer.

Initially, you would use what was called Immediate mode. For every vertex in a triangle, you would make a function call to a function called glVertex. There's a bunch of different flavors of that. There's one called glVertex3f which means you're providing three spatial coordinates x,y and z to describe where in 3D space this triangle is. Then, numerous other function calls later, you call glEnd and glFinish and it splats all those triangles out to the screen.

Something that's very important to note here is that OpenGL is accumulating these vertices as you submit them into buffers that it manages. So it's doing all this mirror management behind the scenes for you. That has a sort of amount of overhead associated with it.

As time went on, OpenGL accumulated additional functions for submitting batches of vertices at once and drawing from arrays that have already been uploaded into graphics memory. Over time, it has sprawled out and become – it's an API with a very very large surface area. There's a modern subset of it that you can use but a lot of the complaints that are currently targeted at OpenGL have to do with the fact that it's a very old API. So it got all this historical croft hanging onto it. That's OpenGL in a nutshell.

Make sense to me.

Of course, there's also a variant of OpenGL that lives on iOS called OpenGLES which is also supported on Android. It doesn't have quite as much historical baggage as OpenGL on the desktop but, nevertheless, it does have this legacy of hiding things like state changes and memory copies from you which reduces performance in very subtle ways that are very hard to get around because so much of this is implicit.

If there are two great sins that OpenGL commits, it's probably implicit memory management. You don't know when memory is going to get copied or how it's going to get copied. You have little influence over when memory gets shuttled around.

whenever you set up state – state might be whenever you switch textures; which texture you're going to be using on a model; whether you're doing blending, that is, whether you need transparency and setting up all these different parameters that affect the scene that you're rendering – OpenGL does a lot of work behind the scenes to validate that set of state. It does this at runtime everytime you make a draw call. That has a certain amount of overhead because it's certainly possible when you're setting, basically, global state from anywhere inside your rendering core to wind up in an inconsistent state. OpenGL has to do a lot of work to make sure that everything is consistent before it actually attempts to draw anything less to crash the driver.

Those are the two woes that Metal actually seeks to address explicitly.

I know you're leading into talking about how Metal addresses those. Before we get to that, I know OpenGL is – I think its C, right? It's a C API.

That's right.

But I think a lot of people think of it as almost a tone language – tone variant of C – because you're just using OpenGL API so heavily. What about Metal? How is Metal – C? Is it Objective-C? C++? How does the API just look at a language level? I think OpenGL is really intimidating to a lot of people [chuckles] who've never done it before.

Yeah. It's extremely intimidating and doubly so just because the API surface area is so huge. There are hundreds of C API calls that you can make into OpenGL. Metal, by contrast, is an Objective-C API. It has very, very small surface area. There are only maybe a dozen or so core classes that you're talking to. The number of concepts, the number of moving parts is somewhat smaller with Metal despite the fact that it is lower-level API and then it provides you more direct access to the hardware.

I like the sound of that. Objective-C sounds good to me.

Yeah. It seems surprising that you wouldn't just write a foundation or core-foundation style C API if you want to get the most out of your GPU. But in fact, the call overhead in Objective-C is dwarfed by things like the bandwidth to the GPU and like the performance cost of actually doing all the drawing. You're actually conducting relatively few Objective-C method calls in order to do a lot of work on the GPU.

There are a couple of other things that I want to talk about namely the programmable pipeline and also what a GPU actually does foundationally. All of this is going to lead into an actual discussion of Metal, I promise. It's just that there is quite a lot of preface that's needed to really talk about it in depth.

Back when OpenGL was introduced, all of the graphics hardware that was available at that time was what we call fixed function – meaning that it's configurable but you don't write the software that runs directly on the GPU. You can, say, for example – in GL you say, – you can configure a light by calling a function called glLight and passing various parameters to it. You can configure the material of the current thing being drawn with glMaterial. All of the actual code that runs on the GPU is implicit in the driver and it takes these parameters that you provide and renders using them basically as parameters in this fixed function pipeline.

Now, I guess in the past decade or so, we've introduced what's called the programmable pipeline where certain phases, certain stages of the graphics pipeline are entirely programmable by the graphics programmer. What does that mean? You are responsible, in fact, for writing what's called a vertex shader which takes all of these 3D points that you specify in 3D space and multiplies them through by these matrices. The purpose of which is to transform them so that they're in the 2D screen space that the eye actually sees. That's now the obligation of the graphics programmer whereas, previously, a lot of that was handled by OpenGL and higher-level graphics frameworks.

Then you have what are called pixel shaders or fragment shaders whose responsibility is to determine the color of every pixel that winds up in the render buffer by doing things like reading from textures and manipulating colors, doing lighting and things like that on a per-pixel basis.

So the role of graphics programmer has changed somewhat from being state machine configurors to being GPU programmers. It's a very important shift that's happened over the past decade. So OpenGL has a shading language called GLSL which is derived from C. Metal has a shading language which doesn't have a different name – it's just the Metal shading language that's derived from C++ 11. The big part of graphics programming these days is writing the shaders that get plugged in at various stages along the graphics pipeline and execute more or less directly on the

GPU.

When we're talking about a shader, what does that actually do?

Yeah, great question. Just to walk through the pipeline. You describe your 3D world in terms of triangles. The reason we use triangles is because they're simple, right? Every three points unambiguously define a triangle whereas if you tried to draw, say, a quadrilateral, you've submitted four points that were non-coplanar, then, that has all kinds of weird pathologies. So graphics hardware from the very beginning has been angled toward drawing triangles and it's very very good at drawing triangles.

You go through this vertex shader stage – like I mentioned earlier – that takes these triangles that'll have xyz coordinates and are sitting off in some coordinate space that you then have to position a virtual camera in the midst of your scene which is represented by a matrix. Then, you have another matrix that represents the actual foreshortening process. Turning a 3D scene into a 2D image essentially has to emulate certain aspects of the human visual system such as vanishing points and foreshortening.

We model all of this with matrices. The role of vertex shaders is to apply those matrices and spit out more or less two-dimensional vertices that are in this special space the graphics hardware can handle. Then, it goes through a non-programmable stage called rasterization where those triangles that are now sitting in this two-dimensional screen space get chopped up and converted into pixels. Then, those pixels more or less are passed to your fragment shader where you consult textures and where you consult various information that you've passed along through the pipeline in order to do lighting and shading and texturing and things like that.

vertex shading and fragment shading. Vertex shading is responsible for basically turning the 3D scene into a 2D picture. Fragment shading is responsible for determining exactly what color each pixel lines up being in the final image. Is that a decent overview?

I think so. If I want to model a car, I would get a 3D model of the car from a 3D designer. I would import that into the app and the vertex shader actually lays it out. Because it's 3D, we can look all around it but we have to pick one vantage point so we pick a camera angle. Is that the right term?

Yeah, the camera positioning, camera orientation would be the parameters there.

That tells us what view to actually render to the screen because that screen is 2D. The vertex shaders or the other shaders, they would do things like “Okay. Since it's farther away, it might be a little bit darker.” Things like that.

Sure. Yeah. What you do, in practice, is every vertex carries along not just its position but any other parameters that you might need to inform how it should look on the screen. For example, you can pass along a color that determines how it looks and you can pass along a special kind of vector called a normal. A normal vector just indicates how the surface is oriented at that point. If you had a sphere, then at every point, the normal points outward. It's orthogonal or perpendicular to the surface at every point. So you can pass along these normals.

Those are very important component of shading because if you look at an object in the real world, for example, how that surface is oriented relative to the nearby lights informs how light reflects off of that surface. You actually have to do – you have to account for these – the physical reality of the universe if you want to create plausible 3D scenes. You can carry along whatever information you need with each vertex in order to do whatever fancy lighting and so forth calculations you wanted to on the GPU.

Very cool.

Let's talk a little bit about what Metal actually does now that we've given an overview of what 3D graphics is. By far, the biggest win that you get with Metal is precomputed render states. What I mean by that is: whereas, previously, I was talking about how in OpenGL each time you do your drawing, you set this global state that informs OpenGL about the lights and the materials and various other things. With Metal, a lot of what you do is offline. You would build what's called a render pipeline state and you specify offline – basically at initialization time – which vertex shaders and fragment shaders and a few other parameters. Those all get compiled into this intermediate representation which you then reference later on when you're doing your drawing. In contrast to GL where you're constantly doing the state validation, with Metal you pay the cost of state validation once up front. Then you can reuse these state bundles for future rendering. That's one way in which Metal substantially reduces the CPU overhead of graphics rendering.

Another big win with Metal is explicit memory management. You already have this big advantage when you're on a mobile device that you're on a shared-memory architecture. The CPU and the GPU are looking at the same RAM. They may have their own on-die caches – their L1 and L2 caches – but, essentially, when they have a pointer that refers to the same physical memory which is really important. What it allows you to do is greatly reduce the number of copies that happen both implicitly and explicitly in the sense that you can allocate a chunk of memory on the CPU and then write vertex data or whatever you need to into it. Then, just hand Metal a pointer more or less and say “Don't copy this. This is where you're going to be looking for your vertex data.” That saves you – there are really no implicit copies whatsoever. You really are responsible for synchronizing access to these buffers in memory which is a big obligation but is also a huge win for performance because there simply are no – there's no copies happening under the covers as there always were with OpenGL.

Do these two things having – I think you said functions right? They're called functions in Metal – having those get compiled down and then also this making memory management so that's not implicit. These are obviously done for performance but how much of a performance benefit is there? So what's the win over OpenGL?

Sure. In order to answer that, I have to recourse to this abstract notion called a draw call. A draw call is basically what happens when you submit actual geometry to be rendered. You configure your pipeline and then you submit geometry in a draw call. The marketing speak around Metal when it debuted was you will get the ability to submit 10 times as many draw calls. That's an upper bound but that is a way of expressing how much Metal reduces the CPU overhead.

For example, I think that Unity engine – they said they saw something like a 40 to 60 percent reduction in CPU overhead versus OpenGLES on modern iPhone hardware. What you can do with that savings in CPU time – which is basically waste because it's going to state validation in memory copying – is you can submit a whole lot more geometry and more varied geometry because you can substitute in new precompiled render state bundles at very low cost. Then draw – for example, you might draw geometry that represents a tank and then draw a geometry that represents terrain and then draw a geometry that represents a particle system. You can rapidly switch among these precompiled state bundles and then submit your geometry. The big win with Metal is that you can, as an upper bound, submit up to 10 times as much – not as much geometry – but as many – make all these different draw calls with different state configurations which gives you – it's a big performance win. It also allows you to be a little bit more flexible and creative with your art because these state changes are cheaper basically.

Okay. It sounds like this is not just a marketing speak theoretical thing. It actually does make for a big performance boost even in the real world.

Absolutely so. Yeah. You could just take a look at the Epic Zen Garden theme that was released around the time of WWDC last year and see just this amazing gorgeous dense particle system that would not be possible to render with OpenGLES. There are – not a whole lot of games – but there are definitely a handful of games on the App Store that have just gorgeous graphics that are actually enabled by the savings in GPU time and efficiency that you get with Metal.

How do you know if you're app is going to be well suited to something like this?

Yeah. If you're currently draw call-constrained, as we say, so in other words, if you feel limited by the amount of geometry that you are able to render in your game or if you are having to be ultraconservative about the number of different, say, materials or the number of different state changes you are performing; and if you're pegged on the CPU is another really important thing because, again, OpenGL's going to get a big chunk of CPU time whenever you're rendering in order to do all of these overhead. If you're CPU-bound or if you're draw call-bound, then it's possible that Metal will provide you some benefits.

Is there a way to test it out without completely converting your app over?

Good question. I don't know. I guess you could use a profiler that indicates – because in Xcode, you can open up that side pane and it will tell you if you're running at 60 FPS or not; and it'll show you the allocation of time between the tiler, the renderer and the CPU. If you're pegging the CPU and the GPU, then there's a pretty good chance that Metal would be a win because it's going to reduce your CPU consumption.

Something I'm just wondering about, I'm not really sure how this works out but Metal, of course, is iOS only, it's not cross-platform the way OpenGL is – but it seems like games, in particular, are one kind of app that are, in general, historically have went themselves very well to being cross-platform partly because OpenGL is cross-platform but also just because games are typically not really tied into a particular platforms, UI conventions, or anything like that. It seems like a hard sell to lock yourself into using Metal – meaning you can only support iOS and a lot of games want to support Android and whatever. Is there any solution to that? Is there writing all the graphics stuff twice?

Yeah, absolutely. I think it bears mentioning by the way that not only is Metal iOS8 only; it is constrained to devices that have A7 and A8 processors. It's really – anything prior to the iPhone 5S is not capable of running Metal. There is a very small subset and I recognize, as a full-time author about Metal that I'm hawking this stuff but I'm really occupying a niche of a niche. I'm speaking to low level graphics programmers and people that are exclusively targeting iOS8 and above. That's a pretty small market. The way I think this actually works out in practice is you have these Metalware authors like Epic; like Unity; like all other Metalware and game engine producers who already have this proliferation of rendering backends that speak to DirectX; that speak to whatever the graphics library on the Play Station is; that speak to OpenGL. They're architected so that these renderers are pluggable. When Metal came out, within a few months, Unity had a Metal backend. By using one of these Metalware solutions, you're getting all the benefits of Metal without actually having to write a line of Metal code yourself. Whenever I'm talking – whenever I'm writing about Metal, I understand that I'm addressing two different markets and that is: Metalware authors for whom it makes sense because they operate at scale to write Metal rendering backends and also very curious amateurs because Metal is never going to – I shouldn’t say never – Metal, in its current incarnation, is not a cross-platform API; will not be a crossplatform APItargeting, say, Windows and Android and so forth. The major benefits of Metal are where you can get a Metalware author to incorporate it and then use it implicitly in your own work. I don't imagine that they're going to be just a huge number of people writing Metal code themselves but they are the people that I most care about, right?

In the second group outside of the Metal authors like the crease amateurs. Is this something that only for game developers? Or can I take advantage of Metal – getting any advantage of using it for any other types of applications on iOS?

You want those Wicked Fest Table Views, right? [Chuckles]

3D table views. Go for it.

Writing a Metal backend for async display kit, that'd be something else – [inaudible] or whatever I don't know if anybody is doing that. That'd be pretty cool. Actually, yeah, you bring up a really important point which is what else can Metal do besides 3D graphics? We've been very, very fixated on 3D graphics so far but it's important to realize what a GPU actually is in contrast to CPU.

A CPU and the operating system that sits on top of it are good at task parallelisms. You got all these myriad processes and threads running in parallel but they're all doing different things. Whereas a GPU is specially designed to have dozens or hundreds of processors or cores that are all doing the same thing simultaneously and this is really good for graphics processing because – by the way this is called data parallel programming in contrast to task parallel programming – when you're rendering, say, a triangle onto the screen, you got maybe thousands of pixels that comprise an especially large triangle on the screen and they're all going to be running the same shader code but the data is going to be different for each pixel.

GPUs are special-purpose designed to basically do floating point operations in parallel really, really fast but each of these little shader cores is – for example, you have an if statement in a shader, it's not as though this little shader core takes one branch or the other. It actually executes both and the masks out the result of the branch that should not have been taken. This is in contrast to a processor that when it sees conditional, it takes on branch or the other. This is a quirk of GPU architecture that's in service of the fact that GPUs are designed to do basically the same operation in a massively parallel fashion.

All of these leads into what we call compute programming or GPGPU which is General Purpose Processing on Graphics Processing Units. Metal actually has explicit support for compute processing that has nothing whatsoever to do with graphics programming. What you can do is put all of your – all of the data that needs to be processed in parallel into buffers and write a special kind of shader called a kernel function which operates on every unit or every element in a buffer. You can perform any arbitrary computation that can be expressed in the C language pretty much on each element in the buffer. Then, you can write that out to another buffer or you can go on to manipulate it in whatever you see fit. When you talk about “What else can I use Metal for?” You could, for example, do physics on the GPU by casting your dynamics equations as a GPU program; you could, in theory, do massively parallel processing of rigid body dynamics and fluid simulations. You can do image processing, fast Fourier transforms, matrix calculations at very, very large scales and really use the whole animal by involving the GPU in these large scale computations. It shouldn't be missed that Metal is not just 3D graphics; it also has a very important compute component.

Okay. So you could just – by using Metal for doing FFTs and that's basically sound or wavelengths. Okay.

Yeah.

So it doesn't have to be graphics.

Not at all. You can definitely do single processing. What's nice about that is that it's going to be substantially faster than running equivalent processes on the CPU once you cross the threshold – once you cross the bandwidth threshold of basically loading this data into the GPU, if your problem size is larger than, say, 10,000 elements as it often is in audio signal processing, then compute shaders can be a win for you in cases like that.

Okay. 10000 samples in a quarter of a second or something like that? [Chuckles]

Yeah, exactly. I don't know exactly what the story is as far as if you were to, say, process audio samples with Metal and then try to feed them to an audio unit or other core audio thing. There's maybe too much latency to actually do – to try to use the GPU for real-time audio processing. From what I've just heard from people that are tried to do it but, for doing – in theory, doing transcoding or doing effects offline, it could be a good fit for something like that.

If I feel like doing a frequency analysis, FFTs.

Frequency analysis would be another example. Absolutely, yeah.

If I wanted to build my first Metal app and do something 3D, how do you get started? What's the first step?

It's really hard to talk about the “hello world” of Metal because there are so many moving parts. First of all, I do have to plug my blog – metalbyexample.com – where I do try to – I wrote an article series starting it in August of last year that tries to lay bare all of the different things that go into building very rudimentary Metal app. I can try to give you an overview.

If you want to – Let's say you just want to draw a one triangle to the screen. That's the equivalent

of “hello world” in a 3D graphics API. Just to give you a rundown of that. What you would do is you would create a Metal device which is a type of object that abstracts GPU and it conforms to this protocol call MTL device. Then you would create something called a command queue. A command queue is a thing that serializes work that's going to be executed on the GPU. It's a serial queue. What goes into a command queue is a sequence of command buffers and these command buffers are – they consist of encoded instructions. One element of a command buffer might say “Change the background color to this.” and another might say “Draw this sequence of triangles.” but it's expressed in a very terse binary format that the GPU can easily unpack.

The way you translate between human speak like “Draw this triangle.” and what actually goes into a command buffer is through an object called a CommandEncoder. A render command encoder has a number of methods on it that allows you to express say “Draw vertices out of this buffer with these indices.” and so forth. That's what gets encoded into a command buffer in place on a command queue.

That's one aspect of how you get stuff into the pipeline. But then, in order to inform Metal of which shaders to use, we build one of these things that I mentioned earlier which is a render pipeline state. You've written your vertex shader and your fragment shader in this C++-derived shading language. You pass the source code to Metal. It gets compiled and you get handles to those functions and you pass those to your render pipeline state. The render pipeline state is something you set on your command encoder to inform it which shaders should be run as part of the processing. That's two components.

Then, in order to actually get anything on the screen, you have to interface with UIKit and there is a special CA layer subclass called CAMetalLayerthat is able to basically hand you these textures, really, that you can draw into these renderable textures. That's another thing that you configure on your command encoder, roughly speaking, when you're doing drawing.

At the end of all of this, – by the way you also have vertices copied into this thing called MTLBuffer which is basically a raw pointer that you can copy into that you can then tell Metal about. You basically say, at the end of all this, “Draw triangles from this buffer using the draw table texture that I specified using the render pipeline that I specified”. Then you basically end your coding on your command encoder which packages all it up to GPU commands. Then you present the texture to be drawn and you commit your command encoder.

At the end of all these 25000 steps, that draws a triangle on the screen. That's sort of the relatively high-level view of all the work that goes into drawing one triangle with Metal. There are only a few other related concepts that you need to understand before you're doing fancy things like lighting and texturing. It's very intimidating to write a “hello world” but I try to, in my introductory articles, make it as comprehensible and as simple as it can possibly be but no simpler.

Sounds good. As far as “hello world”s go, I'm expecting to drag this button under the U controller. Done. That's great.

I sincerely wish it were that easy.

But if you can get over the initial intimidation of there being like all of these unfamiliar concepts especially if you're coming into this with a not particularly strong background in linear algebra, matrix multiplication and fundamentals of 3D graphics; if you can just face that and learn bit by bit, it's actually really, really cool. You can do some really awesome stuff.

Now, what are the wins or how the performance gains with Metal? Is that –we actually share the memory between the CPU and the GPU? Are there any patterns for multi-threading? I get worried if I say, “Hey, I get this many memory in the state I want it. Here you go use it.”

Yes.

How do you handle multi-threading?

Yeah, excellent question and a really important topic that I probably do not write enough about. Okay. You have these buffers and they contain your vertex data. The last thing you want to do is be writing into a buffer while Metal is trying to draw it onto the screen, right? That's sort of the concern that you have. What you can do is basically keep several buffers in memory and write into one as you're drawing from another. You can control access to these buffers by use of a semaphore so that you just use whatever your usual multi-threading, multiprocessing primitives are to essentially control access to the data that will be used by the GPU.

Metal is pretty GCD-friendly in the sense that you can tell a command encoder or command buffer rather ''When you're done executing, call this block.” That block callback that you get is where you can basically signal your semaphore and say “Alright, I'm done with this buffer that was on-queued for rendering. Now I can start writing new stuff into it.” And just basically run it as a circular buffer or just switch among these various disconnected buffers in memory. That's how you make sure the trains don't run into one another. It's just really Multi-threading 101. Just make sure that you're not writing into memory that's being read elsewhere. It's actually fairly straight forward as long as you are submitting rendering calls from a single thread. For example, for the main thread, it's not too terribly hard. You can do it with just one semaphore that signals to you when a buffer frees up to be written into.

Do you have to worry about what thread you’re – obviously UICode has been run on the UIThread but does the code have to be run in the main thread?

Not necessarily. A Metal command queue which is one of the fundamental objects that I mentioned is an inherently thread-safe object. If you're in a background thread, you can ask for one of these command encoders and command buffers and then whenever you end encoding, you can commit to the command queue that will be handled in a thread-safe fashion. It gets a little bit tricky if you want to perform rendering in the same render path across multiple threads. There is a specialized render command encoder called a parallel render command encoder that allows you to do that; but that's a fairly advanced technique and it really only buys you performance when you're really constrained by the CPU and you need to be able to do work across multiple CPU threads in order to submit all your geometry to the GPU.

But, yes, you can definitely perform these rendering commands on a background thread. But obviously, then that entails whatever additional complexity is inherent in every multi-threaded app. All of that is incumbenton you. Metal provides very little in the way of safety when it comes to being thread-safe.

Very cool.

Warren, you're writing this book but do have any plans or are you actually using Metal in an app that you're going to release or anything like that?

I toyed around with the idea of writing an engine or a game but, frankly, where I derive my pleasure from Metal is just learning about cutting-edge rendering techniques and exploring the ways in which those could be cast into one of Metal projects that I can then share with other people. I haven't really thought too much about what comes next. I haven't actually built any really substantial large-scale game or engine project with Metal yet. I don't have any particular designs on that. But I hope I'm helping other people do it. The feedback that I receive seems to indicate that I am.

So you are hearing from people that are using Metal then? I'm trying to get a sense how widely adopted this has become because it was only announced less than a year ago.

Right. You can check out the Apple Developer forums. There is a Metal-specific forum and the conversation there is pretty vibrant. It's always going to be – at least in the mean time because it's iOS8-specific and specific to these newer devices – it's not going to see as broad of adoption as something like OpenGLES. But for a certain class of people, it was a god-send. It's been a lot of fun to play with it and push these devices to the absolute maximum. It's important to realize that Metal was created in order to unlock the potential of the GPU that sleeps inside these devices. There really is no other way to get that low-level and that high performance – you're not going to get there with OpenGLES. For the applications that absolutely demand it, Metal really is essential.

Right now Metal is only on iOS and I'm curious as a Mac developer if there's any reason for it to come to OSX and I'm sure we, of course, are just at Apple's mercy on this but is there any certain demand for something like this on OSX? Or does it not make as much sense?

I would love to see Metal on OSX. There would have to be some changes to the API because desktops, at this point, have multiple GPUs. For example, you would need a way of selecting among them or distributing work onto them. I think that could be done in a fairly elegant way.

One of Metal's big wins, of course, is the fact that there is this shared-memory architecture on iOS devices. You would need probably some way of explicitly shuttling memory around if you were to create Metal for OSX. But there's no fundamental reason why the API couldn't be ported with a few changes.

But there are all these other interesting exciting directions as well that's going on in the 3D graphics world. With DirectX 12 and with modern OpenGL – the OpenGL 4.5 – coming around the corner and Mantle and Vulcan and all these entirely new low-level graphics APIs coming down to pike, it's really an exciting time to see how these APIs are going to evolve in the future. In my happiest world, Metal becomes an OSX API, I have no idea if that's actually going to happen; if that's the direction that Apple intends to go in.

Alright, well, we're getting toward the end of our time. Is there any other aspect of Metal we haven't talked about that's critical to understanding it?

I think I've probably spoken my piece. I hit all the bullet points that I had although not necessarily in the intended order but I'm happy to address any further questions.

The only question I have is it seems that you have this deep knowledge of how graphics rendering works. We really couldn't do it justice within this hour. Is there a place or a list of places where people can go to bone up on that if they really want to know a little more about it?

Yeah. Just to speak to the broader question first. There are definitely easier ways to get into 3D graphics programming. You have this bevy of APIs on both iOS and OSX that can provide a much smoother entree to 3D graphics.

You've got SceneKit on, I guess, it's definitely on – I think it’s on both OSX and iOS. You've got SpriteKit. You've got – heck, just play around with Core Animation and you learn some things, right? What you can do there is explore what it means to create a 3D scene and do some rendering without actually having to get your hands dirty with shaders right off the bat. Then, when you want to get more serious with it, there are a lot of really good books on the subject. The sky is the limit as far as how you want to go with this.

I'm looking, in my bookshelf; I've got a book by Eric Wingle called Mathematics for 3D game Programming and Computer Graphics. That's a fantastic introduction to all the mathematics that goes into graphics programming. Then as far as learning APIs, there's the OpenGL Programming Guide and some higher-level books by people like David Eberly, Mike McShaffry would be another.

Basically, seek out tutorials that actually cover some of the fundamentals and then you can go from there. Because I've been working with this stuff for almost 15 years now and I still feel like there's so much more that I have to learn than I have already learned. Some days I feel like I forgot more than I know but I think in order to actually get anywhere with low-level API, you actually probably need to have a project that you're trying to achieve. It's very difficult to learn this stuff in the abstract because there's just really little motivation that comes from that.

If you got a game idea, prototype it out in something like SceneKit then drill down into something like OpenGL, GLKit or Metal – whatever suits you. Unfortunately, there is no fast road. But if you want to get in touch with me, I'm happy to provide tailored lists of resources to help you learn.

Then, did you say you'd written a book on this?

I am writing a book on this. It will hopefully be out relatively soon. I'm actually waiting for Dubdub 2015 so that I can whatever late breaking changes need to occur; do some rewriting around what comes out in the next iteration of iOS and then hoping to publish both as an eBook and a soft cover probably around the middle of this year. But I'm continuing to write articles on metalbyexample.com on a more or less weekly basis. That's where you can see of that material.

The book is going to be based, in large part, on whatever I've already published there.

Alright, the book, the blog. Anything else we should know about as far as finding you or getting a hold of you?

You can email me at wm@warrenmoore.net or you can – feel free to leave comments on metalbyexample.com. Anything that's unclear; anything that trips you up with the content, please let me know. I want to make sure it's as clearly as it possibly can be. I'll try to help you when we try to get to this stuff. I try to help them along as much as I possibly can. I provide a lot of hands on support and I think that's a valuable use of my time because this is my passion and I want to help as many people as possible learn about it and use it to its fullest capacity.

Alright. Let's go ahead and do some picks. Alondo, do want to start yourself with picks?

Sure. My first pick is actually in service of learning some of the – one of the simpler APIs. I'm actually working on a game with my nephew – I’m writing now. We are using the iOS Games by Tutorial – a book from wenderlich.com – to get started. He's already story-boarded everything so I'm pretty excited about it because I'm a newbie when it comes to graphics programming and gaming in general. I think, so far, it's been really an easy read and great tutorials there.

My second pick is the new Raspberry Pi. I actually thought that my – I ordered a couple – thought they were actually going to show up before the show started but I'm still waiting for it to be delivered. A local software dev group – we have a project that wanted to do with some students so we ordered a few so we could get started. The new Raspberry Pi has more memory. It's faster and I think it's going to serve us as quite well in this project.

Those are my two picks.

Very cool. Andrew, do you have some picks for us?

Sure. I've got a few picks today. My first pick is predictable because it's an article written by Warren but it’s an Objective-C IO article that he wrote with Max Chris about Metal. It gives a good overview for somebody who doesn't already know all about graphics engines and OpenGL so this is just a great introduction if you want to learn more in written form. It's in issue 18 which was all about games.

Then, my second pick is actually another Objective-C IO thing which is the newest issue which just came out a day or two ago. It's all about audio which is a subject near and dear to me. In particular, there is an article by Chris Liscio called Functional Signal Processing using Swift that's all about doing audio signal processing using functional programming in Swift. There's some pretty cool stuff in there. It's actually used for functional programming that I hadn't exactly thought of but it's pretty perfect. So those are two Objective-C IO picks.

And then my last pick is something that Warren just mentioned which is SceneKit. SceneKit's a framework for doing 3D graphics on. It was on OSX and then they put it on iOS a couple versions ago too. It's on both platforms and it's an Objective-C API. It's very Cocoa-like. If you're already an iOS programmer, SceneKit is actually really easy to jump into and get started with and yet it's a fairly deep and powerful API. It's a good way to learn some of these basics of 3D graphics and not so much the low-level stuff but in terms of putting objects in the scene and lighting them and camera and animation and physics and all that stuff. But there's plenty of power there too when want, including custom shaders which I've never tried but you can write shaders for SceneKit.

Those are my picks.

Alright. Jaim, do you have some picks for us?

I've got one pick and I'm going to do a beer pick. People who've been listening for the show long enough; they realize that I'm a big fan of the German beers – most of them. I like them a lot but it's hard to get – you're really not sure what you need to get a lot of times – either a Spaten or a Paulaner or the light beers. Maybe sit around for a while. A lot of times they're in green bottles that are skunky. Most of the American breweries just don't really do that great of a job with the Lagers, especially the German-style ones. But I had one, about a week ago, from Victory who'd done a number of really good Lager. But the Victory Helles Lager is fantastic. They're out of Pennsylvania I believe – something like that – but they're available in quite a number of places. If you want a good German-style beer brewed by an American company, Victory Helles Lager.

Cool. I've got a couple of picks. The first one is: I'm putting on another remote conference for Rubyist. If you're into Ruby, go check out rubyremoteconf.com. I also picked up the RubyMotion book which is a way of writing iOS and Android apps with Ruby. They're native apps, natively compiled apps and I'm really enjoying it. I don't know why but for some reason it seems to make a

bit more sense to me than some of the Swift or Objective-C stuff. I can do that stuff but this is just – I don't know. Maybe it's because it's Ruby. Anyway, I'm really enjoying that so I'm going to pick RubyMotion.

Then I'm going to pick a podcast. If you're new or newish to programming or if you're just interested in hearing cool interviews with cool good people, go check out the CodeNewbie podcast by Saron Yitbarek. It's terrific. She interviews all kinds of people; asks terrific questions. The last one that I listened to was a round table with her and Scott Hanselman and one of the other people that she was in the boot camp or the startup camp that she was in. Anyway, it's just – there are just great conversations about code. Go check that out. They also do a Twitter chat every Wednesday.

Those are my picks.

Warren, do you have some picks for us?

I have one pick for you today. I'm going to keep it simple. You may have heard of a game called Smash Hit. It got a lot of accolades last year but the team that created it who's called Mediocre have put out another game called Does Not Commute. It is this zany driving simulation top down where you are responsible for conducting people as they go along their daily commutes. The twist of it is that you control one driver at a time but as you progress through the game, the drivers that you have instructed along their path interact with the drivers that you are currently instructing. It builds every time. Cars can crash into each other and it’s a race against the clock to get everybody to their destination safely. All the crazy physics high jinx contrast really cutely with that 70's pastoral pastel game aesthetic. It's a lot of fun. New from Mediocre – Does Not Commute, highly recommended.

That just sounds like fun.

It’s brilliant.

Alright. Well, thanks for coming. Thanks for talking to us about this and, hopefully, it helps a few people figure out how to make their apps faster.

Yeah. It's absolutely been a pleasure sharing with you guys.

Alright. Well, we’ll wrap up the show. We'll catch everyone next week.