Check out JS Remote Conf and All Remote Confs!

 

02:32 - Brad Urani Introduction

04:01 - Immutable/Persistent Data Structures; Advantages

07:30 - Tools for Debugging

08:23 - Why do Rubyists care about things like Elm?

09:39 - Persistent Data Structure Use Cases; Functional Programming

12:07 - Testability

13:51 - Where does “functional play a role in a typical CRUD app?

Active Record, 

22:49 - Coworker Receptiveness of Ruby + JavaScript Style of Programming

26:41 - Service-oriented Architecture (SOA) vs Monolithic Applications

30:21 - Monoliths as a Necessary Stage in the Development of a Mature Application

33:23 - The Repository Pattern; Terminology & Naming

37:40 - Structured Query Language (SQL)

Picks

Dan Carlin's Hardcore History (Coraline)
Stuff You Missed in History Class (Coraline)
Buffer (Avdi)
New Belgium Brewing Accumulation White IPA (Avdi)
Saramonic SmartMixer Professional Recording Stereo Microphone Rig (Chuck)
LaunchCode (Brad)
Turing's Cathedral: The Origins of the Digital Universe by George Dyson (Coraline)
VAT19 (Brad)

Special Guest: Brad Urani.

 
CORALINE:

When my daughter [inaudible], whenever she got upset about something, she'd say, "You're ruining my life about this!"

Hey everybody and welcome to episode 237 of the Ruby Rogues podcast. This week on our panel we have Avdi Grimm.

Hello from Tennessee.

Coraline Ada Ehmke.

Hi from Chicago.

I'm Charles Max Wood from DevChat.TV. A quick reminder to go check out JSRemoteConf.com. I’m actually doing 12 conferences next year and you can get tickets to all of them at AllRemoteConfs.com. We also have a special guest this week and that’s Brad Urani. You want to introduce yourself?

Yeah. Hi! It’s an honor to be on the show. I’m a long time listener, first time caller. My name is Brand Urani, I’m a Web Architect at Procore, in Santa Barbara, California. I’m very pleased to be joining you all today.

What do you do at Procore?

I’m a Rails Architect. So, Procore's interesting, we make construction management software where in Santa Barbara, we're a company that is going real fast and it’s a company that is for the most part of giant Monolithic Rails app.

In fact, we were a Rails app since version, I believe, 0.9, so in the very, very early days. I didn’t work here back then, I’m fairly recent here. But I worked on various third party integrations. I helped us design basically our [inaudible] architecture class design, testing strategies, education, on-boarding, recruiting, training and various other things.

Architect is not a job that we don’t hear very much anymore.

Yeah, I say architect with the lower case A. [Inaudible] working at Procore is kind of interesting because our team is split into squads and the amount of actual “architecture” we do is actually rather minimal. The Procore team has found over the years that you can get an awful lot done basically with basic MVC Rails and there’s -- it generally pays to err on the side of less architecture, not more. So, it’s been interesting because I come from a background of pretty heavy [inaudible] design and the dynamic works out surprisingly well.

Recently, I was at Ruby Conf and you gave a talk there on Immutable Data Structures. What was that all about?

It’s about persistent Immutable Data Structures and that’s an idea that many comes from functional programming. So first of all, we have to basically differentiate between just persistent and immutable. Immutable is simply something where [inaudible] versus taking array. If you create immutable, an immutable array, you cannot change it afterwards.

Ruby has that with persistent array that freeze which, you know, once you created that array, you can’t [inaudible] add an element or convene an element or erase an error, that you know, allows you to avoid or surpass the bugs but it’s not that interesting or that useful.

Persistent is a little bit different. So in Ruby, you can get a persistent data structure by using a gem called hamster. It’s a really, really great gem. It’s got a number of [inaudible] data structures about persistent list, persistent caches, persistent sets, persistent vectors.

Persistent means, is, if you have a reference to arrays, so for instance if you say, A = [1,2,3] and then you go and say, “I’ll shift to 0 and I do it”. So, you push the 0 on the front using B = A [0]. Now, B has a reference to a list of [0,1,2,3] but A is still just [1,2,3]. In other words, the original reference you have is pointing to an array that was unchanged. And if you think about that for a second, you scratch your head a little and say, "Well, how do you implement that?"

And the first thing that comes to your mind is, “Okay, I have this array [1,2,3]. I added 0 but left the original [inaudible]. I must have cloned it, right? I must have made a whole copy in memory.” But that’s actually not the case. Persistent data structure, persistent list is actually a link list. So what you end up with is a reference A that’s pointing to the second node in the list which is 1, and B pointing to the first node in the list which is 0. You tackled the load on to the front. And that way, by having a single list with link list [0,1,2,3] and having A point to the 1 node, and B pointing to 0 node, you’ve managed to both alter a list and not alter it at the same time in a way that is memory efficient that doesn’t involve any cloning.

So a little tough to describe but it’s actually a very elegant solution. The point being you allow changing of the list without cloning and keeping all references to a list, don’t see any changes, they’re effectively immutable.

What’s the advantage of an immutable persistent data structure over just, you know, messing with hashes, messing with arrays?

Admittedly, the applications for them are somewhat niched but the primary benefit you get out of the immutable persistent data structure is that it allows you to keep perfect history. So any program you have that uses this data structures as you change things, as the state of your programming changes. If the state is created out of persistent data structures, you keep references to all the previous states.

So, this gives you, probably, the neatest feature is the time travel ability so that it’s really easy, for instance, to program a state manager that keeps references to all the points of time which allows you to travel back and forth, which is very, very clever. Of course, if that would work, you need to adhere to certain architecture. Basically, all your state needs to be in one place.

If you take all the state of your programming and hold it in one object in one place, and you don’t have other immutable state in various objects, instance variables and various objects, you get that time travel capability which allows you to do some new things, time travel debugging, among other things.

Are there tools for that kind of debugging?

There aren’t in Ruby. This is an idea that it hasn’t really come to Ruby because Ruby being, you know, a purely OOP language. OOP to the extreme, in fact, so much so that everything is an object, of course. It’s somewhat of niche application in Ruby. It is very, very prevalent in the functional programming world. Any functional language because it’s generally designed to avoid as much as possible, you know, Rails state changes. It is designed to isolate state, to isolate changes into state to as few places as possible. It’s much more easily time-travelled. The bulk of the functions are pure functions which don’t hide state changes in various instance variables.

The [inaudible] implementation I’ve seen of time travel using persistent data structure is Elm. Elm enforces an architecture of single state such that any programming you build in Elm is time travelable by default.

Now, we did a whole episode on Elm. I’m kind of curious. We got the kind of a viewpoint, I guess, of Evan and I think it was Rob, you know. We kind of got their viewpoint on things but why do Rubyists care about things like Elm?

Well, first of all, we live in a world of worldwide web. [Chuckles] So, just about everyone does Ruby these days is also doing JavaScript.

Yes.

So, like it or not, at some point in your career, you got to end up doing front-end development and Elm works front-end development, a new paradigm with just a new level that’s not precedent, that’s is just, in my opinion, [inaudible] any JavaScript solutions.

So, if you want to do front-end development, you know, look at Elm. But, it can also, I think, a very instructive and perhaps where the future Ruby should go, people want to get involved in something that’s completely different from Ruby, you know, to further their careers, to further their own understanding, to further their own knowledge. There’s a kind of polar-opposite paradigms.

Although there are some interesting cross-sections, there’s kind of opposite paradigms and it’s just a great way to become well-rounded developer and to understand, maybe, something that you’re not familiar with if you’ve done Ruby for most of your career.

Is it incompatible between use of persistent immutable data structures and programming in an object-oriented way or do you think that there’s some meeting ground there?

Well, I think the data structure use cases are admittedly niched but there are some interesting ones taken only in algorithms. Algorithms where you have to keep a history, or sort of steps you’ve taken are kind of useful. But even if persistent data structures are a little bit niched, I think there’s a lot to learn from functional programming in general, definitely a lot that relates to basically everything from class design to database querying to application architecture in Persistent Rails app.

I think there’s a lot you can learn just from functional programming and the techniques involve there.

What are some of those lessons?

For instance, you know, I spent most of my days working in Ruby on Rails, and Ruby on Rails is somewhat interesting because if you look at the web app, it can be described pretty simply as a

data transformation and it’s a sort of data-in-data-out flow.

Consider a web request comes in and you have programmers. You got your request parameters, you take that data, you set it down like a pipeline of data transformations to the database. Either you are saving or you’re querying, you can take that data back out of the database, you send it through functions, back up to the front-end again.

In a nutshell, that’s for the most part, all web [inaudible]. That’s for the most part where the web is. A Rails app is a set of functional transformations between a request coming in and either read-orwrite from a database.

Functional programming also describes programs as data transformation pipelines. Now, the difference there between sort of what the Rails paradigm is or what the OOP paradigm is or the functional paradigm is that along that way, an OOP program will hold intermediate state instance variables.

In other words, you may take an object. You might set something. Later, you might retrieve that. You might change that instance variable, you know, and do many other number of sort of things related to state changes within that request before you reach your response to the user.

What functional programming teaches us is that you can see a lot of benefits by minimizing the amount of those state changes you do and understandability and readability, and especially, testability.

So, let’s talk about the testability aspects a little bit. Can you elaborate on that?

You know, if you look at Active Record, for instance, when you make a query, you store in data in an object and you may have methods on that object that mutate the instance variables, that change the instance variables in there. It’s really hard to write a test for that. In fact, it’s really hard to write a test against models in general because in order to properly test something that changes an instance variable in a model, you need to sort of reach in there using any number of [inaudible] to ensure that the state has changed.

And as you're programming, when you commit this code and your colleagues put along and look at it, keeping track of in this web request pipeline sort of where you’re expecting that object to be as it was original where this change can be real hard. And it makes it real hard to unit test.

If you design your web application architecture, kind of you design such that your functions are pure so that you pass a parameter to a function and you get a result back and it doesn’t change any state, that is so easily testable. I call it “embarrassingly testable” because if anyone calls you not testing your class like that, you’ll be embarrassed because it is so easy to test. Then you can write a single unit test against a single method in distinction to, for instance, an integration test, controller test.

It helps your testability greatly just to write pure functions because every pure function is very, very easy to test. So, you know, what I call sort of a functional [inaudible] design, the truth is not so [inaudible]. I mean, my application architecture has typically this amalgam of OOP and functional programming and good old procedural programming by doing as much of it in pure functions as possible. Other parts that are pure functions are so much more easily unit testable. And I mean unit testable in the strictest sense, meaning one test, one function.

We take like a typical Rails CRUD application. Where does functional play a role on that?

The first thing I’d like to do, looking at a Rails app that if you start by reading beginners’ tutorials, there’s a sort of like idealistic world where you have one view, one controller and one model. So, you’ve got a user view, a user controller, a user model which corresponds to a user table in the database. But if you’ve done enough web development, you come to learn pretty quickly that the world is rarely so simple. That often times a single view is going to be displaying data that is taken from multiple tables joined together and that is going to undergo a number of transformations based on your business rules. And then also, the way it’s displayed in the view might be in a drastically different structure, often hierarchical structure than it is stored in the database which is a relational structure. So, you realize that sort of the ideal MVC which is one-to-one-to-one doesn’t quite hold. So now you’re forced to sort of come up with a way of, "OK, how am I going to manage this sort of changes in my data?”

First of all, put query level joining combining. Then at the business logic level, change arriving data, deriving stuff based on my business rules. Then at the view level, arrange the things so that they are reasonably presented to the view.

And functional programming can help us. The first step I do is try to separate the logic from the queries. If you just use, basically, models can make that hard because even if you go about jumping into a model and writing methods that have both business logic and queries combined, then when you go to test, you're forced to stop those queries or you forced to plant data. Both of which sort of violate the idea of the unit test. But if you can separate logic, you make your models rather dumb and do very little but treat them more as like just structs, like something that just holds data and you pass those into a class that actually does the transformation that can make those class do the transformations, pure class. They take data in the form of models or hashes or structs or what-have-you and then spit out what needs to go to the view.

Now, those are pure and now, it’s really, really easy to unit test those. You end up with cleaner tests, better tests, getting at the heart of what you’re testing by separating the logic from queries. And now, you come up with something that’s informed by functional programming because of that, more testable, less likely to have bugs in them.

I’ve always taken issue to Active Record being called active record because it seems like an active record model violates SRP and that it is responsible for persistence and business object. But I actually went back to the patterns book and saw that that’s what Active Records supposed to be so that kind of diffused that argument. But I think it’s a great idea to separate persistence from business logic. Now, do you do that through, in a case you just mentioned service objects or view objects? How is it implemented?

I use service object. I also like to distinguish between objects that read and objects that write. So, command queries separation. So, in my Rails apps, I have controllers call either services if they get requests, if they’re fetching data and I call them commands if they’re writing data. And actually, there’s a precedent for this. Actually, Martin Fowler, he calls it the Transaction Script pattern where you have a single class that represents basically a single transactions of things writing to the database.

So, the basic answer is yes, using service objects but with an additional command query separation. You get something really neat when you do that. In a lot of web apps, you want to offer the user’s feature that shows sort of a history of actions they’ve taken. You open an account on this city, you deposit money on this state, you withdrew money on this state.

If you do your program using a service layer in a separate command [inaudible] services depending on whether you are reading or writing, you can have like a command-based class where you’re basically keeping an event log which allows you to have that sort of transaction history that’s a user feature. So if you need that capability in your application, that comes very, very useful and then of course, every app, every web app these days is reporting analytics, heavily reporting analytics so you can do data-driven analysis of your users, what they’re using, what features they want to do.

Having a command-based class for all your commands allows you to, basically for free, send-off analytics events to your mixed panels, your Goggle analytics, maybe your own internal solution so that you can basically much more easily build a BI solution. So that’s really neat user really useful.

And the other thing I find is that one of the issues I have with Active Record is that it tries to handle database transactions for you. For instance, when you hit that ‘save’, anything that appears in a callback, [inaudible] start with a callbacks, automatically you’re in the same transaction but that can lead to trouble because say for instance, you have ‘will callback’ that does some database updates, a simple update, then you’ve got another one that goes and fires up API requests, third party API. You’re doing that API request with a third party inside an open-database transactions which begins potentially, you’re locking a table in Postgres.

In other words, relying on a framework like Rails to handle database transactions for you can often lead to trouble in ways that if you’re in a large-scale, a broad-scale, because it is trying to do what you need to do explicitly which is handle database transactions the most efficient way possible and making sure that all your creates and your updates are all together in one transaction with little else happening which becomes a [inaudible]. So that’s another benefit of that kind of command pattern.

What’s wrong with callbacks [inaudible] sarcastically? [Laughter]

Right?

Yes, we unload that when we go into Rails to make a real data update and then the [inaudible] on the side you created your trigger apps of massive cascading chain that we forgot about the effects. 15 different things, you know, that there’s plenty of literature on the topic. I don’t think I need to force that one heavily.

The fun never starts.

Yeah.

Oh, man. And it’s so funny too, because I know it’s there so every so often I go in and I create a callbacks just like, “Oh, I’ll just create this real quick because I just need some quickie little thing done.” And almost always comes back around and bite me.

I'll allow myself a before save but that’s about it. I want to set a default value or something along those lines because getting to initialize in the Rails object is not easy.

I find that I think with OOP, if you look at the early literature of object-oriented programming all the way back to the Gang of Four, a lot of that was written in the world of desktop applications. And desktop applications are inherently stateless. Picture, for instance, Microsoft word. You may have multiple windows open, various things, [inaudible] paper clips and what-not on your screen. It kind of makes sense for all of those to have their own state.

Imagine in this kind of separate components in the single application and those things, they stay on screen. That makes sense without [inaudible] programming because you have objects with instance variables. It seems to me kind of that we took those paradigms and we transport them to web where perhaps in server-site web, I mean, or perhaps things were a little different because the web is inherently stateless. The only state in a web app is the cookie and what’s stored in the database.

So, to have objects which are created and destroyed with every request, kind of weakens the case for that just a little bit, which you start to realize when you get involved with functional program. It’s just that function represent actions. And classes can also represent actions. So typically, we think of classes as things – a user, a shopping cart. But you can also use them for things like a sign-in, and add to cart, something like that. And in the context of web, it can make a little more sense.

I know. That is something I learned from working with Corey Haines years back. I had always been taught that nouns are objects and verbs are methods. But they have classes that represent processes that represent verbs can be very freeing and really keep your controllers nice and fun too.

I saw somewhere there’s a researcher invented a language that was verb-focused. English is a noun-focused language typically. We start with the subject to the sentence and only then produced the verbs. But it’s interesting to think about what would our language be like if we put the verbs first and represented the world around us in terms of the interactions of things, the actions rather than trying to picture the world as a frozen state of matter.

I can tell you how it would be if we had verbs first, “Go to the store, I did. Yeah.” [Laughter]

That’s it. So functional programming is like Yoda.

That’s right.

Everyone loves Yoda.

Hey, Brad! I have a question.

Yeah?

You started a new job about a year ago? Was it why you moved out to Santa Barbara?

In about six months, I left St. Louis, moved to Santa Barbara, California.

Oh, it seems a lot longer without you.

Oh, Thank you.

We missed you in St. Louis.

When you got there, how receptive were your co-workers to this slightly different style of Ruby and JavaScript?

Well, I’m really lucky enough as I came out here, I found a lot of my co-workers are writing Ruby on Rails by day and they’re very interested in functional programming and spending a lot of times on nights and weekends being interested in these things in Elixir and Haskell and Elm.

So, it’s been neat because I’ve come from a background of very small, early stage start-ups starting with either greenfield apps or very small apps that could be more or less completely refactored or rebuilt. And here I've come to a place with a giant Rails app that's 8 or 9 years old from all the way back from Rails, like 0.9 or something like that. So what’s interesting here is it’s impossible to come in here now and re-factor the entire app and things I’ve talked about, command principles and service principles. So there’s a new challenge now in siding on a case-by-case basis on a request-by-request basis, whether it makes sense to input these patterns in various small places. So it’s been nice because everyone has been very receptive.

And we’ve been able to very slowly, in a few places where it makes sense, implement it where it helps us without broad-sweeping dictums about, "Alright, we’re going to switch to this pattern everywhere." And that’s kind of a new challenge that I've really enjoyed.

Do you find any negatives from the inconsistency in the code-base that the change and coding standards and styles lead to?

No, we don’t have standards.

In a good way. I’d say that very positively that we're built in the teams. We split off in the teams and everyone is allowed to do things in the way they want to do without an architect saying, "These are our standards; these are our styles."

And what that means is that all the ideas we have live on their own merit so that if certain teams don’t see the value in like a service or command style pattern and they want to do their Rails basically MVC then they do that and it works and they knock out features and everyone’s happy. And then another people apply it where it seems like it’s really going to help.

So for instance, if I want to model something as an action, for instance, take some kind of request that makes them write, that does multiple updates and creates. You can model this as a single transaction; it's some kind of command pattern. And you may get the benefit of not only can I call it from control or I can call it from a [inaudible] task, people see that. And rather than having some of top-down architecting things, every idea lives by some merit which is good in any process where you need to see.

When you see your own merit, is that like, do you measure that by how much people like it? People on that particular team?

We try not to over-architect. We try to do things where they are useful. So, it’s basically…

No, that makes sense. I just kind of want to de-clog the part where merit is always subjective.

Oh, yes. So, in programing, most of the decisions about designs aren’t really objectively measureable. People have tried to do empirical studies of how changes in class design, even in testing practices can be measured and how you can prove that - certain philosophies, certain methodologies better than the others. Most of those I find to be of rather -- most of the studies, most of those sort of [inaudible], I find to be rather to be [inaudible] interesting things of cloudtesting practices with a number of [inaudible] produced. So, it is definitely subjective, but I don’t think subjective quality just because something is subjective in quality means that we should not attempt to measure it.

Yes, I agree.

So, I have a question. You talked about, Brad, working on a monolithic Rails application and I know a lot of companies are dealing with their monolith problem by refactoring to service-oriented architectures. Do you think that SOA lints itself well to functional programming?

.You know, I don’t know that SOA versus Monolith has too much of a profound impact on that. It seems like within a monolith or within a service, you can just as easily use a more [inaudible] approach or more functional approach. I know you have some experience with that. I saw a great talk you did at Kansas City Developers Conference two years ago about Smashing the Monolith. I don’t know. What do you think?

I think re-factoring to a service architecture lets you undo a lot of early decisions that you might have made in the architecture of your original system. And I don’t know, I’ve been thinking about this a lot and I'm wondering if service-oriented architectures are harkening back to the RPC days Remote Procedure Call days - as opposed to implementing REST, more of a REST approach to architecture.

So, it seems like if we’re doing RPC again, that a functional style would lend itself really well to that.

In Scala, there’s definitely at least in Twitter and people who use the Twitter finagle library for service center actions, it is much more of an RPC attempt than REST.

Yeah, the alternative is to invoke an object with a lot of parameters and let your controller do its magic. But I think being more direct and more upfront about what it is that you’re doing within point is an advantage.

I completely agree in them. I’m glad you brought it up. I think, first, service are not, I mean, you could look at the Rails app and using kind of the architecture I described [inaudible] services [inaudible] things and command a lot of verbs, not nouns, definitely lends itself much more to RPC than REST.

It goes back to actions versus things if you’re…What about this fundamental model around things which is trying to what you get if you follow kind of the Rails way. REST doesn’t make more sense but RPC is definitely more [inaudible]. So, I completely agree.

Even when you’re not trying to make an RPC, there’s things like GraphQL that are – we're moving past REST. REST is not the most efficient data access method. And I think you’re right. REST is very thing-focused and we’re kind of getting past that as a community.

But Brad, I’d like to a point about you have a monolith, you’re not breaking it up into services necessarily but you’re still able to change and evolve your coding style team by team at will. So you don’t have to move away from the monolith to gradually improve your code.

Yeah, that’s right. It’s something that has taken me a while to adapt to. Before, I come from a world from [inaudible] to sharpen other things, of sort of real kind of tough, heavy, Gang of Four style OOP world. And the thought, if that’s what you’re used to, the thought of a lot of different teams working on the same app, it’s that with different styles can strike your [inaudible] are you sure that works? And I found it really does, I think it’s really fascinating and I think it’s been a smash huge success.

Cool. I think part of what our programming philosophy is changing is to realize that change is constant in our programs, to not expect to be able to architect the perfect, pristine OO application and have it stay. Once we embrace change as the normal state of things, then yeah, that sort of constant improvement feels more natural. But if you can break out services, please do.

[Chuckles].

[Chuckles].

So there’s an interesting article. I can’t remember who wrote it and I want to see if Fowler wrote it but I’m not positive, about monolith as a necessary stage in the development of a mature application, as opposed to going straight to services where you have no idea what the service boundaries are. You have no idea what your domain objects are and you’re going to make a lot of mistakes in the architecture and your data model. Then, monoliths are really great for that. And once the monolith is built, you can easily separate out concerns and see where those scenes are and then refactor to services.

I actually originally started out that way with DevChat.TV. I started out with [inaudible] services. And yeah, I felt all of that pain pulled everything inside, figured out where things are and now, I’m starting to spin things back out.

Wow!

So I can attest to that.

Currently at D.C., you’re to refactor within a program than a cross services.

Definitely. And it teaches you to get started too because you don’t have 15 different things you have to spin-up and check 15 different logs on.

Yeah. The way I definitely am very interested we're [inaudible] the transition to services by breaking off little things around the edge of the monolith, breaking [inaudible] services so we’re evolving that way. So, it’ll be really, really interesting experience to see how that goes. You may have to invite me back in a year or two when further along, I can tell you about that.

Definitely. And I think that the service approach might have another side effect, Brad, that harkens back to something you talked about earlier and that is treating Active Record as a persistent [inaudible] and keeping the business object out of it. If you have a persistent service that’s free of business logic, I think it falls that pattern pretty well.

Yeah, and what’s interesting is if you look at… I just barely – I started [inaudible] service. I’m just trying to look at a little bit at the Phoenix framework which is of course, the web framework for Elixir. You can hardly see that they are taking that lesson to heart. So, the Phoenix ORM which is called Ecto is interesting because first of all, it gives us the repository pattern instead of the Active Record pattern. So, you say basically myrepository.users. and then you kind of string together basically kind of SQL commands .where or whatever. Whatever it turns is just structs the simplest data structure possible with low methods.

So that’s a pretty radical departure. Other parts of Phoenix are very, very, very helpfully inspired by Rails. You can tell that they definitely didn’t [inaudible] in the persistence layer. What’s kind of funny is that when I started talking about models that do very, very little, people accused me of, “Oh, you’re following what Martin Fowler calls the anemic model pattern.” He calls it the anemic model anti-pattern, turns out the sort of next generation of ORMs is sort of not only turning in new models, they are not really models at all. They’re just structs of data. It’s just data. In other words, that separation and using an ORM as a thin wrapper is enforced from the very get-go.

That’s awesome. I love Elixir, I love Phoenix and Ecto is pretty interesting.

I would push back a little on the idea that the repository pattern supports any [inaudible] models. I mean, at least as documented in Fowler’s book, repositories are often there to feed your [inaudible], which are a way to map data into rich domain objects. So I’m kind of confused about how that supports anemic [inaudible]?

Can somebody define the repository pattern?

Yes, please.

So the repository pattern is a little bit contrast to the Active Record pattern. From what I understand, those are the two types of Object Relational Mappers. The two primary types are active record style and repository style.

In active record style, you kind of have this classes that say, “Okay, here’s the class of user which corresponds to my user table.” When I run a query basically, I’m using the database to hydrate the user object with data which then has the methods. Hanging off that, it allowed me to operate on that class.

Whereas the repository pattern says, “Here’s a repository that represents my whole database."

And I’m going to basically run SQL queries off of that,” start chaining methods together to generate SQL query that returns an object. So in my opinion, just aesthetically speaking, repository seemed to me – some people might disagree with me on that – but they seemed to be a little more focused, a little more explicit as a querying mechanism. Whereas, active record models seem to try to hide the implementation details a little bit more and try to allow you to do object-oriented programming with less knowledge of actually how the database works or how the querying has happened.

I'm going to be pedantic for a moment. So I want to make it clear that we’re not arguing with your choice of how to structure applications. I think that’s kind of a separate question. But just in terms of keeping terms clear, what you just described is sort of the exact opposite of the repository pattern. [Chuckles]

The repository pattern is actually a higher level, less SQL focused pattern than the active record pattern. And again, when I say actually, obviously there is no 'God did not set these patterns down in stone’. But if we’re going to go by the terms that Fowler’s talking about, repository is much more abstracted from the SQL layer and it basically ties in a bunch of the lower level patterns in order to interact with the database without really thinking about the database. It actually gets a little closer to the object-oriented database’s view of the world. It’s an intermediary layer between your data store and your domain objects that uses things like query objects so you don’t have to have like raw SQL.

One of the things that confuses all these and makes it more difficult is that as active record has sort of grown and metastasized, it has grown to – it's this blob that has slowly eaten up like the entire Patterns of Enterprise Application Architecture book. It started out as an implementation of one of the patterns in there and now, it’s like all of the patterns are in there. And active record actually includes a good chunk of some parts of data mapper and basically all of repository at this point.

Interesting.

Active record is the worst branding ever. [Laughter]

[Chuckles] Because it now includes the concept of taking a query object. In other words, something that abstractly represents a query without actually writing SQL and applying that to a repository and getting the result set and using the result set to populate domain objects and all that stuff gets rolled into active record. Of course, that means it isn’t really active record anymore. So, it’s very confusing. So yeah, terminology gets confusing when people start naming their libraries after patterns and then [inaudible] the libraries.

Naming is dangerous.

I think what’s interesting with what you mentioned is my understanding of repository versus active record may very well be totally wrong because I tend to use object-relational mappers' ways and they weren’t necessarily designed from sort of an ORM [inaudible] .

But you mentioned hiding SQL and allow you to do OOP in a way that hides its implementation details then encapsulates your queries. My big question or my contention of that is why do we want to do that?

In particular, SQL is a fantastic language. It is one of the great success stories of computer programming. It was created in the early ‘70s. It’s immensely powerful during the whole NoSQL binge that everyone went on from 2005 to 2012 and everyone was saying, "We're finally past SQL. We're finally on to something better with all these little SQL databases." Everyone was crying to death now for SQL. Now, the newest, greatest database is looking like who [inaudible] one and some of the open source, sort of [inaudible] people are trying to do the same thing have all gone back to SQL.

In other words, it’s the only thing we all still use every day besides Unix, but [inaudible] we use every day or at least the old programming language that we all still use because there's still nothing better.

I always thought it was a mistake to try to hide the SQL from the web app. In fact, I look at web apps as basically the code between the browser and the SQL query engine. And I've particularly found that it becomes problematic hiding SQL when you're on-boarding junior developers, when you're teaching junior developers. There seems to be a trend to try to allow people to write a web app with the SQL database without knowing SQL. And quite honestly, I think it’s a big mistake. Not only because does it result in poor SQL N+1 queries and things like that, solutions that aren’t scalable which is a problem at a [inaudible], but also SQL is just great in itself and I don’t know why you want to hide it.

One of the arguments I've heard against using raw SQL as database portability and that’s something we’ve talked about in the show before. But if you’re going into raw SQL, it’s hard not to use database-specific functionality in your queries because SQL is not – well maybe, it's SQL standard but the implementation is not standardized.

Yeah, I’ve heard that one often. And just to be clear, I don’t necessarily mean writing just raw SQL. You could just as easily be using something like active record and where that’s very SQL forward, like for instance, in the service object being user.joins.joins.where.limit, in a way that makes your calls to active record look like a single SQL query.

In other words, SQL [inaudible] use of active record. I thought that was the other argument. The portability of what I have heard but [inaudible] you have never been on a project in a commercial project where a business that we're trying to support or it’s like we get in there, “Oh my god, you got to switch to Mongo or Postgres. You got to switch to MySQL.” You know what I mean?

I think some people are facing that now with Amazon’s database service which is based on MySQL instead of Postgres.

I think that’s definitely valid. These things are hard because we encode variability based on what we thought was going to vary at one time. And then, down the road, it turns out that’s really not the thing that we vary at all. I think a lot of the sort of the need to have abstract SQL, abstract the way from proprietary SQL, maybe not as many projects need that anymore, but like Coraline just said, some may discover that, “Oh hey, we need to move to Amazon’s SQL store.” And suddenly it is an issue.

Of course, that’s going to be painful no matter what.

But I mean, there are different – yeah. And there’s a limit to how far you can abstract. You’re not going to suddenly switch from a SQL store to Mongo and have your magical data mapper just support that because you’re talking about – often, you’re talking about moving not just from the swapping out your data store but you’re swapping out a lot of semantics of how you think about your data and that’s got to be [inaudible].

But I mean there are definitely intermediate layers between the really high level object mappers and the low-level SQL access. One of the things that I love using these days is the Sequel Ruby gem – the spelled out Sequel Ruby gem which is wonderful because it lets you execute queries against your tables. Using an abstracted interface but only very lightly abstracted. I mean, you’re not necessarily writing raw SQL but you’re writing something very close to it. And it will work just fine against your SQL or against your Postgres or against your MySQL but you know that what you’re writing is against some kind of SQL. And then it's just returning arrays of hashes back.

And I think, Brad, it sounds like that’s a lot like what you’re talking about and the patterns that I relate that to go back to POEAA book are like the raw data gateway and the table data gateway patterns along with the data transfer objects which, you know, really dumb objects from moving data around.

Basically, structs.

Exactly.

You had a good point there, Brad, about exposing SQL whether it's slightly abstracted or literal SQL, being useful for on-boarding new people. I’ve experienced that at new jobs. There’s like all these code and I don’t know what, especially the dynamic language, I don’t know what is in this, if it’s Clojure, map of maps and then I get to something with SQL with it. And I’m like, “I know what this does. I can go look at it in the database and see exactly what’s coming back.” I think that’s really good for code readability.

Yeah. I tried to argue my architect applications in a way to prevent sort of [inaudible] that I call ‘database denial’.

Yeah, database denial is this thing that affects a lot of the web applications which is when you use high level abstractions over SQL to hide querying and brings you problems like N+1 queries, lack of explicit opening and closing of database transactions and just various other things.

Querying in multiple layers, for instance. I've seen a lot of people who are new to Rails for instance start to sort of naively write active record and end up with queries in the model, in the controller, in the helpers and in the views. So I try to pick architectures that acknowledge the presence of SQL there that work with SQL, make it sort of, even if it’s [inaudible] active, even if it’s doing ORM as a thin layer, that make SQL query as a first class citizen to avoid all those symptoms of database denial.

I think one of the things that we have been taught by the whole NoSQL movement and what started that was recognizing not all data is the same - not all of it needs to be stored the same, not all of it needs to be updated the same and has the same constraints. And then we have all these different alternative databases and now, we’re like, “Put those SQL databases. They’re all so good,” which is true. But also, I think we can keep that realization that the database doesn’t just magically work. We have to think about what we're putting into it and what the requirements are under different circumstances. And then yeah, like you said, explicitly start and end transactions so that people can see what’s happening.

I only have with not being with databases, I guess, especially active record lint itself to this is persistence first thinking. Not every domain object needs to be persisted. Like you said, just getting the same way but I think that when you’re starting an object with the migration, you’re thinking data first. You’re thinking state first. I think that can sometimes be a problem.

Yeah. When I was first programming, that is totally how I learned to do it. First, you design the database model, the relational model for the things. And then you layer on the program from there and yeah, you’re right. We can kind of move past that and think about the flow of data and actions because there is more than one way to store the things. The relational model is not our only model anymore.

Avdi, I’m so glad that you’re like on it with these patterns and defining them.

Just to be extremely clear, I think the questions, Brad, that you’re bringing up are really important ones. I think they are the ones a lot of people are asking and I think these are things that are very important to reconsider. So, I’m not arguing at all about do we need. I think it's good to ask, do we need the sort of high-level denial of the database? I only bring up the definitions of patterns from the books just for clarity’s sake. That’s all.

Exactly.

But just because the pattern exists, it doesn’t mean it’s necessarily something you need to use. [Chuckle]

Is there a name for the just barely abstracted SQL style?

Like I’m saying, the patterns that are interesting to look at for that, I mean, if you’re going to go to the POEAA book with I always recommend going to just because there’s so much wisdom in there or the table data gateway, the raw data gateway and then like the data transfer objects which, you know, represents a very lightweight way of moving data around without building a big model around it.

Those are very familiar from my [inaudible] days.

Yeah.

Yeah. I’m just curious to hear you say that, Avdi. I watched recently, it was your keynote from – I forget which Ruby Conference – [inaudible] about formalism. It’s funny to hear a gentleman who was, prior to the end of formalism, talk about in the most formal way possible, the correct definition of design patterns.

A foolish consistency is the hobgoblin of little minds.

This is the perpetual contradiction that I have to live in…

And I try to…I guess my personal mission right now is sort of the constant recognition that formalisms are things that people invent, that they are always contextual, that terms for things, well-defined terms that for things can be tremendously useful, especially if everybody, like, if you’re familiar with the source material and actually understands what the term they’re using means.

But bottom line, they’re always invented. They’re always contextual, they always embed, unstated assumptions about what is important and what isn’t important and what things are things and what things are not things because the things that have no names in your pattern language are things that you're implicitly denying and that can be a problem too.

Sapir Whorf effect.

Yeah. So, know your context but do not be limited by your context.

What’s the Sapir Whorf Effect?

Sapir Whorf Hypothesis states that language influences and constrains our thinking about things. So, if you have no word in your language for different shades of blue, you don’t perceive those shades of blue, basically.

Right. And Avdi mentioned something there that also relates to what Brad said about on-boarding. These names are great if everyone knows them but if you are using them in a context where people don’t know them, then they can actually add confusion. For instance, [inaudible].

It's [inaudible], right? Exactly.

It's basic.

But you can establish those. If you consciously recognize that then you can say, "Okay, when you work here, if you don’t know SQL yet, please learn SQL." When that's expressed and it’s explicit, then you’re not adding confusion. People know what context they need to gain for it to make sense.

Alright. Well, I’m going to push us some picks. Before we got to the picks, I just want to acknowledge our silver sponsors.

Coraline, do you want to start us off with picks?

Sure. Lately, I’ve been getting into history and I don’t know about you but I didn’t have very much history in high school and college. At least college before I dropped out. So I feel like I have a lot to learn in this domain. And so, I went looking for a couple of history podcasts and I found a couple that I really like.

The first is Hardcore History. It’s a podcast by Dan Carlin. It's been seen by many to be 'the history podcast'. Carlin does really, really well research episodes and the production value is extremely high. There’s more like an audio books in a podcast. So that means he doesn’t come out with new episodes very frequently but when he does, they’re super high-quality. So, Hardcore History.

The other one I enjoy which is a little lighter and a little bit more entertaining is put up by the people behind How Stuff Works, it’s called Stuff You Missed in History Class. It’s full of really interesting trivia. I just listened to the episode on Sir Isaac Newton and I learned a lot about Isaac Newton and they have episodes ranging from the organ trail all the way to the history of peanut butter. So, they don’t take themselves too seriously but there’s always something to learn on that podcast.

Alright. Jessica, what are your picks?

Too much pressure. I don’t have any picks today.

Okay. Avdi, what are your picks?

So, I think I’ll pick a social media tool I’ve been using lately called Buffer and some folks may have heard of Buffer. It is sort of marketed as a way to maintain your social media engagement by always having tweets and stuff coming out all the time and whatever.

I like Buffer not because it lets me be constantly filling my buffer with stuff but because it’s writeonly. That’s why I like it. If I install it on my phone, it doesn’t give me a way to read Twitter. It just gives me a way to write to Twitter. And not only that, the stuff that I stick in there isn’t going to get post for a while which means that I can’t get start obsessively checking Twitter for replies to whatever I wrote because it’s not even up yet.

And by the time it does go up, I don’t care nearly as much as I did when I first posted it. So, I’m not going to check for replies then. And basically, it means I get more done and I don’t think about the social media quite so much. So, I like Buffer for managing my own brain, not for managing my social media engagement.

Other than that, I’m going do to a beer-pick. It’s the season of winter beers and I recently tried the New Belgium Accumulation White IPA and it’s a really exceptional beer. I really, really liked it. So if you can find it, check that out. That’s it for me.

Awesome. I've got a quick pick here. I haven’t had the chance to try it yet but I did get it set up the other day. It’s a Saramonic SmartMixer. What it is, is it goes on your tripod, kind of like for a camera except it has a clip for your smartphone and then it has an audio input-output that has a couple of microphones on it that you can kind of point at yourself while you are shooting video with your phone. And it hooks directly into the phone and it also has an XLR plug if you want to hook it into something like a mixer. Obviously, you can also plug it into a computer. And the reason I’m doing that is because I want to get back into doing Periscope videos every morning and so I got that. I got it all hooked up and set up. I actually did a Periscope on unboxing it. But yeah, it’s kind of hard to explain exactly how it's shaped but basically, right above the phone, it has this little box that has the microphones on it that you point at yourself. And that way, I can get things set up and I can just talk to my phone or talk to my computer and have it pick up and record and do all the right stuff. So I’m kind of excited to play with it. I’ll put a link to in the show notes and we’ll go see how goofylooking this thing is. But I think it’s going to work out. I think it’s going to be fun and I’m planning on recording quite a few videos with it. So, that’s my pick.

Brad, what are your picks?

So first of all, a really neat program out of St. Louis from LaunchCode. LaunchCode is a non-profit that places beginning programmers in their first jobs in pair programming positions. So [inaudible] more of an apprenticeship program called the [inaudible]. It’s a program that has placed almost over 200 people in new programming jobs and it's spread out of St. Louis into three or four other cities. And [inaudible] assisted the White House. If you're interested in the future of tech-hiring, definitely check out LaunchCode. They’re doing really, really neat things.

The Origins of the Digital Universe. This is a history book. Coraline, you’ll probably like it. It’s by George Dyson and it is a story of the invention of the first computer at the Institute for Advanced Studies at Princeton in the late ‘40s and early ‘50s. It features interesting people as Robert Oppenheimer, Alan Turing, Kurt Godel and the star of the show, John von Neumann who a lot of people credit with basically inventing the modern computers. It’s a neat story, it kind of mixes the invention of the atomic bomb with the invention of the first computer.

Finally, since we’re nearly in the holiday season here, I want to pick VAT19.com. It is an ecommerce site. It sells a lot of really, really unique and cool gift things you can’t find on Amazon. It's a small business run by a friend of mine at St. Louis. And every [inaudible] with a YouTube video that garnered hundreds of millions paid views. People love them. It’s a great neat, feel-good shop if you can’t find what you're looking for on a typical Amazon's and Wal-Mart’s of the world.

Check that out for something unique and different.

Alright, I think that’s it. So, we'll go ahead and wrap this up. Brad, if people want to follow up with what you’re doing, or check out what you’re about. Where do they go?

You can follow me on Twitter @bradurani, it’s my name. I have a blog at FractalBanana.com. Those are the two best places to find me.

What does the fractal banana look like?

Well, it’s a banana where you zoom in and you look closer at it, you just see more bananas.

It’s bananas all the way down. [Laughter]

Bananas all the way. [Chuckles]

Gotcha. Alright. We'll go ahead and wrap this show up. We’ll catch you all next week.