Piotr Solnica joins the Ruby Rogues to discuss the upcoming ORM called ROM. It looks to be very promising in that it separates the persistence layer from the object and much more. 

Special Guest: Piotr Solnica.

 

Hey everybody and welcome to episode 123 of the Ruby Rogues Podcast. This week on our panel, we have James Edward Gray.

You all make me want to program wet and naked.

Nice. Avdi Grimm.

Hello.

I’m Charles Max Wood from DevChat.TV. And we’ve got a special guest and that is Piotr Solnica.

That was very good. Hey everyone. I’m Piotr.

I was just thinking it rhymed with pizza.

Alright, well do you want to just explain who you are and what you’ve done?

Okay, sure. I’m a software developer from Poland. I live in Kraków. I’m mostly a Ruby developer. I’m also doing some frontend work from time to time. People probably know me from a project called DataMapper and now it’s called Ruby Object Mapper. So this is my biggest priority in open source right now. And yeah, that would be it, probably.

So DataMapper is dead? Just kidding. Just floating [inaudible] rumors there. [Laughter]

That’s a hard question. [Chuckles]

So are you not following the DataMapper pattern anymore? Or did you just feel like it was time to refresh?

Actually, it’s the other way around, isn’t it?

Yeah, it’s the other way around. We basically started with DataMapper as the project name. However, it implements active record pattern, which is funny. Eventually, we decided to rename the project Ruby Object Mapper because we didn’t actually like having the project called by the pattern itself. So the first project called DataMapper, as I’ve said, is an implementation of active record. So right now what we’re doing is basically building a real data mapper as the [tree] implementation of data mapper pattern. So that’s basically the short story.

So you’re now building Ruby Object Mapper as a data mapper pattern?

Yes, exactly. Exactly. Because DataMapper 1 has some elements of data mapper, the pattern. So for example you can have, we call it properties defined in the model and you can define how attributes from the database are being mapped to properties in your model. So that’s data mapperish. But that’s basically everything. In DataMapper 1, the resources, this is how we call model instances, DataMapper 1 resources know about persistence. They are aware of the underlying database. For example, when you fetch a User instance, you would call User.save, so it basically knows about the database and it will do everything that is needed to be done in order to persist the data. So in Ruby Object Mapper, which is a true data mapper, it’s completely different because the objects that are loaded into memory, a Ruby object, don’t know anything about persistence. They don’t even know that they were fetched from the database. So that’s the huge different between the two patterns, between active record and data mapper, that in data mapper you have a layer of mappers that know about persistence and they know about the objects that they load from the database. But the objects themselves, the main objects, don’t know anything about persistence and this is how we’re going to do it in Ruby Object Mapper.

Is this a totally new thing then? Or is it DataMapper 3?

It’s a new thing.

Okay.

Initially we started working on basically the next version of DataMapper. It was supposed to be version 2.0. Last year we were talking about releasing DataMapper 2. But eventually, we realized that it’s a completely new project. It’s a new codebase. It’s a different pattern. It’s just completely new. So to avoid any kind of confusion, we decided to just come up with a new name and start a new project, which eventually turned out to be a really good idea because it gave us a lot of fresh energy to work on the project. So it was a good move from our point of view, at least.

Can I take a tiny bit of credit for at least pushing you in that direction?

Yeah. Actually, I was going to say that. [Chuckles]

So at some point, yeah. We got an email from Avdi at some point asking us what we think about coming up with a new name. So yeah, it actually started the big discussion. It happened earlier this year. So yeah, thanks Avdi. [Chuckles]

I think you came up with a great name. I love it. It’s short. It’s sweet. It’s memorable.

Yeah, and it’s funny because some people

And it stands for read-only memory, right?

Yeah. [Chuckles]

Yeah, exactly. And the funny thing is that the first version comes with something called memory adaptor. So it was like read-only memory, for real. [Chuckles] So it’s kind of funny.

So talk to us about the different adapters because I’ve been looking through the project and it’s interesting. It’s broken down into separate pieces for the different concepts in it. And I definitely want to talk about some of those as we go. But the big crazy smart backend would be axiom, right?

Yeah, that’s true.

And Axiom has these different adaptors. So can we talk about those?

Yes. It actually all started with Axiom. So Dan Kubb started working on a library called Veritas and it was renamed to Axiom this year. Basically Axiom is a relational algebra library, like Arel. However, it’s much more powerful. And it’s actually not just an SQL generator. It’s actually an abstraction for relational algebra in Ruby. The reason why Dan decided to build it is that when we were working on DataMapper 1 we also wanted to support different kinds of databases and basically any kind of a data store. So in order to achieve that we would have to create a lot of adapters with a lot of crazy complex logic in order to support things that some data store does not support. For example, we have a YAML adapter for DataMapper 1 and obviously, a YAML file doesn’t support things like joins, right? So we would have to support that inside the DataMapper itself and do this operation in memory. So Axiom is basically a library that allows you to perform all those relational operations in memory. So if a given backend doesn’t support something, we can just simulate that on the Axiom level. So that was the main motivation behind the project. You could say that it’s actually like the brain of the entire project. It’s like the very foundation of everything. And everything else is just a very simple layer on top of Axiom, like extending its functionality or providing more convenient interfaces. In Axiom, we also have this idea of adapters. And an Axiom adapter is actually a very, very simple thing which can use Axiom if a data store doesn’t support something, some operation. But if it supports some operation, it will be pushed down to the database. For example, if we have an adapter for an SQL database and we want to perform a join, the SQL Axiom adapter will just run the query, so the data will be fetched from the database. But if it’s an adapter for some other thing that doesn’t support joins, it would just use Axiom to do the join inside memory. So that’s basically the idea behind Axiom and its adapters.

So you reversed the problem. You basically implemented the general set of rules, joins and all that, and now you override them for the things that do support them to use the native way, right?

Yeah, exactly, exactly.

That’s really cool.

It was, like you said, it’s like reverse. In DataMapper, we started with basically supporting SQL databases and then we started building those adapters and it turned out that first of all, active record pattern is limited. And also, we don’t have a solid foundation to actually support all kinds of databases. So that was the reason why Dan started working on Veritas, now called Axiom.

Okay, so the thing I’m most excited about in ROM, if I understand it correctly, is that we can specify, basically it breaks the connection between the Ruby object tree and the database. If you use something like Active Record that ships with Rails, that pushes you toward a tight coupling between the database schema and your Ruby object tree because the classes and stuff have to reflect the tables. And of course, you can introduce other objects and stuff, but the core of that data is pushed to look like the database. Whereas ROM takes the view of your Ruby objects are just your Ruby objects. You tell us how to map that to the database and we’ll do that. So maybe your object tree is one big object and that gets split out into multiple tables or maybe it’s a bunch of tiny objects that get saved in some other way or whatever. But the new thing is that there’s a mapping between the Ruby layer and the database layer. Do I have that right?

Yeah, you got it perfectly right.

Right. I’m going to move my microphone away from my mouth for a second. Yes! [Laughter]

Exactly.

Sorry, I just had to get it out there. [Chuckles]

Yeah, so that’s correct. We basically define entire mapping on the mapping layer and the data that comes from the database is just a very raw set of tuples. We call them tuples. And those tuples are being loaded into Ruby objects the way you defined it. So it’s completely separated.

One thing that I think is funny about this is when I’m talking to people about Rails and you want to just use a regular Ruby object, there’s the term ‘the plain old Ruby object’ and here, the plain old Ruby object could be persisted or not. So it’s just a Ruby object, which is so exciting to me because having that persistence tied so closely to things in Active Record is in a lot of cases, really, really painful. Sometimes, you don’t care. But sometimes, when you’re dealing with a concept that spans, like James said, multiple tables or some concept that you want to break down but you really don’t care about having a separate table for that particular concept as part of a larger concept, having that strict mapping that you have between an Active Record model and its table is a pain.

Yeah, exactly. This is one of the main reasons why people switch from active record to data mapper. I’m talking about patterns right now. Because basically, active record is as you know, great and convenient to use and easy to learn and use. But eventually, you will hit a problem where something that’s represented in some way in the database is not actually something that you want to have inside your domain. So having the ability to map the representation on the database level to something else on the Ruby level is just fantastic. And that’s why we just want to implement a true data mapper and just abandon active record altogether.

There is a bit of a downside. I don’t think it’s big. But the downside is that now that mapping is no longer inferred, you do have to specify the mapping. That’s the trade-off. But I’m looking at the romrb website and the number two on there is great. It shows a mapping. You’ve got a normal Ruby object and in order to set up that mapping for the database, it takes five-ish lines, roughly, to say where it’s going, what’s the model, which fields are being mapped. So I guess we should just say that you have to be more explicit about this goes to this.

Yeah. However, it’s really important to mention that first of all, it’s a very early version and we basically started with the foundation. And we’re working with some primitives inside the system. But eventually, we will provide all sorts of convenient interfaces where you won’t have to deal with lots of explicit definitions. For example, we can totally reflect schema using relational database. You don’t really have to do that. I mean, you won’t have to do that eventually. We could also use, for example, a model extended with Veritas library and just take its attributes and use it as mapping definition by default. So you can do a lot of things that will simplify usage of ROM. I’m not specifically concerned about this problem, that it’s too verbose, that it’s too explicit, because it’s not going to be eventually. So we should remember that.

Now I just want to say regarding that second option that you just mentioned where you take the schema as defined in the model, is really, really nice. I really like that model because it means you have to actually write out what attributes the model has in your class file but then you get a few benefits. You get the fact that you can just look at the class and you can see exactly what attributes it has and what relationships it has. You don’t have to go refer to the schema file to figure out what’s available on it. So it’s much more revealing. But the other thing, and this is one of the biggest reasons that I usually use the old DataMapper on most of my projects is auto migrations. It is so much nicer to just have my models be the golden rule about what attributes exist, what columns should exist, and then anytime I change them I just run auto_migrate and the database is updated to reflect what the models have. And early in development when the models are changing fast and I don’t have any precious data, it’s so much faster than generating migrations and fiddling with migrations.

Yeah, you’re right. Auto migration is a nice feature in DM1. We definitely want to build some kind of a migration library for ROM. But it’s not a big priority, this year at least. However, we definitely want to push it even more forward versus what we have in DataMapper 1. So yeah, this is very convenient when you can just use information from a model and just generate a database schema. So this is something we will build eventually, but it’s not a huge priority right now. But yeah, that’s a lovely feature.

We should probably mention, since we’re forcing you to talk about some coming things, ROM just did 1.0 but you chose an early release with not everything’s in there yet. It’s basically just core functionality. You’ve talked about how [you notice] we could still make schema declaration more automatic. We could still add migrations, et cetera. And then you chose to basically release early, release often, it sounds like. Can you talk a little about that?

Yeah. The reason behind the release is that we’ve got something working. It’s definitely not featurerich and definitely not production-ready. But people can just play with it, experiment with it, and provide us feedback. So this is really important for us right now. I should probably mention that we already had a prototype of DataMapper 2 last year, exactly a year ago, but we didn’t release it because eventually we realized that we don’t like the design of what we created. So we ditched it, completely removed it. Scratch everything and then start from the beginning. One of the mistakes that we did when building the DM2 prototype was that we tackled too many problems at the same time. We really wanted to have the same level of functionality as DataMapper 1 had. So it was just too much and we ended up with a very weird codebase and a lot of weird things, weird abstractions. So we decided to drop it and start with something simple. This is what happened with ROM. We just started with something very basic, something that gives you very basic functionality, and just push it. Just release it right away and see what happens. So once it’s released, it’s also much easier to work on it because some foundation is ready and we can just continue with the development. And we’re also receiving feedback, so we can prioritize things. So this is really a huge difference versus what happened with DataMapper 2, which was in the works all the time and we never actually released it. So right now we released something very basic but we will keep on pushing new releases pretty often. So this is a huge change for us.

I think that’s awesome. I think it’s really great. I wish more people would do that. We always have to keep polishing and gold-plating before we feel like it’s ready for the world. “Oh, we need this one more feature,” and it makes the feedback group so large. So I think it’s a great move.

Yeah, exactly.

It’s tough with a library like this, because there really is, with this pattern there is a lot that goes on under the covers. The whole relational algebra engine, that’s a lot of logic but it’s not very obvious logic.

Yeah, that’s true. That’s true. And especially that Axiom was developed for about a couple of years, I think. So we also wanted to have people actually start using it so we can be sure that it’s going into the correct direction. With the thing released, it’s just much easier to work on it.

Avdi just mentioned how DataMapper is a much more complex pattern than active record. There’s a lot more going on. But when we actually read Martin’s book, Patterns of Enterprise Application Architecture, the whole time I was reading it I’ve seen things like unit of work and identity map and thinking, “Wow, where’s this great stuff?” And if you look into the Rails version of active record it’s like, unit of work, yeah, that concept doesn’t really exist there and identity map, yeah, they sort of put it in for a while, hated it and I think now maybe they’re actually removing it. Anyway, it’s been removed or something. So there are all these other interesting ideas in there that are meant to work together with all of this. And one of the things that got me super pumped up about looking at ROM was here’s those ideas. One of the major pieces of the current ROM is rom-session, which is basically the unit of work right out of that book. And it also includes an identity map.

Yeah, exactly. We used a lot of patterns from Martin Fowler’s book, so we’re closely following those ideas when working on ROM. But the session is probably the most immature element of ROM because we built it this year only, [Chuckles] which is three months ago. So session right now is more of a state tracker that basically builds a queue with all the operations that should be executed when you flush the session. But the actual unit of work is not ready yet. I have a prototype of unit of work that I will plug into rom-session soon. But right now, session is pretty straightforward. There is identity map, so that’s something. The difference between rom-session and using just ROM relation is that when you’re using a session it basically waits with everything it needs to execute until you flush it. So if you’re using plain ROM relations without session, once you insert something it’s going to be inserted right away. If you’re using session, it will just create the entire history of all the state changes that you did inside memory and when you flush it, it will just execute everything in the same order that you actually added it. So with unit of work, it’s going to be different because unit of work will use some kind of mechanism to t-sort all the operations using the dependencies between those operations. For instance, if you have, I don’t know, some kind of a parent object with children, if you decide to save children, it will figure out that there’s a parent that needs to be saved before saving children. When it saves the parent, it will also know that there’s a foreign key that needs to be set on all those children. So it’s going to be a pretty complex beast to implement. However, that’s the idea behind unit of work. It basically resolves the dependencies and executes everything in correct order. So that’s the missing piece actually, right now.

Yeah, I’m looking forward to that. I just think it’s cool that we’re not just getting a data mapper here. We’re getting all the goodness designed to go with that, which seems really great.

Yeah. The ROM relation is actually pretty straightforward. It’s just a wrapper on top of Axiom. It allows you to work with Ruby objects instead of those raw tuple data that Axiom gives you. This is also the mapping part, actually. And it’s a separate part, rom-session is a separate project and rom-relation is also a separate project. And there’s also rom-mapper which is our default implementation of mappers but you can totally write your own mappers if you want.

That’s one thing that I did notice, is that it’s all broken out into different modules. So I really like that as well. If I needed to do something a little bit different, then I can plug something else in.

This is also one of our main motivations so that ROM won’t block you when you don’t like something. It should be really easy to replace each of the pieces. So we’re really focused on how we design our interfaces and how things work together so that the interfaces are just very simple. If you need a custom mapper, it’s just basically an object that responds to three methods, to get the basic functionality obviously. So if you have some kind of edge case where default mappers don’t really work for you, you can just write your own mapper and inject it easily into rom-relation. And it’s very simple.

That does lead me to one question. It goes back to Axiom though, as opposed to ROM itself. And that is that it looks like the main focus right now is on SQL-based servers. So you have the different dialects between MySQL, PostgreSQL and all those. It says in there that work is being done for it to work with databases other than Postgres. But I’m curious about things like the NoSQL families. Will Axiom work with those as well? Just in the sense that it would, for example, generate the query JavaScript for MongoDB as opposed to SQL?

Currently we have a MongoDB adapter so it’s possible to use Axiom with NoSQL databases. But I suspect that quite often, you will have to extend Axiom with some custom functionality that is required be a given database. But Axiom is designed in a way that allows you to extend it. So our main focus is SQL, R-SQL databases. Obviously, we are starting with it. But we will definitely make sure that it’s still possible to use Axiom with other databases. And this is the reason why we already started building other adapters. So as I said, we already have the prototype of the MongoDB adapter. There’s also an adapter for ElasticSearch and a database called ArangoDB which is a NoSQL database as well.

I could see eventually having, the cool thing about the mappers is you can have your Ruby object tree all constructed and then send most of it to Postgres or whatever, but then have that mapper in there that sends parts of it to Redis or something, right?

Yeah. One of the cool things about Axiom is that you can work with many different databases at the same time. So for example, you could have two relations coming from two different databases and join them together and actually write into that joint relation and Axiom will distribute the writes to the correct databases. So that’s pretty neat.

That’s actually really exciting. I’ve played with using the MongoDB example again. I’ve used MongoMapper and Mongoid and both of them, then thing is, their interfaces are usually pretty similar to something that you’re familiar with, Active Record or something else. But they’re not the same. So that’s one thing that really excites me, is the fact that I just have one interface to deal with to persist data.

Yeah. That’s the main goal of the project, to have a unified interface for all sorts of databases. But as I said, we will definitely have some custom extensions inside adapters for various databases in order to basically use the native features of databases. Because we really want to support native features of databases. It’s not like we want to hide databases from you and say that they don’t exist or whatever. We really want to push as much as possible to the underlying database. It’s especially important in SQL databases like using constraints, using check constraints for validation, all sorts of native features that are basically much more reliable than Ruby itself.

We should probably mention, we’ve talked a lot about the various pieces, rom-session, romrelation, mapper and stuff. There’s now a GitHub account for rom-rb and you can find all the various pieces there. So that’s a nice central place to start looking around. Taking the discussion in a slightly different direction, in addition to all those pieces you’ve made that are under the rom-rb that we’ve been talking heavily about, session and stuff, you’ve made a lot of other little projects. And when I was looking into ROM, I just went and looked at the dependencies on the gems and started walking through those dependencies and a lot of them are simple little projects useful outside of ROM itself, just useful in a more general sense. So I’m thinking of concord and equalizer and adamantium, which help just build simple Ruby objects and stuff. Has that all just come out of what you’ve needed as you’re building the project? Because there are some cool libraries in there.

Yeah, exactly. This is basically the outcome of all the development efforts while building DM2 and now ROM. We really want to promote building small libraries. We’re not big fans of things like Active Support for example. So rather than having a huge support library, which we wouldn’t need anyway, we decided to just split things into smaller pieces that you can use in your own projects that don’t necessarily have anything to do with ROM. So we created a bunch of small libraries that we use in basically all of our projects. Right now, they’re mostly helper libraries that remove a lot of the boiler plate code. And also, we’re using this thing called adamantium which helps us building our objects that are not mutable, by basically freezing them.

I thought adamantium would allow you to shoot knives out of your fist. [Laughter]

What are some of these other projects? Equalizer?

Equalizer does the, actually this came up in a discussion we had recently. Remember in the episode with Tom Stuart where the code in the book is a little weird because basically everything inherits from Struct and when you look at his major reasons for doing that, one of them is to get a reasonable equals method everywhere? That’s what equalizer does. You can mix it into your object and it defines a reasonable equals method in hash and other comparisons.

Yeah. It also adds inspect method.

Nice. I’m going to use it and then I’m going to tell people how smart I am for using it.

It’s pretty cool stuff. A lot of them are really cool. And then another library we’ve mentioned on this show before is virtus. Do you want to talk a little about that, Piotr?

Yeah. Virtus was actually the first so-called DataMapper 2 piece that I started working on. So the idea initially was to come up with the same interface as the property API in DataMapper 1. But we wanted to have an interface available in plain Ruby objects. So I extracted the entire property API from DataMapper 1 and just put it into a standalone library and it’s called virtus. So that’s how it started. But eventually it turned into a pretty powerful library. It allows you to define attributes that also support coercions and you can also use some more advanced features like using embedded values or embedded collections. When you include virtus inside your model, you can initialize the instance by passing a hash with attributes and those attributes will be coerced into correct types. This also includes embedded other virtus objects. You can also use it with OpenStruct or Struct if you want. Virtus is already a pretty stable library and I’m currently working on its final 1.0 version. One of the things that I did recently was extracting the entire coercion logic from virtus into a gem called coercible. This is also following our philosophy with small focused libraries. So I removed a lot of code from virtus and just put it into a small library that you can use whenever you want to use coercions. Another thing that we’re doing is integrating virtus with a library called axiom-types. This is another small library extracted from Axiom which gives you just a bunch of data types that you can define along with some extra constraints. For example, you can have an integer or string and provide a minimum length or things like that. So I’m integrating virtus with axiom-types which will allow me to just remove a lot of code from virtus. Virtus started as part of DM2 but it’s really a standalone library. It has nothing to do with ROM in terms of dependencies. It’s just a small library that you can use for all sorts of use cases. I think that people in most of the cases were using virtus for a form object, for parsing and sanitizing user input coming from web forms and things like that. So this is a pretty common pattern for virtus. I also remember that some people are using virtus to prototype models. So that’s one of the use cases for virtus as well. But yeah, as I said, it started as a piece of DataMapper 2 but eventually it’s just a standalone library completely [inaudible] from ROM libraries. However, I do plan to build some kind of an integration between virtus and ROM. I think I already mentioned that we could, for example, use a virtus object and use its attributes to automatically generate mapping for ROM. That’s something that will simplify usage of ROM.

That’s very cool. You’ve basically built up this entire ecosystem of useful tools that can be mixed and matched. It seems like you can take a little of this, a little of that, and can help you in just straight Ruby or you can move into database with ROM and stuff. I just think it’s neat, this ecosystem of tools you’re building up.

Yeah. I actually really like that you used the word ecosystem. We’re doing exactly that. We want to build a huge ecosystem with a lot of small libraries that you can use whenever you need a small piece of functionality. We really don’t want to provide any kind of a huge monolithic library. Because that’s something really weird for me, that whenever people see a library that comes with a bunch of dependencies they are like, “Oh wow, this is so complicated. Look at all those dependencies.” But when you install just one library, you’re like, “Oh, this is a lightweight library. It has no dependencies.” [Chuckles]

But it comes with 1,000,000 lines of code. So, cool. How lightweight is that, right? So this is our philosophy. We like small libraries. It’s also much easier to develop a small library. It’s way much easier to develop a small library versus things like [inaudible]. Even DM1, which was split into smaller piece, dm-core, the core library of DataMapper 1 is still a pretty huge project even though it’s called dm-core. So we are really focused on building small libraries. I think it’s just much easier to maintain them. Another benefit is that once something is done, it’s done and it’s not going to change for quite a lot of time. This is something we prefer to do.

So I’ve got a question here. I’m looking at your example on rom-rb.org. It sets up the ROM environment to use a memory mapping or memory to store the objects. How exactly does that work?

This is the very basic adapter that doesn’t give you much, actually. It gives you a lot, but you can’t really use it in anything real because it’s just an in-memory store. You cannot persist the data in any way. You could, for example, dump the data into a YAML file or something like that. But we decided to build the memory adapter as a basic reference adapter where we could experiment with [various] interfaces. The thing about axiom-memory-adapter is that we can actually build ROM, add all the features that we need, and just test it against the memory adapter because Axiom basically guarantees that it’s going to work with other adapters. So this is what I meant by saying that ROM releases very early, because it only comes with the in-memory adapter. If you want to use it, you should actually build your own adapter that does something real. However, it’s still pretty cool because we can continue working on ROM, adding all the missing features like, I don’t know, session with unit of work for example, and just use the memory adapter.

Have you considered doing something like using the memory adapter for in-memory caching or things like that?

Not yet, to be honest. At least I didn’t think about it yet.

One of the things that I really liked about DM1 was that when I did go to create an adapter I found that there were basically four methods that I needed to write in order to create an adapter and they were create, read, update and delete.

How am I ever going to remember those?

[Laughter] And if you implemented those four, you had yourself an adapter. Are you trying to make creating an adapter similarly simple with ROM?

Yeah, that’s the plan. We still haven’t come up with the final interface for adapters. But it’s going to be as easy as in the case of DM1. Actually, it’s going to be even simpler, because in DataMapper 1 whenever you wanted to support some native features, I’m sorry, whenever you had to support something that your database doesn’t support you had to do it on your own. And right now with Axiom, it’s all supported out of the box in-memory. Actually, the simplest thing that you need to do in order to build an in-memory adapter is to implement each method. So that’s the simplest thing if you want just to read data.

So that you’re basically saying, if I understand correctly, is assuming I wanted to write the Redis adapter I would teach it how to write and save to Redis. And then things like joins would just work because they’re provided by Axiom.

Yeah, exactly.

That’s crazy cool.

Yeah. That’s definitely crazy cool. Whenever you need something that’s natively supported by your database, you will probably have some kind of driver that gives you that, so it’s also going to be easy. For example, MongoDB adapter, it’s using a MongoDB driver which provides you with lots of functionality that is native functionality in MongoDB. So you still don’t have to do a lot of work in order to support native features of the database.

So that’s one thing that’s interesting then. Let’s say that you just have to implement, you said, two or three methods in your adapter in order for it to work with Axiom, how do you make it work with all of these other features? For example, in Postgres let’s say you wanted to save some attributes as JSON and some attributes as just regular column fields, how do you tell it to handle that?

You can add custom attributes to Axiom itself. You could also define some special mappings on the ROM level and basically have the data dumped to a form that a database driver can digest. So that’s also related to mapping, how things are being mapped from database to Ruby objects and the other way around.

That’s awesome. Then one other thing I’m looking at here. It looks like most of the attributes, are they stored in their own little attributes spot or are they instance variables generally? It looks they’re instance variables here. And I know in Active Record, it’s generally pulling them out of an attributes hash or something that looks or acts like a hash.

Are you asking about ROM objects, your domain objects loaded in memory?

Yeah, your objects, yeah.

It’s currently handled by rom-mapper and it’s pretty easy. We actually have three strategies for loading objects from the database, or I should say from Axiom, from the Axiom layer. You can have an object that implements a constructor that accepts a hash with attributes. Or you could also allocate the instance without calling the constructor and just write into its instance variables.

Eventually, you have an object with instance variables.

Oh, I see. I’m looking at the example on the ROM main website in the second setup mapping. Yeah, it looks like you have some attr readers. So can you map any method that’s defined on the instance?

Right now the object will be instantiated with the attribute hash or it will use the attributes as defined in the mapping to write the instance variables.

I think Chuck was asking there about going the other way, when we’re going from Ruby land to database land. Can we just map anything that’s a method?

Ah yeah, sure. Yeah, that’s totally possible.

That’s what’s so awesome about it, both of those sides. It’s that when you get up to Ruby, it’s just a Ruby object. It’s just an object. But the trick is that there’s this mapper in the middle translating from the database and moving them into these objects using different strategies like Piotr covered. It’s making these objects. But when you’re done, you’re just playing with objects. It’s pretty cool stuff.

Yeah, and you can fold them around however you want and then you just tell the mapping what you’re interested in, which is really exciting.

Right. So if you have auto-generated fields that only matter while the object’s up in memory or whatever, just don’t map those back to the database and you’re good.

Exactly.

Very cool stuff. I think it’s very interesting. So I have seen one big thing missing. Maybe we should talk about that. Looking at all the pieces and stuff, what’s there, what’s coming or all that, the thing I’ve seen absolutely zero mention of is validations. Can we talk about that?

Yeah, sure. We actually started working on a project called, I forgot what it’s called right now. However, it started as an extraction from DataMapper 1. We took a validation library and refactored it. So it’s a standalone validation library that you can use with plain Ruby objects. And I’m not exactly sure how we’re going to integrate it with ROM itself yet. But I suspect one of the things that we want to throw out is actually using [form] objects, when you’re talking about web development and building things like Rails apps. Personally, I would rather use some kind of a dedicated object that deals with a specific input in a specific context instead of having some global validations that are being run everywhere in all sorts of contexts. So for instance, you could have a form object with for example virtus that will act like something that’s called in Active Record attribute accessible. It was moved to strong parameters recently. So you could have that inside your form object that will basically reject an entire input that is not known. Then you can have validations on that object. So once you instantiate it, your actual domain object, it will revalidate because only valid data will be passed down to the actual domain object. So this is how we imagine validations with ROM, like a separate layer on top of ROM objects.

I really like the idea of a separate layer. I often find myself wanting validation somewhere where I’m not active record-ish or even just not yet. Like in my current project that I’m working on, we have one model that’s particularly expensive to instantiate into the database because it’s very complicated and we have to look up a lot of stuff and things like that. So we handle it as a background worker. We take in the request, “Okay, we’ve got what you want,” and then just queue it to a background worker to build the object that we need. But in order to do that, we want to make sure that the thing we took in is okay before we tell them, “Yeah, okay, we got it. We’re good to it,” so we need that validation. So in the Active Record world, that’s weird because we basically build the Active Record object, ask if it’s valid, it is, then okay, then we take all the attributes out of that, queue them to the background worker, which basically ends up building the Active Record object and actually doing the same, which is weird and stuff. It’s that validation is like you said, we don’t want to use it with the persistence right now. We just want to know if this set of data is valid. Cory Haines and I have actually talked about that one time. We were sitting in a restaurant in Hawaii, I think, and just having a conversation about it. And he had mentioned that he had experimented with the idea of validations as a service, as a service object, where I can hand it this data and say, “Hey, is this okay?” or something like that. And I think what we’re talking about here, about it just being a separate thing, that sounds really appealing to me.

I really like your story. It clearly shows that the Active Record approach can be quite confusing and difficult to manage. And also something worth mentioning is that dealing with some [inaudible] input so close to the persistence like it happens with Active Record is just dangerous. So encouraging people to do things like, hey you can do User.create(params[:user]). It’s so insecure.

We should just have a separate layer that would just digest all that input and make it secure.

Right. You want to check input right when it comes into the system and then not worry about it anywhere else, which is what you were talking about with form objects. That seems like their domain. The data comes in from the form, populates some kind of form object, and we say, “Yeah, this is good,” or “Thumbs up, thumbs down.”

Yeah, exactly. And it’s also easier to manage validation context because in my opinion, validations are always contextual. It’s not like you always have the same context of a validation. You can have validation for data coming from a web form or you can have validation of some kind of [inaudible] sent by some client application that sent a request to your application. So it’s always contextual. Sure, there are a lot of common things between those validation contexts, but still those are separate contexts. So it’s just better to handle them separately.

That’s an excellent point. If you have the form on the user-facing interface, then there’s what the user can do. But if I have the form in the admin interface, then I should be able to tweak a lot more that they can’t touch.

Yeah.

That’s a cool thing. One other question I had is I’ve often wanted, Active Record takes MySQL as what I would say is the canonical representation and then frames everything in terms of that. But Active Record more so seems to take the view of lowest common denominator. These are things that all the SQL databases do, so that’s primarily our focus. And I realize there are some exceptions. And Aaron Patterson has actually been doing a lot of work to sneak in Postgres niceties lately, which is really cool, I think. But I’ve often wanted an ORM that just said, alright, Postgres, we’re going all the way down the rabbit hole, as many features as we can grab. Because there’s a lot of cool stuff in there that just has no support in these layers. And how is ROM going to address or deal with that? Or is it?

As I mentioned, we definitely want to support native features of various databases. I’m not exactly sure how it’s going to be done, because that’s basically Axiom related stuff and unfortunately I haven’t committed anything to Axiom yet. So I just don’t know how exactly it’s going to be built. But definitely, one of the most important things for us is to truly support databases and their native features.

That’s cool. There are just so many neat features in there. One time I had to do this horrible, I

guess it’s a data transform where I had to put it into a different format and it was complex to build up the format and stuff and then I wanted to just cache it somewhere. I tried to do it using a SQL database but it’s so complex to build up and I had to do so many queries back and forth as I did it that it was just horribly complicated and slow. And I switched to using SQLite and I made an inmemory SQLite database and did it that way, which was plenty fast enough and easy to manipulate. Then when I was all done, I just took a snapshot of that database, which is something SQLite can do and stuck it on disk. And boom, fully [hydrated] database. And it’s just sometimes those database-specific features are really cool and can allow you to use some pretty powerful stuff.

Yeah, that’s true. And unfortunately, in the Ruby community, it’s changing but for quite some time people actually didn’t use very basic features like constraints in databases. It’s changing. I’ve noticed that people are actually starting to use certain features. The good example is unique as validation. A lot of people used to just rely on the Ruby validation of some unique field. But then we obviously had issues. So people learned that you should actually use native features of your database. Don’t treat it as some kind of stupid data store because it’s a pretty powerful database engine after all.

Exactly, yeah.

Alright. So I’m a little curious. What features are coming up that aren’t in there yet are you most excited about?

Personally I’m really excited to work on session and unit of work because it’s going to be very challenging and it’s something completely new. At least I haven’t seen anything like that in Ruby. So this is something I want to add as soon as possible. But our focus is on SQL right now. So we decided to focus on SQL generator and integration with Axiom itself because right now we’re working again on a new small library called SQL and this library is a pure SQL generator coming with its own AST. So once that’s done, we’re going to integrate it with Axiom. And there are also some missing features in Axiom itself. For example, it doesn’t support server-generated keys. This is for things like auto-increment integer fields. So we don’t support that. Right now, if you want to use ROM and you want to save something, you need to provide the key. Another thing is support for relationships, and this is also something I’m very excited about. So one thing that’s going to be added to Axiom is support for nesting and unnesting data tuples. So when we load data that comes from, for example from a join query, Axiom will send it to ROM as a data tuple with nested tuples. So we could load the data in the same way using mappers, in the same way for relational databases as well as in case of any database that supports embedded objects. For example, MongoDB, where you can have a user with an embedded address. So Axiom with ascend the data in the same way to ROM in both cases, even if it’s a join. This is called nesting and unnesting data tuples and this is going to be a pretty exciting feature because we just need it to build relationships that work in ROM.

Nice, very nice. Well, do you guys have any other questions before we get to the picks?

I’ve got one last one. Tell me how to get mutant powers.

Oh, right. I almost forgot about it, actually. [Chuckles] Something that Dan started doing while building Axiom was something called mutation testing. Initially, he was using a tool called Heckle which gives you some basic mutation testing functionality. The concept of mutation testing is that you have a tool that changes your code at runtime, runs your tests, and if your tests don’t fail then you have a problem. Basically, it expects that your tests will fail if the code was changed. So we initially used Heckle for that. But it turned out it’s not very flexible and it required a lot of extra work when writing tests. So we started looking for alternatives and we found mutant, which is a new tool. Right now it’s maintained by Markus, one of our core team members. He rolled out the tool last year. Initially, it was only running on Rubinius and right now it runs on MRI as well. Mutant gives you mutation testing. So as I said, you run it and it changes your code and then runs your test. And if your tests fail, then it means you don’t have mutation coverage. So this is what we’re doing when building ROM and it’s super exciting. It’s a practice that really helps you in building super-focused tests that actually test things that should be tested. Then also, it’s also a great metric tool which is far better than line coverage tools. So something that we do is that we have a set of rake tasks that we run on master branches on Travis and one of the tasks is using mutants to run all the mutations against a given library and see if all the mutations are killed. So yeah, that’s mutant.

Awesome.

Again, just overall, just impressed with this kind of ecosystem that you guys are building up and I think I’m going to start dipping into the water more and trying these pieces outside of things and just get used to the whole thing and see how much of it I can use. It’s pretty cool stuff.

Yeah, I’m excited to play with it, too.

Mutant will be released. I mean, it’s already released but we’re working on a new version 0.0.3 and it’s going to be ready really soon. So check it out. It’s pretty cool.

Awesome. Alright, well let’s go ahead and do the picks. James, do you want to start us off?

Sure. As I said last week, I’m working my way through a whole bunch of picks that I’ve found recently and this week it’s blog posts time. Cool blog posts I’ve seen lately. Here’s one from Nathan Kontny called ‘Tracer Bullets’. And this is cool because I use this trick all the time and this is the first time I’ve seen any writing about it. There’s actually a gem to go with it. But basically the trick is just to grab a timestamp somewhere and then as I’m running along I’ll just periodically output how much time has passed since that timestamp. My co-workers find it strange because I output these outputs and they’re always increasing. So instead of saying I spent x amount of time here, x amount of time there, it’s just now I’ve spent this much time, now I’ve spent this much more time. But it really helps. I like the title he came up with, Tracer Bullets of just being able to find, wait, why did we have this huge jump over here? What’s going on in there? Then you can narrow it down. It’s often easier than having to wrap sections in start and stop so you can figure out the time and stuff. It’s really easy when you’re doing command line stuff because some line on the command line to test something out, you just grab that timestamp at the beginning and then subtract from current’s timestamp whenever you want. So that’s a cool trick. I like the blog post. Another blog post I found from Ryan Bigg is ‘Bundler local paths’. And I have no idea how I did not know this feature exists. Avdi knows because I see him in the comments, so I’m really mad at him for not telling me. But this is a great feature. If you have a project pointed in your gem file at GitHub but you have a local checkout because you’re making changes to that project, you don’t have to push up to GitHub and then fix the ref. Instead, you can just twiddle this Bundler config and tell Bundler, oh by the way, I have a local copy of this project and here’s the folder it’s in. Then when you bundle, it’ll just use your local copy. So that’s really awesome for when you’re fiddling with another project that’s a dependency of your project. Then in Postgres, we talked today about how I love all the native features of Postgres. Here’s another cool one I don’t see talked about too much, which is ltree. It lets you do hierarchical structures. So if you have something like comments and people are replying to comments or whatever, it forms this tree structure and this extension for Postgres called ltree can handle that for you. That’s pretty awesome stuff. And this shows you how it works and how to use it. It’s a good blog post. Then finally, the not to miss blog post, this time from Ernie Miller. He recently was unemployed and he talked a lot about what he wants from his next job and what he’s looking for and stuff. The joke I made at the start of the show, want to program wet and naked, that comes from this article. And it’s really great stuff. Ernie’s just a classic great guy who goes around telling us why we should be happy all the time and just really infectious attitude. So it’s a great blog post. You should go check it out. That’s all I got.

Alright. Avdi, what are your picks?

I'll pick a blog post too. There's a blog post called 'the church of interruption'. Actually, I don't know if it's a blog post. I think it might just be an article that somebody put up, but it's called 'the church of interruption'. It's about human communication and it's really good. I will also pick a game. Pretty much the only gaming I do anymore is on my mobile devices and I started paying Asphalt 8 recently. I've played a number of the Asphalt games because I like racing games, casual racing games. And I've been really impressed with Asphalt 8. It is I think the first time that I felt like I was having the Burnout experience on my phone. And Burnout for me is the top arcade racing game. Yeah, really fun. Very heavily into the whole try to get as much money out of you as you can philosophy of offers you lots of opportunities to pay for new cars and stuff like that. But you can ignore all that. Just have a lot of fun racing.

Awesome. Alright I've got a couple of picks. One is a blog post. It is the holy grail of ecommerce. It's a 91-point checklist for ecommerce. So if you're building a product, it's worth at least looking through these and seeing which ones you're doing and not doing, and which ones you're doing well and not doing well. And I've been enjoying it so far. It's really long so I haven't finished it yet. But so far, so good. I also have been working on this feed app that is supposed to fix all of our feed problems. And I needed a background job but what I needed was I needed something where I could say run this job at this time and just run one off jobs. So I’m using Resque and I’m using Resque scheduler. And the scheduler actually allows you to say enqueue at this time and then do this job with these arguments. And it's awesome. So really, really happy with that. And we’ll throw it over to Piotr. Piotr, what are your picks?

I had two, but now I have one. [Laughter]

My first pick was Ernie Miller's blog post. [Chuckles]

It's a great blog post, isn't it?

It's absolutely great. And the part about coding wet is just, I can relate.

It's awesome.

Next week, we're going to be podcasting wet and naked. [Chuckles]

And if you need an employee, go hire Ernie Miller, for goodness' sake.

Yeah, exactly. So my pick is an app. It's called Calca. It’s Calca.io. It's a pretty cool document editor. It's a hybrid between a text editor, a spreadsheet, and a functional programming language. So you can create documents with it and you can use its tiny little functional language. You can define variables. You can define functions. And I found it to be really, really cool for things like calculating some financial stuff for my private budget as well as my company's budget. So it's really cool. For me, it's much better than using just a spread sheet. So it's an online app. Also, there's an OX X app and an iOS app. I don't think they have support for other platforms unfortunately. And yeah, that's everything I have.

Awesome. Alright, well looks like we're going to wrap the show up then. Thanks for coming, Piotr.

Thanks, thanks. Super happy that you invited me.

Well it's been a fun talk. I'm really excited to see where ROM goes. I do want to remind everybody about our book club. We're reading 'Confident Ruby' by Avdi Grimm. Anyway, it'll be in October. We're going to be reading that book so pick it up and read it. And was there a discount code, Avdi?

There is. It's ROGUESCLUB, all caps, all one word. 20% discount.

Woohoo!

It's a good book. Go read it. I love it.

Alright, and with that I guess we'll wrap this up and we'll catch you all next week.