Riak Core and Partisan with Mariano Guerra - EMx 269

In this episode of Elixir Mix the panel interviews Mariano Guerra. Mariano wrote some wonderful tutorial blogs outlining how to use Riak Core in Elixir. He begins the episode by sharing a little about himself and his work. Mariano then defines Riak Core and tells the story of where it came from. He explains what he loves about Riak Core and dubs himself the unofficial cheerleader of Riak Core. Mariano tells the panel about his blog articles and what listeners will find in them. He explains to the panel what inspired him to write them. Mariano then gives the panel examples of the problems solved by Riak Core and the best use cases for it. Partisan is the next subject the panel asks Mariano about. Mariano shares the story of where Partisan came from and explains when you want to use it. Finally, Mariano tells the panel about his work for the Erlang Ecosystem Foundation. Right now he is working hard to unify the documentation for all the Beam languages. He shares his admiration for the Elixir documentation and explains that Erlang documentation needs a lot of work. The panel discusses how unifying the Beam will help the community and make their lives easier.

Hosted by:

Show Notes

In this episode of Elixir Mix the panel interviews Mariano Guerra. Mariano wrote some wonderful tutorial blogs outlining how to use Riak Core in Elixir. He begins the episode by sharing a little about himself and his work. Mariano then defines Riak Core and tells the story of where it came from. He explains what he loves about Riak Core and dubs himself the unofficial cheerleader of Riak Core.
 
Mariano tells the panel about his blog articles and what listeners will find in them. He explains to the panel what inspired him to write them. Mariano then gives the panel examples of the problems solved by Riak Core and the best use cases for it. Partisan is the next subject the panel asks Mariano about. Mariano shares the story of where Partisan came from and explains when you want to use it. 
 
Finally, Mariano tells the panel about his work for the Erlang Ecosystem Foundation. Right now he is working hard to unify the documentation for all the Beam languages. He shares his admiration for the Elixir documentation and explains that Erlang documentation needs a lot of work. The panel discusses how unifying the Beam will help the community and make their lives easier. 

Links


Picks


Transcript


Welcome to Elixir Mix, your weekly Elixir podcast talking with members of the community. My name is Mark Erickson, and today we're joined by Josh Adams. Hi. Sophie DiBenedetto. Hi, guys.

And today we have our special guest, Mariano. Hi. So, Mariano, before we jump in and talk with you, which I'm excited to do because we have a lot of cool stuff. He's just to give, an idea what we're talking about, he's done some one awesome articles about React Core and Partisan, which are two extremely cool technologies that are in the Erlang e and Elixir Ecosystem. But before we do that, we gotta say hi to Sophie.

So Sophie was a guest last week and we're, inviting her to join us as a a guest host today and see and and see if she see well, she'll get to test out if we're nice enough to hang around with for a little while. So so if you want you yeah. So why don't you, just kind of give a little introduction to maybe a little background to just, for people who may be new to you? Yeah. For sure.

First of all, thanks for having me. Super, super excited to be joining you guys. I guess a little bit about me, I've been working with Elixir for a couple years now, both just kind of personally for fun side projects when I was first learning and then more recently at the Flatiron School where we've been doing some really cool things, I think, with Elixir in order to provide our students with an in browser, sort of, like, very full rich development environment. I am also part of the MPACS NYC organizing committee, so I'll probably plug that every now and then. You guys will hear a little bit about it.

And, yeah, currently sitting here in Brooklyn, staring at my dog, who is staring at me, and that's gonna be happening throughout the course of this episode. I know your dog is is somewhat famous, makes a number of appearances in your presentations. Yes. Very true. That's, credit to my partner who is an artist, and she draws all of the illustrations of him for my, slide decks, which is a really nice way to get, like, a very polished look for free.

Very cool. Alright. Well, Mariano, I would love to hear a little bit about where you work and kinda what you're doing. Could you give us a little bit of background? Sure.

I'm cofounder of a really small start up, called InstaTech. We build, tools that let non programmers or people without skills be consumed data, analyze it, aggregate it, change it a little bit, and build visualizations, live visualizations, and publish them on the web. And, our back ends are all the the first version was Erlang. The second version was FNA, and the third one I'm starting to develop is Elixir. I'm I'm going through all the languages on the Beam.

And so, yeah, that's kind of where I work. I work from home. So Nice. Alright. And so I'm curious because you have had a lot of experience with the Beam, are you currently doing anything in your, projects that you're working on that you're experimenting with React Core and Partisan?

Yes. I the previous back end was built on React Core. And so, yeah, it run for 2 years in production. And, the the the current one is, not using a it's, it runs on partisan, but it doesn't use any, like, cool features of partisan. And the new one, the one I'm building with Elixir, I'm planning to use partisan, heavily and maybe not record.

I have an idea for for a new way of of doing distribution of load between nodes. I will that's why I'm also interested in TLA because, when you're working with the, like, distributed system algorithms, you really want to be sure that not only the implementation is right, but also that the the algorithm is right. So I'm going to be basing, the new architecture, on on parties analytics here. Cool. So what is TLA?

TLA plus is a a a programming language, a tool, for temporary logic of actions is created by the creator of, of, LaTeX and also the creator of Paxos, the distributed consensus algorithm that kind of kicked off all the movement about consensus algorithms. And, he was fighting to to get to marry, like, the the succintiveness, do you say in English, of mathematics and expressivity of set theory and set algebra with, something that can be run and can be used to prove algorithms. So he created this language called TLA plus, which is a language used to model algorithms, and the the the tool has it's kind of like a programming language, but it looks a little bit like math. And, you use a a model checker that checks that the the model you specified of your algorithm is actually right. And it has, like, a brother or sister called, well, I forgot the name, calc something, which is a language that looks more like a programming language that compiles to TLA.

And it's, it's I find it useful to as a there's an expression that says that, you should learn a programming language if you change the way you think about programming. And I've experienced that learning, like, c, Python, Erlang, Lisp, and now TLA plus. I I I've been following, like, I I've read 2 books. I've read all the blog post online. I follow the the series, that Lampert provides on his website.

But I never used it to specify anything, but already learning about it and seeing examples already makes me think, clearer about the design before implementing it. And I want to actually try to use it to specify algorithms. Interesting. I that that's really cool. I I think, I I totally agree with you that, there's a big benefit to learning new languages or new frameworks or anything if it causes you to think differently.

Just kind of challenges your mental model. So I would love to jump into, React core. Maybe you can give us, so maybe I think it might be a new a term for a number of people. It's so maybe you can just give a little background about what it is and kinda how it works. Sure.

So I will go with, like, a little bit of history of why it exists. Many years ago, in in software terms, Amazon published a paper core called called usually the Dynamo paper, which specify the architecture they use for for their, scalable, highly available, not consistent, data store, key value store, and they specified what it's is called the architecture. And later, the people at Vassar, a company that no longer exists, decided to create a key value store, an open source key value store based on that architecture using. So they created React, which is a key value store that still exists. Now it's, like, in in a in a foundation, in a community, and it's still being developed and used.

And and they they used this paper and implemented, the the the architecture in Erlang. And at some point, they said, like, there's value in in in this architecture outside of React. So they decided to extract the the core of React into a framework of its own. And so they announced and published React Core as an open source project that's separate from React, which React uses, of course. And, other people used it.

You can check, the one from the top of my mind. But on the last chapter of of this series resources, there are there's a list of of many things that use reacore, but DalmatinoDB, which is a time series database, uses it. And so many people, started using it to to build distributed systems that follow this dynamic architecture on the Beam. And me being interested in distributed systems and being interested on the Beam, I started trying to use React Core. And at some point, like, it was, hard to get documentation.

So at some point, I started doing it. So, basically, it was, like, extracting the framework or the the the architecture out of React, which, which implemented a paper called Dynamo, the Dynamo architecture. And, yeah, after that, it became its own thing, and, I've been part of trying to make it more popular because, I don't know if people like it, but I call it like the the Rubion rails of distributed systems. It's the the the easiest way to build together with Microsoft Orleans. I think they are the 2 easiest ways to start building actual production ready distributed systems on top of giants.

Like, you get the foundation, you implement, like, the callbacks. You have, like, a skeleton of the project, and you just put your code in places, and magically, you have these properties of of these systems. That is really cool. So thank you for that little, background. So Basho was the, an Erlang centered, consultancy, and you mentioned that it went out of, kind of they went out of business as a a as a going concern, as a as a as a business.

Yeah. But what I'm really impressed with, like so you we have a link to the show notes, to the GitHub for React Core. It is under the Basho GitHub organization, but this is still actively being developed. Like, as of this recording, the last commit was 18 days ago. And so I just want to people to be aware that even though it was created by a consultancy that doesn't exist any longer, it still is actively, moving forward, and and progress is being made.

Yeah. I can add some some context. So, basically, Briak is being used by many big organizations, like the NHS in the UK and BED 365, to name some I can remember. And I think also the Danish health ministry or something like that. So when when Baso went out of business, these these organizations wanted React and React Core to keep being developed.

So Bet 3 65 bought Basho, like, what what was left of the company and open sourced everything and created kind of, like, a structure. I don't know if it's a foundation or what that employs people to keep developing React and React Core. So React has new features and has new releases, and React Core has new releases. And what what happened, when, like, when Basho went silent and, up until a couple of months ago is that some people did some forks. And now the the, like, the work that's been done in the communities to, like, to merge all these forks and to have, like, one official fork, which is the Basho ReCore before people may if they search for ReCore on HEX, they will see ReCore, ReCore NG.

And I have some forks too, and many people have forks. Some research, universities, one in Germany and one in Portugal have their own forks. And also, Christopher Michael John has his his own fork. We all have, like, our own forks. Now we are trying, like, to merge it back to 1 because it's really confusing for people.

We are trying to do it the right of these 3 systems, and when they search for record, they find, like, 5, 6 active forks. So you'd say you're in marketing? Yeah. I'm I'm, like, the unofficial, PR of Christopher Meiklejohn and, like, the unofficial, cheerleader of Reactcore, let's say. Yeah.

That's that's awesome. He needs he needs PR. He needs a cheerleader. He's awesome. So one of the things I wanna highlight and one of the reasons, we wanted you, come on and share some of this information with us is you did a a a series of blog posts talking about React core and part on partisan.

So using partisan is, like, the node connection, mechanism, and doing an Elixir tutorial, which I think is really great because there might be a lot more documentation around using React core with, Erlang just because of its history. So I think it's great that you're doing some coverage of how we can use Elixir with that too. So maybe you could just give people an idea of what, if what they would find by looking at the series of blog posts. Yeah. So my the the biggest, I think because I I I started doing when I was implementing the work in on real core, I I started, like, documenting my discovery process because it was for me also learning.

And that was, like, a really long read me at some point, and then it became, like, the little reacore book. And, at that moment, reacore, with all these forks and with, also with Erlang evolving really fast at that point. Now it's kind of stabilizing. The code examples I was publishing were not building anymore. People had problems with dependencies and stuff like that.

And, so I I I did like a new update that was called the record tutorial was which was basically just the code examples of the little record book but updated so that they build and run again. And, I noticed that all the time the the the difficulties of people was basically setting up the environment and making, like, the ping pong example. After that, the hard part was just basically knowing what code to put where and what each callback meant. And the 3rd big big, roadblock, but if you arrive at it, you're almost done is, how do you handle call what's called hand off, which is how do you might live migrate data between minutes. And so I always focus on the on the setup, like get a project, running and understand the concepts.

And, when when people were trying to to use my code examples in Erlang by migrating to them to Elixir, they they mentioned that they had really weird problems that I didn't have, and I didn't know why. Then later, I learned why. So I say, okay. There there's this need. And and there was a a tutorial, a series of medium blog post, by it's I think it's GPAD, the the handle on Twitter, on real current Elixir.

But also they got update outdated, and also they were just the code of the callbacks, not like the project that you can clone and try to run. And so I tried to update it again and gave a tutorial at, Elixir conflate in in managing this year. And next week, I'm giving it to this one too. So I already forgot what you what your question was. Yeah.

So we were just discussing kind of, what some of your blog posts cover. Oh, yeah. So so what what it covers here is still a short mention of what it is. I I will I will add this link to to that. And and then how to set up a new project and and then what the callbacks are and what code to put to build that in memory key value store is an ETS as a as a back store.

And the the last one is how to implement and hand off to migrate these, keys, from 1 window to the other. So that is to be really practical and to to give you code, but complete code, not just like snippets that you don't know where to put or you put somewhere and it doesn't compile because something is missing. I want to give you a whole project. So while I also build a companion project that's called CVV, which is a a value store I explained, but each each step on the guide is a commit on the project. So you can just check out that that commit and see the history and see the difference, see everything I'm telling you is actually a complete project that builds.

I hope it keeps building. For that, I build a dependency that's called I don't know why it's called, because it's, because the the fork from Christopher for React Core is what's called, which is Spanish means to join. And means joining, just a play for it. Which which this library does is like an umbrella dependency that brings all the dependencies in the right order, in the right versions with the right overrides so that you just get one dependency and your project gets record. So I think that was one of the biggest, additions to to all the documentation that there is.

And the other is, what I'm trying to do right now is to build like a Docker image and maybe a Docker compose so that people play with clustering without without having to do the tricks of of having different environments to have a cluster on the same machine without clashing ports and stuff like that. Something I think people would like to understand is when they hear about React Core and they're thinking about their own business situations or their own projects or problems, their the big difficulty may be understanding how does this fit and what kinds of problems is this really good at solving. So I wonder if you could speak to that. Sure. Yeah.

Since it's set to, like, a generic architecture, it's like, if somebody says, like, solve this problem using, I don't know, 3 layers architecture. It's really generic. It doesn't tell you much. And to know if it applies it and it doesn't apply, I think, examples and, like, description of generally what you want to do. The problem that you should have when you try to use React Core is, you want to build an application that you can horizontally scale, without masters.

Usually, there are there are masters of primaries. There are 2 type type of distributed systems. One's where each node is equal to all the others. And when you want to scale, you just add a new node and the nodes, like, rearrange, who owns what. This is the dynamic architecture.

And you have other architectures that are in the in the middle if you read the big table, paper by Google, where the cluster is split in 2, like, in 3. You have, like, a, which is a consensus cluster with 5 nodes. You have, the primary or master. It's called master in in that, paper that knows and then you have, like, worker nodes, which handle work, but, the one who decides who works on what and who wants what is the master. And whenever you want to to do something, you you have to ask the the primary, like, who owns this?

I I need to do something on this key, who owns it. And it gives you the reply. It says no number 5, and then you can cache that resolution and and talk directly with no the with no number 5. And at some point, the node number 5 might tell you I'm no longer working with that key, and you can ask again. And so that's like like recapitulating if that's an English word.

Like, you have this, primary worker architecture that's also distributed systems. But and but you have this more distributed system architecture where where all the nodes are are equal. And, basically, they they work, with each other to distribute the load and migrate the data and decide who owns what. Reactcore is the later. You don't have this primary worker or primary replica, distinction.

And and you want to you want to build something that, scales horizontally, as I said. You just basically, when you want to scale, you add a new node. This new node joins the cluster, and the other nodes give them slices of the, let's call it, namespace. Like, you are working with keys, then maybe a key value store, then maybe a pops up system. It may be an MQTT broker.

It may be be like a Kafka clone. It it may even be like a like a chat system, like a video game. How do you call it room where, like, you join and you find people and stuff like that. Usually, something where where you have a key that identifies what you want to do, and you you dispatch an operation on the key and the the data for the operation. And, and then the system decides which node works on that.

So I already mentioned some examples like, you could implement something like an MQTT broker with it. Something like RabbitMQ, key value store, a pub subsystem, a web socket system, a chat room system. Anything that you can you can say, okay. When I'm doing this operation, this key identifies like, it's like another identity of the thing. It can be the username.

It can be the session. It can be the channel. For for this for this podcast, it's the the meeting ID we are using. So we will all land on the same server, and that server server will keep the state of of our meeting. So it sounds like so it sounds like it's a, masterless, top ring topology Exactly.

Where it's like, it's using, like, a hash space. Like, so I might hash the keys and those get distributed among all the different nodes. So I can just scale up by adding more nodes and there is no master. They just kind of, like, repartition themselves. Exactly.

And so then when I'm able to so, like, is the is a data value stored in multiple nodes in case one of those nodes goes down or is temporarily unavailable? Yes. That's called quorum based reads and writes, which is support where we occur. Luckily, you you have to implement, like, one thing and you are there. Basically, what you say is, okay.

I want to do this operation, and I want to send this operation to 3 nodes. And I consider that this operation succeeded when at least 2 nodes replied. Or you can say, I want to send this operation to 5 nodes, and I want I consider it a success when 3 replied. Or you can say, I want to send to 1, or you want well, you can say, I want to send to 3, and it says when the 3 replied. So you want to be really sure that it was written.

And that's, that that level of consistency you decide. It's your it's your, like, from your business, domain. But, the the framework gives you, already this. You don't have to think a lot about it. It's already supported.

Yeah. I always I always thought of, React Core as back in the day, there was the joke that, any Basho presentation must contain the golden hash ring slide. Yeah. It's a secret. I always think of React Core I think of React Core as the library that implements that slide.

Exactly. That that that ring is, maybe people will understand it if they never saw the, like, the ring or don't know what consistent hashing is. Some systems or some architectures support what's called sharding, which is basically when you scale, you decide, like, a slice of your key space. For example, the users from the a to the, I don't know, f, go to this server, and from f to c, go to this other server. You do the partitioning, the sharding manually.

You specify, like, the the ranges, and you configure it, and you bring new servers, and then you migrate it somehow. Like Reactcore, this ring is basically it's like auto sharding. And how it does auto sharding, trying to maintain a property which is each server on the ring should theoretically, receive this almost the same amount of requests as any other server. You don't want one server to be handling, like, twice or thrice the amount of requests the other one. So what it does to because React Core doesn't know if your key space, for example, your usernames, you may have more usernames with a than with, I don't know, x.

It what it does is you give the key, the username, and it hashes it. And this hashing operation gives you an, uniform distribution over the, like, the the space of of possible values. And that means that, like, statistically, all your notes will receive the same amount of requests unless, like, Justin Bieber appears. And and, yeah, it's like out of sharding if people understand what sharding is and don't know what consistent hashing is. Maybe it'd be good to jump in, just mention what partisan is.

So that's another library that you're using with this. And this so partisan, I just wanna make sure people understand, can be used without React Core. It is a standalone library, but maybe you could help us understand what problem that's solving. Sure. So, a little bit of history again.

I I like to to to go over why something exists. So, the beam provides distribution using something called distributed, which is a protocol that allows you to cluster, Veeam virtual machines. And it provides this property that you get a PID, a process ID, and then you send it a message, and the distributed will do the work of finding where this PID is and deliver the message. How it's all this when you join a virtual machine, a virtual machine to a cluster, it will do what's called a full mesh, which is every virtual machine knows about every other virtual machine. And, there's a property called, Metcalfe's law, which is like the the like the number of connections between nodes grows, I I think exponentially with the number of nodes.

So for a small number of nodes, you keep, like a small number of connections because each virtual machine keeps an open socket with each other virtual machine, and it sends pings to know if they are alive or not for monitoring and stuff like this. And this has it works great. It's transparent. You send a message. It's it's there.

You can monitor on other machines, and you get a message when they crash or where the node crash and stuff like that. But this this has a theoretical or practical limit that depending on the workload you're doing between 50 and a 100 nodes, your cluster will start to the great performance because, like, if each node has 50 TCP connections to other 50 nodes and they are all pinging all the time, it's a lot of it's a lot of work just to be a cluster, and you have to add the workload of your application on top. And other problems, like it's just one all the messages from one node to the other to queue the serialization. Order of the messages is sent via one socket and up until, airline 21, each message was sent as one thing. So if you send, like, a 5 megabyte binary, it generated something called head of line blocking, which is, like, all many small messages will, like, block behind this huge message up until it was delivered and and confirmed.

And this was improving in our in our 22 where they split messages in in smaller chunks. So this is, the performance of of this unit that are not improved with this. But still, if you if you go to parties on the cloud, you can see the paper and the benchmarks where, I'm getting ahead of myself. So partisan says, look, we want to try different, like, different ways of clustering NERLYN nodes, different architectures. And 1 we want to try different algorithms and different distributed, strategies because sometimes you don't want to keep, ordering between messages that don't care about each other.

For example, you may care about all the messages from one node to the other to be ordered, but only for one username or for one session ID, but for the others, you don't care. Or you want to have 2 channels open, 1 to send big payloads and the other one to certain hardbeats and stuff like that. And so what Christopher built on his ongoing PhD thesis is like a artisan, which is reimplementing distributed as a platform where you can plug different algorithms and different strategies and different membership, algorithms, at runtime. So you you always use the same API, but at runtime, you can change, how do you want to, like, cluster the notes and how you want to keep the membership information and how you want to route the messages. And and with that, they, you can choose, like, fine tune, at run time, what, what's better for your workload.

And also, because this is implemented in CEE. It's and it's on the VM. It's not a place where you will go to make changes or put, like, a logging line or put, like, a trace thing because it's, like, it's it's you don't want to be touching the VM. Also, because if it crashes, it crosses everything. And the partition is implemented in our line.

So it's, like, it's cool when when the foundation you are on is in the same language you're using because you want to, like, start traces and you can trace and you can touch the code and try something new. And, this is, like, how partition started. It was like, what if we make a pluggable distributed airline? Basically, you we can try different things. We can use it as a research instrument for people to try new algorithms, new papers.

What if we try this workload? And what what he did later to to run the benchmarks on these papers is to migrate real core to partisan so that that he could implement a key value store and benchmark, like, the key value store running on real core on distributed against, running on partisan and what was the difference. And the difference is crazy because you're you're comparing, c implementation of distributed airline tune for 30 years against an implementation of distributed airline built by a PhD, and the performance is better on the partisan, which is really cool. That is cool. So that is there's a lot there.

And so I'm I will make sure we put links to all of that in the show notes. There are just so people are aware, I encourage them to go check out your articles. You've created, 6 at least so far. And, so I'll have all of those linked to those blogs. But also, I want to make sure people are aware of you you did mention this already, but you have an open source project, Civil DB that people can check out and experiment with so they can actually just get it running in a and see the code how it's all put together.

And so I think it's an awesome resource for people. Like, if you are interested in starting to play with Partisan or Reactcore and you'd like to understand how these pieces work together and how you could, benefit or build a system like this. Just be able to more quickly experiment and see if this is appropriate for your situation. It's a great resource to check out. Yeah.

I'm actually really excited that that you did that and also that apparently it's not nearly as much trouble as it used to be when I was first playing with React core, because, I had to use, like, a 3 release old version of Erlang to even run anything that anyone had online. Yeah. That my my main contribution is to try to make it easy and more accessible. Like, I'm not I'm not the guy writing the papers. I'm not the guy, like, doing the crazy implementations.

I I'm I just when I when I see something that's really good and not many people are, like, finding, I my my part is just trying to make it more easier to try, to understand, easier to follow, and stuff like that. So Accessibility. Accessibility. Yeah. Well, maybe we could transition a little bit.

I want to talk to you about, what you're doing with the Erling Ecosystem Foundation. So maybe you could just mention a little bit what you're doing there. Yeah. So, I'm a member of the working group on documentation. We are trying to to unify a little bit the the toolset and the and the work of to start Erlang and Elixir so that you have the Erlang documentation available on Elixir tools.

Also, to try to because we we I I come from the Erlang community. We learn a lot from leaks here, and you have beautiful documentation. So we want to try to make use of all these tool chain tool chain, tool sets, and styles and and and guidelines and stuff like that. So, we are working on ways to translate because the airline documentation is, like, so shows it is built on XML and and, with really long and weird tools toolset. So we are working on translating those XML files into something that can be consumed by the Elixir toolchain so that they it can embed documentation on the Elixir ecosystem, but also so that we can render the Erlang documentation on the kind of Elixir style.

Not not necessarily like ARPUL, but maybe red or something, but, but as nice as the Air Leakster documentation looks like. And this is, like, the the the thing we're working on because the working group is is young and new. But after that, we are going to work on on everything that has to do with, making it easier to consume and build and share documentation on the beam, basically, and not duplicate, effort when there's no sense to do it. And the second is, the language interoperability working group of which I'm well, I will be if approved, the the chair. And this is similar, but instead of, instead of being about documentation, it's it's, about, making it easier also to avoid duplication.

Like, the the Airline language already has a standard library, and the Elixir library has a smaller standard library, but really powerful. And, what to I I'm talking personally here, not for the other members. But what I would like to have is that nobody else on the ecosystem has to create a new, like, enum, or a new list data type or a protocol that they can reuse the Elixir ones or they can reuse the Erlang ones so that whenever somebody publishes a library that's useful for me, I don't care which languages. Maybe I care, but I should care the least possible so that we can move forward faster without reinventing the wheel. And, also, personally, I I think it's, interesting for the Beam to be a a platform to try new things.

Like, I really that's why also I do the reoccur and the parties and things because I I would like Erlang to be, like, the target platform by default to try new distributed system things. And I would like also it to be, the the target platform to try new programming language paradigms. What, how would it look like a programming language that abstracts over distribution and stuff like that where, Christopher work on last, which is kind of like that. But I would like to push it so that if you want to try a really crazy programming language concept, you have so much already fit, like, like, kind of like, but for languages. And what we are working at the beginning is to make it easy for people to use Elixir libraries on on, for people to use, Elixir structs and protocols on other languages in general.

And, what else? Those are, like, the and also the unified there there's many people working on the on, IDE integration for languages, for Elixi, for Erlang, try to get them all together and also not reinvent the wheel so that there's a platform. And when somebody does, like, a Hobi programming language for the Beam, they just plug into some callbacks on the language server, and they get instant integration with, with an IDE, with resource studio code, for example, and they plug in something, and they have nice documentation. So we want to make it easier to share, inside the platform. Really cool.

Yeah. Having read your, blog post on Reactcore just ahead of today's conversation, I'm, like, not at all surprised to hear that you're involved with the documentation working group because I feel like your writing had this really nice, like, sort of didactic flavor to it. Like, all the setup stuff and all the dependencies were taken care of. And, you know, having been a teacher myself, I'm, like, always trying to strike this balance between giving you everything you need so you can focus on the actual hard part, you know, without making it too easy. Yeah.

So it's really cool and exciting to hear about what you guys are doing by way of documentation and some of this, Alexa Erlang, interrupt. I mean, I think you sort of already spoke to this, but I'd just be curious to hear your take on, you know, why this is something that we need in our community and the Alexa community and the Beam community. I think, like, I was really I, as I said, I was, part of I'm I am part of the Erlang community before Erlang community existed, actually. And, I was, there was a lot of people and new people coming in, like, complaining about the the problems that Erlang had, which were kind of all solved by Elixir in a sense. But at the beginning, some people on the Erlang side got defensive, unlike, they are stealing our thunder.

Let's say, they are stealing our our virtual machine and all our libraries, and they are saying that it's like a new language. And there was something to that. Like, there were people, like, wrapping libraries, with, like, 15 lines of code and not not mentioning the the underlying library and stuff like that. But after that, I think the airline community understood that Elixir was this breath of fresh air and this impulse. I I I I I don't think I'm saying something new when I say that the Elixir community is bigger than the airline community.

And, and this process of, okay, they have something to teach us, and they are building stuff that's really cool, like like structs, like protocols, like documentations, like, also culture, like, attitude and and attitude to beginners and and ease of use and stuff like that. And, I I guess, to generalize it, to to make it work also for smaller communities like LFE and any other new language like clean, like alpaca, and closure, which is a closure implementation from Erlang. We want to everything that that the Erlang community has and learned and everything that the Elixir community built and knows and learns to to unify or refer to not duplicate, to not fight, to to push forward as a platform, and to provide all of these learning and resources and tools to these smaller communities that are trying some cool stuff, so that they don't have to, like, implement the language server, implement the documentation tool chain to implement to implement, like, integration with the documentation from airline and stuff like that. And to make, like, like, the beam community be something that where we are all like, we are somebody may be part of the gleam community, but we are all part of the of the beam community.

And and we don't waste it for because there there's a lot yet still to do, and and it will be a waste of effort if, every single community is reinventing everything. And if we are also, if you are if we are noninteroperable in the sense that I see a cool library that implements, like, some crazy algorithm, but it's implemented in our pocket. So I have to, like, implement it again because I can't use it. Yeah. I'm definitely pro anything that's good for the community and allows me to be lazier.

So Yeah. Me too. Alright. Well, I think we're up on time. Thank you, Mariano, for coming on and talking with us.

So let's go to go ahead and transition to pics. Josh, do you wanna go first? Sure. I need to grab the link, but there's a talk that I found out about when we were talking right before this podcast, and that's my pick. I have not watched it yet.

But, it is rewriting GitHub Pages with React Core, and so it's on, like, today's watch list. Anyway, that's it for me. Sophie. Yeah. I have 2.

So one is a blog post that a coworker of mine just published, on our sort of engineering team's blog on Medium about how to compose queries in Ecto. And I just thought it it really made something quick for me because I feel like it just took me a really long time to kinda grok Ecto and and kinda have it feel as natural as working with someone like ActiveRecord. And I feel like I've built, you know, composable queries in ARAL and, you know, really loved how flexible and dynamic that that kind of was able to be and didn't really feel like I could get to that same place with Ecto. And this was just a really nice post that felt like a fun read and helped make a lot of things click for me, so definitely recommend checking it out. And, secondly, non programming pick.

So I should preface this by saying I I really love to cook, but I don't like to bake. It's too exact. You have to follow a recipe, and I'll mess it up every time. But someone recommended this, very fancy looking burnt basque cheesecake recipe to me, and it is the absolute only thing that I've ever baked that came out looking exactly like the picture. So whether you're new to baking or you just really wanna impress your friends at a holiday party, definitely check out this recipe.

Cool. Alright. I've got 2. 1, it was a video I watched yesterday. It was a it's a 10 minute video on YouTube that was with, Francesco Cessarini.

He's with the Erlang Solutions. And it was a great art a great little video. He's talking about immutability. And immutability is especially when people are coming new to, functional programming and that kind of mindset, they can really struggle with it. But he kind of goes through how it works, the benefits, and along with mutability there are times when you want to use mutability or immutability in a different situation.

So he kinda talks about those. So it's a great resource. The other one, I got this from a a coworker. I love this. Is it's a a declassified short CIA manual on sabotage from 1944.

So immediately, there's a whole bunch of it that is just not applicable. You know, it's like, you know, talking about how cars work and you can how you can pull certain cables off to make it not work or something like that. But this there's a a section titled General Interference with Organizations and Production, and that is totally relevant today. Now I'm not suggesting you sabotage your organizations that you're in, but instead, check it out and think about how the of the people and the places where you've worked and see if you see some of these same behaviors. So I'm just gonna list off a few of them because they're they're funny.

So one says, insist on doing everything through channels. Never permit shortcuts to be taken in order to expedite decisions. Or when possible, refer all matters to committees for, quote, further study and consideration. Attempts to make the committees as large as possible, never less than 5. So if you start to see these kinds of patterns in companies where you work or anything, it's like you can kind of read this and say, oh, I should do the opposite.

I should make sure we're having small committees and people can take shortcuts to make sure things get done faster. And, so it's just a fun one. Like, another one is bring up irrelevant issues as frequently as possible. Maybe you know that guy or gal who does that in the meetings, you know. Maybe you're that person, so maybe But also, if your goal if your goal is to sabotage, it's also a useful resource.

Yes. So those are 2 fun ones to check out. Mariano, how about you? Yep. So, as I mentioned previously, I have to say first, I'm not a TDD guy.

I'm not a test first guy or a heavy unit tester. But I'm really interested lately in in, like there's, like, a spectrum of testing from unit test, integration tests. But I'm interested more in the in the other side of the spectrum, which is a property based testing and, specifications. And so for property based testing on Erlang, you can use proper, and I think there there's a wrapper for Elixir too. And there's a really good book, called property based testing with proper Erlang and Elixir from Fred Hebert.

I hope I pronounce his last name correctly. Which covers how to use proper to use property based testing with the library called proper. And farther right on the spectrum is, TLA plus, which I already mentioned. And the book is called Practical TLA plus by, Hillel Wayne, and, it teaches you how to use TLA Plus and, this other language that I'm gonna remember the name. Let me see.

Plus calc plus calc, to specify, algorithms in in, like, increasing levels of complexity and to use the the model checker to check if your implementation is correct. So, is as I said, you I guess everybody should explore the whole spectrum of how to test and specify specify, your algorithms and find, like, the sweet spot. I think all of them are useful. I think the the farther, like, to the specification side you go, the more, it's a a tool of thought of design. More maybe you don't, like, write the specification or maybe you don't keep it up to date.

But the fact of trying to fit your mental model into, like, a a model that can be checked mathematically, even if it doesn't even compile, should give you a better, a better model to start with than, like, some boxes and arrows on a on a blackboard. Great. Alright. Well, Mariano, thank you for coming on and sharing with us, how people can be using React, React Core and Partisan. And for your blog articles, I appreciate that.

So if people would like to connect with you online or follow you, where should they go to do that? I'm, Wariano Guerra, like my name, but with the w at the beginning, on Twitter. And I'm mariano Guerra at GitHub, and I'm mariano Guerra dot org on on the web. I in my Twitter account, I talk mainly about software. Like, it's another I don't talk about other things.

Maybe I love music and politics sometimes, but mostly about software. So it's a a central point to follow. And my my blog is mainly post technical content, so both of those. And my GitHub is full of repositories that are long dead, but you can check there are some active ones. Great.

Well, that's it for today. Thank you for listening, and we hope you'll join us next week on Elixir Mix.
Album Art
Riak Core and Partisan with Mariano Guerra - EMx 269
0:00
46:19
Playback Speed: