Takashi Kokubun is a staff developer at Shopify. He has been working with Ruby’s MJIT compiler for over 5 years but has been recently maintaining YJIT as well.  JIT compilation is a method of running computer code that involves compilation after a program has begun running rather than before. He joins the show to talk about these topics alongside Chuck and Valentino. He also explains their importance as this contributes to running Ruby applications smoothly. He also shares his experience working with rust and creating HAML 6.0.0. 

Hey there and welcome to another episode of the Ruby Rogues podcast. This week on our panel we have Valentino Stoll.

 

See ya now.

 

I'm Charles Maxwood from Top End Devs, and this week we have a special guest, and that's Takashi Kokubun.

 

Hi.

 

Do you want to remind people who you are? I'm pretty sure we've had you on before but

 

Sure, sure. So

 

Been a little while.

 

yeah, I'm kind of known as a template engine maintainer. I'm a maintainer of EOB, and also I kind of authored a Homo 6. Homo 5 was replaced by another version called Homolyt. So I'm kind of the authored Homo now. And I also became a maintainer of Suimasenho this week. So I'm maintaining all those things. And after those kind of template engine works, or is maintaining a JIT compiler. So Ruby programming like C Ruby, or also known as MRI has two different JIT compilers. One is MRI, sorry, MJIT that was merged to Ruby 2.6 and the other is YJIT that was developed by ShotFi and merged to Ruby 3.1. So that's two different JIT compilers and I've been maintaining MJIT for five years and I'm currently developing YGIT full-time, so that's basically what I'm doing.

 

Good deal. And I think we were planning on talking a bit about the JIT. Just to kind of kick it off, I'm a little curious. So is YJIT going to replace MJIT? Is that the plan?

 

So currently we are not planning on removing MJIT, but basically

 

Okay.

 

for production usage, we do intend to let people use YJIT instead of MJIT in production. We still have MJIT for experimental purposes, because we rewrote the MJIT in Ruby this year, we could possibly explore more aggressive optimizations or optimization that take a lot of time for compilation, because YJIT compiles everything during the runtime, unlike image during the compilation partly, it's harder to introduce some, or it takes careful consideration to introduce two complicated optimizations. So it's kind of easy to experiment with complicated conditions in images first, and then think about how we could adopt that in YGIT. So it's kind of two different ways to experiment with JIT compiler. So I don't think we are going to completely replace But still, we think that MWYGT is for future or longer term difficult, I think.

 

All right, sounds good. So, boy, this goes back. We don't do this as much anymore, but we used to always start with the definition. I think that's a good place to start here. Do you wanna just explain what the JIT is? Like, for the normal Ruby person who doesn't ever write JIT in their text editor.

 

So JIT is like an abbreviation of just-in-time. And just-in-time meaning we kind of skipped apart the compilation. So JIT is, if you say JIT, it usually means just-in-time compiler. And then by just-in-time compiler, it just means that we are compiling the code on the fly. After you started the interpreter, we start compiling the native code. So if you say JIT, it usually means native code instead of so-called byte code. Byte code is something that's executed by the Ruby interpreter usually. Like if you provide a Ruby program to the interpreter, what it does first is to parse the Ruby program into a syntax tree and then the syntax tree is compiled into a so-called byte code that could be kind of easily interpreted by the Ruby interpreter that is kind of specific to the interpreter and not reusable like a Java virtual machine. So the Ruby bytecode is specific to Ruby, but then we could also further optimize that into a native code for, for example, for Intel and ARM, YGT supports Intel and ARM architectures for 64-bit machines. And so basically what JIT compiler does is to optimize Ruby specific virtual machine instructions into a native code that's specific to your machine. there.

 
Charles Max_Wood:

Awesome. So yeah, so just to kind of break it down for folks and maybe say it in a little bit different way. Yeah, when you run your Ruby program, it basically converts your Ruby code to a syntax tree. The syntax tree then is basically interpreted into byte code that'll run on the Ruby VM, but the Ruby VM is not as fast as having native code that's compiled to run against your, processor architecture. And so the JIT basically then comes in and just in time or you know more or less while you're running your code at some point compiles it down to that so that you know when you call a specific method or things like that all of that functionality

is already compiled to run optimally on your CPU. And that makes Ruby fast, which makes us all happy.

 

Yeah, that's our plan.

 

So, you said that you've been maintaining MJIT for the last while and now you're working on YJIT.

 

I'm going to go ahead and start the recording.

 

You said MJIT's written in Ruby and YJIT's written in Rust.

 

Yes, yes.

 

So what's the difference and why the different approach?

So first of all, YGIT has a unique architecture, which was sort of proposed by the lead developer of YGIT, Maxine. So Maxine's PhD thesis was about the compiler design that is named, called lazy basic block versioning. That is to compile each basic block. Basic block is a simple block, broken down. Whenever you have a branch instruction, like if you have a if else end, a block between if and else is a single basic block and under block between as an end is an under basic block. So each basic block is compiled lazily. Like if you reach a basic block boundary, you basically stop compiling it and then just execute that just compiled native code. And then when you reach the point that ends the currently compiled JIT code, then you start a JIT compiler again and then compile only that basic block and then go forward. And then if you reach another non-compiled basic block, you compile that again. So that kind of lazy compilation is the design of YJIT. And that requires you to execute a compiler runtime at runtime, like you are not compiling the code in a different thread. Like MJIT used to be using a native thread that is running parallel with the main thread of Ruby. So you don't need to worry about the compilation time for MJIT. or YGIT because of that kind of assumption that is actually used by a JIT compiler. For example, you can peek into the actual self object or arguments and open whatever you want to actually look at. You could do everything at the runtime. So because of that assumption, YGIT is kind of hard to parallelize those compression process. So you don't want to be slow at compiling the Ruby code in the YGIT compiler. So C language is obviously hard to implement. That's kind of complicated logic. So we wanted to get away from C. However, it needs to be fast so that we don't need to let Ruby store. So for making Ruby fast, we kind of needed to use a fast language, unfortunately. And then Rust was kind of a good alternative because that doesn't need a GC. Having two different GCs in a single process complicated, so we wanted to avoid that. So it's a pretty good option to make it fast while still allowing the complicated process. So because of the synchronous compilation assumption, why did we need to use Rust? But for EmuJIT, because we were using native threads, first of all, and then we also switched the architecture to use a forked child process to do the compilation. Because of that, we needed to give up about Windows support, but now the image compiler is running as a child process of the main Ruby process. You don't need to worry about the speed of the compilation kind of. So we could just use a Ruby to implement whatever complicated logic we have. So that's the difference between.

 

So, I'm curious on that, because you seem to make the distinction of mjit and yjit there of one being more thread-friendly than the other, right? And that makes me think of, okay, well, like the whole jit purpose is the more you run it the faster we'll get because it caches more of the things and uses the bytecode rather

 

Right.

 

compile each time, right? So how is that kind of like stored and cached at the system level and how does that interact with the processes for both MJIT and YJIT? Is YJIT basically say, hey, if you're doing multi-threaded stuff, we're not going to take advantage across the thread space?

 

So the thread thing I talked about is more about the compilation process and not about the actual runtime code. The runtime code actually do support the multi-threading. Like when you run the compiler, it's not kind of multi-threading friendly because you want to actually know about the actual main thread information. Like as I said, the receiver or arguments, if you run that priorly, the receiver or the arguments might be changing So we don't want to use that threading there, but after you compile the code, that's kind of compatible with the multi-threading. Like for example, if you use Raptors, you could use multiple native threads running concurrently, but that just works. So for the JetCompiled code, so that's not a problem, but it's basically compilation time threading versus the runtime threading. And I think it's not a problem for actual users.

 

Yeah, I mean, I'm looking forward to it. I saw your talk just recently on YouTube, now that RubyKaigi has their videos out. And I really liked, yeah, Ruby4 was in your title of your talk, which I thought was great. And talking about getting Ruby to be hopefully as fast as Java,

 

Yeah.

 

right? Which I think you said 10 times faster is the goal, right?

 

Yep. Right, right.

 

from having to worry about all of our data people using Python instead of Ruby. Right?

 

Yeah.

 

So, I mean, what is on that trajectory toward that Ruby 4 goal?

 

Great.

 

Because I love just upgrading my Ruby and all of a sudden all my code is running faster.

 

Hmm.

 

What are some top things that you see as making that jump up that path? you

 

So first of all, about the goal, the goal I said was not only Java, but also like JavaScript. So like Java uses a Java virtual machine, which also allows something like a class loader that is pretty dynamic, similar to Ruby. So if you can optimize Java, which has a class loader or some sort of dynamic things, you should be able to make a similar kind of dynamic language as fast as Java as well. Ruby should be able to be that fast as well. And similarly, if you have a V8 or like something as fast as JavaScript, because JavaScript is also a dynamically typed language, Ruby I think should be also be able to be as fast as JavaScript basically. That's the idea. So there's technically no blocker for doing that. We basically want more people to do that. And fortunately, Shopify provided the funding for developing YGIT with a lot of many people. we should be able to do that as long as we continue this effort. And for discussing the actual things that are missing currently, I think we first of all need to be on 100% JIT compiled code on running production applications. So our current actual immediate work is like we have the metrics called ratio in YGIT. percentage you are on the JIT code. So, like, whenever you hit something that JIT compiler cannot handle, you fall back to the interpreter. So, that means you need to be slower than the actual, ideally, JIT compiled code. And then the percentage currently, like, on the benchmark called Rails bench, we are kind of, like, 89% on the YJIT. I think it's better than the 3.1 nearly 100% so that almost everything runs on JIT compiler. So once we do that, we then need to optimize the existing JIT code further. So the first step is to bring us towards 100% ratioing YJIT in the real world applications like Rails, and then optimize the JIT compiled code further. And I think that there are pretty big things that could be optimized in the already JIT compiled code. local variables, local variables are on the interpreter, we store that on the so-called virtual machine stack, that is not native stack, so it's kind of slower, but we could at least use a native stack, and also we could use registers that are native to the actual machine. So if we leverage or allocate the Ruby's local variables to the registers, we could be more faster, although you always have to think about how we deal with the dynamic things like finding local variables for get could prevent that kind of optimization from happening. But we should be able, we have a pretty good way to deal with that kind of optimization. Like we have a way to patch existing JIT compiled code. Like why JIT has a way to dynamically update the existing code. For example, that is used by so-called like a feature called TracePoint. we have to enable the debugger and we cannot reuse the optimist code anymore. So to enable that, we patch existing JIT compiled code and that is not easily achievable in MJIT. That's why I'm kind of favoring in the YJIT currently. So in YJIT, we could do that kind of pretty dynamic optimization. And so we should be able to deal with dynamics like binding local variables get. So we should be able to do the local And the other big thing is method inlining. So whenever you code a Ruby method, that takes a lot of cost because in every Ruby method call or block call, you have to maintain a lot of metadata and that's a lot of assignment to the registers and stack, so it increases a lot of costs. And obviously Ruby uses a lot of method calls in the program. Once we can analyze that each block doesn't need a call frame for that kind of method, or this block doesn't do any like a like inspect expectation on the call frame, then for example, if you don't use the like puts caller or backtrace related information, you don't need to push the call frame necessarily as long as the method doesn't raise anything either. So for that kind of case, we can just keep pushing that frame and skipping that will optimize almost every method called faster. So if every method called fast becomes faster in Ruby, it means we make it a lot more faster. So by doing so, the performance of Ruby could be much closer to other very optimized languages. So I think those two things are kind of like on three, the currently developed version is 3.10.2.

 

Yeah, I did see at 3.2, you mentioned you have the bytecode monkey patching, which lets

 

We are trying to make it stable by the end of this year. So we are not going to introduce those two optimizations this year. But for the next version, we are planning on doing those two big optimizations.

 

you to make your own JIT on top of the JIT.

 

Yeah.

 

Which is kind of funny. So how do you keep track of your optimization progress? What kind of monitoring do you do while you're running stuff and testing specific things? Do you have a special tools or visualizer?

 

Yep.

 

I'm just imagining a dashboard for instruction sequences just running, right?

 

Right, so we do have some tools to monitor that. The first of all, we have a benchmark harness called YGBench. It's a GitHub slash Shopify slash YGBench. And we have a bunch of real world benchmarks, such as, first of all, RailsBench. And another thing is recently added with Ruby, LSB. So we developed a language server by, like, developed by Shopify. And we added a benchmark harness measuring the performance of the Ruby LSP. So if you make it faster, that your local development experience because faster, because we also enabled widget by the Ruby LSP language server. So those kinds of real world applications are kind of compiled into a open source benchmark there. And so we want to, we can easily measure the metrics using that benchmark harness. And we have a metrics generation system by YGIT that is called YGIT stats. So if you pass, or like first of all, you have to enable the stats in when building Ruby interpreter. And then once you get that, you can pass dash dash YGIT dash stats to enable stats. And when you pass that flag, you can either instrument the stats by coding a method to return a hash that includes a stats statistics, or at the end of the interpreter stop, you see that metrics, every metrics at the end of the interpreter exists. If you run the YG bench with dash dash YG stats, you can see the metrics at the end of that program. And then you could also see that the overtime change through the method code of the YG stats method code. So the two things we are currently doing is, first of all, we run YG bench on Rails bench on every single Ruby commit on the GitHub Ruby repository. we can easily check what is changing in the Rails bench. Like for example, we often monitor the change of the ratio in YGIT for Rails bench so that at least we don't become like slower than the previous revision at least on the Rails bench. And we also have like website called speed.yg.org, that is to monitor the visualized graph of every Rails bench, So if you open the speed.yjit.org, you can see the graph or like historical changes over those benchmarks. And finally, we also have internal deployment of YJIT. So we have two different monolith, or like we have a monolith and also like a pretty big, highly optimized usage of the Ruby application. So for that application, on production in a so-called canary cluster. So there is a small cluster that deploys YGIT and we can monitor those metrics there. So like not just a pseudo Rails application that is called RailsBin, but also we also monitor the actual performance on production as well. So that way we can kind of track not only easily monitorable metrics, but also the actual metrics on our business. applications.

 

That's really cool. So can other businesses contribute these kind of canary stats?

 

Currently, we don't provide a way to make it visualized in the dashboard, but maybe what you could do might be, if you have something intensive in your application, and if it happens to be open source, you could probably contribute that kind of code to YGPenchant. So if you do that, we can easily learn that. If it's just a proprietary application code, even if we can see the graph, in depth and so we cannot help you anyway. So just having a graph is sometimes not helpful. We should definitely be able to see the code. So if you can carve out your workloads into a YGBent, that would be really cool. Yeah.

 

Yeah, that'd be awesome. I would love to see kind of how other companies are taking advantage of the JIT and how they progress, right? I mean, Shopify is big enough that I could get a lot of feedback from their services, I imagine.

 

Yeah, speaking of that, is there, because you're talking about measuring benchmarks and stuff like that, one thing that I'm curious about is that, is there a beyond sort of monitoring and seeing what's going on with the JIT? Yeah, is there some practical thing that I can do on my apps if I have a performance concern somewhere that I can say, OK, JIT this, JIT it to death?

 

So I think if you don't, if it's hard to carve out your workload into YGBench, what you could do is to, again, about YGStats, so if you build your, maybe we should actually make it easier to build the interpreter with stats enable, but there is a way to enable YGStats, and then if you build Ruby that way, you could run your application with YGStats enable and then see how the stats are looking at, And then if you can just provide the YG stats to us, and then we could see what kind of thingz are fading, or at least exit into the interpreter. So for the first milestone, we want to make the ratio in YG 100%. And then if you just provide 
the YG stats, we could at least see why the ratio in YG could be low in your application. So we are kind of prioritizing our YG compilation development process by looking at those stats. So like, if you have the stats that look different from our stats, then probably that could be like allowing us to prioritize optimizing your workloads. So that might be a good first step, I guess.

 

 

Shopify, somebody pointed me to Shopify's Boot Boot program where you could dual boot Rails and dual boot even Ruby versions. I kind of want to test out some of this JIT stuff on it and see. I don't know if it allows doing that or if it's just versioned or yeah, I think having that ability to see even just like what your CI does, it would be nice to have something I could just snap in. can kind of do for somebody? Could you like hook into widget bench in like a CI against a specific portion of your Ruby?

 

If you want to run your local code without like publishing your code into the internet, maybe you could use a so-called harness in the YGIT Bench to hook your application specific logic to the YGIT Bench. Like if you want to see the comparison between non-YGIT version versus YGIT version, then you could definitely plug that into the YGIT Bench locally and then without publishing that to the internet. that because we already have a Rails bench, maybe you could use some of the code to like call the Rails application code. Like if you are interested in plugging your Rails application to watchbench, then you can definitely look at Rails bench as an example to call the Rails endpoint. And then you could measure what some of these stats look like there and also compare between GT enabled and GT disabled version.

 

So how do you feel about rust?

 

All right.

 

Ha ha ha 

There we go.

 

So I started using that in my private project before joining Shope and then now, like almost full-time writing Rust. And first impression was like, it's too hard to write, basically. So Ruby is pretty easy to get in, but Rust is too hard to even start writing because operation failure, and it's a lot of failures if you start writing that. And the largest difference to me was the move semantics. So if you have a variable or a reference to something, you cannot share that between different places. If you reference that variable, it basically is gone to that function, and you can't use that variable anymore unless you are doing intentionally sharing that as a reference. of semantics is hard to understand unless you have experience in C++ that also has some sort of move semantics partially. Rust is a pretty well designed language in a way that allows you to have no GC. It feels to me like Rust allows you to solve two complicated problems and that is exposing the problems to users instead of hiding that by having complicated logics like garbage collection. So if th program language itself has a garbage collection, you don't need to solve every single program in hard problems, but Rust allows you to solve it by thinking about essential problems. And so it's kind of like a trade-off. Like if you are interested in writing a fast code, maybe modern options are basically a go or a rust or a zig. But if you choose Rust, it's harder to write compared to at least a go. But it's like, you can't, enjoy solving difficult problems, then you might as well try Rust for doing that. So at least it's an interesting problem, but like experience, but if you are just someone who gets things done, Rust might not be an option for you, I guess.

 

So I'm curious how your experience has been tying back into the C code portion of Ruby. Is that experience pretty smooth with Rust? Like I imagine at this point, hook into. Is that smoothing out or is it getting more complicated over time?

 

So this is kind of also another major challenge in using Rust. So like Rust is not like kind of hard to integrate that with C basically. So whenever you use a new to integrate a C code, you at least need to declare unsafe. So if you integrate your Rust code with C, you basically have to like say unsafe block every single time. And at least it's not hard. And like, so there's a language called Zig that's also a kind of a modern language. And Zig is designed to be easily integratable with C. So if you use that language, you don't need to do a lot of things for encoding the SQL directly. But in Rust, you just have to generate a binding similar to what you are doing in Ruby. So like you first of all call the bindgen tool to use LLVM to pass a C code and then convert that to Rust declaration. you can link the SQL and RAS code together, but again, you need to use the unsafe block. And also you have to do the type conversion between C types and RAS types. So like, it's, I would say it's almost as hard as using C from Ruby. So it wasn't like a good experience. And like we at least have already have an interface for calling major C to be C functions already. So it's not getting worse over time, but still we sometimes have to add another declaration of C functions to the YGIT. So it's not easy, but it's not too difficult either.

  

So what kinds of optimizations are you putting in now? And I guess what does Rust offer in that way? I'm just curious, you know, instead of writing it in C or Ruby like the other JIT is.

 

Yep.

 

So I think we are not really leveraging Rust optimization, but we are assuming that Rust is fast enough.

 

Okay.

 

Because we don't have a garbage collection, whenever I write Rust, I kind of assume that every single extract is allocated on stack so that we don't need to use a heap for heap management that is slower than stack allocation. So because we have mobile semantics, I think we are transparently always leveraging the stack allocation optimization, which is at least not possible in Ruby, or like, yeah, it's not possible, almost always not possible in Ruby. So yeah, we are kind of leveraging that, but basically we assume that Rust is fast whenever we.

 
Charles Max_Wood:

And what's coming, I mean, what kinds of things do you look at now? Because it sounds like your JIT implementation's relatively stable and mature, so yeah.

 

Sure, So for YGIT, the recent work that I've been doing was a code GC. So in the Ruby 3.1, the first release of YGIT in Ruby was basically like, if you start Ruby interpreter, it allocates the physical memory of 256 megabytes at once, and then use the code, like start writing the code blocks a block of the memory. However, obviously it consumes a lot of memory even if it's not needed for the process. So we changed the architecture this year to lazily allocate memory pages one by one. So if you start the interpreter, it doesn't allocate a lot of physical memory. It just allocates a virtual memory, which is not actually consume your physical memory. So we allocate virtual memory and then use physical memory over time.  in 3.2, however, we still need to leave code used by initialization without garbage collection. So like, for example, if you have a Rails application, you do a lot of initialization proc zess in the Rails initializer, for example, and then you will not use that code anymore after the initialization finishes. So it's still like waste some memory usage without garbage collection. So we introduced a code CC, to garbage correct your JIT code as well. And that's, I've been a primary developer of CodeZC so far, and then we first of all needed to change the diff, like layout of the code or the order the code put in so that that could be easily garbage corrected. And then we also implemented the algorithm to garbage correct everything in the JIT code. And so what it does is you have an option called size, which is 256 megabytes by default. And then if the memory hits that limit, we start the code GC and then it basically frees every single code, including onStack one. So like onStack means if you are calling something in the caller, then it's onStack. And so we cannot free that immediately, but we can at least invalidate the code so that when you leave that stack and then come back again, So by freeing every single code in the 256 megabytes you can start recompiling only what's needed after the code zc is triggered so that you have more compact code and you have no initialization code if the code zc happens after initialization. That way, you save more memory as well as having more localized code that is faster than less localized code. So that's the kind of optimization I've been recently working on.

 

Cool. So is this what you're doing full time? You said at Shopify?

 

Yes, right. So I'm hired as a YGT member. And so even while I'm also a maintainer of MJIT, I don't really work on MJIT at work. However, sometimes other folks are working on something else, such as object shapes. So object shapes is a recently merged idea to CRuby. That is to optimize instance-supportive access, basically. So for implementing that, they needed some code in YGIT and also MJIT as well. And fortunately, now Shopify has a MJIT maintainer, so they could talk to me and do some pair programming to fix in something in MJIT when needed. So I've been sometimes helping other developers change MJIT, but for developing MJIT, there is rarely necessity of changing MJIT, so I didn't need to MJIT for my own work. So that's that, yeah, my work at Shopify.

 

Mm-hmm.

 
Valentino_Stoll:

So I'm curious about the object shapes in it. How did that, what ended up needing to get corrected in JIT and, or YJIT and MJIT to accommodate object shapes?

 

So object shapes is an idea to sort of compact or compress the information of every object into a single 64-bit value. And so let's say there is an object that has an instance variable named A and instance variable named B, and also it's frozen, then that kind of information could be compressed into a 64-bit value. And so if you can do that, JIT compiler can only check those 64-bit by a single instruction. than checking, for example, like this object has an instance variable name A and this is frozen, then you cannot set that instance variable because it's frozen. So that kind of check needs to happen by multiple instructions in the previous implementation, both for YGT and MJIT. And because of that, because we want to leverage the new, the architecture or the object shape, because Object shape is to optimize instance variable access. And also, because we wanted to not complicate the implementation, we basically replaced the previous implementation with a new implementation. So you are kind of forced to use the new optimized code in both MJIT and MJIT. So as long as you are removing the old implementation, you have to support object shapes-based access in the MJIT and the MJIT. So that's why it was needed to be.

So is it faster still to freeze everything? That was a joke.

 

Well, like,

 

Haha!

 

that check is... Freezing... Yeah, right, right. I mean, like, so when you do the, like, when you freeze objects, yeah, that's, now it transitions the concept of object shapes into a frozen object shape. So it's not optimized, but like, at least for accessing the frozen object, that's supposed to be a little bit faster. So that's, yeah, currencies.

 

That's good to know. So I'm seeing here Ruby 3 has arm support.

 

Yep.

 

Have you run benchmarks to see if M1 actually does perform better with that adaptation?

 

Great, so yeah.

 

And tell me, I'm curious, that seems like a huge challenge to change, adopt another architecture. What is that process to?

 

Right, so our initial implementation that was released in Ruby 3.1 was only supported in Intel. So whenever you see Ruby's virtual machine instructions, you can just compile that into, convert that directly to Intel instructions. So that was a previous implementation. However, you now need to switch that to Intel ARM. So you need to basically have an indirection between Ruby virtual machine and the native code. IR, internal representation, well, intermediate representation. So there's a so-called YGIT IR in between those two things. And we kind of, like, first of all, we introduced that layer, and then we added the backend, like, Intel backend, ARM backend. So if you, instead of directly generating the Intel code, but doing the generating the IR, you can automatically support Intel and ARM by using that. process was we first implemented the IR and also two backends to that. And then we gradually replaced the interbase generation to IR base generation. And then we did that for each Ruby instruction one by one. So by completing that, we then supported the arm for every single instruction in the interpreter. So that's how it worked. And I think it took a half year this year. Early this year, we worked on rewriting YGIT into Rust everywhere. And then we've now started working on IR. And so I think, yeah, overall, I think it took almost a half year. And so it was a long process. And then I think the hardest part in doing that was the difference between Intel and ARM. So when you write something, when you write native code in Intel, that will take care of the so-called instruction cache. native code in memory, it's cached by CPU. And so CPU will take, Intel CPU will take care of invalidating the cache so that when the code is updated, it will run the new code after that. So that's easy for Intel, but for ARM, CPU will not take care of that. And so you basically have to, like compiler needs to take care of what kind of code needs to be flashed from the cache of the for flashing the code that we've written immediately. So because of that, because we didn't need to do that in Intel and we nearly needed to do that, we had a lot of segmentation faults in the ARM port. By just converting the Intel to ARM, you don't introduce the invasion process. Whenever we do that, we sometimes see the segmentation fault because of that kind of thing. And we've even worked on that this week as well. even while the ARM support was months before at this point. So yeah, it's been hard for supporting ARM. And then once we did that, I think the performance improvement was good. Like on Rails bench on M1, I think that was like 30% better. Like you can also see the ARM's performance benchmark on speed.budget.org today, thanks to Noir developing the ARM support on the Vigimetric website. And so, yeah, for applications like Rails, you could see some 10% or 30%-ish range of performance improvement using the ARM architecture. So it's not like worse compared to Intel. Like you can get fair bit of optimization in both Intel and ARM, which is good. But one interesting thing was that if you run Intel code on Rosetta on M1, you could also kind of jit the, execution like YG generates intercode and then Rosetta will convert that arm and surprisingly it kind of ended up being faster than the already arm generated code. So like it was kind of like unfortunate but I think Rosetta will not be maintained for a long time. So it's

 

That's really funny.

 

still needed to support arm. So yeah, that's interesting experience.

 

So I'm curious, are there architectural frameworks or hooks that can be taken advantage of for each individual architecture with Ruby? Like are there optimizations that can be pursued for each architecture? Reaching for those?

 

So because we have like a YGIT IR to support those architectures, we have a separate backends to each of the IR introduced by YGIT. And so the way we do optimization, like architecture specific optimization is to basically split everything after IR for each architecture. So we have like a so-called concepts called passes. You generate IR first by the YGIT compiler go through concepts of the passes, and then when you go through the passes, that would be generally more optimized IR. So you have passes for register allocation. So if you go through the register allocation pass, the IR will, instead of using the memory every time, that will use registers. So it's faster. So by transforming the IR through the widget passes, you can perform architecture-specific Because, for example, register applications also architecture specificity, because the number of registers is also different. So by doing so, we allow architecture specific in operations such as choosing more optimized instructions for different cases. For newer machines, we have single instructions for doing atomic operations. So we could do that kind of thing by splitting the backend for different architectures over the YGTIR.

 

That's really cool.

 

I'm kind of curious if somebody is listening to this and it sounds interesting to them, not from the standpoint of, oh, this is what it's doing for my Ruby code, or this is how I take advantage of the YJIT, but more along the lines of I'd like to contribute to this or I'd like to understand it better, right? Because we can only talk about it for so long and then, you know, we all have to go back to life. And so, you know, if somebody wants to dig deeper, contribute, what resources are out there for that?

 

So we have a document for contribution such as a, we have now on the Ruby.git repository, we have a directory called doc. And then under doc, we also have a YGIT directory. And under YGIT directory, we have YGIT.md and YGIThacking.md. So those two things can be referred to for using YGIT and also contributing to YGIT. And that has a section called YGIThacking. And then you could use that as a starter learn about the YGT journals if you are interested. And also, for YGT, we have a lot of external contributors, actually. So you may look at what other people are doing there. And also, you could maybe try doing something similar to those external contributors. Like we have some folks from GitHub that contribute to YGT as well. And so as I said before, we have YGT metrics, YGT stats generated by the dash-ygts-stats flag. to improve the metrics when it's not reaching 100%. Like that will show, like either show the reason why it is not 100% or it will show that we are not collecting the metrics correctly. So if it's not 100%, then you can at least add the metrics or if it's already added, then you can fix that metrics by looking at a particular place that is incrementing the counter. So I think that's a great way or that we get familiar with the YG.

 

You have anything else to ask Valentino? I think I'm tapped out.

 

I'm curious about the Haml updates.

 

Oh yeah.

 

I mean, what is it? 1.7 times faster than the last version.

 

Reha.

 

It's now faster. I mean, Slim is the slowest template engine, which is kind of funny because it's one of the more popular ones for the syntax.

 

Right. So yeah.

 

Perf bump.

 

So first of all, the moment...

 

They put it through the Y-JIT.

 

I actually, I haven't tried enabling YGIT, but that's because we have ERB benchmarks or YGIT bench. I think it should be also make it faster as good as ERB. So it should make it faster, but for derping Hamo, I often don't use YGIT. So I'm not sure about that yet. But still, without YGIT, 1.7 times faster than Hamo 5 and Hamo 6. So yeah, it's a kind of major improvement. But the moment when I started working on Hamo was the Hamo 4. And that was like, Hamo was seven times slower than Slim, basically. So it means

 

Oh wow.

 

that, yeah, from Hamo 4, Hamo 6 is seven times faster today. So yeah, that's a huge improvement. And the reason why we weren't able to reach of Slim for a while is that Hamo has two different blockers for doing that. One is a Hamo helper. So unlike Slim, Hamo provides some helpers that can be used inside of a place like a Hamo tag and capture and capture is a pretty complicated mechanism that has users do whatever they want. So, making the Hamo as fast as Slim, mechanism or just throw in a way that slow design. So I was kind of doing the optimization that are possible without fixing or breaking the compatibility in Hamo 5. Like Hamo 5 became like four times or five times faster than Hamo 4 without breaking compatibility a lot. So that was made without breaking everything. idea of making it faster without breaking the compatibility, but still dealing with the hack or removing the complicated mechanism for cases that are possible. But the problem is it's pretty hard to implement that kind of complicated process. So, like, I was not doing that because it takes a lot of time to do that. But then the founder of Hamo called Hampton reached out to me saying, So we ended up removing the features that are blocking Hamo to be faster. So now we don't have Capture Hacker, unfortunately. And we don't need to deal with that kind of complexity. So you don't have features that Slim doesn't have, which is why it's now as fast as Slim, basically. So now Hamo 6 doesn't have some features, but still it's as fast as Slim today.

 

That's really funny. Just remove, for everybody out there, just remove things that are blocking you and suddenly everything is faster.

 

Nice. Mm-hmm. Yeah, all your slow code. I'm gonna do that with all of my code. This is slow out. So

 

Yeah.

 

this is slow too, I don't need it either.

 

So I'm curious at Shopify, they primarily lean on ERB.

 

Yeah, I think so. Yeah.

 

Yeah.

 

The Shopify is using ERB mainly. And also, for users, the CEO, Toby, authored another template called Liquid Templates. So users

 

Mm-hmm.

 

are using Liquid Templates. But for our own definitions, I think we are using ERB. So my contribution is basically that UTA is there, but it's good.

 

So what got you, I know you're the author of Hamlet, which was like kind of the jump to make Hamel faster from the beginning.

 

Yep.

 

What got you into that being like, oh, I should make this faster?

 

So as of the Hamo 4, I was kind of university students at the time, and then I was into compiler stuff, like I was writing C compiler and the schema interpreter, and also like generally I was into compiling something. And then Hamo was the template language that the previous employee or yeah, the previous era where I was a student there was using. So like the, I was in Cookpad at the time, and Cookpad is using Hamo everywhere. If I were to work on temperate language, then a hammer should be the go-to for making the impact at the company. So I started writing that. And funny enough, there was another person doing that at the time. So he was writing another implementation of hammer called fast hammer. That was later renamed to farmer. But so we ended up having two different faster hammer invitations at the time. And we were kind of competing with each other. meant was like I skipped implementing some feature that Hamo did have and so because of that Hamo Hamlet ended up being faster than fast Hamo because Hamlet was basically broken at the time and so Hamo became the fastest implementation because I forgot to implement something and then we I thought about like the the other author said oh Hamlet is fast because it's broken and so I was out, we could change the design a little bit to reasonably allow that kind of optimization, and then we incorporated that incompatibility in Hamo

 

You have a history here of just making everything run faster as it upgrades, you know?

 

6 finally. So that's how it happened. Hamo-Lit became faster because it's forgotten. We ended up using that in the Hamo 6 as well today. So that's the story. Yeah.

 

I'm a long-time Haml user. I prefer it over Slim.

 

Thanks.

 

I know a lot of people really love Slim. But I don't know.

 

Yeah, now I'm also

 

Yeah,

 

maintaining that, so it's okay.

 

the issue that I run into and the reason that I like Hamil is that I am not a designer at all.

 

Hmm.

 

And so I can play with spacing and stuff a little bit. Maybe you switch a color here or there, stuff like that. But making something look the way that it ought to look, making it easy, making it flow, it's just not really my thing. And so what I wind up doing is I wind up going to themeforest.net and buying templates,

 

Yeah,

 

HTML

 

yeah.

 

templates, right? And so the first step is you put your HTML in a view and then you put all of your assets in public, right? And then you can eventually move them into some kind of asset manager, but

 

you

 

Webpacker or whatever. But you don't even have to do that. And sometimes I just leave it and then CSS file that I just load in after everything else and say, this is me clobbering what, anyway. But what's frustrating is, is a lot of these layouts, it's like this div hell. And it's fine, it's fine you have that slant going in, right, for all your different components.

 

Right.

 

And yeah, eventually those will all move into like web components or things like that, or partials. I don't use partials anymore. Anyway, but the issue I run into is finding  those stupid close tag  It's, it's  such a mess. And then the other thing is, is like, they use bootstrap or something for the basis of it. And so it's got like 18 classes in it. And I'm trying to figure out, okay, this div has, you know, 18 classes and, you know, where does that actually, you know, open and close cause it's the hamill that's easy, right? Cause it's just indented then. More than anything else, it's that frustration of, okay, where do of, okay, where do I find the close  tag on this? And I'll edit something and I'll delete a close tag or add an extra close tag and then everything shifts, right? It is, ah, anyway. So yeah, so thank you for making that fast. That makes my life better.

 

there.

It's funny because that kind of reminds me of Python, the arguments where people that come from Python doing Ruby and they have to add the end to the definition and they expect everything to be neatly indented.

Sure.

Yeah, of course I'm a jerk and I just want to then screw with my indentation on purpose. But I have a plug-in for Rufo on most of my Rails apps. And so I save the Ruby file and it auto-indents everything and cleans up all the stuff.

 

I'm curious, is there any optimizations you've taken out of Haml that you can apply to ERB? Or are the designs kind of just too completely separate?

 

So what I've been kind of thinking about was to sort of some helpers. So like Hamono is now using the template engine framework called Tempo. I'm also like a manager of Tempo as well and like also became a demo recently so I can just reset. But basically what it does is similar to YG kind of so like there's a some concept called Tempo IR like similar to YGTIR. And then similar to Passive we have Feature. their filters, the RL becomes more optimized. So as long as your template engine uses a Tempo IR, you could optimize that similar to Hamo or Slim by using the filters. So we have ERB implementation using Tempo IR inside Tempo Gem. And so we could just insert IR filters used by Hamlet and Hamo to optimize the ERB further. I think we could also use that in Slim cases, although for major benchmarks that might not be impacted by that, but still you should be able to share some of the optimizations. For example, when I was writing Hamlet, I wrote some filter to split the string interpolation into the segments so that you could concatenate that with surrounding strings by compile time instead of doing that on runtime. So by doing so, And obviously, that could be used for other temperament agents as well. And so currently, the ERB implementation inside the temple gem is slow, but we could improve that if we want. But basically, the ERB implementation used by whales or other places is usually a gem calle LV, and it's not used in temple.  So it's kind of hard to leverage the capability there. But if you want to compete with that, we could implement eve in the temple and then sort of replace Airbnb in the future, but that's the current situation.

 

 Charles Max_Wood:

Awesome. So is it as fast as the RB then or does it matter? Cause that

 

Uh, go ahead.

 

I always ask the one and then I'm like, is it fast enough? And it seems like he's fast enough, but yeah, is it as fast as the RB?

 

So yeah, I think we could also do some optimization such as removing local variable in the direction inside of Hamo or same as well. So like ERB sometimes becomes faster than Hamo in some cases, but generally

 for most cases, there's almost no meaningful difference between those temperate edges at this point. So you shouldn't need to worry about that if everything is as fast as those things. And so it's a pretty minor problem. could make it even faster, but still it's like a point, like one or like a 2% each changes at this point. So you don't need to worry about that. Like how many ERBs are fast enough, I would say.

 

Yep.

 

So how are your studies going? How do you juggle all this? Ha ha ha ha ha ha ha ha.

 

So,

 

I know

 

uh.

 

you maintain half of Ruby at this point, right?

 

Right. So that's kind of part of the reason why I joined Shopify. So like, prior to joining Shopify, I was doing the proprietary distributed system maintenance and the previous employer. So like the open source maintenance was only possible after work time. And then I got that child two years ago. So like, I needed to spend more time taking care of the child. So like, maintaining the open source requires a lot of time, but still, I don't have time for doing that. So by making the full-time job open source maintenance, I was now able to spend more time in open source. That's one way to approach it. And I guess the other approach is to sort of switching the projects over time. Like a panel doesn't really need a lot of time for like all the time. Like you only need to do something when releases or changes or other like a dependency update happens. We do need to do something, Basically, the way it works is I work on Hamo sometimes and then do that and then go to another project, do something, because during that period, Hamo doesn't need any help from me. So I kind of work on a lot of things, but in a different moment, basically.

 

  Charles Max_Wood:

Cool.

 

How is your child? Have you found yourself trying to optimize fatherhood as you go along?

 

Uh. Well, yeah, like,

 

Is that a KJIT, like KidJIT?

 

she's... Yeah, yeah, I'm optimizing myself for dealing with that kind of problem. But she's now two years old, so it's getting easier and easier over time. And now I can just happily play with the kid when I'm not working. And then I can also go to the park together and spend happy time compared to the early days where the kid was basically... crying all the time. So now it's optimizing both sides. Like the kid is kind of optimized towards playing with me myself. And also I'm also optimized towards taking care of her better. So that's better.

 

That's great.

 

Yeah, you know, at two, if you leave food out, they won't starve.

 

Heh. it.

 

All right, well, yeah, we're kind of getting to that point. I have a hard stop in about 18 minutes. So I'm gonna push this toward picks. But before we do that, if people wanna follow any progress or see what you're working on online, where do they go?

 

I am available on Twitter as a Kokubun, all becomes a zero. And then I'm also available on GitHub as well, and the same username. So I think those are good resources. I basically announced the major open source work on Twitter. And also you can see everything on GitHub. So those two would recommend to people.

 

 

Awesome. All right, Valentino, do you have some picks for us?

 

My pick is, go see Takashi's great Ruby Kaigi talk. There's a hidden gem in there where he switches to Japanese without noticing. I thought that was great. Kudos to you for being able to think and speak in both languages. It was a great talk. So I recommend seeing that. It was Ruby 4 and Waijit.

 

Awesome. I'm gonna throw out some picks. I don't have any board games to pick this time around. I am gonna, I'm gonna pick a board game convention instead. It's coming up in like two weeks. It's here in Utah. It's called Timcon. The big one in the Salt Lake area is called Saltcon, but that one like sells out every year and it's hard to get into. Utah County and it's pretty awesome. I've wound up going for the last, or I went last year and I'm going this year doing the same thing, a friend of mine is part owner in a game board game shop. And the board game shop does the game demos over in the corner for like five games. So we all learn how to play these four or five games and then we teach them to people. Right. and sit down and we'll teach them how to play the rest of the conference. You can either bring your own games or you can check out games from their game library and then you play it with other people, right? So you're just paying to be at the the conference and you know play with complete strangers which is kind of fun in some ways and you know you see some stuff you haven't seen before if you play with the same people on a regular basis like I do. So It's really fun, it's really cool to do it, and I recommend that people go find their own local board game conventions and find a way to connect with people. Besides that, I'm starting a book club for developers. And the first book we're gonna do is Clean Architecture by Uncle Bob. And Bob has actually agreed to come to some of our book club meetings. We're starting it in December, the first week in December, and we're just going to have a call every week and talk about the book. If you want to come and talk about the book, then

feel free. We're going to use some kind of conferencing software that allows you to have so many people kind of on stage, for lack of a better way of putting it, and then we'll just rotate people through. If you have something to share, some insight, some question, if Bob's around, or some question You know, let's do it. And then what I'm really hoping to do is focus some of the discussion, cause we'll have some discussion group too, like a forum or something around, okay, how does this apply to what the code I write or, you know, things like that. So looking forward to all of that. And that's going to be a ton of fun. Yeah, just excited about that. And then I did push back Rails Remote Conf to February. So if you were looking forward to that, February. And yeah, those are pretty much my picks. Takashi, do you have some picks?

 

I have a couple of picks. The first one is so-called Jiro ramen. So Japanese, Japan have a lot of various types of ramens and one of them is called a Jiro ramen. It's because there's a ramen like a franchise called Jiro and it's kind of a sort of religiously famous or popular ramen franchise. And it's kind of harder to get that kind of Jiro ramen in the U.S. because it's, I'm not sure if it's a popular in the U.S. But recently in the Silicon Valley Bay Area, we got two places of Jiro Ramen recently. So like I was able to enjoy that at this place. So it's very good. Jiro Ramen is basically a lot of garlic and a lot of meat. So like, and also in Japan, it's known for being very cheap. Like you only pay five bucks to get a huge chunk of cabbage and sprouts and meat and also a lot of garlic. And it's a strong taste. that but because you often get up end up being pretty fat by eating that it's like a famous only for like not fat people. That's not a good word to say but yeah so I like that so like it's good that I can eat that near here. Another pick is I speak I bonds so like I bonds is a like US Treasury's So it has now a 9.62% interest rate guarantee, which is pretty good, although it's gonna end this week. Maybe listeners will not be able to buy that in the same way, but still, this information is pretty insane. My rent cost is gonna increase 6% this year. It's unfortunate. I need to fight with that. So it's good that we have a 9.6% interest rate, and I bought that also. I set up a savings account percent APY, so it's good too. So I'm leveraging those kind of things to fight with inflation disease.

 

Very cool. Yeah, I have to say, I grew up just with the kind of the ramen packets that you get at the grocery store here. And so I was like, oh, ramen's just cheap, kind of mediocre food. And then I went to a conference with some friends and one of my friends had been a missionary in Japan.

 

Oh.

 

And so we went to Japantown in San Francisco and went to a ramen place and I'm like, ramen place? Yeah, right. So we go eat there and I'm like, this is the best thing ever. Come back and find out their ramen places here in Utah. But it also turns out that my dad was a missionary in

 

Oh.

 

Japan and he had had that kind of ramen and he had neglected to share it with me while I was growing up. And so he kind of caught a little bit of attitude from me for that because I felt like and unfairly deprived as a child. You know, I mean, we ate gyoza and, you know, fried rice and a whole bunch of other stuff, you know, the way that he had it in Japan, the way he learned to make it, or, you know, we'd buy it from the store. But yeah, never ramen. So anyway, I love that stuff. It's so good. So yeah, and what kind of ramen was it? Can you just type it into the chat so

 

Sure,

 

we can get

 

It's a J-I-R-O,

 

it in

 

it's

 

the?

 

a zero. It's a very normal common name of the Japanese first name. So yeah, that's why. So, yeah, that's why. So, yeah, that's why. So, yeah, that's why. So, yeah, that's why. So, yeah, that's why. So, yeah, that's why. So, yeah, that's why. So, yeah, that's why. So, yeah, that's

 

Oh, Jiro

 

why. So, yeah,

 

ramen.

 

that's why. So,

 

Okay.

 

yeah, that's why. So, yeah, that's why. So, yeah, that's why. So, yeah, that's why. So, yeah, that's why. So, yeah, that's why. So, yeah, that's why. So, yeah, that

 

Cool. Yeah, I'll have to check it out. I think most of the ramen that I see around here is the tonkatsu.

 

Tonkotsu, I guess. It's

 

Yeah,

 

funny that there are two,

 

the pork.

 

yeah, so there's tonkatsu and tonkotsu. Tonkatsu is a cutlet of pork, and then tonkotsu is like a soup made by pork. So like many people got confused and say tonkatsu ramen, but it's basically wrong. And I see a lot of that here, so it's fine.

 

Nice. Well, I started learning Japanese and then my daughter decided she wanted to learn Chinese. And so we are, I'm learning Chinese with her. And when we're done with that, then I'll go back and learn Japanese because I think it's cool. But anyway, well, thanks for coming to Kashi. This was a lot of fun. And yeah, hopefully it helps people understand what's going on under the hood. And if you're dealing with things on the level of, hey, I want to optimize my VM or understand what's going on in there. Yeah, there's plenty of stuff here to look at. Good deal. And then of course, Hamel. And Hamel saves me headaches. All right, folks, we'll wrap it up. Till next time, Max out.