Obie Fernandez is the Chief Scientist at Olympia. They dive deep into the fascinating world of AI and its transformative impact on music production and application development. Our special guest, Obi Fernandez, joins Valentino Stoll to share his incredible journey of living in Mexico City since 2017 while working on innovative startups like an NFT platform and Olympia, an AI-powered consultancy growing rapidly every month.

They also introduce us to a cutting-edge music production tool that can upload music, write lyrics, and generate vocals — a boon for lyricists and singers who might not have the best singing chops. Alongside this, they discuss pragmatic uses of AI through a new book, which bridges the gap between beginner-friendly concepts and advanced AI patterns, illustrated through real-world examples.

Moreover, they explore the intricacies of AI models like GPT-4 and Gemini Pro, discussing their capabilities, limitations, and practical applications. From insights on GitHub Copilot Workspace to the challenges of incorporating natural language in development tools, this episode is packed with valuable knowledge for developers and AI enthusiasts alike.

They also touch on the innovative work being done on platforms such as Olympia and MagnaChat, and explore the burgeoning possibilities of AI in runtime environments. Whether you're interested in the technical nuances of fine-tuning large language models or the potential of AI in game development, this episode offers a wealth of information.

Tune in for a fascinating discussion that promises to broaden your understanding of how AI can be integrated into diverse applications, all while keeping it accessible for both seasoned developers and beginners.

Socials

Charles Max Wood [00:00:04]:
Hey, folks. Welcome back to another episode of the Ruby Rogues podcast. This week on our panel, we have Valentino Stoll.

Valentino Stoll [00:00:12]:
Hey now.

Charles Max Wood [00:00:13]:
I'm Charles Max Wood from Top End Devs. We have a special guest this week, and it is Obi Fernandez. Obi, it's been a while. How are things going?

Obi Fernandez [00:00:23]:
Pretty good, man. Thanks for the invite. It's an honor to be back.

Charles Max Wood [00:00:27]:
Yeah. So do you wanna just, I know you're writing an AI book, and I see you active in the Ruby AI Discord. But, what else are you up to? What else should people know about you?

Obi Fernandez [00:00:42]:
Well, I've been, I've been living mostly in Mexico City since 2017, so people usually find that interesting and ask how that's about. Because, I grew up outside New York City and spent 20 years in Atlanta, you know, in the southeast US. But, yeah, Mexico City is super cool. It's a great place for for expats to live. Great lifestyle. And I've done a couple of startups, in recent years. 1 1 was an NFT platform that that kind of went took took me in a 100 other employees on a on a big roller coaster ride with Venture Vacuum and Crash and Burn along with the rest of the crypto market a couple of years ago. So that was, interesting.

Obi Fernandez [00:01:25]:
And, yeah, the for the last year, I've been working on another startup, called Olympia, which is a a platform, that gives you AI powered, consultants, to add to your team. It just kinda human that humanizes, AI. So yeah. That that that's been a lot that's been a lot of fun, and and it's doing well. It's growing about 10, 20% every month to month. So,

Charles Max Wood [00:01:54]:
Oh, wow.

Obi Fernandez [00:01:55]:
That doesn't seem to be letting up.

Valentino Stoll [00:01:58]:
Good deal. I love Olympia. Oh, I'm a happy I'm a happy user here.

Obi Fernandez [00:02:05]:
Oh, super cool. I didn't actually realize that. That's a surprise. That's awesome.

Charles Max Wood [00:02:12]:
So where where did you find it? Just olympia.com or something?

Obi Fernandez [00:02:16]:
Olympia.chat. Dotchat.

Charles Max Wood [00:02:18]:
Okay.

Obi Fernandez [00:02:19]:
Mhmm. Yeah. I need to I need to bite the bullet and, spend on the on the AI, or.com, the right name there, but they're expensive.

Charles Max Wood [00:02:30]:
Yep. Well, I'll have to check it out. In the meantime, we're I I kinda wanna get back to the book, you know, that you're writing about AI and Ruby. And I'm I'm kinda curious. It sounds like you've been working in this for a while, but, how did you get started with this? And Mhmm. You know, what what have you done that makes you the guy to write this book?

Obi Fernandez [00:02:56]:
Well, this this isn't my first, startup that I I've tried to do in AI. It's, like, back in 2018, 2019, as a side project at Kickass Partners, which was my my consulting company that I merged with Magma Labs, back in 2020. I I did a a project called Demos Guru, and the idea is so I'm also involved in music and music production and so on, and was running a somewhat active record label at the time as as a hobby. So 1 of the 1 of the problems that I identified for myself, and other people, that run record labels is that you get this flood of demos that have nothing to do with the kind of music you put out. Or they they're just not up to par in different ways, you know, not mixed properly and whatnot. And, I didn't I didn't actually realize that, machine learning stuff was was a craze back in that era, 2018, 2019. I was just reading some analysis of it. And, and I don't know.

Obi Fernandez [00:04:08]:
Maybe COVID scrambled my brain, but but I I thought it was pretty unique. What I what I was thinking of doing back then is, like, basically trying to to use machine learning to detect different qualities of a piece of music including its genre, mastering, you know, quality of its mastering and mixing. And and then other things that you could use to identify music. So worked on that for for nearly a year. Didn't get anywhere. Really, really, really hard math and, you know, just kind of difficult problem that wasn't cracked yet. I'm not sure that it's cracked yet right now, actually, now that I think about it. I haven't really thought about it in a while.

Obi Fernandez [00:04:50]:
But the, people are doing the reverse, which is taking text, you know, and turning it into music. And I was trying to take music and turn it to text, which I haven't seen yet. So if anyone's working on that, reach out to me. I'm super interested in it. But, yeah, like, we're talking about Olympia. Right? Like, a like, a year ago, I started working on this this startup. Actually Actually, it goes before that. So the last RailsConf I I went to, which was the first RailsConf in a while, was in Atlanta Mhmm.

Obi Fernandez [00:05:20]:
Last year. And I was super excited about launching, like, a chat tbt clone in Rails, at the time. So so I scrambled, and I made a little platform called MagnaChat. And I open sourced it during RailsConf and kinda announced it and showed it to a bunch of people. Even at that comp even at the rails comp, I mean, a lot of people I was showing it to had never seen chat tbt yet because it was still recently mute, you know, fairly recent. They'd want to know what's going on. So blew some minds just in terms of showing people what, you know, you could do with gpt 4. And then I, my girlfriend kept complaining about her copywriter at work.

Obi Fernandez [00:06:06]:
So she was she was running partnerships for a large conference series, and she had this copywriter in India, who was paid very little money and produced very low quality copywriting, so she was rewriting everything. And, I said, well, hey. I got this MagmaChat thing deployed. Can I make you a copywriter? But, And she said, sure. And then I gave it to her, and I watched I watched her use it. And she's just started using this thing for hours a day, basically started doing all her copywriting herself. And then, very soon thereafter, she said, can you can you make me a whole editorial team? I said, well, what do you need? And she said, give me a copy editor. Give me an SEO, expert.

Obi Fernandez [00:06:51]:
Give me a marketing expert. You know? Give me a PR expert. So so I put it together and very quickly realized, okay. There's gotta be a startup here. Because she was willing to pay for it, And I started showing it to other people, and they were interested in paying for it. So I I said, do you wanna do you wanna be my CEO? Because I was CEO of my last startup, and I hated it. I'm more of a natural CTO as you might imagine. So, so yeah.

Obi Fernandez [00:07:17]:
So I brought her on board, and, we started Olympia together. And I had AAA beta product ready in a couple months. We launched it in July into closed beta, and then we launched to the public in September. And, it's all it's not using Python or TypeScript or or any of the typical, you know, usual suspects you'd you'd wanna use for something like this, or you'd expect to use Broad and Rails right from the beginning, kind of forked, MagnaChat when I started and then eventually ended up replacing almost all of it. Have not done a good job of keeping LanguageAd up to date. Sorry, whoever picked that up. Really screwed you over on that. But my partners at MagnaLab, have, you know, also kind of fork from where I started, and they have a commercial version of Magna chat that they'll implement for people, you know, behind the firewall kind of thing.

Obi Fernandez [00:08:20]:
So, yeah, you were asking about the book. I mean, essentially, I took, you know, I took notes, given I've written technical books before As you know Just a few. So, yeah, I was taking notes along the way. And, right from the beginning, I was like, wow. This is comp to me, completely novel kinds of programming. You know, it's just it's just totally new paradigm in the way just things I haven't seen before. So I was like, okay. I started documenting what were essentially patterns and approaches and things.

Obi Fernandez [00:08:55]:
And III guess I did a little bit of blogging about it as well. I mean, some of the early blogging I did on the subject went viral. When something when the topic is so hot, like, the the d and d post I did for AI got 100 of thousands of of, reads on Medium. So that was kind of insane. D and d. And yeah. So so that was that was what first took me on it. So, like, when I first got access to, the OpenAI API, my kids were actually here.

Obi Fernandez [00:09:30]:
So I I have a 27 year old daughter and a 24 year old son and a now 16 year old son. And the 16 year old is really into Dungeons and Dragons. And and as a family, we we did a lot of gaming growing up, as well. So so they were here for spring break or or I guess it wasn't spring break. Oh, I guess it was, like, a late holiday break. I can't remember Exactly. But they were all here in Mexico City. We're all here at the same time.

Obi Fernandez [00:10:00]:
We were playing Dungeons and Dragons. And I said, hey. I bet you that this, this AI thing, could be a a good DM for a role playing game. And then so so I started playing around with it and I tried to make an app for it, and then I described it in the blog post. And that blog post blew up really big. And I started getting that, like, you know, spidey sense of, like, okay. Here's another big wave Uh-huh. To to ride.

Obi Fernandez [00:10:28]:
You know?

Charles Max Wood [00:10:33]:
So oh, go ahead. Go ahead, Valentino.

Valentino Stoll [00:10:36]:
Oh, no. I I mean, I have a lot to say here.

Charles Max Wood [00:10:39]:
Yes. What do I?

Valentino Stoll [00:10:41]:
I mean, I love the Dungeons and Dragons use case, which is kind of interesting now, with chat GPT. It's like multimodality, like, just talking with it. I had, like, a long car ride car ride recently where I was just, like, talking through a business idea, and it's pretty remarkable. But yeah. I mean, the whole Magma chat stuff, I love that. It's actually really inspirational, like, for how to use AI in a lot of ways. And so you're out there wondering, like, that's definitely, like, a great place to start to just, like, pick apart a, you know, how to do XYZ. Be you know, an until your book comes out because a lot of that is conveyed in there.

Obi Fernandez [00:11:24]:
Well, the well, the book's available. We'll we'll talk you can get it right now on Leanpub, and there's actually much better code out there now than than MagnaChat. So if you're listening to this, don't Don't go. Don't go use MagnaChat.

Valentino Stoll [00:11:38]:
But I

Obi Fernandez [00:11:39]:
was Use rate use rate for rails or 1 of the other newer, you know, kind of things for this, but, yeah, thanks for Yeah.

Valentino Stoll [00:11:45]:
I mean, I'd love to chat through some of the, just the concepts in there. We don't have to talk about the details of it. But, like, you know, kind of just like, giving memory to, like, agents. Right? Mhmm. Which I thought was really interesting idea. And, you know, how because a lot of people think, like, you know, to your point of, like, oh, making a copywriter or a dungeon dragon master, right, like, these are all kind of, like, just roles. And a lot of people think, oh, like, you could just do that with chat gpt, give it a system message or precluding user messages to say, hey. You're now this role, like and a lot of people just say, okay.

Valentino Stoll [00:12:23]:
Well, why would you need any other service or any other, like, you know, advanced thing where ChatChippity is just gonna do that for you. Like, you know, what are you doing on top of that? Right? Like, what what is because there's, like, a lot of, like yeah.

Obi Fernandez [00:12:38]:
The key insight there and it which is really the foundation of why me and many other people can do startups on this stuff, is that, prompt engineering to us as programmers seems like no big deal. It's just

Charles Max Wood [00:12:55]:
Mhmm.

Obi Fernandez [00:12:55]:
How you how you use the tool. But, to a normal person who's not a programmer, it really, really seems like programming, which means they're not interested in doing it or not very good at it. You know, it's just not their the way their brain works. So when you give them an option it like, our our profile, at Olympia, like, our heaviest power users that that we've interviewed, a lot of them are nontechnical. They're not they're not programmers. Mhmm. They are early adopters. They're very excited by the technology, and they follow it religiously.

Obi Fernandez [00:13:31]:
But they're not so interested in learning the intricacies and ins and outs of of prompt engineering. And if I mean yeah. Go ahead. Sorry.

Charles Max Wood [00:13:39]:
No. I've just I I kinda wanna bring it up a level or 2 because, you know, we we we went right into kind of the the GPT largely with model and, you know, and chat g t GPT like things and prompt engineering. And so I'm just wondering as far as, you know, your book and some of these other things, is that the scope of your book is covering that stuff? Maybe you don't go into things like, transcriptions with Whisper or, you know, computer vision or things like that. And the other question that I have is is is that where people can jump in, and how do they do that? Right? Because because III wanna make it very approachable for people to go, This is the okay. I'm gonna do AI. This is this is where I jump in. And and it yeah. I don't know if prompt engineering is the way to go.

Charles Max Wood [00:14:27]:
And anyway

Obi Fernandez [00:14:30]:
So here here's my answer to that. The the re the thing that kinda tipped me over to, like, okay. I should start this book now was that, an old friend of mine, Tom Morini, who might be listening to the show, We we were at a conference. I was giving a talk on AI, and he was in the audience and he raised his hand at the end of the talk and said, where do I get started? Like, this is all so overwhelming. And and that was the that was the spark where I was like, okay. I gotta start working on this. And I literally I literally started working on the way home on the flight. And, and then just dedicated a huge amount of hours to try to get as much done as possible on it as quickly as possible.

Obi Fernandez [00:15:17]:
The reason is, my my belief, and I'm not sure how many other people necessarily share this, but, I mean, I think it's starting to spread, is that you don't have to start an AI project in order to use this stuff. Like, you don't have to have, like, an idea for an AI startup or something like that. Like, almost everyone who's currently maintaining or developing an application on on Ruby on Rails can probably get some benefit from taking a particularly complex part of their their code and dropping in a discrete AI component to to handle it. So that is Do

Charles Max Wood [00:16:02]:
you have an example?

Obi Fernandez [00:16:07]:
So the the example that I the example that I like using, so so okay. Anything that involves user generated content, for instance.

Charles Max Wood [00:16:23]:
You mean like a blog?

Obi Fernandez [00:16:25]:
Like a blog with comments and whatnot. And I realized that there's services that that you can use to to do whatever I'm about to describe. But, I mean, basically, anything involves evaluation of user generated content. Right now, you can do very, very easily with, using large language models and, you know, and existing APIs. That that's kind of a no brainer use case. But but when I go into the book, and I've been I've been using it as an example in talks and whatnot, is that you you can actually take parts of your code that are implemented in code. But if you had a little person that you could stick into your code, then you might wanna use, you know, a little person instead. So in Olympia, the example I give is is the example of, the account manager class.

Obi Fernandez [00:17:14]:
And the the account manager class handles, a few different operations, involving changing a customer's account, most notably adding, an assistant to your account. So internally so in the user interface, this is this works with a button. So, like, you you look through a catalog of of users that are available to add. We call them clones. Yada yada. But when the user presses that button internally, the internal API uses plain plain language. The internal API tells the the account manager, user with this ID would like to add this bot config with this ID to their account. And, if if I hear I'll just I'll just pull up the the actual prompt so I can read it to you.

Charles Max Wood [00:18:13]:
So it sounds like, though, you've got some form of interaction that I, as your user, do, and that triggers essentially a prompt to be sent. Right? It's not this explicit function or method call that does

Obi Fernandez [00:18:30]:
There's a com yeah. There's a component in my app which has the role of account manager. Uh-huh. And it and it has, a selection of functions that it can call. And those functions can mutate the customer's

Valentino Stoll [00:18:47]:
account. Right?

Obi Fernandez [00:18:48]:
It also has functions for communicating with the customer or with an account service rep or both. And it also has a finalize because the the request is encapsulated in a change request object, which is tracked in the database. So when it's done handling the change request, it calls finalize, which persists that along with the transcript or what happened. But it gives you a tremendous amount of flexibility, And and, also, it saves you time, development time. So I wrote this very, very quickly. I mean, once once you identify the pattern, you start wanting to look for places to use it. And then the prompt reads, like, the requirements for the code.

Charles Max Wood [00:19:35]:
Okay.

Obi Fernandez [00:19:35]:
Right? So let's so so, the way that you describe this I think it's okay to do this on a podcast so many people are listening. So this is the this is the prompt for my account manager component. K? You are an account manager for Olympia, a platform that provides AI powered consultants and personal AI clones. Your role is to assist Olympia clone owners by answering their questions, helping them configure and manage their AI clones, and executing tasks related to the clones in Olympia platform. That itself is an approach or or pattern that's described in the book. You're basically giving a role to the AI. So and then now here come the instructions. When interacting with clone owners, you should provide clear, concise, helpful responses to inquiries about account account management, billing, platform features, and clone configuration.

Obi Fernandez [00:20:25]:
The reason I can do that is because I give it functions to be able to look up that information and convey it to the customer. Assist with tasks such as initiating client conversations with their AI clones, modifying 0II think I picked up the wrong 1. I was like, I'm reading the prompt from the, clone manager, not the account manager. Let let me let me take a step back for a second. I I think if you're listening to what I was just saying, you kinda get the the gist of it. You can give instructions to these things. There's another 1 which has smaller scope. This this one's to help specifically with people that run clones on Olympia.

Obi Fernandez [00:21:01]:
There's 1 that is an account manager for regular customers. And 1 thing you can do is add a particular bot config to your account. We roll this out. And within a week, week and a half, JP, our customer service guy goes, hey. We have a problem, with the the clone adding. Someone added a clone and their account is inactive. Would you like me to create a Notion story for it? And I opened the code while I was chatting with him in Slack about it, and I looked at it. And I said, no.

Obi Fernandez [00:21:35]:
I think I got it. And I added in, a statement to the the prompt, which said only allow addition of bot configs if your account is active. I literally just typed that into the prompt of the account.

Charles Max Wood [00:21:55]:
Oh, interesting.

Obi Fernandez [00:21:57]:
You get what I'm saying? Uh-huh. I didn't have to add a conditional to the code, you know, extra checks or unit tests or, anything.

Charles Max Wood [00:22:06]:
Modify your pundit scope or your pundit, rules.

Obi Fernandez [00:22:10]:
Yeah. I I literally just, enhanced the instructions to the account manager and, did a quick manual test, and, yeah, it worked, and we're good to go. And in fact, you know, if you think about what you would do traditionally with that, like, you might, you know, if you have a state which is invalid like that, then you might have to actually, like, write an email template for it, you know, in a mailer and and whatnot. And in this case, you don't because it's instructed to communicate with the user and let it know what happened with the operation. So if their account is not active and it doesn't add the bot configurable, send them an email and say, sorry. Your account's not active, so I I couldn't do the thing you asked me to do.

Charles Max Wood [00:22:58]:
So going back to the idea of getting started then, I mean, is that is is this the kind of thing you're you're encouraging people to get started with? It's just, I

Obi Fernandez [00:23:08]:
don't know. I mean, this

Charles Max Wood [00:23:08]:
is gbt and start writing prompts and

Obi Fernandez [00:23:12]:
it doesn't use chat gbt. It uses the it uses, you know, the API, and there are there are I have my own framework called Ray, and there's, other, you know, frameworks for for how to use this. You could, you you could go at it at different abstraction levels. Right? So, I mean, for for the most part, you need to understand the chat completion operation. And so this this concept of you you give a system directive that kinda sets the the stage for what you want the AI to do, And then you have a conversation with it, where where you say something, and then they say something, and then you respond, so on and so forth. So you have to understand that modality. You can engage with that modality at a couple different levels of of abstraction. The those abstraction layers are provided to you by a library.

Obi Fernandez [00:24:06]:
So for instance, Ray, the the 1 that I described in the book. And, I mean, you can go whole hog, which is what how I would characterize the account manager example I've been talking about. But you can also, do much, much smaller things. So, like, 1 of the patterns in the book is is the the predicate pattern, and that's just a small component that answers a question yes or no.

Valentino Stoll [00:24:27]:
Mhmm.

Obi Fernandez [00:24:29]:
We answer yes, no questions all the time in our code, but they're usually very, very narrow scope. And if they're not narrow scope, then they tend to be brittle or difficult to maintain because pretty much any unanticipated state will break it. Right? 1 1 of the great things about AI is that if you implement a component like this with with an element back end, it can sometimes handle unexpected situations very, very, smoothly. You know?

Valentino Stoll [00:25:02]:
Yep. I think you bring up a great abstract point in it's a lot see, there's a lot of people that are like, oh, no. Like, you know, all this AI stuff is coming from my job in x. Right? Like and the programmers for sure, like like, you see this and you're like, oh, why don't we just have a programmer bot? And it takes requests, and we just, you know, chat to it, and it'll, you know, make our application for us. Right? Like which is not totally unreasonable, but is unreasonable in a lot of ways. You know, hallucinations come to mind. Like, how do you make sure that the account manager, right, like, just doesn't start deleting things and, you

Obi Fernandez [00:25:41]:
know, it's

Valentino Stoll [00:25:41]:
like a it's like a rogue function that is created along the way, and, now it's part of that tool chain. Like, I I know there are a lot of checks that you can put into place, you know, to prevent that kind of stuff. But, you know, a lot of it is just, like, getting natural language, though, brought to whatever that you are doing to complement it. It's how I've seen it work most effectively, which seems to be how you've kind of created your framework here for Ray. Right? Whereas it's complementing the experience that you already have as a real developer.

Obi Fernandez [00:26:16]:
Right.

Valentino Stoll [00:26:16]:
Just making it easier to take advantage of this new tool, which is really AI. Do you see it that way, or do you also see, like, maybe it is coming for your job?

Obi Fernandez [00:26:27]:
No. I don't think I well, I don't think it's coming for any of our jobs anytime soon. And most recently, I got I got access to something called GitHub, Copilot Workspace. You gotta sign up for it. There's a waiting list. It's a preview product where the interface is super cool. And, like, kind of the promise of it is super cool. Essentially, you you attach it to your repo, and then you start a, a session.

Obi Fernandez [00:27:01]:
And in the session, you go, I need to fix this issue, you know, or I need to add this feature. You describe what it is in plain language, and it scans the code, and then it comes up with a plan of action. So a checklist of, like, I need to do this and this and this and this. I need to change this this class and so on and so forth. It kinda looks like what a disciplined developer would do when you sit down because you picked up a feature or an issue. Mhmm. So so far so good. I was really impressed with that part.

Obi Fernandez [00:27:34]:
And then I said, okay. Go. Right? Like, I I asked it to to change a particular class. So maybe I was too ambitious. I wanted to refactor, you know, 3 3 different, very similar implementations into 1, thing. I just I won't get into bloody details. But, so I had go, and then I thought about it for a while, and then it pulled up the the code, and I saw the changes in action. So that part's super cool.

Obi Fernandez [00:28:07]:
And then it highlighted a bunch of the code the existing code, and it deleted it. And then it put in a bunch of nonsensical methods to replace it that had nothing to do with what it was supposed to do. And, I was most definitely underwhelmed. And now it is very, very possible that this works much better with Python and and TypeScript. But it doesn't know the real's way. That's for sure. Because it it was garbage. It was it was hot garbage.

Obi Fernandez [00:28:43]:
I tried to work with it a little bit. You know? And so now this this needs to get much better. And there there are, I think, very credible reasons to think that it might not get that much better anytime soon. I mean, I think I think right now, 1 1 of the lessons I've learned at Olympia is you have to throw a lot of brute force to get, to get a usable result, you know, in terms of the the these things being, quote, unquote, intelligent and having memory and and figuring out what you want and being able to recalibrate. And the reason I'm able to do that is thanks to the race to the bottom and the price of commodity AI models. So a lot of these open source models that you hear about, oh, Mixtrill and Llama 3 and, you know, WizardLM from Microsoft, so on and so forth. So it's all there's all these model providers competing to, you know, to, in this market for AI services, and they're driving the price to 0, the performance up. So that's great.

Obi Fernandez [00:29:47]:
You can access a lot of this stuff. And I do think that the price will keep dropping for a lot of these things to the point where it'll start to make sense to actually incorporate it into your run time. You know, like, in at the rails level and then maybe in the future at the Ruby level or or even virtual machine level for for doing things like error recovery or, you know, like, you if you wanna crank the dial up, like, if you're okay with uncertainty and non deterministic behavior, which some of us are, you know, you you can crank the dial up and have it do things like, you know, cover for you if you give it the wrong type or, you know, try to automatically handle more pointers and, you know, you know, thing things like that. There's a lot of really interesting, magical, dangerous shit that I I think we'll we'll start getting into, you know, as we get more comfortable with these technologies. And as the cost comes down, because right now, the cost tends to be somewhat prohibitive, especially especially if you're using Frontier models, which is what they call GPT 4 or, or Claude. But, but the smaller, faster models, man, they're so cheap. I mean, it's like point 0004¢, you know, per request, Which for Olympia, you know, the scale that we're at is okay. You know, that's a ban that's a manageable bill.

Obi Fernandez [00:31:12]:
You know, if you're if you're a much bigger company, then maybe some of the things that I'm that I'm suggesting literally don't scale. I mean, they, you know, end up costing you $1, 000 a month or more.

Valentino Stoll [00:31:25]:
On the topic of model choice, like

Obi Fernandez [00:31:27]:
Mhmm.

Charles Max Wood [00:31:28]:
I guess

Valentino Stoll [00:31:28]:
I have 2 questions around that. Like, how do you go about finding the right model for what you're trying to do? And how do you like, what is that expiration process like?

Obi Fernandez [00:31:43]:
What is the what process?

Valentino Stoll [00:31:45]:
Like, what's the exploration process for, like, 3 on that model? Yeah.

Obi Fernandez [00:31:49]:
Yeah. Well, I'm a big fan of a service called open router, and I wish I had an equity stake because I'm evangelizing them all the time. But, they, they power they power a number of really, popular AI based, services like Silly Tavern and, a new product that I'm using, which I'll plug called Novel Crafter. They're super awesome. And the expiration process is basically going to open routers, playground, and and being able to compare, you know, run prompts in parallel and see what happens with different, in, AI models. You know, like, basically saying, okay. Here's gpt 4 as my benchmark and then try to run the same thing against LAMA 3 and Mixedral and whatever. So I'm down the line.

Obi Fernandez [00:32:46]:
And the main reason I use OpenRouter, and I feature it in my in in most of the open source I've done, you know, half dozen open source things I've done, is that it's 1 API key, you know, 1 endpoint, 1 account to manage and and pay for, and it gives you access to everything. OpenAI, Claude, Gemini Pro, you know, Cohere. Like, all all the major models and then a whole bunch of open source models, including ones you never heard of, are available there. It's they probably have a couple 100 now. And, for the ones that are offered by multiple providers so so, like, the example of provider would be, like, gpt 4. It's offered by OpenAI itself, but it's also open, offered by Azure. So, 1 of the things I really like about OpenRouter is that if gpt 4 is down for whatever reason, it can automatically fail over to Azure, which means I don't have to have a, Azure account, which is great because I don't like premature aging. So the, you know, Azure is very, very difficult to set up and work with and everything like that.

Obi Fernandez [00:34:03]:
So yeah. But you were asking that this the selection process is trial and error. I mean, I think there's some there's someone in the Ruby AI Discord that's working on some I think it's called prompt garage or or something like that, which gives you some sort of framework for for doing this in a methodical fashion. But but it's pretty easy to like, once you once you got it going, it's pretty easy to just plug in a a model name and see if it works. And I do tend to TDD these things. I don't leave them in my suite. Or if I do leave them in my suite, they're backed by BCR so that they run quick. But I do tend to tbd my a my AI components.

Obi Fernandez [00:34:46]:
So I'll just sit there and and iterate, you know, red, green until I get what I need. And 1 of those things that you change while you're trying to get it to work is is the model. Trying to get the cheapest, fastest model that will consistently work.

Charles Max Wood [00:35:03]:
So do you walk through this process in your book?

Obi Fernandez [00:35:06]:
Yeah. Yeah. That's fair. Yeah. What The whole the whole first half of the book is kinda approaches, and then the second half of the book is just, like, concrete. This is, you know, this is a pattern. This is the name of it. This is how you use it.

Obi Fernandez [00:35:22]:
Here's example code.

Valentino Stoll [00:35:25]:
Yeah. I will say I I haven't made my whole way through the book, but I I've made a decent amount. Mhmm. But I do like, how you keep reiterating on, like, the discrete components aspect of it, which I think is reflected well in your, like, Ray framework, right, where you're just trying to, you know, encapsulate whatever the task at hand is and keep things small. And I think that's kind of reflective of how the models work in general is, like, you give it too much and it's gonna make more mistakes, which I think is true of people too. Yeah.

Obi Fernandez [00:36:00]:
Yeah. I think of them like little people. There's a chapter in the book called multitude of workers because that that's the approach that I advocate. It's 1 way of thinking about it. It's like, you know, if you had little people that you could put, not little people, but, you know, if you had, you know, human little humanized bits of code, you know, that you could give jobs. But they have to be narrow jobs at the moment. You're absolutely right, Valentino. You give you give these things too much surface area, you know, too much to think about, and they they mess up.

Obi Fernandez [00:36:34]:
The only model that I know right now that can reliably handle more than 1 or 2, tasks or functions or, you know, just that kind of latitude, that kind of discretionary behavior is GPT 4. Supposedly, Opus from Claude can do it, but their tool yeah. Their their framework for invoking functions is more immature. But, like, some of our some of our bots in Olympia have, 20, 24 functions available to them for every single request. So anytime you're talking to them, anytime you send them a prompt, they have 20 or more functions that they can call for things like checking system status, for sending email, for recalling things from memory, real time search, Google search, image search, image generation, you know, so on and so forth. This is kinda like their palette, their tool belt of things that they can use to invoke. And there's yeah. Go ahead.

Valentino Stoll [00:37:42]:
I'm curious, like, the the testing approach here. Right? Like, is it does it start to, like, take a different shape than traditional web development? Right? Like, is it more like because once you get to the multitude of workers, right, and you have more than 1, I could see, like, encapsulating 1 as, like, a thing, like, maybe, like, a Rails model or something like that. And you have, like, unit tests that make sure that it says what it what you want it to in a certain format or whatever. But how does, like, the orchestration aspect, does that also translate the same way to, like, the controller flow, like, to, you know, feature tests or something like that? Is there a parallel to that multitude of workers working together? Or do you kind of just, like is that not really something worth doing?

Obi Fernandez [00:38:28]:
I haven't gotten too far down the rabbit hole in terms of trying to do automated tests for intelligent workflows. There's no reason that you wouldn't, you know, if you were so inclined. Trying to think of how to intelligently answer that question. Speaking of a rabbit hole, I mean, you, yeah, you can start combining these things, and it can start getting pretty, microservice feeling. IIII realized that a couple months ago. It started reminding me of some of the serverless, approaches. You know? Like, I was really, really hot on, microservices powered with Lambda when the serverless framework came out. So this is back, like, in 2015 time frame 2016.

Obi Fernandez [00:39:23]:
I started writing a book on it. And it starts to it starts to feel like you have little microservices inside of your monolith when you incorporate these in rails. Because they have, you know, they have a very, what do I want to call the way you interact with them, like, they're well encapsulated. Because they're black boxes. You know? They're you literally can only do input and output of messages with them. So, it I think it's cool. I mean, I love that approach. And I don't I I tend I'm a hacker at heart.

Obi Fernandez [00:40:05]:
I started as a hacker when I was young. So I don't have a lot of hang ups about, like, this has to be absolutely correct. And, actually, I think you're probably a little too worried about hallucinations, Valentina, just from what what it sounds like. Because the for a lot of these use cases for discrete AI components, I mean, it is well since since you've, since you've put a lot of constraints constraints on what you want it to be, You know, what you want it to do, what kind of output you want it to give. And you you crank the temperature parameter down to 0, which reduces the amount of creativity that the it, like, constrains the possible choices of the model down significantly.

Valentino Stoll [00:40:50]:
That brings me to another question, that I feel like I always get is, like, you know, how do you find out what it's good at and what it isn't? Like, are there, like, definitive things, or is it kind of just, like, learn as you go, aspects of it?

Obi Fernandez [00:41:08]:
Well, I've definitely learned as I go. I mean, there's, there's some things which are, I think, a little too meta. So, like, to maybe I just haven't hit on the right way to to do this, but, like, we do automate, automated evaluations of the conversations in Olympia. And you you said you're Olympia users. So, like, basically, there's a process that kicks off after a conversation idles for, like, 15, 20 minutes. And there's a component that will look at the conversation and give it a numerical grade up to a 100. And, say, this is what the bot did well, and this is what did not so well. And if it drops below a certain number, then it gets flagged, and that goes to a dashboard so we can see kind of, like, the overall performance of the system.

Obi Fernandez [00:42:02]:
And, also, so we can be aware in case that person that user is having a lot of issues, then we can reach out to them and say, hey. It seems like you're having some issues with your Olympia assistance. You know, can we help? And, what we find with these evaluations is that the the model is very, very generous. This is like the nicest teacher you ever had. I mean, almost everyone gets 95. Doesn't matter how how bad like, the the worst I've seen it get is 70, you know, and that's where everything was falling apart completely. And it's very kind and generous to itself, which is, like, kind of a weird thing to say. But, you know, it's like, the bot really, really tried.

Obi Fernandez [00:42:48]:
Yeah. So, that I couldn't get to work right. I mean, there were other things. Used there used to be a lot of this has gotten better. And I think this is just mostly because gpt4 has gotten better. But there used to be a lot of, hallucinations around what we call stalling. So you'd ask your Olympia assistant to, help you research something, and they'll say, on it, I'll get back to you in 3 hours. That that's a hallucination.

Obi Fernandez [00:43:20]:
They have no ability to go do a sync work yet. I mean, we're giving it to them, but 6 months ago, they did not. And, then the person would be like, great. How will I know when you're done? They're like, I'll I'll get back to you. Or sometimes say, I'll email you. What is done? So they're really taking this human nice thing to heart. Right? But that's not that's not good behavior because then you're sitting there. And we used to joke around sometimes that your Olympia assistant could turn into the worst employee you ever had because, I mean, you, we had some users that just keep asking them.

Obi Fernandez [00:43:52]:
Is that done yet? No. I'm still working on it. It's been a week. Are you when are you gonna be done with it? So, thankfully, a lot of this stuff has gone away, but but I wrote a bunch of components to try to detect it. And, it it gets into it gets to the limitations of these statistical models with regards to detecting patterns. You know? Because what you're really asking to do is to to detect a certain pattern, but the the pa it's you can't describe the pattern concretely. You have to describe it very subjectively. You know, like, and it just doesn't do well.

Obi Fernandez [00:44:37]:
In fact, this there's this ARC challenge, right now, which is to try to squeeze state of the art, pattern matching, you know, behavior out of AI. And there's there's, like, big prizes for whoever can get it because they suck at it. And it's really, really interesting to see. If if you think GPT 4 is super intelligent and can reason and stuff like that, you wanna you want you to knock down your expectations a bit, just give it give it 1 of these arc challenges.

Valentino Stoll [00:45:11]:
And just I was I was messing around with this arc challenge, the other day, and it yeah. I was I was like, oh, yeah. Like, the large language models, they they should be able to handle this. No problem. And it's it's remarkable. They can't. Like, they they failed.

Obi Fernandez [00:45:26]:
It's like they're kryptonite. You know? It's like you see it's like you see Superman, like

Valentino Stoll [00:45:32]:
Which is funny because, like, you give it, like, the whole thing is, like, you you can give it arbitrary structured data and, like, it has no problem, like, reason or not reason. I don't wanna use that word, but, like, you know, pulling out the pieces that fit together of that structured data. But anything where it's like, it has no reference to, like, which I guess makes sense because that's how it's built is based on reference material. But you give it anything, like, you know, that it has no understanding previous knowledge of, and it it can't really fit the pieces together. Even though, like, to us, it just seems so obvious. Right?

Obi Fernandez [00:46:07]:
Which Yeah.

Valentino Stoll [00:46:08]:
You know, maybe that we need another modality of, like, pattern matching.

Obi Fernandez [00:46:15]:
Yeah. I mean, there's theoretical modalities that, include, like, real understanding of language. But I don't see anyone working on that yet. So, all of them statistical models are the flavor to Azure.

Valentino Stoll [00:46:35]:
I I would like to just quickly, like, dive into, like because you're you mentioned a lot of it just like, you know, you learn as you go, and it's trial and error. Like, is there any, like, standout pitfalls that people should avoid, like, as they're diving in? Like, to just stay steer clear from doing x. Right?

Obi Fernandez [00:46:59]:
Yes. Absolutely. I have the I have the perfect example of something you absolutely should not do. In in your AI journeys, in your programming using AI journeys, you will inevitably write something that loops. So, basically, it it, does a chat completion, and the chat completion will probably call a function. Mhmm. Right? To to do some behavior external to the model, and then you have to loop so that the model can take the results of the function call and incorporate it into its final answer to the user or to the rest of your system in some way. You definitely want to put some sort of, limit on that loop, AKA circuit breaker, Because you can run you can rack up a huge bill very, very fast if, if you make a mistake there, where it just continues calling, you know, calling again and again, especially if you have large context like we do at Olympia.

Obi Fernandez [00:48:15]:
I had a embarrassing failure like that where some subset of our conversations would, just keep looping and the retry the circuit breaker was broken, which is all that. Test your circuit breakers really well. So we had, all of a sudden, my my consumption shut up from, you know, like, 2 deviations. And then I I was, like, looking at that going, uh-oh. And I go into the logs and open router oh, open router shows you kind of, like, tokens in, tokens out, and the cost associated with it. And it's just the same number of tokens in and out. Just boom. Just, like, multiple per second all the way down just filling blogs.

Obi Fernandez [00:49:03]:
Yeah. With GPT 4. I was like, oh, no. Oh, yeah. Luck luckily, the open router guys kinda helped me out a little bit. But, yeah. That was bad. So yeah.

Obi Fernandez [00:49:19]:
These things are expensive. I mean, you're using you when using the expensive models, they are expensive

Charles Max Wood [00:49:24]:
in

Obi Fernandez [00:49:26]:
a in a way that many of us are not used to. But, you know, because we're used to hero you know, Heroku Dyno costs an x amount and you kinda know what you're getting. You know, a lot of other SaaS APIs, you know, kinda cost a certain amount. Like, you get it. This thing kinda has, like, a it's like a it's like a short squeeze. I mean, it's got, like, unlimited. Boom. Like, you're down a few $1, 000 just because you've made a bug, you know, in your code.

Obi Fernandez [00:49:56]:
So a lot of us are not used to that. So, yeah, be careful with that.

Valentino Stoll [00:50:00]:
Yep.

Charles Max Wood [00:50:02]:
1 thing that I was so you've mentioned journeys and context, and I've heard people talk about training on top of kind of the large language model, which I'm assuming is sort of the same thing. Maybe I'm missing the boat on this. But, yeah, if I have a specific use case where I wanted to, I don't know, refine or summarize, you know, some of the information that it pulled in or things like that. I mean, how how do I get it to be more specific? So let's say that I had a text that I was routinely putting into it, and, you know, I wanted it to summarize, you know I I'm thinking here's what I'm thinking. So I was talking to some friends of mine, and, they were somebody mentioned that it would be nice if there was something that would summarize the bills that go through the legislature every year in in Utah. Right? And there's usually, like, stricken out language, new language put in, and it shows you the context of the, Utah code that it's gonna modify. And so, you know, it looks like the models that you've built on Olympia, right, it can go and it can read this stuff off the Internet and, you know, and there's a way to consume it. So how do I get it to be proficient at getting through that stuff and then giving me what I want to know without having to go and read the entire bill?

Obi Fernandez [00:51:34]:
So the you you might you might be thinking of, of a technique called fine tuning.

Charles Max Wood [00:51:42]:
Mhmm. And, you

Obi Fernandez [00:51:43]:
know, so

Valentino Stoll [00:51:43]:
a lot of

Obi Fernandez [00:51:44]:
people think a lot of people think of training, and there's different ways of make of making models better at a specific purpose.

Valentino Stoll [00:51:51]:
Right.

Obi Fernandez [00:51:52]:
And I I experimented enough with it to kind of know how it works and what it's for, but I I've found that you don't really need fine tuning for most use cases. Like, what you just mentioned, you wouldn't need fine tuning at all.

Charles Max Wood [00:52:07]:
Okay.

Obi Fernandez [00:52:08]:
To to do that.

Charles Max Wood [00:52:09]:
To teach it how to read a bill. I could just No. Give it the link language, and it would could say, oh, it takes this out and puts this in.

Obi Fernandez [00:52:18]:
GGPT yeah. GPT 4 and Claude and Gemini Pro, like, what they call frontier models. Uh-huh. And I call expensive ones. They they know all this stuff already. Okay. In fact, what in the book, I call the I call the process of applying, a large language model to your problem domain narrowing the path. So when you're when you're thinking about the the power of your AMI model, 1 very useful thing to have clear, especially if you're completely new to this, is that it literally knows everything for all intents and purposes.

Obi Fernandez [00:53:00]:
Okay? So

Valentino Stoll [00:53:01]:
Mhmm.

Obi Fernandez [00:53:02]:
The the space of of its knowledge, the universe of its knowledge is so so big that if if you present it with something that's which is your prompt or what you're asking to do, it unless you constrain the scope, it it is literally drawing from all of that knowledge.

Charles Max Wood [00:53:24]:
Okay.

Obi Fernandez [00:53:24]:
And if you think about all of that knowledge, it not only includes knowledge of the Utah legislature and, legislatures in general and procedures of, you know, governing bodies. It also includes legislative bodies and governing procedures of science fiction novels, just to give you just kind of a clear, example of why there's a problem here. Right? So multiply that 10 times, you know, a near infinite amount of different, you know, depending on how general what you're asking it to do, it could draw upon way, way more than what you wanted to.

Charles Max Wood [00:54:03]:
Okay.

Obi Fernandez [00:54:04]:
So so what I call narrowing the path is what probably, essentially, what you mean by training. It's but that's you, you know, figuring out what constraints to give it. And 1 of the main 1 of the main ways you can give it a constraint, which is the most popular way right now, is called retrieval augmented generation. So if you're dealing in factual data, let's call it nonfiction. You know, if you're in a nonfiction domain, so, you know, your legislature example is a piece of that.

Charles Max Wood [00:54:35]:
You could make no. I'm just kidding. Yeah. I guess so. That's weird.

Obi Fernandez [00:54:40]:
I didn't say non fantasy, but not you know, non Yeah. So the you you can tell the model, like, hey. You're gonna operate on you're you're going to to generate output based on this specific content that I give you.

Charles Max Wood [00:54:59]:
Okay.

Obi Fernandez [00:55:00]:
And you add that content dynamically at runtime. That's what retrieval augmented generation

Charles Max Wood [00:55:04]:
means. Okay.

Obi Fernandez [00:55:05]:
Retrieval means you're retrieving it from somewhere other than the latent space of the model. So other, you know, outside of the model. So you're you're you're going out, you're retrieving it. So that might be, like, the existing legislation and then the revision. And then you you you you structured data pattern, which is covered in the book, to present that in the prompt. And then because of their training, the way that these models are trained, they understand that 1 of the main use cases that they're used for is to operate on data that they're provided to from the prompt, And, therefore, they don't hallucinate it. I gotcha. You get it? In fact, there there's a I haven't played around with this yet, but but your listeners might find it really interesting.

Obi Fernandez [00:55:55]:
And if I can't find it in the next 5 seconds, I'll definitely give it to you to put in the show notes. But, essentially, there's a notebook there's a notebook product that my team was was kinda geeking out over. I'm I'm gonna have to find that and give you the link. But but I can describe what it does, which is super cool. Like, you you start out with a blank slate. And when I say blank slate, I mean literally blank slate. So you Uh-huh. You can you give it a prompt.

Obi Fernandez [00:56:24]:
You say, I I want you to help me make a high school curriculum. And it goes, what's a high school? And so Right. You you have you have to provide that information. This is what I mean by high school. And then he goes, great. What's a curriculum? And then you you you have to describe okay. When I say curriculum, this is what I mean by curriculum. And I and I actually think that, this I hadn't seen anything like this until a week ago, and I think it might be new and novel.

Obi Fernandez [00:56:56]:
So it's just interesting to think about whether this will take off. It's actually really, really super useful in my opinion. You're you're constraining it all the way down to 0 and then starting from 0. Right. So that so that the possibility of pulling in other things that you don't care about or that will mess it up are different. So like I said, I'll find the link and give it to you. I forget what it's called. But

Charles Max Wood [00:57:18]:
Right. So I guess the other question and and I think I've kind of inferred this from what we've talked about and some of the other things I've read, but you can have a long running context. And I know context means something in this world, but, you know, a long running context that says, okay. You know, I've restricted what you're gonna run on, and so I'm just gonna run everything else through this context. Right? Then that's effectively what you're doing with your AI assistance that you're providing to people with Olympia.

Obi Fernandez [00:57:52]:
Yeah. Their context, includes everything you've ever talked about Okay. With them. So they're able to, most of the time, you know, somewhat know what you're talking about. I mean, this is the limit. It has fundamental limited

Charles Max Wood [00:58:09]:
Yeah.

Obi Fernandez [00:58:10]:
The technology, but it keeps getting better. And, worst, like, we're, like, so in the infancy. It's so obvious that we're in the infancy of this. And, like, you know, those of us have been around long enough and remember IBM PCs and, you know, Converse like I do, you know, actually live through the expansion of RAM and whatnot. So, you know what I mean? Right. And it feels like there's parallels there. Like, for like, I had a lot of trouble with, with uploads in the early days of Olympia because the context was too small. And for the most part, no matter how good my my text extraction was, let's say, from a PDF, some cases I was using, like, Azure Document Intelligence, which is a really, really good way of extracting structured info from a PDF even if it includes tables and diagrams and stuff like that.

Obi Fernandez [00:59:00]:
But then I couldn't fit it into the darn context of the model. It was, like, together with everything else I was trying to find the context, it would blow up the context and then it would fail. Kinda like running out of RAM. Mhmm. And then, somewhere along the way, I don't remember exactly when, Google Pro came out and it has, like, a 1, 000, 000 tokens of context. Now it has 2, 000, 000 or more. I can't remember. So in the matter of an afternoon, I switched it over so that now, when you're referring to a document, it uses Google Pro, Mhmm.

Obi Fernandez [00:59:31]:
And it just shoves the contents of the document, which could be like a full length novel or more. I mean, 2, 000, 000 tokens is a lot. Right. Yeah. So that it just gives that to Google Pro, and then it asked Google Pro the question that it wants to know. So it it offloads to a model that's really, really good at doing that sort of thing. And I've seen it do incredible stuff. Incredible.

Obi Fernandez [00:59:54]:
Like, I I've seen it be able to extract figures from from a payroll report. But, like, that that had, a couple 100 employees, you know, and, like, basically, help me do some ledger entries related to moving these employees, you know, expense from 1 category to another, and it goes into and pulls it out correctly Mhmm. Which is something GPT 4 doesn't do reliably. And there's we go back to trial and error. You kinda have you you know, you start it is its own field, that it's it's own has its nuances. It has its nooks and crannies of of expertise and things you have to know to do or not do. Like, the the Google thing, I couldn't have told you before I figured it out.

Valentino Stoll [01:00:41]:
Right.

Obi Fernandez [01:00:42]:
That it would fit at that. Right? That was a trial and error. Cool. But you were ask you're asking about context. I don't know if I answered your quest your question correctly. But, yeah, it operates on context. Mhmm. Context is key.

Obi Fernandez [01:00:54]:
So 1 of the main things you need to learn is just how to build that context. The book goes into that part. There's different ways of doing it. Actually, x I think XML will make a comeback to some degree I posted on Twitter the other day. And I talk about it in the book because, XML happens to be a really, really good way of representing data to the, to the model, because it's really easy to parse out afterwards. You know, XML in, XML out. I can't even you know, Obi from 10, 20 years ago is cringing. But, you know, it's like, it's really, really easy to

Valentino Stoll [01:01:29]:
Yeah.

Obi Fernandez [01:01:29]:
To pull information out of of XML, compared to other things that I can give you.

Valentino Stoll [01:01:37]:
Yeah. You know, it's really funny. You know, everybody always asks, you know, how does how does it adapt and to learn new things or, you know, how does it learn x? And it doesn't.

Charles Max Wood [01:01:50]:
Doesn't. Right.

Valentino Stoll [01:01:52]:
You have to you have to tell it, and then it can say it's it's kinda funny because, like, the process is, like, you know, okay. You give it the prompt to do this, and then you have a prompt that just, like, matches it with the data that you already have that can then give it to the prompt again and then be like, oh, yeah. It's this thing that you told me that it is.

Obi Fernandez [01:02:14]:
Yeah. I don't think we'll get to AGI personally with the current approaches. I mean, I could I I could be wrong, but I mean, the the the way I see it, it it really real like, in order to be a full human replacement, it has to really learn on the fly. Like, there there has to be a modification to the way that inference works based on new information. And, you know, some some people think you can just brute force your way out of that problem. So, like, if your context is long enough, you can just add everything you've ever learned to the context.

Valentino Stoll [01:02:59]:
Mhmm.

Obi Fernandez [01:03:02]:
I would happily do that approach in Olympia, except that, it's so damn expensive that that it's unfeasible, you know, to do it. Like, I could pro like, I don't know this for sure. I haven't analyzed it very well. But but just kind of based on my gut, and the amount of information that I know that that customers, put into Olympia, I could probably fit all everything I know about a customer into a single Google Pro Google Gemini Pro context, which would be awesome, except every time I ran that, it would cost me, like, $4. And, you know, and a monthly plan is is, you know, ranges from 20 to $67. So we're still orders of magnitude away from being able to do that. Mhmm. When we get to the point where you can do that, where you can fit all the email correspondence you've ever made done or maybe just all the sent messages, let's say, into a single context, then, yeah, you got something.

Obi Fernandez [01:04:05]:
But at the moment, it's too expensive.

Charles Max Wood [01:04:10]:
Makes sense. So I think we're kind of getting toward the end of our time, and it feels like we're kinda winding down anyway.

Obi Fernandez [01:04:23]:
I'm trying

Charles Max Wood [01:04:23]:
not to

Obi Fernandez [01:04:24]:
ramble. Just be good podcast guys.

Valentino Stoll [01:04:27]:
I I could have you on for another hour if you want.

Charles Max Wood [01:04:29]:
I know. I'm sitting here going. I mean, I definitely have more things to ask. But,

Obi Fernandez [01:04:35]:
I mean, 1 of the

Charles Max Wood [01:04:36]:
things we did cover,

Obi Fernandez [01:04:38]:
which, you know, which, in in retrospect, I I wanted to cover because it's it's for this audience is, I have occasionally been discouraged by the amount of Python you know, how strong Python is in this field. Right? Uh-huh. And and, like, fundamentally, there's there's no reason you know, there's no, like, technical reason why it couldn't have been Ruby. And, you know

Charles Max Wood [01:05:10]:
I hear people talk about the math libraries and the data libraries and, you know, how those feed into training some of the systems at the lower level. But where we're talking about some of these other models, you you're not necessarily building those models. So, I mean, is there a reason why

Obi Fernandez [01:05:27]:
it's pipeline? When it comes to you. No. There's not. It's just cultural. It's just Okay. It's just inertia. So, while sometimes I found it discouraging, what what I've managed to flip that around to as the eternal optimist is, it's a positive because there's a wide open greenfield to do all sorts of cool new gems. And you can, in some cases, draw inspiration directly from an existing project in the other world.

Obi Fernandez [01:06:01]:
Like, the the most, I mean, super clear case of that is langchainrb. So, like, 1 of the main libraries that people use to do this kind of stuff in in Python and JavaScript is called langchain. Mhmm. There's even a startup around it that's made a lot of money, and then, this Ruby is named Andre, created Langchain RB and just kinda lifted the whole approach and and wrote it in Ruby, and that's become a thriving project of its own. I I personally don't use langchainrb because III want it to feel like rails. I don't want it to feel like langchain. Like, when I'm when I'm when I'm using these things in my app, I don't want a whole new paradigm, a whole new architecture, a whole new set of things to learn. Right? Like, I wanna apply it to the paradigm I'm already familiar with.

Obi Fernandez [01:06:52]:
But, nonetheless, I mean, that's a pretty good example of a project that got very big and very popular very fast because it basically just said, okay. Well, that's popular over there. Let's make it in Ruby. I I think that there's dozens, if not 100 of of things that could be done in that way. The smallest, you know, kinda lowest hanging fruit is just client libraries and SDKs for popular, providers and SaaS apps and things like that. So, like, Alex, the owner of the the Ruby AI Discord, got really famous with his Ruby Open AI library, which is, you know, used by millions of people, I would say, is probably not a stretch at this point. So super, super popular library very fast. I've done a bunch of little, you know, client libraries for, like, for OpenRouter, for Anthropic, etcetera.

Obi Fernandez [01:07:44]:
Like, there's so much greenfield opportunity to to just come in and do cool stuff. And that that's why I would rec you know, if you're interested in that, especially if you're starting out your career, you know, and you want a popular open source project under your belt or, you know, to be associated with, I think the timing couldn't be better for that. Really, really couldn't be better. And the other thing too is that it's not like it's not like years past where the scope of innovation was so narrow that if, like, you missed the boat, you know, doing 1 of these open source projects or gems, you know. Oh, well. That's it. Gotta wait 5 years for, like, the next big thing to come along. I mean, there's literally a big thing happening every 2 week 2 or 3 weeks.

Obi Fernandez [01:08:30]:
So, you know, that also just makes it even more wide open. You know? I think it's the coolest place to be. I mean, it's not the end all be all. It's not a silver bullet. It's definitely not a hammer you should hit every, you know, nail in your application with. But but it's incredibly cool. And and, again, I go back to this, like, you know, most of the people I've seen that have objections to using AI in their projects or whatever. They, you know, they they'll say, like, it's a sarcastic parrot.

Obi Fernandez [01:09:03]:
It's not actually intelligent, or I can't get deterministic results, or it hallucinates, things like that. And I can tell you as a practitioner with it for last year that a lot of these a lot of these objections don't really make sense to me. Right? Like, I do come at it from a hacker mentality, so it's kinda like whatever's clever and works, you know, is cool with me. Doesn't necessarily have to scale. Doesn't necessarily have to be the most, pure or, you know, correct, quote, unquote, way to do things. You know, but when you come at it from that mentality of, like, hey. What what can I do with this that's cool or magical or amazing or, you know, really impress my friends and my grandma? You know, like, this is, I can't hype it enough in other words.

Charles Max Wood [01:09:50]:
Nice. So tell us about your book a little bit because Mhmm. I mean, I found patterns of application development using AI. So is this not Rails specific, or is this not the right book or what?

Obi Fernandez [01:10:02]:
No. It's not it's not supposed to be Rails specific. I mean, I I'd I've been doing Rails day in, day out for for almost decades at this point. So that's what I chose to do my examples at. Also, I I was very, very happy to, like, lift code directly out of Olympia because, I mean, 1 thing I personally hate in in textbooks is, like, you know, the car example if you're doing out to, you know, o 0 or whatever. It's like nonrealistic examples or stuff that's super academic or just does doesn't have practical applications. It's like, okay. Well, 1 way to prove that something has a practical application is, like, here's literal code that I work, that's in production.

Obi Fernandez [01:10:47]:
Here we go. Here's an example. You know, especially for something like patterns, instead of being all hand waving, like, you know, like, providing a concrete specific explanation. So I try to do that as much as possible in the book. You could probably rebuild Olympia with with what's in the book. You know, I kinda give away a lot of our, approaches and secrets and things like that, which is cool. And, I'm planning a Python edition and a JavaScript edition down the line for people who just can't get over the fact that the the examples are in Ruby. But the the the patterns are not Ruby specific.

Obi Fernandez [01:11:24]:
The patterns are patterns for how to use AI in your normal application development. You could use those same patterns in in Django or Labarel or whatever you want, Java, etcetera. They're they're not specific to Ruby, and that's why there's no Ruby in the title or anything like that. And I I mean, I'm I'm very influenced by the Gang of 4 patterns book, by Martin Fowler's book on patterns of enterprise, application architecture Mhmm. By Greg Hopi and Dave Rice's book on, enterprise messaging. You you know, the Mhmm. Or integration rather. That those were, like, the formative books of my career coming up doing what I did.

Obi Fernandez [01:12:14]:
So they they were very influential on me. And, I like the approach. I think that they were invaluable for couple generations of of people doing what we do. So that's what I went for. You know, I said I said I think there's an opportunity here to write an evergreen book that really kinda covers the fundamentals of how you integrate this technology with your normal application. So things I don't go into in the book are, will AI replace you as a programmer and, like, all the philosophical Not like not at all. I just give 1 sense it's mentioned in in the preface. So I don't talk about this.

Obi Fernandez [01:12:52]:
The other thing I don't talk about is AI generation code. It's cool. Whatever there's awesome projects like, Scott Werner's blueprints is cool, so plug him. Copilot is cool. I use it all the time. I've been experimenting with other things. I think Copilot is probably still the best. But I don't get into it because to me, it's not it's not interesting and it's not evergreen for a book.

Obi Fernandez [01:13:18]:
You know, it's literally just a tool. You install it. You get good at using it, and you're done. And if they're gonna continue changing it every year. Right? So there's no point Right. Blocking book. The stuff that's in the book is truly meant to be evergreen. Like, it should be good 5 years from now, which is why we're actually gonna go ahead and make a a print copy of it.

Obi Fernandez [01:13:38]:
So you'll you can get it on Leanpub right now. It's 1 of the best selling books on Leanpub right now.

Charles Max Wood [01:13:47]:
Yeah. I put a link in the comments. So if you're on Twitch or Facebook or YouTube, you could see it.

Obi Fernandez [01:13:55]:
And, and, yeah, I mean, it's, like, 400 pages. It's I mean, it's meant to be a good solid reference book that you that you refer to. And you you should be able to pop it open on any page and find it interesting. I mean, it's not, the the the first half of it is meant for people who are relatively new, I would say. You know? The second half is really, like I said before, the concrete patterns, the things you'll refer to time and time again. I mean, you know, when I say I'm influenced by those books, I mean, they're literally sitting here next to me.

Charles Max Wood [01:14:28]:
Yeah.

Obi Fernandez [01:14:29]:
You know, as kind of my go to for I I thought there might be some overlap, actually. And and I if I remember correctly, the only 1 that this is, the Fowler book. But but there's, there's a pattern for proxying, your API. It is called am I gonna be able to find it real quick? Yeah. API facade. So so when I started the when I started working on the book, I was like, oh, I'm probably gonna find patterns that are very similar to the Gang of 4 or, you know, enterprise integration patterns and things like that. And then there ended up not really being that many, if anything. I mean, there's some that kind of allude to older patterns.

Obi Fernandez [01:15:19]:
An API facade was the 1 that was kind of, like, very, very similar conceptually to the integration pattern.

Valentino Stoll [01:15:26]:
Mhmm.

Obi Fernandez [01:15:28]:
But at the same time, it's completely different. This is essentially early in the episode, I mentioned that, like, you know, when you're talking to an Olympia assistant, it might have, like, 20, 25 different functions and can call. You can't, at least with the current state of the art technology, you can't give it, like, hundreds of functions to call. Basically, you're you're using up too much context. It's too expensive. It'll get confused, etcetera, etcetera. So for instance, how do we do integration with GitHub or Gmail? You know, APIs that have surface area with, like, dozens or hundreds of of endpoints, you you use what's called an API facade. So there's basically 1 function, which is the Gmail helper function that's exposed to your main assistant.

Obi Fernandez [01:16:14]:
So anything it wants to do with Gmail, it can just send a request to the Gmail helper function. Gmail helper function in turn is another AI that has all the different operations.

Charles Max Wood [01:16:25]:
Oh, interesting.

Obi Fernandez [01:16:27]:
So you literally go to the g you know, if the user asks our assistant, hey. Check if I have any urgent messages from Charles, The assistant will call the Gmail helper function and say, do I have any urgent email messages from Charles? So it's proxying it. Yeah.

Charles Max Wood [01:16:46]:
Right.

Obi Fernandez [01:16:46]:
And then that AI has the ability to scan your inbox, to search your inbox. Right? And that AI has all the full documentation in the prompt directs and things that you need to in order to be able to use that API. Same thing with GitHub. So we have, like, a GitHub integration so you can actually look at open PRs and and and do operations and things like that. So that that's, like, 2030 functions itself. So you do it with a helper. So so that that's what I in the book, that's what I mean by patterns. Right? So I identified that this is a pattern that's used more than once, you know, and should be reusable by other people.

Obi Fernandez [01:17:23]:
So I generalized it, describe it, document it in the book.

Charles Max Wood [01:17:28]:
That that is really interesting. I'm just imagining, I don't know, a podcast assistant that's, you know Yeah. Hey. Hey. Obi said something he didn't want to say at about 15 minutes in, and it sounded something like this. Can you go take it out? And then it, right, it goes to the audio editor assistant. The audio editor assistant knows how to open up the file and scan and understand, okay, this is where it will be said the thing. And then

Obi Fernandez [01:17:58]:
Right. That that's the that's the premise of multitude of workers. Uh-huh. So we can call it multitude of workers because in like, in your example, you'd have a worker who is the podcast editor worker. So that, you know, has a specific set of tools. And it just sits there waiting for a request for that, which is why it's reminiscent of microservices, you know. And architecturally, I think you think about it in a similar way.

Charles Max Wood [01:18:25]:
Yep. Cool. Well, I wanna be mindful of time. And we scheduled this to end in, like, 3 minutes. We're gonna go over a little bit. But this has been really fascinating. I just went and picked up your book. Awesome.

Charles Max Wood [01:18:40]:
I didn't know I could have it yet. So as soon as Oh, okay. As soon as I went and looked for it, I was like I was like, oh, I'm buying that sucker. Yes. It it is mine. It is on my computer. So Oh, awesome. Yeah.

Charles Max Wood [01:18:54]:
So I'm gonna go ahead and move us to pics. Before we do that, though, Obi, if people wanna connect with you on this stuff, I see you all the time on the, Ruby AI Builders Discord, and I'll I'll get a link in there for that. But are there other good places to find you?

Obi Fernandez [01:19:12]:
Just, you know, usual Twitter, l b fernandez at Gmail if you wanna email me. The product is olympiadot chat. We got a little community of users there. Yeah. That's pretty much it. I mean, some some of my other channel like, if you find me on Instagram and stuff, it's all gonna be my music career stuff. So, I mean, if you're interest you like electronic music, look me up on Spotify. It'll be Fernandez.

Obi Fernandez [01:19:39]:
I just recently made a spreadsheet of all the music I've ever made, since I started. And I'm I'm over a 130 original compositions. So about half of them never been released. But but, yeah, it's come come a long way with that as well. So

Charles Max Wood [01:19:57]:
Awesome. Alright. Valentino, what are your picks?

Valentino Stoll [01:20:04]:
Along the, subject of today, I'm just gonna do 1 pick. I saw this talk from doctor Jim Fan, who's part of NVIDIA's new gear group, which is Generalist Embodied Agent Research. It's fascinating stuff. But he gave this talk at the last NVIDIA conference on, the new foundation agents that they're building, around robots and teaching them to learn using virtual dojos. So there's, like, a very manual process involved in training a robot to do tasks where they have to, like, do a lot of trial and error of their own to autocorrect themselves. And so they've basically found a way to create virtual environments using, like, some previous projects they did with Minecraft to, like, teach skills to robots in virtual environments so that they can do the repetitive process of trial and error virtually. It's super fascinating stuff. And he gives a great talk on, how these foundation agents work, and I recommend you check it out.

Charles Max Wood [01:21:15]:
Nice. I'm gonna throw in a few pics of my own. So my friends came over last night. We were playing board games. And, yeah, I think people are pretty used to me picking the board games. And, we played lost ruins of Arnak. So I'm gonna pick lost ruins of Arnak. I just pulled up the wrong thing.

Charles Max Wood [01:21:39]:
So, essentially, you have, like, a technology track. You have exploration you can do. You can do deck building. All of those things play into your score at the end. And so and it's it's a little bit more involved than kinda your your regular game. Board game deep weights it at 2.96, but it does say that you can play it with the players that are 10 10 plus. The community says 10 plus. The game says 12 plus.

Charles Max Wood [01:22:15]:
That's probably about right. There were 3 of us playing last night. We had a great time. So, anyway, it came out in 2020. There is an expansion, and I haven't played the expansion in a while, so I'm not gonna shout that out. But, anyway, lost ruins of Arnak. It it's a super fun game. It's just a little more involved than sort of your settlers of Catan or whatever that, you know, comes in about a 2.

Charles Max Wood [01:22:44]:
So, another pick I have. So I've been working my way through, Ayush, who's not here today, but through his book, the rails and hot wire codex. And I actually got a working version of my I'm I'm basically rewriting top end devs because they're just things I want different vastly different. Right? You learn all your lessons, and then it's like, okay. Well, I could refactor all this, but that would be more work than just, you know, writing it new. And the other thing is is I can add in things that I didn't have before from the start, like internationalization and accessibility and things like that that I didn't do last time. Because adding that in after the fact, I miss stuff everywhere. So this way, I can just be complete on putting it together.

Charles Max Wood [01:23:36]:
But yeah. So he has his chapters on turbo native. And so yeah. So I have an iOS version of the app running. I have, most of an Android version running. I just got it running so that it brings up the website in the in the native view, but it doesn't have the navigation or anything else on it yet. Turns out that's a little bit of work to get that to go, but it's very cool. So I'm very happy with that.

Charles Max Wood [01:24:05]:
I'm gonna pick, the rails and hot wire codex by Ayush Nwatiya. And then, yeah, there was something else I was gonna pick, and I just can't think of what it is at the moment. Oh, I've been watching the Euros. I'm I'm a soccer nut. And so, yeah, been pretty happy with the way things are going so far. So, my team, if they're in and they weren't in the World Cup and it broke my heart, is Italy. Just because I lived there for 2 years, Actually, went to a couple of soccer soccer matches while I was there, and I just I just love watching those guys play. If it's not Italy, then it's usually England, France, US, and honorable mention is Netherlands.

Charles Max Wood [01:24:51]:
And, so all those teams are playing in the Euros because it's European championship and except for US. And, all of them won their matches, so I'm I'm enjoying so far watching them win. I think the next game on the schedule that I'm gonna be watching, though, is France versus Netherlands, and so, of course, I have to root for France. So we'll see how that goes. But, yeah, it's it's been awesome. So if you enjoy soccer, let me know. You might see me tweet about it some. But, yeah, those are my picks.

Obi Fernandez [01:25:24]:
You a pick?

Charles Max Wood [01:25:26]:
Yeah. Do it.

Obi Fernandez [01:25:28]:
I've been playing with this, thing called Yudio. It's like audio but without the a. Uh-huh. Except I'm you I'm so I I think generating a song is a, you know, like, a full song, especially if you use AI lyrics is is a just a novelty. I don't know that this will have any sort of, transformational impact on anything. However, it's really fun for doing things like impressing your barber, which I did here in Mexico City a week ago. Because we were he asked me about AI, and I said, did you know I can make music? And then I made a, like, a sono like, a banda sono sonora, you know, kind of, thing about Marcos, the barb the barber who had a seat Nice. And and he was so floored, that he was laughing his ass off and then it literally interrupted all the other barbers to show them in the shop.

Obi Fernandez [01:26:25]:
So it it has good novelty value, you know, making operas about your coworkers and stuff. But what I what I'm using it for as a as a semi professional producer is to to generate, vocals and to get ideas based on my own music because a week ago or so, they added the ability to upload, bits of music. So, now now that turns it into a bit of a a professional tool. Because for instance, I can take an instrumental that I've made. I I make mostly instrumentals. And I can upload it, and then I can put in my lyrics, and it will sing my lyrics. And I can iterate on that the same way that you would iterate an image with Midjourney or or Dolly until you get something usable. And then you without getting into the workflow too much, you can extract the vocals and actually integrate them into your music.

Obi Fernandez [01:27:17]:
So, for any of you out there listening who, like me, you know, love the idea of writing lyrics or singing but you can't sing worth a damn, then, this is a really awesome tool to use for that. And it's a it's a lot of fun. Awesome. Good. That's my pick.

Charles Max Wood [01:27:39]:
Cool. Alright. Well, thanks for coming, Obi. This was

Obi Fernandez [01:27:43]:
was great to catch up again too. Likewise. Thanks, Valentino.

Valentino Stoll [01:27:48]:
Yeah. I look forward to to reading the rest of the book for sure.

Obi Fernandez [01:27:52]:
Right on. Yeah. Let me know what you think. And, oh, yeah. That's it. I had something else to say, but I forgot.

Charles Max Wood [01:28:00]:
No. It's all good. We'll just wrap it up, though, and till next time, folks. Max out.