Genetic Algorithms With José Diogo Viana - EMx 211
José Diogo Viana is a Full Stack Engineer. He joins the show to talk about, Genetic Algorithms to optimize an Asset Portfolio and his company, "Finiam". He begins by discussing his company, what clients they cater and the services they provide. Being a Fintech company, he also tackles their projects in Finiam and what frameworks they usually use.
Special Guests:
José Diogo Viana
Show Notes
José Diogo Viana is a Full Stack Engineer. He joins the show to talk about, Genetic Algorithms to optimize an Asset Portfolio and his company, "Finiam". He begins by discussing his company, what clients they cater and the services they provide. Being a Fintech company, he also tackles their projects in Finiam and what frameworks they usually use.
Sponsors
Links
- Genetic Algorithms to optimize an Asset Portfolio
- Finiam Blog
- zediogoviana
- GitHub: zediogoviana
- LinkedIn: José Diogo Viana
- Twitter: @zediogoviana
- Reach out to Adi for work opportunities + Founding Engineer roles: aditya7iyengar@gmail.com
Picks
- Adi - Temu
- Adi - Roborock
- Adi - Build Your Own Framework
- Allen - World War Z: Aftermath
- Diogo - The Last of Us
- Diogo - Code Bullet
- Diogo - Range
Transcript
Allen_Wyma:
Hello, welcome to another episode of Alix Mix hosting today, this is Allen Wyma. today, I have Adi Iyengar on the panel. Hello adi,
Adi_Iyengar:
Hello, hello,
Allen_Wyma:
And we have the go. The I say that wrong again.
José Diogo_Viana:
Yeah,
Allen_Wyma:
I did it
José Diogo_Viana:
slightly,
Allen_Wyma:
in so
José Diogo_Viana:
we
Allen_Wyma:
we have.
José Diogo_Viana:
know.
Allen_Wyma:
You're a man of many names First of all, so it's hard
José Diogo_Viana:
Uh.
Allen_Wyma:
for me to keep check which
José Diogo_Viana:
uh.
Allen_Wyma:
name to call you, because you keep telling me you keep changing your name because too many people around you choose the same name. So you're You're a trend center. basically by name.
José Diogo_Viana:
Yeah, if you open my website, the first line that you probably see there is that people call me one or the other, so it's actually a pretty common thing for people to miss miscall. me, pere.
Allen_Wyma:
Yeah, I mean, first of all, I belive you. Your your first name is actually Jose, Right, But that's you said. That's a very
José Diogo_Viana:
Yeah,
Allen_Wyma:
common name. It's no relation to the other Jose. We know, Right
José Diogo_Viana:
What you'll say?
Allen_Wyma:
Joseph Line, you don't know Josehalin,
José Diogo_Viana:
Oh, it's the same name. Yeah, but no relation.
Allen_Wyma:
But no, you're not related. Okay, Just want to
José Diogo_Viana:
Though,
Allen_Wyma:
confirm you weren't named after him? Right?
José Diogo_Viana:
probably I'm a few years younger. I guess not sure if my parents knew him. but
Adi_Iyengar:
How old do you think Joseph?
José Diogo_Viana:
yeah,
Adi_Iyengar:
Aim is man?
Allen_Wyma:
I didn't say like that. I mean, you know, maybe he's just that powerful when he was young. Everybody you know
José Diogo_Viana:
Everyone
Allen_Wyma:
knew
José Diogo_Viana:
who knew
Allen_Wyma:
him.
José Diogo_Viana:
him.
Allen_Wyma:
Yeah, it's not easy
José Diogo_Viana:
I don't
Allen_Wyma:
to create
José Diogo_Viana:
know.
Allen_Wyma:
a programming language Is awesome as elixir. right, so I think
José Diogo_Viana:
Yeah,
Allen_Wyma:
you know he
José Diogo_Viana:
true.
Allen_Wyma:
definitely has some skill. Um,
José Diogo_Viana:
but
Allen_Wyma:
Yeah,
José Diogo_Viana:
yeah, I think I don't think my parents are that much the kanosofs on that sense of not sure if that was the case.
Allen_Wyma:
But I mean kind of going back to to you, right? You're saying you're a full stack developer. Right?
José Diogo_Viana:
Yeah, true.
Allen_Wyma:
What kind of company were ctuallyworking at? What are you doing every day
José Diogo_Viana:
Okay. good question. So my company is called Finium. We work as a pinte consulta Cy team. We are a pretty small team and we help other businesses or founders to get their their projects from the ground Up to being out there. We usually work with in small bursts outside like six to nine months of the project, which Ust get the product out there and help the clients build an internal team and we jump on to the to the next thing from on the last eighteen months side say we've been working pretty much on blockshan and Web, three related stuff, but yeah, we do a lot of things. I'm a full stick, but I spent most of the time on the vicend Being part of a smaller team. We need to be everywhere, so we always have that thing.
Allen_Wyma:
Now for? for you guys? You're saying that you go to companies and you help train up their their team? Is that what I heard correctly?
José Diogo_Viana:
I wouldn't say, train. We help. If the client starts by himself, We help him recruit and and build their own team. We can give them training. We have earned that in the in the past, because at the time the client started with a couple of I would say, friends and they were this, but they didn't have the specific knowlege of the technology we're using at the time, so we spent a couple of months training game into into.
Allen_Wyma:
Are you specifically only elixir or using some other technology? Or how does that work?
José Diogo_Viana:
Yeah, good question, We, we use a bunch of things, but I utsalakseries top the top of the stack. We have used no j s for forbakends or Ruby on rails, Depending on the on the situation and the front end, we most mostly pick reactor or spelt, We have never used full Mona application with a like realixeries. Most of them I was only in the back end with with an a P. I.
Allen_Wyma:
Now? How is a conversation to to throw up a lixir, Because I know I always have this issue. People say. a what's that like? Is there like a lot of reservations on using Elixir, Or I mean what's the experience like
José Diogo_Viana:
Yeah, it has happened before mostly because people don't don't know it. Not really sure why I would say that it is pretty much a lot of exposure nowadays, but still a lot of people don't don't know it in the space, not even by when you say elixer, compelled to their long vital machine, et cetera, just to make sure that people get to know most of the time To one they don't know the other also, and you kind of need to show what advantages you have. all that resilience and faltallernce. The getting stuff quick out of the out of the ground Because you can, I think we can spend much time talking to a prospect, client talking, We can do this. You can do that, and in a couple of days you can actually get them something to to see on With a liker. You can. You can get it. so I think when when they get that and you get that blow you, you actually end up in. Yeah, okay, we can go with this.
Allen_Wyma:
Now? I mean, elixerisreally. not traditionally like a Fin tech kind of language. You usually think it's think like job or C plus plus or things like that. I mean, for people that do know it, I mean, is there pushed back to say? Well, this is not like traditional, and you know we're worried about this and that, even if he do know the language,
José Diogo_Viana:
Uh, yeah, that's That's a good question. To be honest, I don't know what to answer it, because I never experience that those types of arguments or questions, I know that physically used everywhere, So that's where most of the time people tried to choose. I had one personal experience once and But instead of it was instead of using jabe script, so as no farther back, and which we end up going with it, and it wasn't the best idea, so looking back, we kind of regret it. At least, I kind of regret it. but that's
Allen_Wyma:
What
José Diogo_Viana:
an ol
Allen_Wyma:
actually happened, I'm kind of curious because usually that's Job is probably the biggest language when it comes to Fin tech, right, I
José Diogo_Viana:
Jascipjobs.
Allen_Wyma:
mean, it was always
José Diogo_Viana:
orry,
Allen_Wyma:
sorry. Job of script. Okay, So job of script was the issue, was it? I mean,
José Diogo_Viana:
Yeah,
Allen_Wyma:
what was the exact issue? Because even that's really really popular. right that usually is something people would be really interested to use.
José Diogo_Viana:
Yeah, the biggest issue for us was background managing. Background job is managing. We used bullemqifif, I remember correctly, but it started to get to be a bottle like in our, In our application pretty pretty fast. so yeah, that's mostly where we were. We regret on it.
Allen_Wyma:
So what was the? What was the fix? Was it switching over to Alix, Or how did you manage to solve this issue?
José Diogo_Viana:
Yeah, we, at the time we actually, when, when it started to become a bottle neck, it was when we were ending off to to the client, because we are already finishing up the project. The The where other lady is internal team. So to be honest, I don't know. Ow. that went through. That's one of the sites parts of being in a consultancy of this type where you actually don't Keep your. You don't keep your stuff, so I don't believe they changed to elixir, I mean, I'm totally sure they didn't change to elixir, because they didn't know it. They know it, so I don't know how that went.
Adi_Iyengar:
I actually, Allen actually feel elixer and erling the whole beam because them is probably the best for a fintech and I think at last almost like three or four years, people have started to realize that An there's so many fintecplatform tande built in Lixor, they know that Okay, the raw speed may or may not be there. but with the capabilities of horizontal scaling today you can minimize lag. You know, you can be completely even driven, which, which is probably what you want to be, even source, rather for for fintech, Ah, have resiliency of like something like Rabbit, M, Q, or even like half wrapped around and looks, a ship which gives a lot of resiliency, and then the scale enough to minimize the lag, Like I know block five when I was when I was there. I mean, an, you know, talk to it of the like banrupanstuff now, but they were dealing with a good amount of scale and around the time when I left, and you know there was basically no legs for each transaction like each Currency, up dates, account up dates. like you know, because you know some people were using block fires like a regular trading platform. Not like a bank, which is what we supposed to be people date trading on block fire, and it was able to handle that load pretty pretty easily without even spending. I think for Remedy was like less than eight hundred thousand a month. infrastructure for that level of scale, And that's another hing. it looks like provides right. like if you write this in like Jos Job, all these Abs, you would have to kind of over index on horizontal scale Ability spend a lot more money and infrastructure to handle less scaled Because the systems aren't as resilient right, so I actually feel like or line re the best right now for Fintech.
Allen_Wyma:
No, I don't disagree with you right. I'm actually working on a fan tech project at the moment and we're using. Obviously, we're using a lixer for the back end. Um, it's actually now that you reminded me there was a guy who worked at Goldman Sex who got prosecuted for stealing some of the source code and handing it over And think he was working at full on trading system and Erling, which is kind of interesting. So I mean, it sounds like gold man's actually using our languach is, which is pretty cool. M. Yeah, I mean for everything you said, Of course I think it's It's fantastic, right because I always hear about you know low latency trading Using Job, and how you can You know they keep fighting the G C and you know how to eat Is A little bit of memory is possible. Until you, you just almost hit the G C limit right as opposed to something else.
José Diogo_Viana:
Yeah, I knew that block fire, as you mentioned, used elixer, and we also add a previous client is actually a close business to us. It's called U. Trust the process, payment from crypt to to Fiat currencies. We worked with him. We work ith them and we helped them. Actually do not the full thing way and were just worked on a specific part. I can talk Much about it, but they also used elegusarall in all of their all of their platform. Yeah, as you said, they have the same advantages. It was pretty. It's pretty scalable. It's reliable. All that All those things that are cool in the system, the Co system with their long, and also they have. they used event source, S. Q, S, Also very easy to to implement it for keeping track of transaction states, etcetera, etcetera, So yeah, I think Eelixeriis gaining up grand on Fintexplace Yea, personally, I still see a lot of Java script skip stuff on the back end. Mostly,
Adi_Iyengar:
I'm also going to do like a plug here, like my current company. The cover is a sports company, but I'm working in their bet part of the system and we are completely pedal stack, uh, fintackcompletely, on peddlstack we handled. I think millions of us wont say like five or six million users active with placing bets during Super Ball. This last subeball perfectly with liker commanded even Source was, I think, maybe one or two minor bugs in production. and Yeah, the resiliency of sheer resiliency and ease with which you can spin up such a high performance application of the Lixirit's, kind of mind blowing. You know you cannot do that in another application or any other language.
José Diogo_Viana:
Yeah,
Allen_Wyma:
Yeah, yeah, I agree. it's amazing that like nobody else really expert looked at this and there's so many inderestries using earlyingit's. It's really crazy like Erling, or even a lixure. Of course, And now you mentioned about the betting right, Can't figure about simple bet. I mean so many people,
Adi_Iyengar:
Yeah,
Allen_Wyma:
but now we're drifting off into. you know, so that everybody the audience Im sure already knows about. But what now everybody does know about So sadly you know, I don't get to play around with too much data science. Literally, data science stuff happening, Elixir, which is probably one of the hottest parts going on right now. You wrote an article about how to use. I forgot which part of it but how to use some data science within Elixir, For like figuring out how to spread out your I don't know at even called. it's like spread out your portfolio. Isn't that right?
José Diogo_Viana:
Am I losing you? Hello,
Allen_Wyma:
Think we lost our guest for a moment?
José Diogo_Viana:
Hello.
Allen_Wyma:
Maybe I should I pause it, or when I can't really pause the recording.
Adi_Iyengar:
Yeah, I don't marker as I don't know. Just I know markets like persist right. So hopefully, Man, this week of fifth I would have finished. I would have put fifth lixarpedlstack, Ap, and production a product in production for different different company. Post n v, P, U and I can have to find them founding engine. Hopefully I can give a plug. It's crazy man. It looks there so easy. It's so easy to build awesome products. Like number of requests Or And this Ap is making two different apis Is like hundreds. high performance Hy, Performance like that That takes like Yu know, hundred thousand requests, and like an hour or so, and we're paying like
Allen_Wyma:
Yeah,
Adi_Iyengar:
four hundred bucks and platform.
Allen_Wyma:
Yeah,
Adi_Iyengar:
it's crazy.
Allen_Wyma:
I recently I recently got yell that because I did. What did I do? I did, Um, like a hundred million tokens for an integration like in their data base,
Adi_Iyengar:
M.
Allen_Wyma:
And they were like freaking out, Like what happened because I never kept the token. I kept it up in memory, and every time we're up were we were updating. not to mention
Adi_Iyengar:
Hm,
Allen_Wyma:
we boot up the system locally, But
Adi_Iyengar:
M.
Allen_Wyma:
it was just me and other developer, and we only had one environment running. And somehow he said that we Like a hundred million authentication
Adi_Iyengar:
Something
Allen_Wyma:
tokens
Adi_Iyengar:
is off.
Allen_Wyma:
within a couple of days. Yeah, I don't know if there was really that powerful or or, or maybe I don't know. It's crazy.
Adi_Iyengar:
Well, I mean the problem is in authentication. Tokens should not have that many like I mean it. just t. T.
Allen_Wyma:
We never.
Adi_Iyengar:
L is not
Allen_Wyma:
We
Adi_Iyengar:
being
Allen_Wyma:
only
Adi_Iyengar:
used
Allen_Wyma:
kept
Adi_Iyengar:
correctly.
Allen_Wyma:
him in memory, so
José Diogo_Viana:
Hello,
Adi_Iyengar:
Hey,
Allen_Wyma:
I think
Adi_Iyengar:
you're
Allen_Wyma:
that was
Adi_Iyengar:
back.
Allen_Wyma:
the issue.
Adi_Iyengar:
Let me mark
Allen_Wyma:
Hello.
Adi_Iyengar:
the clip
José Diogo_Viana:
I'm
Adi_Iyengar:
again.
José Diogo_Viana:
sorry, I'm sorry, my father, who was in the house because we had some issues with the eating, and I turned off the power so the so the construction guy could actually work on
Adi_Iyengar:
Got ya?
José Diogo_Viana:
on the machine. So yeah, I'm still a bit breathless from running upstairs.
Adi_Iyengar:
No problem?
Allen_Wyma:
No like that, right,
Adi_Iyengar:
I can add another mark Allan whenever you already. when you can continue
Allen_Wyma:
Yeah,
Adi_Iyengar:
recording, Let me know.
Allen_Wyma:
yeah, so I forgot where where were we? Oh yeah, so kind of going back to. I want to bring it back from our previous discussion. It. We're talking about how awesome elixer, how you can use in so manydiferent places, But you used it in a bloggarticle, which I thought was quite interesting Where you're talking about how you can use some of the data science parts to help, and I believe let me know if I got this one wrong. Right is like how to use some simple statistics and other piece Is to kind of distribute your portfolio to kind of avoid risk and also maximize profits.
José Diogo_Viana:
Ah, I wouldn't say that was the main goal. Actually, it was enacted on project started in an Eaton project, and to be honest, I just wanted to use geneticalgrithms in any place at a time I was reading the book, and for for a few, for for for a few days for a few weeks, I've been complaining and nag Everybody to repay. Let's use jetcamsorithems in the in that in the acaponllet's use Gentcosibmsr. All that all that stuffs? I tried to basically get. Where can I fit this because I really wanted to try it. We have that idea for actually the project called Ma, Don't know even how to say. It doesn't really matter. But the idea was to make it a pen like a pension fund for criptewasets. The main goal was to you don't get fat currencies, You, you, you invest it, And you you get in return. What assets should we make our quote Investment portfolio and quote? So the main goal was to actually have just a couple of assets. Don't really matter which, but actually then we saw well, this is an opportunity for me to to play in genetic algorithms and we can actually get something out of out of the way out of that. so yeah, that's that's try. and we Actually ended up selecting selecting a few assets randomly. The only one that was pre selected was Etherium staking, because it counts the the zero risk or the free risk investment as you actually get in government bonds, and et cetera et cetera. So we wanted to, and that one as our metric, And the others were pretty much selective randomly and we just wanted to make a few. Yeah, I got to be honest. this was a funny prices involved. wanted to have the most best word, all the cool things around, So genetic algorisms and making predictions for for the best assets to be there, was actually the final, But I wouldn't say that it started that way, so notify answer the question. I think I got lost a bit.
Allen_Wyma:
Well, I mean, you basically said that you wanted to kind of play around with it and use genecalegrims as as an excuse. right,
José Diogo_Viana:
Yeah, pretty much to be honest, for for a few days I try to basically make everyone just to genetic algrithems on thine. so, instead of making it I wanted to part the elixir cold to something that would work on train just because it had genetic algorithm. But then no one wanted to listen to me and they gave me like this, so Play with this and be quiet for for a second. That's pretty bad cheat. Because I already have the lixercalled. At least the framework that's given to you by. by the book, you and obviously need to make a couple of changes that we can talk about next. You always need to change your epirator. mold the framework to to make sense in your use case.
Allen_Wyma:
I mean, I did. I do have the book. I just never cracked it open. Right? What exactly doesn't mean to be some kind of genetic algorithm? No idea what that means. Sorry.
José Diogo_Viana:
Yeah, yeah, a good question. I had it before because I have a background in data science from back from college and my master's degree. We never studied directly genetic alrithems with. But it's one of those things that you actually end up knowing that exists and where it can be used that. so starting from the beginning, genetic alerim is optimization solution to a problem to specify Problem most of the times, or you can use eneticalems everywhere, but most of the times they are used for finding solutions to problems that are really difficult to find a solution. To The main example that I like to to use is the Napsac problem. Problearth Of it, it's that one that then it can be extrepulated into a lot of different situations. But you have a container is a nap, And you have different sets of things that you want, start their different sizes and you want to optimize the best space to amount of things on. Start on the on that knapsack. Every word of it. Should I go with deeper? Now? Can I go to the next thing
Allen_Wyma:
Yeah, you can go ahead.
José Diogo_Viana:
So you can use genetic ovens to solve it as a optimization problem? You don't have a guarantee that you will find the optimal solution. You just have a guarantee that you find a pretty good one and then genetic calgrims as the name. So the name says, It works pretty much as genetic evolution in Our. In the Nature works. So you start with population, ideally initialized Randomly, Because well, you can actually make it evolve from something else. You need to start it some some way You started randomly, and then each population, So you can each generation, you anviluatetht that that population and check the best element of that population already solve your your problem. Most of the times you have a final value. You want to uptimize up to you even know that it's solution, but you don't know how do you get to the solution, So that's how you know when you actually end up stopping it. But every time you don't get to the solution you make make your children so you cross over parents. The best parents get crossed over. You also introduce couple of mutations that I can also talk a bit about later. And and then you, You always repeat the process So simple, Six steps. If you, If you look on the original on my blog post, and also it's present on the on the book, you initialize, you evaluate, you select parents, you create children and mutte, and you start over to evaluate until you actually end up stopping. Uh, does it make sense?
Adi_Iyengar:
Yeah, yeah, it's it's basically, I think a cool way of saying M. Yeah, I mean, really finding a set of combinations of U in puts that yield a desired result right.
José Diogo_Viana:
Yeah,
Adi_Iyengar:
So so like that's like what you meant by Like kind of fitness right, Like the. The. The fitness criteria Is you know is when you meet that you stop your Mutations and stuff like that. An initial selection can can still happen randomly or in a specific way. that something, this that everything goes under the same umbrella. Genetic algrthm. Like you could go, You could have a random one. You could have. I don't know whatever algorithm of selection itself, So it could just be a pure brute force algorithm that optimizes for you know a function In. Think you put it really well, it's an optima
José Diogo_Viana:
Yeah,
Adi_Iyengar:
That
José Diogo_Viana:
yeah, I wouldn't say it's
Adi_Iyengar:
solution
José Diogo_Viana:
brute force because the ideally you, after a couple of generations, you mostly discard. You start discarding the worst possible solutions. But yeah, as you said, the perfect
Adi_Iyengar:
Right,
José Diogo_Viana:
word is and optimized brute force algorithm. Yeah, yeah, the publition can be entalized randomly, but The part that you need to tweak for it. use case is, I'll actually you initialize, and in all your input to make it make sense for the actual solution you want to get. For example, in this case it was for a portfolio optimization. So our in puts we have. Let's start from what is what? So you can think of gems here that belong to a chromos, And the ram is is a member of our population, and each sin in our use case was includive as the percentage or of the location we wanted to give to that asset. So if we, if we have ten assets in our portfolio and we want to optimise the distribution of our funds, we can have, let's call it a are, or at least of ten, ten numbers, initialized random, Leave From zero To do an hundred. Obviously everything Eton needs to be at the maximum of one hundred, So you can locate more than then what you have, and then you pretty much go go. You start with that, and you evaluate always on those numbers, but for example, other problems can be into binary input, so you either end this or you don't. So it's The only a zero one. That's one of the toughest part that was one of the toughest parts on our use case was to actually not initialize, but to work with with a range of numbers and not something that was either on or of so you, vitoryoudon't have it, and into this I couple of slight works into our evolution. that toward pretty hard, too To the work I can give an example, but you have, as I said you, everything needs to to be under on der per cent because we don't want to be supposedly investing more than what we have available. So when you initialize you need to make sure that everything is under one, Andre. That's a pretty simple thing to. To do. you can select random numbers, Normalized them, and make everything oh under one hundred and making sense. But to be honest with it and start that way because well, it was an act on all of that rush everyone trying to do to stuff. So we just did we started us. Let's initialize this randomly and we know that everything is below and under. Because some stupid metric data, I remember not, things are not normalized And you actually get to the second study when you evaluate people. When were he population? We? actually, the, looking back, it was very stupid. We said, So if you are over one undred you're considered invalid, so you aren't fit. So you have fixing fitness function of zero. It's the. It's basically saying you, If you say this is zero, you don't survive. And our our fitness function was pretty much the beginning. Very simple thing. So you have expected return for specific asset. If we want to locate ten percent to this asset, Our fitness function is ten percent times this expected return, The next percent of the second asset times It's expected return. And then we just some everything, pretty simple account. So the cromosoms that would give us return are your number of expecting returns would be the best ones. But the thing is that the ones that were one under percent, we will just give them the slip, them as zero and I ever evaluating to the next and evolving to the next To the next generation. Everything will be zero, basically or third generation. For generation After a couple of generations everything was zero and they are the part here of the Ling is that this is always running random stuff and a lot of them. So you just see like ten thousand generations, And it's I would compare this to the plugging distributed systems, because you will know why things Are the way they are, In the sense of where this, where this is coming from. You don't know. How is this getting to zero? How is this giving such bad returns? And then we end up discovering that we were penalypenalyzing too much. the good, the good cramasons, Because when you were reaching one hunder percent, so this is getting very good. You are actually killing them. so Oh, you're too good. Went over one hundred per cent. You are now involved, so you get a zero on the fitness function, So eventually, when we actually get it to start getting better, we destroy everything. So it's that was one of the problems.
Adi_Iyengar:
Nice, Yeah, I mean to me, it sounds like. I mean, the fitness function itself could have been, Could have accounted for that. If there was possibility for fitness or more than hundred. Maybe you know, the fitness function itself should kind of bit more dynamic. but yeah, I mean it's It's also hard right when our playing with only indigious right like, Have you? have you tried using Um, an x or something like that? like you know. Also Also been very Curious. like since the begin the episode Like, how would these algums go with? you know, and that ill alow, like also replace your in digit operations with just any data right
José Diogo_Viana:
Yeah, that's that's
Adi_Iyengar:
Like this
José Diogo_Viana:
a good
Adi_Iyengar:
censor.
José Diogo_Viana:
question. Yeah, that's a good question. Actually, you, you were right in on the first part saying that our fitnexpunction was was wrong, at least the first couple of versions And that's one of the things that you. you need to also always be, be constantly changing and tweaking to make it to make it work for for your situation about The next part of the n, X and the inter operations. At the time, we wanted to make the incolving of the M puts as simple as possible, so we just say so, Let's go with with incisor solutions, and we don't get intermediate, so you either put one percent, two percent, three percent. Let's ignore the rest. So pretty much like in physics were, let's ignore All those variables that make this Are. we have twenty four hours to make it work and let's go. Let's go with it. So Yeah, probably other other solutions would make it easier. and also answering the second part of the question. I never use Annex. I tried to use it when back back when as was very, very very, but I, I started working out on Fridays on Every Friday. when I came back. The eva is different so I end. I end up giving up for a first for trying it. Now I've heard it. It has already
Adi_Iyengar:
It's
José Diogo_Viana:
been
Adi_Iyengar:
got a chap.
José Diogo_Viana:
in deploy. Yeah, do you have any
Adi_Iyengar:
Yeah,
José Diogo_Viana:
question
Adi_Iyengar:
it's
José Diogo_Viana:
specifically?
Adi_Iyengar:
very. It's been very still for a while.
José Diogo_Viana:
Sorry,
Adi_Iyengar:
Sorry, I think there's a lack, but I was just saying that year X has been very stable for a while. I have not used axonibut. I've used an ex for just like a mathematical problem to get like a more accurate data. like I had this project that like products like planets, like their motion and their position, and obviously we can do that with numbers, enexhislike ideal for that, and leseftikethat like actual trick operations and stuff. So that's what like? I think. The first that came to my mind is I'm also very near to genatical. It is like Alan, like I basiclydn'teven. know what they were before this episode and as soon as you started talking about him like, Yeah, there is definitely something there with N. and just like using it for an living, any kind of data? So yeah, it's it's it's I would definitely be interested in, like you know, at least googling. if someone has given her a try or not.
José Diogo_Viana:
Yeah, To be honest, at the time, we had to pre calculate bunch of values, for example, the expected returns for for assets, we selected a couple of tokens, and we had to to actually select or calculate the expected returns, and also, for the most complicated part, at the end of of the day, we had to also calculate standard deviation. All that stuff and I ended up with patent, because that's what I used back in the when I was learning all this stuff in college. I alveready knew how to do it. As I said, It was a nice thong. All that shall that. we need to make it twice. So what can I get? So I have that
Adi_Iyengar:
Yeah,
José Diogo_Viana:
patent script. I can. I can calculate Sundardiviation pretty easily, so to be honest, I didn't go back to statistics to see. Although I calculate standard deviation, I just know that in it and you can do it Pretty much with one liner, and that's what I did. But if I had more time for sure that I would like to test and ex or even explorer, Is it called that way? Right? Elixer is now the explorer where you can actually do things like panes.
Adi_Iyengar:
Right, yeah, life book. Yeah, I think Nix also has a standard deviation function.
José Diogo_Viana:
Okay,
Adi_Iyengar:
It's It's basically like that empire, right, Like whatever,
José Diogo_Viana:
Yeah, yeah,
Adi_Iyengar:
But
José Diogo_Viana:
yeah,
Adi_Iyengar:
but
José Diogo_Viana:
he's
Adi_Iyengar:
yeah,
José Diogo_Viana:
ugly.
Adi_Iyengar:
totally makes sense to you. spot on. because, because already like familiar with that yet?
José Diogo_Viana:
Yeah, that's one of the things that. when you actually end up trying a lot of things, Be honest. I start to lose a bit of Where can I do what? Because I don't remember where where I live That specific thing, but yeah, we ended up with with Python. It's not my perfect perferred language, but I can complain.
Allen_Wyma:
Yeah, I had this guy over at my office to give kind of quick introduction about data science because I know nothing, and then I have one guy over here who's really into doing. I think they're all kegles of this kind of problems in data science.
José Diogo_Viana:
Yeah,
Allen_Wyma:
And it was. It's pretty interesting like once, he kind of gave me like a baby talk like you know, from very like twenty feet up in the air, and then like this, kind of gave me a very quick introduction about how it all works and it's definitely advance, but the same time like, The more you learn about it, the more it's like. Okay, you're just building upon previous ideas And that's what makes it really complicated. Basically like a Lego kind of thing, So I'm still like assembling my two pieces at the beginning. I'm still
José Diogo_Viana:
Eh,
Allen_Wyma:
trying to catch up to everybody else,
José Diogo_Viana:
yeah,
Allen_Wyma:
but I do understand it's not that complicated. And so that's something I want to try to get into soon.
José Diogo_Viana:
To be honest, I think Neticaloris are a pretty good start on Ecsense, because they don't touch all that complicated part of machine learning and deep learning. Nero networks, you don't need to understand function like Iationactation functions. How many layers do you need on your on your model? Ow to train it, You don't need not need to worry about just playing mathematical stuff, And then you need to Lose a couple of of time to make your your feet next function, and your mutation functions adapt to your your situation. But then you don't need to to worry about. or should I use convolution network, or should I use this type of different training system? I would say they are a pretty good start For you if you have interest in statistics and all that stuff. But yeah, now that you have Bumble Bee and all that urging, face things with in the elexraecosystem it's pretty easy for you to to play with it. I've been wanting to try, but I haven't at the time yet for sure that all those image classifications, a text, the prompt suggestion, Ll, all those Things are pretty amazing, to be honest, and I think shown son moriarity, I expect to be pronouncing his correctly. Also the same author of the desionatic Algrithm's book, S very active on on On the data side Part most of the projects are are pushed on his side, at least from what I see, follow him on Twitter, and also Get to look at Gato contributors, et cetera So I assume Yeah, he's the expert on this, so uh, yeah, I'm only a follower.
Adi_Iyengar:
Like, Yeah, it's actually been on my raider. I think we had someone from nerves, like at the end of last year and I have this project that I've been wanting to do with umbleby and image recognition with nerves as an embetterd system. I create the rib at the end of last year, December twenty ninth, and been willing to commit to that, but hopefully hopefully I'll have something up and running by end of March. Yeah, it is. I think you're right. I mean, it's so easy with you know all these abstractions now when it looks sorted to basic machine learning. It does you know? It does feel like a lot shorter bridge to cross than it was earlier.
José Diogo_Viana:
Yeah, even with and I don't think you have all this. Don't have these psycho system and this at facilities To get something out of the dear. You need to understand the Algaritems or you need to at least copy specific lines of gold from some stock overflow question Here. it's pretty much you just set up the life book, You, You figure the sales, and you get things showing and running and it's a very rewarding experience. To be honest, To actually see your your models, or uh, your, The thing you're trying to predict, making so, making responses or getting predictions. Even if they are wrong. You actually see all this this thing is learning. I'm so now I'm making this This thing to predict or written result. For me, it's very rewarding experience. and yeah, to this is like a we think, because I've never had like the use case to to use it on a professional side. But yeah, we always try to to make those side Projects that we never commit to like. Like you said that, and we go from project to project.
Allen_Wyma:
I mean, Do you ever think that Elixer can get to the point where we can just remove the need to have Python? Because I'm aware. Like like again, that guy came over and he basically said, If you want to do Dat science, you, you pretty much have to get into Python. I mean, Julia is doing pretty well, but I think that no matter what, there's so much stuff written with Python, there's so much stuff kind of done Because of it's just been like the forerunner in the space. Do you think we could ever catch up? I mean, would it take a long time to even get there?
José Diogo_Viana:
Yeah, I would say
Allen_Wyma:
Aud,
José Diogo_Viana:
it would
Allen_Wyma:
saying
José Diogo_Viana:
take
Allen_Wyma:
you're smoking
José Diogo_Viana:
up.
Allen_Wyma:
something. It's noways going to catch up.
José Diogo_Viana:
Yeah, it's difficult. The amount of things that are written for Pat and tenser flow. All those first of Flo. It's not technically piton, I guess, but all of the the things that you have running on top of it to interact with with tenser flow, and all of those training mechanisms, and then I'm missing the Words for the. actually, when you mix in model, so you can do extra things, so you train the model. But then you can mix them and use bagging, and et cetera, et cetera you, all of that you all love that things. All of those things in parton, Outside, it requires a lot of work in this place from your league. More people, also more hands to get that, because in parting We have some outside ten years, fifteen years From this from the beginning, I remember from classes teachers mentioning the Netflix. the Netflix competition back in the beginning like Oh, something of one or two, I can remember, But it's something to look up where they made competition for people to make predicts Ans of of what other users would like to to see. And yeah, it's been around from from for a long time.
Adi_Iyengar:
Yeah, I mean, I actually don't think the purpose here is even like Aim for replacing both on it to protect Elixir. I think that's what I really like. but Jose. he's approaching building Alixe community as if he's running a business. You know, he is minimizing or maximizing the likely that elixir will be relevant in many areas in the future, and machine leaning was an easy. easy when there, like literally only three languages that do that. You know. Only one of them is general purpose. so Lix, being a general purple language of extracting out in that domain where you know about eighty five percent of machine learning stuff now can be done with Elixir, without using Python. You don't need the last fifteen percent in the Lixurforelii Urvive, Like you're doing crazy machine learning. You're probably building on top of something is already built and Python is fine for that, but the goal has been achieved now Elixir has expanded and it's Easy to do, you know, moderate level machine learning with a Luxor, and I think on to the next problem for Josey. That's what. That's what. I figure. I think he's maximizing the overall effectiveness of the language.
Allen_Wyma:
Yeah, that's that's a good point. I mean, I guess maybe my, my wording was too strong. I mean because I'm always. I mean. Okay, every language has really good benefits in certain areas right, so I mean, of course you can never replace a language. I would say that really excels in some specific area. For me, I kind o look at it as I don't want to introduce another language. Another kind of run time or whatever, unless I really have to, right. So you're saying basically, even right now, you're saying that Elixer can do eighty four to five per cent of data science,
Adi_Iyengar:
Yeah,
Allen_Wyma:
so it could probably fit for most cases.
Adi_Iyengar:
Right, and like with Bumblebee and like, integration with like other models, Ritenanppith, on being able to interact with models or Python, you've already crossed that bridge. You know you don't need. I think Probably there's more stuff you can do in lixturebut. It won't give you as much return for the effort you have to put. right,
José Diogo_Viana:
Yeah,
Allen_Wyma:
Okay,
José Diogo_Viana:
I
Allen_Wyma:
Well,
José Diogo_Viana:
think
Allen_Wyma:
then the
José Diogo_Viana:
one
Allen_Wyma:
sounds
José Diogo_Viana:
of the
Allen_Wyma:
like we're really there approaching.
José Diogo_Viana:
I was going to say that. one thing that I think that got Patan out of basically winning the race is that the race, if there isn't a race, but ll say that it's the the most selected one. It's because it has easy syntax to understand for people that only have mathematical background. I experience that we had a lot of people joining us on The masters degree. That wouldn't come from the computer science background. They would come over from statistics or mathematics, and pitenorare for them was very easy to understand. get a grasp of it and make quick models and get things running even if it wasn't the best solution. and to be honest, even inside the more tech Call and computer science background, people or developers, I think that the fact that elixer is functional and all all those things, it's a great advantage for those understand, but for people that that come from different backgrounds, I think it's a bit more difficult to understand, So that's why Piton is always the first Selection on that case, Because you may have a physical background. You want to modulate something. You probably won't pick a big elixir, Because I would say it has more concepts to understand before you actually start doing things while invite you. You don't really need much to to understand. At least that's why. Why, the way I see it.
Allen_Wyma:
Okay, Yeah, I mean, I don't have any more questions. I think I just need to kind of brush up on my data, Sin still, and finally crack open that book.
Adi_Iyengar:
Me too, man,
José Diogo_Viana:
Yeah, it has a cool.
Allen_Wyma:
I mean, Is there anything else that you think that? sorry? Go ahead,
José Diogo_Viana:
No, no, no, no, I don't think that it has a cool chapter back back back in the end where you actually make you don't make it, but you can play tires with genetic al viriums. You know, the book doesn't teach you out to play the game with using Genetic Arid. What it gives you a taste of how you could do it, And you can also test it out with different games, so you should deftly read. We can get your get your thinking. How can I use this to play to play to play games or other things that it might interest you.
Allen_Wyma:
Kay sounds
Adi_Iyengar:
Yeah,
Allen_Wyma:
good.
Adi_Iyengar:
I read a few. I very briefly remember reading that book Right when I bought it again. It's on my library. What I really like is, I think there is a chapter there about test till not every book does that. That is like many. there's many books that cover a very specific domain, but they don't even have a chapter for tests. I like to add a section for test in every chapter, but I appreciate that in a, in a book that did not need as much testing. that was like a chapter for Tests, and also like you know a lot of example code and stuff. So uh, that's my two sense of the book. It sounds like a good book.
Allen_Wyma:
Yeah, it sounds pretty cool and I'm happy to hear the testing part. Yea. I love books that testing in there. I mean, sure, it may be distracts a little bit, but at the same time, I never even thought about that, but it would be nice to. How can I actually test to make sure that I coded this properly for my model? Right?
José Diogo_Viana:
Yeah,
Allen_Wyma:
That's huge.
José Diogo_Viana:
I would say that
Adi_Iyengar:
You having
José Diogo_Viana:
that
Adi_Iyengar:
this
José Diogo_Viana:
part
Adi_Iyengar:
property
José Diogo_Viana:
of the month.
Adi_Iyengar:
based test Right from what? Remember?
José Diogo_Viana:
Yeah,
Allen_Wyma:
Okay. I mean, Is there anything else you wanted to state before you move on to pick?
José Diogo_Viana:
Well, you know, I think I've said everything. Definitely, it's a difficult concept to to explain in general, genetic algorisms, or even later science. In general, it's probably better written. Gen. then explain, because when you're writing stuff you can think and re write and do the process hiterativitly, but here, yeah, I hope that people actually understood and you all understood the gist of it and that's pretty much it.
Allen_Wyma:
Yeah, great, and with that I'm transition is over to pick. Uh, maybe I think we always go to. I think the host always goes last. Right in us to go first is fine. but all right De what you got for us?
Adi_Iyengar:
Yes, I've got a few picks today. So first one is this app called Tamu, Or it's like an you know commerce, I guess like online shopping. What, Gonna call it? It's very. It's gamifyed. It's very gratifying if you like to online shop for a bit of happiness, which everyone finds it gratifying right to get the stuff. Obviously, But it also nice to spend money, but you don't want to spend too much time, because Amazon is like An can sometimes be like. You know, I don't like spending in a more than ten minutes on Amazon at once. Like searching things right, Tamu is like very user focused predicting your needs Looks, Ike. I just got a lot of I and they don't share information with a like g, p. R. complain stuff, so check that out. It's It's interesting, but I'm having a nice time with it so Well to share that with everyone. Another cool piece of this is like a hardware. Um, this is like a robovack. It's called Robe Rock. Have been. I've been using this for like four months now. I wanted to use it enough before recommending it. It's really cool. I'm a lazy person. I hate vacuming and stuff, so we have like you know, six robot vacuums in the house for different regions, and like they also self empty. After cleaning, We replace the bag once a month, and they are pretty efficient. You have to clean their rotors very often. Like one someone cleaning needs to be done and it's very simple, Takes like fifteen minutes. for like all the vacuums, great thing about roborock is they also use a liker learned, which they're not the only urobovac that uses lix need nito. Also is a lixernow, So it's quite interesting that it looks serious like you know. the whole smart home nerves thing is like expanding that too, And it's very Exciting to know that the part that I bought and I used. I learned later that they use. It's really cool. So that's another one. I got some good news. Three people of the seven who reached out to me last two months got jobs. Unfortunately, the other for are still in the hunt, but it works if you guys are looking for jobs. you know, a few wantreferls instead of hit me up. Also speaking over furls, One of the companies that I've been advising is going life this week, so like when the episode all published like a week or two before and they desperately need founding engineer. I built there, M v P. Well, it's not an M. P. Anyways, A product they have a, They have revenue and it's got a bit of a. I got a bit of Boston, but mostly it's peril stack, hundred percent
José Diogo_Viana:
Yeah,
Adi_Iyengar:
goat coverage Because I built it. That's how I do it. So if you want to work in A, If I say to myself, a nice ly built elixir Pettalstack application, I want to be a founding engineer. You want to do the start of grind. Hit me up. It's a great exciting opportunity, H, and that's actually one more pick. self promotion, so my book will be out in two weeks for Bata finally, after two years of work. so uh yeah, Ah, I was gonna say prora, but you can just order it now, so go ahead and buy it to harsh. Well, you know, while judging me on it, but it basically teaches you how to build a web framework like Phoenix, From start. it teaches me how to build a web server. Obviously not profess not production great because you know it's a book. The scope is very limited, but at least mystifies. You know how those things kind of work under the hood a little bit, so you'll know a bit more about how Phoenix works how webs Its work, and also like wrapping everything into like a nice digestible met programming in de face, like how to do that. how to make it testable, How to make it in respectable and stuff like that, So uh, yeah, go head and check it out. I'll leave links to all of these in the description,
Allen_Wyma:
Okay, Yeah, I was looking for for some of the links that you were talking. But so I think that goes on to me next, Right or he said the guest.
Adi_Iyengar:
The guest.
Allen_Wyma:
Okay, So ready? You got your picks ready?
José Diogo_Viana:
Yeah, I think so ll start with very interesting pick probably, and I would have to say the series of Last of Us from O, because I don't see series from a few for a few years. I haven't seen anything. not even Game of Thrones All those things that people actually end up watching. Not really sure why I think it takes much commitment for me to go every week and watch an episode, but as a player, as a former player of the last of us and I still expect for the next game to to come up. I've been watching this series and like to recommend it for people that haven't been watching it yet Next outside, Suggest our pack, Cold Bullet from from you tube. It's actually, and Youtubechanal, with funny, funny vireos for data science stuff. Basically each video is pretty much the mystification of over, model or complex a thing basically playing trying to playing. Try to play in. I'm for example, in tietries are teaching it out to out to walk. Yeah, I would say that it's a very cool U Tube channel. and also for people that want to know more about the topic, it's for sure a good place to to start and have a few. like. It also uses genetic alguirisms most of the time. So it's It's a plus there. Um, I'll say I'll pick last one just because I'd like to suggest this book every time. It's not a technical book, but it's range by David Estin. Uh, it's pretty much a book talking about our people with different skill sets, or basically the T shaped skills can actually performed better, then People that were very, very, very specific trainings on on a specific subject. For example, it starts by comparing. I don't remember if it was not our case, Father, but it was one of one of these starts by comparing them with one of one of them with Tiger Woods. Basicall, Tiger would started by playing golf from a very young age. While the tines player that I don't remember, I've read the book a couple of years ago at a pretty broad start in the sport. He played football in City that, and he still performed at I at a tournament. So yeah, I see myself as jack of all trades. Probably a master of none. Not really sure. I expect I a master of something. I don't know. At least I try to be one day, but yeah, I like to think that coming up to a new subject or a new a new field, the other things that I have encountered will help me get good. A good way. That's it
Allen_Wyma:
No great. some good picks. Yeah, for me, I just have one pick. as usual. I've been picking video games because I'm finally Ettngbck video games These days, my latest field game I'm playing these days. Cold was the aftermath. I mean, maybe I'm little bit late to the band wagon, but it's a lot of fun. I enjoyed the movie Up until the ending. I thought it was not very nice, but seeing a swarm of zombis running up to you and kind of making a perm. It's pretty scary Look And playing the game it's pretty crazy. It's pretty exciting. so if you guys haven't checked it out, I think it's a pretty good game. I'm playing on steam these days. Um, yeah, that's that's my pick. Just that one pick for right now, so go welcome or to come on and explain more about. You know dat science. I really appreciate that and it's cool to see that you know you used it in heckathon, and pretty you know, wrote up an article about it kind of inspired me to sort taking look more at this, So I appreciate your coming on, and with that, thank you so much for coming on and hopefully have you background again.
José Diogo_Viana:
Cool. Thanks for for having me. It was a pleasure.
Genetic Algorithms With José Diogo Viana - EMx 211
0:00
Playback Speed: