
The Impact of Open Source on Business and Development Practices with Daniel Loreto - DevOps_233
In this episode of Top End Devs, hosts Warren Parad and David Kimura dive into a compelling discussion with guest Daniel Loreto, founder and CEO of Jetify. The conversation revolves around innovative solutions in the DevOps world, particularly focusing on the use of Nix and DevBox to streamline developer environments. Daniel shares insights from his vast experience at prominent tech companies like Google, Airbnb, and Twitter.
Hosted by:
Warren Parad

Show Notes
In this episode of Top End Devs, hosts Warren Parad and David Kimura dive into a compelling discussion with guest Daniel Loreto, founder and CEO of Jetify. The conversation revolves around innovative solutions in the DevOps world, particularly focusing on the use of Nix and DevBox to streamline developer environments. Daniel shares insights from his vast experience at prominent tech companies like Google, Airbnb, and Twitter, detailing how Jetify is leveraging AI and agents to enhance software development. The trio explores the challenges of reproducible development environments, the complexities of ML development, and the strategic benefits of open sourcing tools. Along the way, they touch on the impact of AI agents on the industry and the balance between innovation and practical application. Prepare for an engaging episode filled with technical insights and thoughtful reflections on the future of software development.
Transcript
David Kimura [00:00:02]:
And we're live with another episode of Adventures in DevOps. Almost blew the podcast name right there, Warren.
Warren Parad [00:00:10]:
Well, you know, things happen. Alright. I've only done I've only done a few hundred of
David Kimura [00:00:16]:
these at this point. It's it's still a learning curve. So how are you, Warren? Did you bring us a security fact today?
Warren Parad [00:00:24]:
Well, you know, I I think again, I had brought one last time, and I thought, you know, this time for sure, our fellow co host, was gonna be here and share the long awaited one that, I I had been promised. And it you know, coincidence, not here. So I don't have anything today.
David Kimura [00:00:44]:
She's pausing for dramatic effect. Like, a really, really dramatic effect.
Warren Parad [00:00:52]:
It's gonna be the best fact ever, I swear, when it when it actually drops.
David Kimura [00:00:55]:
It is. It is. Right on. Well, also joining us today, Daniel Loreto from Jetify. Daniel, welcome to the show, man.
Daniel Loreto [00:01:04]:
Thank you so much, Will. Happy to be here.
David Kimura [00:01:06]:
Dude, I'm excited to have you here. So you are with Jetify now. Tell us a little bit about your background.
Daniel Loreto [00:01:14]:
Yes. I'm a software engineer by trade. You know, have worked at a lot of different tech companies of all different sizes, so I can talk about, you know, different experiences.
Warren Parad [00:01:25]:
But
Daniel Loreto [00:01:25]:
I worked at Google, Airbnb, Twitter. I worked at, you know, tiny startups that you never heard of. I worked at, Virta Health, which is a telemedicine company for diabetes where I, ran engineering, data science, IT, infosec, kinda like all the tech stuff. And and now I'm founder and CEO of Jetify. We're a company kind of trying to make it easier and faster to turn your ideas into working software. And, anyways, today, we've been playing with AI and agents on how they can help with the software development process.
Warren Parad [00:02:00]:
I mean, you you've got that opening so well nailed down. I can tell this isn't the first time you've had a meeting with yourself.
Daniel Loreto [00:02:08]:
Thank you.
David Kimura [00:02:11]:
Got the elevator pitch nailed.
Warren Parad [00:02:13]:
Well, I mean, it's sort of funny because, like, I meet with a lot of people and a lot of them feel the need to sort of introduce themselves and say a lot about their background even if it has nothing to do with the conversation at all. And some of them just go on and on for minutes. And I'm like, you're mentioning other people's names, and I don't know who any of those people are. You're talking like they're the most important people in the world. And, like, I for sure, like, I don't care. I mean, at least you stuck to company names. So, like, you know, that's great on you. And I think a lot of people know, most of those companies.
Warren Parad [00:02:40]:
I think most of them still exist, by their original name. So, you know, that's what's important. I was raised
David Kimura [00:02:46]:
in a Midwest Protestant community.
Warren Parad [00:02:50]:
Just go really deep into the ground.
David Kimura [00:02:57]:
Florence just not saying anything.
Warren Parad [00:02:59]:
Well, I mean You're gonna mute
David Kimura [00:03:00]:
you're gonna mute me, aren't you?
Warren Parad [00:03:02]:
I got I got I actually got sucked for a second because I spent, not short period of time in the Midwest, after I graduated from a university because it was the only place that I could get hired, as a job. I wasn't actually a software engineer by trade. And so I ended up there. And I was like, do I have a connection for this? Like, can I talk about, the religious values? I'm like, not really. I actually I I actually don't know a lot there. Like, I've never really been a huge into the religious scene. So I I can't say anything. And so I just was like, I'm just gonna keep my mouth shut.
David Kimura [00:03:33]:
I was trying I was thinking, like, in the second Austin Powers movie whenever doctor evil goes to therapy with his son, Scott. And they're like, tell us about yourself. And doctor evil gets up and he does, like, this long rant about how he was raised by his father. And and it goes really, really deep, but I couldn't remember enough of it to try to work it into it as a joke here. So Well,
Warren Parad [00:03:56]:
don't don't worry. You'll have plenty of opportunity to go look up that for, for next time. So today
David Kimura [00:04:02]:
might check for the next episode.
Warren Parad [00:04:04]:
Yeah. Well, I I don't spoil it now. I, I really like the idea of getting back on topic of of dev boxes, environment machines. Like, I I feel like this is a huge challenge that still exists. And, you know, it's interesting because I work personally in, like, over the years in a lot of different IDEs and a lot of different software languages, and I've always hated it. Like, there was never a time where I'm like, you know what? Working in this language, this is the best. And I I feel like, you know, if you're in the space, you've got some horror stories. You've got some, like like, how did you even get into this? You know? Why is why is this where you went?
Daniel Loreto [00:04:45]:
Yeah. So, man. So,
Warren Parad [00:04:48]:
I mean, I I guess
Daniel Loreto [00:04:48]:
a lot of these stories start with kind of scratching your own edge and trying to solve a problem that you have. So, you know, just having worked at different software companies, I I've seen those stories where, things work on your computer and they don't work on your teammate's computer, or there's a bug in production where you can't reproduce it locally. And often that's tied to the environment. And so when we started Genify, we were like, you know what? From the beginning, let's just set up some reproducible dev environments. And we decided to set up some Docker files for every kind of piece of software that we were working on. But very quickly, we ran into some pains, because, you know, Docker uses a lot of resources, on on your computer. And so when you're developing locally like, shipping to production, great. But developing locally, like, I don't know, the fan spinning in your on your laptop, you do a little change and you wanna see the results and, oh, it was the wrong line in the Docker file, so you busted the entire cache.
Daniel Loreto [00:05:52]:
And now you're, like, rebuilding the entire container. And so all of that felt really, really painful. And so, we had heard about this technology called Next. We wanted to try it. We we kinda failed two things at once. One is like, this is awesome. It it kinda creates reproducible environments. And then we were like, oh my god.
Daniel Loreto [00:06:12]:
The developer experience on UX for this thing is not what we want it to be. So, anyways, we ended up building an open source tool called Dev Box that kinda wraps next and lets you create those environments locally. And, you know, we just posted it on Hacker News. Before we knew it, it was, like, number one, on Hacker News that day. And so it kinda told us, oh, people are having this problem, that they want some solution. And that's, yeah, that's how we started with the developer environment work.
Warren Parad [00:06:42]:
I I love the story of, like, pain and suffering through development process because it's such a I mean, it's such a common strategy for innovation. Right? Like, I as a software engineer, you can, like, fix your own tools in a way. And so realistically, like, when that happens, it's like, well, I'm gonna make something better that doesn't it isn't as bad as the thing we're using. It's interesting you brought up Docker. One of the things I've seen a lot is engineering teams attempting to get the development experience on their local machine working through Docker. And this is something I've never understood and maybe this is something you could, jump into if you have some understanding about it. Like, why why why do that? Like, if I have the source code on my machine that I've cloned from a git repository somewhere, why why not just install the couple of tools I need and, you know, hit f five and run it? What does Docker really get me there?
Daniel Loreto [00:07:32]:
Yeah. So, I mean, I I think the the issue forget Docker for a second. Like, why isn't it enough you know, Docker or no Docker. Why isn't it enough just to check out the code, install the tools, and call it a day? What happens is that in many cases, you don't just need the source code. You need the specific versions of the tools that you're working with. So it's not like I need, I don't know, Go or I need Python. It's I need Python two point seven point eleven. Right?
Warren Parad [00:08:05]:
Hopefully not anymore.
Daniel Loreto [00:08:07]:
Yeah. You're right. I mean,
Warren Parad [00:08:08]:
that's that's that's an old version.
Daniel Loreto [00:08:09]:
But just well, but but see
Warren Parad [00:08:11]:
I mean, you say that,
Daniel Loreto [00:08:12]:
but, like, what happens a lot is is you're working at a large company, and there's this project that everybody depends on, but it's not staffed well enough because other other things became more important. And you gotta keep that thing running. Right?
David Kimura [00:08:27]:
Every day everybody depends on it, and nobody's willing to update.
Warren Parad [00:08:31]:
Yeah. Yeah. Exactly.
Warren Parad [00:08:32]:
There is I I I think we should just pause for a moment here because I know a nontrivial amount of our audience members are, like, suffering through some sort of PTSD withdrawal right now. Having said that, like, we're you know, you're working in some environment and, you know, some code that hasn't been touched in a while is not working. Like, I feel like that's, like, 10%, twenty % of your job, whatever it is. You know, you have no idea what's going on there and it's not working. And you didn't write the code. You may be lucky if you knew the person who did write it because, so one of one of my first jobs I had, there was an engineer who if I saw his name on a commit message, I know that there was gonna be a bug with that code that was going to cause me to lose the rest of my day. Until, like, if I'm debugging around and I see a line of code by, by by by Craig, that's it. Like, I'm sorry.
Warren Parad [00:09:25]:
Like, I know. It's gone.
Daniel Loreto [00:09:27]:
Shout out to Craig. Right there. So but but, anyway, I mean, yeah, that that's the point. Like, you often have to work with these projects, and they often need specific versions of software. And if you don't use those versions of those tools, things break. Right? And so, I guess, in a in a project where you haven't set up too much around developer environments, you might just have some read me file saying install Python. And so everybody is installing Python, but somebody installed Python, you know, a year ago. Somebody installed Python six months ago.
Daniel Loreto [00:10:03]:
Somebody installed Python today. They all got different versions of Python. Right? And you compound that by all the tools you might need in a project. Right? So it's not just Python, but maybe, let's say, you're using protocol buffers, you have a product both compiler, you know, and so on. And so, yeah, I think that it's rude. Like, that's that mismatch of versions is kinda what people are trying to solve that is not solved by just the read me saying install these tools and, start working on the on the project.
Warren Parad [00:10:36]:
Yeah. I definitely agree. I mean, there's a huge aspect here, which I think has to do with the languages themselves and their package managers are part of that. I got it working once. Let's not go back and touch it by even the language, maintainers themselves. You know, it's very limited, experience there. And so they they go off, they, you know, the package manager for whatever language you're using, like, it's sort of working. And so there are some that are definitely not as great as others, and then you just compile that on and put that usage as a dependency for literally everyone who uses that language and framework.
Warren Parad [00:11:11]:
And I still think there are languages today that no one is happy with the package manager for that language, and yet somehow they're getting over it. And
Warren Parad [00:11:19]:
I mean, there's languages where you have five package managers that you wanna know which one you're using. So so
Warren Parad [00:11:26]:
I have a feeling that you are throwing, the JavaScript, environment under the bus there a little bit. No. But I'll I'll have to I I have to say that JavaScript is one of no JS specifically is one of the best development communities compared to the other ones. Like, yes, there's n package managers, but all of them, like, work at least 99% of the way. And and and there are problems with them for sure individually. You know, they're not so totally perfectly robust. But, you know, compared to some other ones, like, I don't I don't struggle to pull out a new JavaScript package manager. I do struggle to figure out how to make Maven work even though I've gotten it to work like a hundred times before.
Daniel Loreto [00:12:11]:
And I'm not I haven't touched the Java world in such a long time. I I have no I have no comment.
Warren Parad [00:12:16]:
Yeah.
David Kimura [00:12:17]:
I just had a little bile come up in the back of my throat.
Warren Parad [00:12:22]:
You know, the thing is, like, I I remember c sharp, like, NuGet was not a good package repository. The tool wasn't great. Even dot net improved a lot of things there, but they had a diamond dependency resolution problem where, like, you depend on two dependencies who require different versions. And and Node. Js, like, figured out how to get around that. Npm solved it, yarn solved it, p m p m. They all have solutions there. So yeah.
Warren Parad [00:12:44]:
I mean, the individual tools aren't great and it's not great that there's lots of them, but at least I feel like they're not getting in my way. But, yeah, like, if I'm programming in c plus plus or something old school with for, like, a large company, then not only are there dependencies that are probably committed to a repository. Like, they're not in some package manager somewhere. They're like the the binary dependencies are, like, in the source code, and they require some Microsoft, redistributable c plus plus library from twenty years ago that I have to go and download from the Internet from some suspicious website in order to even, you know, build my project to get it to run somewhere.
Daniel Loreto [00:13:22]:
Yeah. Agreed. So yeah. Anyways, I mean, I think it's all those paper cuts that just slow you down in your development process. And all of a sudden, you are I don't know. You wanted to work on a feature. Right? You wanted to work on something related to the project that you're interested in, and you find yourself in dependency hell and just going deep into package managers when all you wanted to do was, you're working on an actual feature.
David Kimura [00:13:49]:
Well, I think that's I think another aspect of that is onboarding new developers onto your team. Right? Like, how long does it take to get them where they're actually working at on the code that you hired them to work on versus setting up an environment to actually see what they they're actually supposed to be building?
Warren Parad [00:14:11]:
That sounds like a question to air our dirty laundry on the show. So I guess I'll go first. You know, it's like a couple weeks to a month, and then I think we know that the long tail turnaround is like ninety days to get to peak performance. And then it's like six months to do effective reviews at that point. And before that, I think there is a very long time where there's still a lot of training and education that goes in. Not But it for us, it's mostly domain specific. Like, if we don't hire a domain expert for our company, we deal a lot in the, authentication authorization space. So, like, how SAML works, how OAuth two works, how OpenID works, how SCIM works, how audit trails work.
Warren Parad [00:14:50]:
Like, there's a lot of things that you don't wanna get wrong in a way, and so there are some tools. So but it's a long period of time. Will, you're smiling there. So I feel like, you know, you're gonna tell me that, you know, your new hires that I I believe you were interested in, interviewing and, getting into your team. They're, like, couple weeks max, and they're already at peak performance.
David Kimura [00:15:09]:
So I this is definitely an outlier. So I won't say that this is the state of our environment by any means. But, the last guy I hired, about a month ago, he was pushing production commits by the end of the week.
Warren Parad [00:15:25]:
Which is good or bad. Right?
David Kimura [00:15:27]:
Yeah. Oh, okay. He was pushing functional production commits that actually did beneficial things.
Warren Parad [00:15:36]:
Wow.
Warren Parad [00:15:37]:
I mean, that's amazing.
David Kimura [00:15:39]:
Yeah. But it was, like, it was a very unique situation. You know, we hired him, and he came in with a very specific set of skills that we needed. So he looked at our environment and was like, oh, yeah. Here's where it's wrong, and immediately started contributing to that. And then there's, like, the rest of the company, like, the web three space that we live in, where he still has a lot of onboarding to do before he fully understands that. So it was cool to see, and and I paraded it around the company just because that's the kind of person I am. But, you know, and and, like, the big picture of things, you know, he he couldn't have done a domain specific commit in that first week.
Warren Parad [00:16:20]:
Yeah. Are you using something, like, some sort of ephemeral development environment? Or, like, how was he able to get acclimated so quickly?
David Kimura [00:16:29]:
Yeah. It was over it was on our Kubernetes clusters. And so he had a deep Kubernetes experience and, just looked at the way that ours were configured and was like, oh, yeah. Here's here's something that's gonna gonna help you out. And then he also had a lot of OIDC experience, so he removed we were maintaining a lot of permissions and roles via pull request in a Git repository. And so he refactored that to use OIDC so that you just get it inherited when you authenticate.
Warren Parad [00:17:06]:
Yeah. I mean, I think the
Daniel Loreto [00:17:08]:
okay, you guys presented two different, like, onboarding experiences, but I think regardless, for me, you wanna spend the time creating the domain specific knowledge. You wanna spend the time kinda teaching and mentoring and and and having your your employees grow. I think you wanna take away the I'm just fighting with the tools, and this should be a solved problem. For sure. You know, I still have to deal with this now for, I don't know, two days, just to get my environment set up before I can even start, you know, doing a little coaching and and testing it.
David Kimura [00:17:46]:
Yeah. I totally agree. Like, our none of our customers benefit from the time and effort we spend into working on Kubernetes. They only benefit from the time and effort we spend building a better product that that they in that they use. And I think that's what you're alluding to. And and, Warren, to to catch on a point you made earlier there, you know, dealing with ephemeral environments, we recently refactored a bunch of ours because we had this one VM instance with I don't know. It had hundreds of gigs of RAM on it, and it was a bunch of Docker Compose files. And when you needed a new integration environment, you would copy and paste one of the existing Docker Compose directories and then go modify the stuff that you needed in there.
David Kimura [00:18:37]:
And so we refactored that to use Helm charts. So now the launching of a new environment, you just add a new helm value helm values file for the environment that you want and then let, Helm build that out for you.
Warren Parad [00:18:54]:
Yeah. So I I will say that, NIX is something that's always scared me personally. Like, I I feel like I never but wasn't a opportunity where I was like, you know what? I'm gonna reach for that new tool. So, like, you know, congratulations for, you know, seeing that as an opportunity and going at it and then building the UI on top of it to make it happen. What do you feel like the turnaround time is for getting new engineers acclimated to using the dev boxes through through NICS? So they come into a team. They have no NICS experience, no experience with maybe ephemeral dev environments. You know, they're all doing software development on their local machine. What's the turnaround time for them understanding and being able to execute effectively?
Daniel Loreto [00:19:36]:
Yeah. I mean, assuming we're talking about an environment where the tools needed are kind of available in the next package store and and kind of the versions are well represented. I think it's very, very fast because, I mean, DevBox, the tool we built, it presents an interface that's very similar to package managers from different languages that you're already familiar with. Right? So, you know, in you know, we're talking about JavaScript earlier. You have that package, the JSON file. Well, in DevOps, you have the DevOps, the JSON file, and it lists a bunch of packages that you need for your project. And you might just say, you know, Python and the version, and it might say the product of compiler and the version. And it has a command to add a new package, to search for a new package, to remove a new package.
Daniel Loreto [00:20:28]:
And so what we try to do is kind of present that next functionality, but under a UX that people are are already familiar with. And because people are are already familiar with it from other tools, I think it's a a quick learning curve.
Warren Parad [00:20:46]:
How does this affect the sort of software development cycle? So I make a change in in my code. We're talking about, impacting the Docker build cache and forcing it to rebuild. Like, what's the corollary here for, Dev Box? Do I do the software development locally? And then it uses some sort of, remote server somewhere where the Dev Box is being spin up, like, you know, using
Daniel Loreto [00:21:12]:
the The Dev Box is our open source project. It lets you declare your environment in this JSON file. And then the idea is that you can take that JSON file and reproduce that environment in many places. We we offer kind of on on our paid solutions ways to spin those environments on the cloud. But just to clarify, DevOps is just gonna be the open source, tool, and let you, kinda do it locally. And the way it works is it is like I said, I it uses next behind the scenes. In practice, there's no container, locally. I mean, you can you can choose to use DevBox inside a container if you're building a container, but you can also choose to do it outside of a container if you don't need one.
Daniel Loreto [00:21:57]:
And what's really happening is that Nix kinda installs all the versions of all the software that you need in what's known as the Nix store. Think of this as some path in your local drive where every binary or piece of software that's installed is stored under a content hash so that the path for it is unique. So even if you start installed, I don't know, five versions of Python, they will each have its own content hash and therefore will live in a different place in that store. And so, anyways, it like that, it's able to install all sorts of versions of software even if they would be technically conflicting. And then when you start an environment, it kinda sets the paths, to point to the appropriate versions of things, and and now you're working locally. Right?
Warren Parad [00:22:49]:
So is it changing the environment variables and the alternatives that are set on my operating system to
David Kimura [00:22:53]:
make the environment match
Warren Parad [00:22:54]:
what's necessary for that Yes. Necessary for
Daniel Loreto [00:22:58]:
that Yes. Yeah. So it will change for example, change your path, environment variable. And so, you know, you might you might say, okay. In this project, we need in this version of Python, this version of the protobuf compiler, or I know those live over here in the next store. Let me set up the path such that you're gonna have access to those binaries and not the other ones.
Warren Parad [00:23:21]:
Okay. Sure. That makes sense.
David Kimura [00:23:26]:
And is that done, like, system wide, or is that done on, like, a project per project basis?
Daniel Loreto [00:23:32]:
Basis. So so the analogy I make if we wanna, you know, reference existing, tools for particular languages, it's kinda like on Python, you have virtual AMP type of environment. Imagine that, but now kinda like at the system level for, the the tools and compilers that you're using for a project.
Warren Parad [00:23:53]:
So done correctly is what I'm hearing.
Daniel Loreto [00:23:58]:
We think so, but form form your own opinion.
Warren Parad [00:24:02]:
Well, I mean, I I think virtual env is, like, a quintessential example of it doesn't get much worse than that. You know, maybe maybe someone can point me in the right direction there, but I don't remember. Like, usually things automatically get committed to memory for me if they're easy to understand. You know, know, like, I think everyone has this. You know, you see something. It's like, oh, I can remember what that is. There's a couple
Daniel Loreto [00:24:22]:
of other things.
Warren Parad [00:24:23]:
Three months later, you come back and you're like, I gotta relearn this thing over again.
Warren Parad [00:24:27]:
Right. That's the thing. It's like, I don't touch it enough. So the only time I actually touch virtual env at all is when I'm generating new, cert box certificates from Let's Encrypt on, one of my remote servers. And that happens once every 90 days. And, every 90 days, I cannot remember how to do that. So, you know, there's Celsius, something is wrong with that technology for sure. So, you know, it's great when there's an easy way that makes sense to set it up and exit the the the machine.
Warren Parad [00:24:54]:
And that, of course, it has to be process specific and local so that you can potentially run multiple at the same time, so that you can run multiple pieces of, like, different services that have to communicate with each other.
Daniel Loreto [00:25:07]:
Right. Right. Makes sense.
Warren Parad [00:25:10]:
How do I how do I know that, early on in my career, it was always like, I don't need that garbage I'm just gonna do it myself and later in my career I definitely gotten to the point of I should definitely listen to those things and not think that I could do it myself so what like what's the advice here you know how do you know you're in a situation where whatever you're doing needs to be upgraded to using next through DevBox? Like, quintessential.
Warren Parad [00:25:35]:
Yeah. I mean, I think
Daniel Loreto [00:25:37]:
let me maybe start with the cases where I don't think you necessarily need it. I think those tend to be you you're working on a team with this pretty standardized stack. Probably have one particular programming language that you're focused on. And for the most part, all you need are the tools for that library, and you're not bringing into many external tools. Right? I think as soon as you start having more kind of system level dependencies, additional tools that you're bringing in, maybe multiple languages, you know, across the company, then tool you know, things to control the environment start making more sense. You know, just to grab an example, let's say you're coding in Python. If you've standardized on a version of Python, and you're building kind of pure Python libraries, I don't know. You probably don't need the tool like this.
Daniel Loreto [00:26:35]:
But, yeah, AI is the rage old these these days, and you decide to do some ML stuff. Right? And and you need some Python libraries that really depend on native libraries. And so now all those native libraries are gonna get pulled. And you're doing some image processing, and so you need, like, ImageMagick installed in the container. And and so you and it depends on a particular version of ImageMagick. And so now your environment shifting from kind of pure Python to kinda like these extra dependencies. And I think that's where this makes sense because that you can now treat those extra dependencies as something that you track and version and and pin, as opposed to, I don't know, just wild west and, you know, pull whatever is available today. And, you know, it might be different yesterday from tomorrow.
Daniel Loreto [00:27:29]:
I give you you know, some examples I gave earlier were only on the development phase, but, like, imagine you you're doing image processing in production, and the container requires a particular version of ImageMagick installed. Like, now we're talking about production environments too that if you're not careful, are you just gonna changing nilly willing?
Warren Parad [00:27:49]:
How does the conversion from using Nix through DevBox to get your local development environment? I mean, I feel like you opened the can of worms here. So, when you do that locally, how does that convert to using getting those same dependencies ready for deployment to production?
Daniel Loreto [00:28:05]:
Yes. To be fair, I I do think that's an area we're continuing to improve kinda early days on the shift from using the box from development to production. But high level, I mean, you can craft a Docker file in a container in many different ways and and really just kinda running any command you want, right, and and setting up the environment inside the container. So what we think is that, in general, you can have one definition of the next packages that you need that and that one definition can be used to set up your developer environment, but it can also be used to install those same packages in the production container. Of course, you'd have to differentiate between things that are needed in both production and development versus things that are needed just for development. Now I I said it's a work in progress. I I think something that I feel next packages haven't been optimized too much for in in the past, and and I think it is one of the challenges with what I'm saying, is that I I don't think next packages are optimized for size. And so sometimes they pull in a lot of things, into the container, and the container grows kinda kinda bigger than you want it to be.
Daniel Loreto [00:29:28]:
And so I think that's something that still needs to be improved, within, you know, in in terms of using next for creating these types of production containers. Yeah. But, I
Warren Parad [00:29:35]:
mean, I think it's a huge step in the right direction because the dependency management through Docker has always been, oh, yeah. Just install like, use sudo and then install a huge list of packages that are required to run here. And, like, what version those are, whether or not they work with the actual, Linux OS that you're actually running there compared to the one on your development machine or in your or, you know, in your dev box or whatever could be totally different or what's available in production. Because you're not actually running the exact container. There are some differences there. And then there's also your CICD pipeline, which is probably a different version of Linux on it. Right.
Daniel Loreto [00:30:09]:
Right. Right. Yes.
Warren Parad [00:30:09]:
And
Warren Parad [00:30:09]:
this is Exactly. This is just if you're using Linux. Right? I mean, then you throw in ARM and other operating systems, and it's it's for sure a huge mess. So, I mean, I definitely see that persistence is another aspect. Like, Nix doesn't you know, you're focused on using it for dev boxes in sort of a femoral way. Right? Like, you're not making permanent changes to the operating system. But in production, you almost want the permanence there. So there is this sort of disconnect and you're achieving the connection there through using Docker or plugging into persisting it in the Docker container, build file, I assume.
Daniel Loreto [00:30:41]:
Yeah. Exactly. Exactly. Right. Yeah. And, I mean, I think you touched on this, but just to emphasize it, because I think sometimes people don't think about it. A a Docker file is not necessarily reproducible. In fact, I'll argue it's probably not reproducible.
Daniel Loreto [00:30:57]:
People sometimes think of Docker as reproducible in the sense that if you previously built a particular image and you reuse that image, then you know you're getting well, you're getting what you already packaged before. Right? But if you have a Docker file with instructions and you build it six months ago and you build it today, the the output, the resulting image will likely be different because, again, I mean, it's the same scenario I was talking about with with your engineers just installing the environment. Probably your docile file just says install Python. And six months ago, the version of Python, you know, was x. Today, the version of Python is y. And so you build a container six months ago versus today, you get different, images as a result.
Warren Parad [00:31:44]:
I think code dependencies definitely fall code code dependencies fall on this, abyss here. Right? They're they're not quite infrastructure as code, which has been solved as far as deploying what we have to our cloud environment. And they're not the source code, which is stored so elegantly in in Git. There is this middle area where I need to configure the that container or, heaven forbid you're using a virtual machine directly or bare metal in a very specific way. And it's like, well, it's not IAC and it's not my source code. And then what I meant end end up with is some sort of script in bash a lot of times to explain how to actually install that thing. And so it's like, well, why aren't we doing declarative programming here like we are everywhere else? It's like the one area that still is procedural and still prone to a lot of issues. And, like, that's what I'm hearing is like, okay.
Warren Parad [00:32:30]:
This is totally solved now. You know, if you need more than one dependency, how can you not be using this solution?
Warren Parad [00:32:36]:
Right. Yeah. Exactly. I I think you said it, in
Daniel Loreto [00:32:39]:
a way that I agree. Like, it'd be great to just have your developer environment defined in a declarative way. And then that declaration, you know, we can use to build the environment correctly, and you can use it to build the environment in many different places. You can build it locally. You can build it inside a container. You can build it on the cloud. Right? And, you know, take the environment with you where you need it.
Warren Parad [00:33:06]:
Will's processing all of this, I can see.
David Kimura [00:33:10]:
I am, but, I think I failed to press the turbo button on the CPU this morning because it's processing slow. It's running slow this morning.
Warren Parad [00:33:24]:
That's okay. It happens to all of us.
David Kimura [00:33:26]:
Right. So with the, when it comes back to the production dependencies, is that declarative in the manifest file? This dependency is a production one or this one's a development dependency?
Daniel Loreto [00:33:45]:
Right now, in if you want to talk about, like, the implementation we have in the in the open source tool, you would have, like, two different dev ops, the JSON files. One for production, one for development. Okay. But we have talked about, you know, potentially unifying that in one file. Actually, we were making the the analogy to, package dot JSON before, and in a package dot JSON, you have the dependencies and Right. Dependencies that are for both. So, yeah, we're we're considering kind of unifying that. But, yeah, I mean, put aside the exact implementation we have today, the exact version.
Daniel Loreto [00:34:20]:
Like, I think the idea is can we declare our environments by specifying the packages we want and the versions that we want and then specifying which packages belong in both development and production and which ones belong only in development? And then take away that very procedural, like, bash scripts that you're doing to install every little thing, and then let a system that can reproducibly create the environment from that declaration, you know, do the work.
Warren Parad [00:34:55]:
You I think you hit on really well the scenario where you may not need to do something like this. Like, you don't need the I I hesitate to call it complexity of using, next based configuration. But it's just it's one more tool. And so No.
Daniel Loreto [00:35:07]:
I didn't.
Warren Parad [00:35:08]:
I I I would call it, like, I
Daniel Loreto [00:35:09]:
mean, it is an extra tool. Yeah. I agree with you. Like, if yeah.
Warren Parad [00:35:13]:
But, you know, compared I mean, swap that in for something else. I mean, it's pretty much a very simple bootstrap and and clear configuration files that are very understandable of of what they're doing. I'm I'm curious if you see a particular vertical or industry market segment where it becomes super critical still. Like, maybe if we say web servers, web development, maybe that's really simple. It's not too complicated here. But if if you're building a game I I'm just making stuff up now. Like, you know, this is where you can jump in and say, you know, this is exactly it. Like, game or web three, like, maybe there's a lot more value in or doing something with ML, a lot more value in next configurations.
Daniel Loreto [00:35:50]:
Yeah. I think I mean, I I think one of them is when you have a lot of when you need to use native dependencies that are not, native to the language that you're using. Right? So I think Python and ML is a good example, but, you know, that it's not just ML. Like, you know, there's several cases where you might need to install Python libraries that depend on kinda underlying c libraries. I I think those tend to not work as well if you don't have a a tool, like like that box. I think the other case is when you rely I mean, made similar, but maybe it's not a library in this scenario. But, yeah, you're you're relying on a binary that needs to be present in the environment, because you wanna call you know, exec that binary or or do something with it.
Warren Parad [00:36:41]:
How much of a mess is ML development from a dependency management today?
David Kimura [00:36:54]:
Say no more.
Daniel Loreto [00:36:59]:
I think I think it can get complicated.
Warren Parad [00:37:04]:
Yeah. Because and then yeah.
Daniel Loreto [00:37:05]:
By the way, you know, I'm I'm I'm talking about DevOps and and ML, but there's even cases where we are not able to help fully in that vertical, because then kinda drivers come in, you know, for the different GPUs and whatnot. And and there's some things that Nix can't fully control there unless you were using NixOS. Right? But we're, remember, we're we're doing this in in a scenario where you you keep your current computer, current operating system, and and just use that as this added virtual environment. But, yeah, it can it can get real messy, not just with the library dependencies, but kinda like all the related drivers and how they match the corresponding architecture that you're developing again.
Warren Parad [00:37:54]:
So you wouldn't recommend doing ML development today quite yet? Maybe we wait a couple more years, though.
Daniel Loreto [00:38:02]:
Yeah. Yeah. I mean, I I think it's an improve maybe I'm saying I think it's an improvement, but it's not all the way then. Well,
Warren Parad [00:38:11]:
I do remember, and I'll date myself here, when I was trying to, extract Bitcoins from the Bitcoin faucet, when I was in the university, and that included having to, basically getting my operating system to a state where I would just ended up with a black screen with a terminal, you know, and it's like, okay. You know, I really messed something up here. And, like, that was the state of the software at the time to just do anything related in, in the cryptocurrency space, in cryptography and in general, really. Good luck using those tools. So, like, that's definitely improved. Now you can use Python or JavaScript to interact with any of the, chains that are available out there, Ethereum and all the derivatives of it without a problem for the most part. I mean, there are some edge cases for sure if you wanna stake or mine that you're gonna have a problem. And I think that's where ML is.
Warren Parad [00:39:03]:
Right? It it used to be much worse, and I think it's slightly better, but I do feel like maybe we're going so quickly and we've monetized whatever little, amount of value we've been able to create as a society through, model development that we haven't really took the opportunity to innovate with the development pipeline.
Daniel Loreto [00:39:26]:
Yeah. And I you know, I'm sure there's I I I do think there's probably plenty of people working in in solving this given, you know, the the the AI wave and the whole interest in in MLM.
David Kimura [00:39:37]:
Optimistic of you.
Warren Parad [00:39:40]:
I mean, it's I think it's I think it's optimistic, but I also I also think that maybe a little bit realistic because with the, deep state model that had come out I mean, DeepSeek. Right? That it, I thought we were in a different podcast for today. Yeah. I mean, it's a joke that, you know, I frequently go around with some of my colleagues, about, you know, the impact of of these models. But the fact that you are there's now competition requires doing things more efficiently. You can't bring a new engineer on board and have them waste months and months not on on tool set, like, not even on the domain. So that's what drives innovation there a lot when you're in competition, to succeed. And I think, that's gonna be a really important qualifier going forward in the ML space that we continue to have sufficient competition to to drive innovation there.
David Kimura [00:40:35]:
So this is an area where I haven't spent any time or effort. But, Daniel, you have right because you have been working on some ML work to do QA end to end testing. Right? So what's the what's the barrier to entry to get into that space?
Daniel Loreto [00:40:56]:
Like, meaning, like, what's the barrier for engineers to start creating agents themselves and kind of what knowledge and tools might they need?
David Kimura [00:41:05]:
Yeah. For sure.
Daniel Loreto [00:41:07]:
Yeah. So, yeah, I mean, like like you said, we are developing, an agent that we call called test pilot. It's meant to be, an AI QA engineer to away all that boring, repetitive manual clicking on UIs that people often do. Yeah. And we we've been learning a lot. I mean, I think I think agents, are relatively like, big investments in agents are relatively new. And so I think the barrier of entry is something you can catch up on. Like like, I would actually incur if if you're interested in creating agents, I I think it's a good time.
Daniel Loreto [00:41:44]:
I think you can learn. I I think you can do it. I would say I mean, to me, one of the most important things you need to do with developing AI applications is set up a evaluation framework, for the work that you're doing. So in in in non AI, non you know, kind of traditional software development, let's just say, you can often write a function that you can reason about fairly well, and then you can write tests that are fairly deterministic to know whether you your the behavior is what you expect. Right? As as soon as you involve the LLM, well, now you have nonprobabilist like, no sorry. Non deterministic behavior. Right? And so you might call it once, and it might tell you something. You might call it again in a second, and it might tell you completely different thing.
Daniel Loreto [00:42:41]:
And often, because you're dealing with very general cases, you might be feeding it lots of different examples, and you don't fully grasp how it's behaving across all those different examples. So so to me, kinda like one of the first things you need to do is set up a bunch of examples with kinda like the expected behavior, and then be able to measure kind of the changes that you're doing and whether they're improving that evaluation that you have in mind. Alright? And then in this case, it's gonna be not purely pass fail. It's gonna be, like, okay. Today, you're at 50% of being able to handle the cases that you want. And then you do a few changes. And, okay, now you're at 55%, and then then you're at 60%, and so on. So, yeah, anyways, I don't know.
Daniel Loreto [00:43:30]:
I took took I can probably think of more, kind of best practices, but I that's the first one that comes to mind.
Warren Parad [00:43:38]:
I would even maybe take a step back before that. I mean, that that's really great insight. Like, someone who hasn't spent a lot of time in it, is there a particular site or foundational model that is where you would suggest people start going for first? I mean, probably probably I I think from my own experience, I know that building models is super expensive. So they're probably not gonna do that. But maybe there's a service out there that you say, you know, go pull this model and try to fine tune it or use it out of the box. Like, any recommendations on that?
Warren Parad [00:44:07]:
Yeah. I mean, I I I
Daniel Loreto [00:44:08]:
do think you need to start with the with the preexisting general model. I I recommend using probably a kind of multimodal provider, something like open router, that allows you to easily swap models. Because what what's happening is I mean, you guys see on the on the tech news all the time. Right? Every every other month, each AI vendor, like, outdoes each other. And it's like, oh, we have this new new model, and it performs better on this benchmark. And so you don't wanna tie yourself so strongly to one model that when those innovations are happening, you can't test the other one. Right? So I would pick some provider, like like I said, OpenRouter is one of them. I think Grok, I think CloudFlare has something these days too.
Daniel Loreto [00:44:56]:
But, anyways, a provider that lets you specify which model you wanna use, and then kind of build the agent on top of that. I think the other part of building an agent is not just the underlying model that helps you reason, but there's kinda how you prompt it, right, and kinda how you tell it to do what you want it to do. And then the last piece is kinda like the tools that you give it so that it can take action. Right? So high level in my mind, like an agent is you're grabbing the LLM for the reasoning, but you need to kind of give it hands and eyes, if you wanna think about it that way, so that it can manipulate an environment that you're giving it control of.
Warren Parad [00:45:43]:
Right on.
David Kimura [00:45:45]:
Is that something that you've got a significant portion of your team dedicated to, or is it something that you can iterate on quickly with just a few people? Like, how big of a, how big of a manpower
Daniel Loreto [00:46:02]:
Yeah. I mean, I I know most of our team is working on on test pilots. I mean, something we found is so we're we're a a Go shop. Like, we use Go as our primary, programming language. We we've been using it for for a while, and so we wanted to continue using it. And what we're seeing is that a lot of the tools that are readily available in this space are Python based. And, you know, we we can use Python for stuff that's not production, but for our production stuff, we wanna continue using Go. And so we're having to build some of them ourselves in Go.
Daniel Loreto [00:46:38]:
We do plan to open source them later on and kinda give back to the community and hopefully, kinda people that like Go, you know, let let them have some readily available tools for building AI and agents. But, yeah, the the that stuff that's getting built right now, and and often we look for it, and it's like, oh, I guess nobody has, you know, done this type of library yet in in Go for AI. So it it is early days.
Warren Parad [00:47:04]:
Yeah. I mean, I think that's a really good point, actually. Like, it's usually a mistake to rely on Python for high reliability software development in production. And so if the tools aren't there to support your production workloads in a in a different language, then that really tells you how early it is because the people that are utilizing those tools don't care about, like, sustainable code or reliable code because it doesn't offer the core features to guarantee, safety or quality of the of of your value of your service that you're building.
Daniel Loreto [00:47:38]:
Yeah. Like, I I mean, there there's many different ways of building a company, but just I I've had to deal with, like, Ruby production systems, Python production systems, all that stuff. And I'm I'm kinda like in this case, we're, you know, we're picking different language and, kinda moving forward with that.
Warren Parad [00:47:56]:
Yeah. No. I'm totally with you. I used to be, like, early on in my career, I'm just like every language is the same. Right? You know, there's you can swap between one or another if you've had a sufficient years of experience. But then over time, you start to realize the fringe benefits of individual languages or negatives that are associated with them. So things like Go or other well typed languages like c sharp. The the community or the packages that are available, to help drive forward having the the right execution of the right stuff in production is, like, super important.
Warren Parad [00:48:26]:
Not which is when you're building something quick and hacky, but when you actually care about it working continuously. And, like, if you're at a company and you're one of the core things you're building is, reliable builds, I I feel like, you know, that goes hand in hand Right. Right. Using other I mean, it's sort of like a different mindset. Right? Like, if you're hiring an engineer and they care about using, scrappy tools, then ones that potentially break from day to day, the nightly build version of that tool is broken by default. They're probably not in the right mindset to be working on something where you need high reliability or reproducible builds, or, you know, these other technologies. It's just a complete other side of the spectrum.
Daniel Loreto [00:49:06]:
Right. Yeah. Totally agree. Yeah. I mean, what what we've been doing, maybe where we kinda allow Python to come in is, you know, I mentioned we have this kinda evil process. We're able to try ideas. And so I'm like, okay. We're prototyping a direction, and there happens to be kinda like a Python library that does that off the bat, and we just wanna check if that's gonna improve our performance, you know, the performance of our agent.
Daniel Loreto [00:49:31]:
Like, you know, prototype something quickly with Python, pull that library in, run it against our eval framework. Let's see if this direction has length. Oh, it does have lengths. It is promising. Okay. How are we gonna build it now for production usage and, you know, handle the load we wanna handle and take care of all the edge cases we wanna take care of and and so on? And then we and at that point in time, then we spend the time developing the the Go, software and infrastructure.
Warren Parad [00:49:58]:
Makes a lot of sense.
David Kimura [00:50:00]:
So that's, one of the things I wanna ask you, you know, you talked about, open sourcing some of the tools that you're making and contributing back to the community. That's a huge commitment, and it's a it's a conscious decision by companies because a lot of companies are like, yeah. We invested this. It's a core part of our business. We're using it for the business. But then you're saying, I wanna I wanna give it back to the the community.
Daniel Loreto [00:50:31]:
How do you
David Kimura [00:50:34]:
how do you tie that back into your business model and and, like, justify or account for the time that and the salaries that you're spending on people working on an open source product?
Daniel Loreto [00:50:49]:
Yeah. It's a great question. And, you know, I'm sure everybody will kinda make different judgments there. But for us, the way the way we think about it is, okay, we're developing Test Pilot, this kind of QA AI QA engineer. You can think of the software that makes up Test Pilot as being in two different buckets, at least in my hand. Right? One is, like, the truly domain specific logic and and, kind of proprietary aspects of it that make it behave like a QA engineer, right, and that we're not open sourcing. But then there's the tools that we wish we had had that that we had to develop for for ourselves that are applicable to developing all sorts of AI agents and all sorts of, kinda, AI related products. And, you know, we're we're a start up.
Daniel Loreto [00:51:40]:
We are trying to get more eyeballs. We're trying to kinda become more prominent and get people to hear about us. And so I actually think of open sourcing, almost as a marketing exercise. Right. We don't have marketers. We're all you know, we're very technical teams, like, basically engineers, but sometimes I think you can look as open source as as as a way to kind of bring in that audience. And because our audience are developers. Right? I think we're in a different vertical, you know, serving different customers.
Daniel Loreto [00:52:15]:
Maybe open sourcing doesn't work that way, but our audience is developers. And if we can create a reputation of, wow, those guys at Genify really know what they're doing in terms of AI agents, and have you checked you know, have you seen the tools they've made and the libraries they've made? I think that all just kinda comes back to to the type of business that we're building.
Warren Parad [00:52:38]:
Right. I I I totally agree. I mean, we went down the same path, a while ago with some of the things that we've we've open sourced. It is, like, unfortunate to say, it is like a huge marketing story. It's a marketing story for, brand awareness. It's a marketing story for hiring engineers. So they know about your company or what you do before that, you know, they show up in the interview. They can research it and see that you have something technical rather than a website with a couple of links on it and a knowledge base.
Warren Parad [00:53:06]:
Like, that's not a lot. Right? You know, this gives them more ability to have insight in what you're doing. I mean, there is there is a pessimistic side of me that says, like, no one cares what you put on the Internet. Like, you're no one's that special. Right? Your thing doesn't matter. I remember, like, a bunch of times, like, Netflix went and open sourced a bunch of huge technologies. And I'm just like, I don't care about those things. But now some people maybe are using that in production, I guess, could be, potentially, because I think they wrote them for Kubernetes or something.
Warren Parad [00:53:35]:
And, you know, so that's great. You know, you you get that experience. I I think it the one thing we try to be really careful about, that I think is important here is that the thing that you're building needs to be really close to your your value, your actual product. Because even if it's, like, one or two hops away, then it's the people that are utilizing it. They don't care what that thing is that that you're actually doing, which is super unfortunate. But I will say that rather than a marketing tool, there's a second, step here, which is if you're gonna open source something, I feel like you feel like it's more important to do it well. And things that are done well will live longer, and people will see and start to utilize more. And that give brings, sort of like a your own usage levels it up because you actually invested in doing that thing well.
Warren Parad [00:54:22]:
And so, like, even if you weren't Dev Box wasn't sort of out there as a a cloud thing that you could utilize and pay for, the fact that you've open sourced it means that that tool is now serving you hopefully better, for your ML development for your own agent. Right?
Daniel Loreto [00:54:38]:
Yeah. Absolutely. And, actually, I it's it's important for, like, different benefits that open source might bring, I'll I'll give you a third one, at least in my view, which is, you know, when you're a start up, I think one of the most important things is be talking to users. Right? Be talking to users and learning from them. And we found that when we open source things, we often get to talk to more users, but we also take get to talk to users that normally would not have talked to us. So, yeah, take that box.
Warren Parad [00:55:11]:
I I
Daniel Loreto [00:55:11]:
don't know if I can share the specific companies, but there's some biggish companies using that box that I think if we had just kinda pure proprietary solution or this little start up, they never would have looked at it. I think what gave them confidence to look at it is like, oh, there is this open source project. Kind of, you know, worst case scenario, the start up goes nowhere. We can rely on that open source project, and we can kinda continue improving it and whatnot. And so that's given us at least a channel to communicate with, those types of potential customers and users, and and it brings in a ton of learning for us in terms of, okay, what features are important? What what is that class of user or customer asking us about, and how should that affect our product road map and and whatnot?
David Kimura [00:55:59]:
Oh, that's huge. Like, in my opinion, like, one of the like, the primary objective of every startup is to figure out the difference between what you built and what you cussed what your customers thought you were building. And so open sourcing gives you a way to have that conversation without trying to force them down through a sales pipeline.
Daniel Loreto [00:56:21]:
Right. Exactly right. Yeah.
Warren Parad [00:56:22]:
It's it's scary though because, you know, when you put it out there, it's like, well, people start will start to see my actual bad code.
David Kimura [00:56:30]:
Right. Or,
Daniel Loreto [00:56:31]:
like It is scary. Yeah.
Warren Parad [00:56:32]:
My commit messages, you know, that maybe I should have worded differently.
Daniel Loreto [00:56:36]:
And, I mean, it is tricky too in, like, you know, I gave you our kinda, like, our heuristic for figuring out what parts of test pilot are open source and which ones aren't. But sometimes you're in some gray lines, and you're kinda like, should I open source this or not? Like, is it do I get more benefit from opening it up to the community and getting that audience and maybe getting some faster adoption? Or do I get more benefit from keeping this as kinda like the secret sauce of the company? And, you know, there's all those stories like Elasticsearch and AWS, and everybody gets big and starts changing their licenses. So, like Right. So
Warren Parad [00:57:19]:
I don't know how I don't know how right I am here. We've done some initial research on on the topic, and it does actually seem like customers that fall into a particular bucket never leave that bucket and go to a different one. Like, a free user never converts to your lowest tier paid plan. And the average business plan doesn't escalate to enterprise. It just doesn't happen. People self identify with whatever bucket they're in. And so whether or not you have an open source solution really only functions as advertisement. The people who are using it aren't gonna be more likely to get to like, start to pay you.
Warren Parad [00:57:53]:
And one of the things we recognize, which is one of the reasons why, our product, Authorus, is not open source for 90% of it, is because those free users actually generate 99% of the support cases that you need to deal with. So there's, like, a huge cost investment that goes into it, and you don't get any reward coming back from that. I mean, of course, there's some gray areas where there's highly coupled technology and where it's being utilized effectively. But often, the one the customers that are gonna pay you the most money or be your best customers, like, they don't particularly care about running it themselves in most cases. Unless they're like a government entity, or
Daniel Loreto [00:58:32]:
Right. Right.
Warren Parad [00:58:33]:
You know, they have some regulations they have to deal with.
Daniel Loreto [00:58:36]:
Yeah. I mean, usually, if they really carry some compliance requirement, but that might just be pointing to, like, you just need to solve that compliance requirement as well. Right? Like
Warren Parad [00:58:45]:
Oh, yeah. I mean, compliance doesn't mean security. So, like, that's a completely different story.
Daniel Loreto [00:58:50]:
Yeah. Absolutely.
David Kimura [00:58:51]:
It's an additional tax. Right on. That was that was intense.
Daniel Loreto [00:59:06]:
No. I don't in a good way.
Warren Parad [00:59:07]:
Yeah. Yeah. Yeah. Definitely in a good way.
David Kimura [00:59:09]:
Definitely in a good way. I didn't mean otherwise.
Warren Parad [00:59:13]:
I mean, I I know we we jumped, through a lot of different topics, Daniel. I if you feel like there's something you know, one last thing you wanna share, regarding either Dev Box, or the QA copilot, test pilot, you know, have at it.
Daniel Loreto [00:59:29]:
Let let me think.
David Kimura [00:59:30]:
Famous last words. Go.
Daniel Loreto [00:59:33]:
Oh, man. A lot of pressure. I mean, I don't know. I I guess I'll just add, maybe this is less about the specific products and just kind of things I'm excited about, but I am bullish on on AI agents. I mean, obviously, I'm, you know, building one. But, I just think there's more and more parts of software development that could be taken care of through agents. And I think it can be done in a way where we take away that kind of repetitive work, and we still leave a lot of the fun and creativity to to the humans. I know there's more doomsday scenarios that people talk about, right, where it's like, oh, we're all out of jobs and whatnot, but I I'm actually hoping it's more, like, now we can all achieve more, because those less interesting parts of the cycle are taken care of automatically.
Daniel Loreto [01:00:25]:
And and now you can focus more on system level thinking, on product level thinking, and and so on. So, yeah, maybe I am an optimist like you guys were saying before. I mean,
Warren Parad [01:00:36]:
I'm just gonna say to that no comment.
David Kimura [01:00:40]:
I'm gonna agree with you, Daniel. Like, regardless regardless of what technology does, I know there's always gonna be problems, and I've I've made a career out of solving problems. So I'm I'm not too worried about it.
Warren Parad [01:00:54]:
Okay. I'm super worried about it. Should I share my opinion? Yeah. Absolutely.
Daniel Loreto [01:00:58]:
So No. You you can't be worried about it, but, like, let's make the world we want.
Warren Parad [01:01:02]:
Well, I I feel like it's not a technology that's right for being democratized for access for everyone to utilize, equivalently. The people with the most money and the most power will have the most powerful, agents at their disposal and that just entrenches their own power and prevents them from being ousted. And it doesn't leave a lot of opportunity for those that are already being disadvantaged to rise up and, fight against that oppression.
David Kimura [01:01:37]:
There's a a documentary on this, called Terminator. Check it out.
Warren Parad [01:01:42]:
I I think I've heard this.
Warren Parad [01:01:44]:
I think it has a well well known, narrator. Right?
David Kimura [01:01:48]:
Yeah. Yeah.
Warren Parad [01:01:52]:
No. I mean, that's that for me is, that's optimism, that that documentary.
Warren Parad [01:02:01]:
That that that's a good scenario.
David Kimura [01:02:06]:
Alright. Oh, man. Now that we've all made our way onto an FBI watch list, let's move on to pics.
Warren Parad [01:02:13]:
Yeah. Sure. So I'll go first. Today, I my pick is gonna be The Martian, the book by Andy Weir. He has a whole bunch of books. I don't know if I'll say this one is the best, but it's definitely really good. They're all science fiction adjacent. I mean, they're not they're reality, played out in a hypothetical future where, well, it's a new scenario.
Warren Parad [01:02:38]:
And they're they're great. I think it's really well written. One of the first things I ever read from Andy Weir was actually something called The Egg, which is a very short story, which I find also really good. And I and many, many years later, I read The Martian, and I didn't realize that it was the same author. And it's quite interesting to have read two pieces by the same author and be like, wait. There was this other really good thing that was written in a similar way. And, like, going back and and actually seeing it. It's like, I actually read that thing.
Warren Parad [01:03:06]:
So if you haven't read it, I haven't seen the movie yet, actually. But if you haven't read the book, I highly recommend it.
David Kimura [01:03:11]:
Right on. I have done both. I've read the book and seen the movie. But, I'm happy to hear you say that he, that his other books are better because that's
Warren Parad [01:03:24]:
Yeah. That's not true. I I really like the short story. He has, a drawn comic with, like, an overlap between Alice in Wonderland and, Dorothy from Wizard of Oz. And I forgot who the third character is. That's okay. The Martian's great. There's two other books.
Warren Parad [01:03:44]:
I think there's a fourth one coming out. They're not related at all. The one, Luna, I think, for the moon, it's not as good, but still interesting. And then there is one with, Deep Space, which is also really fantastic.
David Kimura [01:03:56]:
Right on. Cool. Daniel, what'd you bring for a pick?
Daniel Loreto [01:04:00]:
Awesome. So I have a book as well. It's it's Wild Robot. So I think I mentioned earlier, I have two young kids, an eight year old and a five year old. And so I I do a lot of bedtime with teen and and reading with them. And so Wild Robot is, one of the books that I'm reading with with one of my kids, by Peter Brown, there's a movie as well, kinda similar to the example you gave. But, yeah, I I I just find it really interesting to read with my kids. Yeah.
Daniel Loreto [01:04:29]:
Your high level, it's about a robot that is pounds itself in an island, doesn't know where it came from, and eventually needs to take care of, an animal. And it kind of makes you reflect on all these questions like, you know, where where where is parenthood? Is this robot taking care of this animal? You know, is that apparent? What what's the relationship between that animal and the robot? And, I don't know. I guess I get a little sentimental because I'm reading that with my kid, and I I'm just fine. I've just found it, like, really, interesting for both of us to go through the story.
David Kimura [01:05:05]:
That's cool. That's super cool. Alright. Well, we're gonna make it a three peat. I'm changing my pick, and I'm going with a book as well because, Warren, you said something about, you know, recognizing the writing style a while back. I read a book and, really enjoyed it. And I was like, wow. This guy writes so much like Stephen King.
David Kimura [01:05:30]:
It's just insane. And the author's name is Joe Hill. Turns out it's Stephen King's son. So Oh, wow.
Daniel Loreto [01:05:38]:
Oh, wow.
Warren Parad [01:05:38]:
Yeah. Yeah.
David Kimura [01:05:39]:
Yeah. The book I the book I read was Heart Shaped Box. And if if you like Stephen King's writing style, Joe Hill just seems to have picked up on that style and continued running along with it.
Warren Parad [01:05:52]:
Is it also, like, thriller suspense stuff, or is it a different genre?
David Kimura [01:05:56]:
Nope. Same same stuff. Same, like, sick and twisted thing where you're like, oh, wow. I I did not see that one coming. Yeah. Good stuff.
Warren Parad [01:06:07]:
Do you ever find it a challenge to, like, pick up a book that you know is gonna be like that and go through it and be like, I'm I don't know if I can keep going. Like, I'm really afraid of what I'm gonna read next.
David Kimura [01:06:19]:
No. I don't I don't think so. No.
Warren Parad [01:06:26]:
He's like, I love it. It's great. That's what I look forward to.
David Kimura [01:06:29]:
Right? Yeah. I mean, I I guess the rest of that conversation is for me and my therapist, but
Warren Parad [01:06:41]:
Oh, don't worry. We'll shut the camera off here in a moment and,
Daniel Loreto [01:06:43]:
we can
Warren Parad [01:06:44]:
do the conversation.
David Kimura [01:06:47]:
We're gonna be like, no. No. I promise I stopped recording. Tell us all about it.
Warren Parad [01:06:51]:
Well, you see this little red light in the top left hand corner.
David Kimura [01:06:54]:
It's just a UI glitch. Don't worry about it. Daniel, thank you so much for joining us. This has been a blast. That's fantastic.
Daniel Loreto [01:07:03]:
I've enjoyed it as well. Thank you guys for for having me.
David Kimura [01:07:06]:
Alright. And to all of our listeners, hopefully, you enjoyed it as well. Than
And we're live with another episode of Adventures in DevOps. Almost blew the podcast name right there, Warren.
Warren Parad [00:00:10]:
Well, you know, things happen. Alright. I've only done I've only done a few hundred of
David Kimura [00:00:16]:
these at this point. It's it's still a learning curve. So how are you, Warren? Did you bring us a security fact today?
Warren Parad [00:00:24]:
Well, you know, I I think again, I had brought one last time, and I thought, you know, this time for sure, our fellow co host, was gonna be here and share the long awaited one that, I I had been promised. And it you know, coincidence, not here. So I don't have anything today.
David Kimura [00:00:44]:
She's pausing for dramatic effect. Like, a really, really dramatic effect.
Warren Parad [00:00:52]:
It's gonna be the best fact ever, I swear, when it when it actually drops.
David Kimura [00:00:55]:
It is. It is. Right on. Well, also joining us today, Daniel Loreto from Jetify. Daniel, welcome to the show, man.
Daniel Loreto [00:01:04]:
Thank you so much, Will. Happy to be here.
David Kimura [00:01:06]:
Dude, I'm excited to have you here. So you are with Jetify now. Tell us a little bit about your background.
Daniel Loreto [00:01:14]:
Yes. I'm a software engineer by trade. You know, have worked at a lot of different tech companies of all different sizes, so I can talk about, you know, different experiences.
Warren Parad [00:01:25]:
But
Daniel Loreto [00:01:25]:
I worked at Google, Airbnb, Twitter. I worked at, you know, tiny startups that you never heard of. I worked at, Virta Health, which is a telemedicine company for diabetes where I, ran engineering, data science, IT, infosec, kinda like all the tech stuff. And and now I'm founder and CEO of Jetify. We're a company kind of trying to make it easier and faster to turn your ideas into working software. And, anyways, today, we've been playing with AI and agents on how they can help with the software development process.
Warren Parad [00:02:00]:
I mean, you you've got that opening so well nailed down. I can tell this isn't the first time you've had a meeting with yourself.
Daniel Loreto [00:02:08]:
Thank you.
David Kimura [00:02:11]:
Got the elevator pitch nailed.
Warren Parad [00:02:13]:
Well, I mean, it's sort of funny because, like, I meet with a lot of people and a lot of them feel the need to sort of introduce themselves and say a lot about their background even if it has nothing to do with the conversation at all. And some of them just go on and on for minutes. And I'm like, you're mentioning other people's names, and I don't know who any of those people are. You're talking like they're the most important people in the world. And, like, I for sure, like, I don't care. I mean, at least you stuck to company names. So, like, you know, that's great on you. And I think a lot of people know, most of those companies.
Warren Parad [00:02:40]:
I think most of them still exist, by their original name. So, you know, that's what's important. I was raised
David Kimura [00:02:46]:
in a Midwest Protestant community.
Warren Parad [00:02:50]:
Just go really deep into the ground.
David Kimura [00:02:57]:
Florence just not saying anything.
Warren Parad [00:02:59]:
Well, I mean You're gonna mute
David Kimura [00:03:00]:
you're gonna mute me, aren't you?
Warren Parad [00:03:02]:
I got I got I actually got sucked for a second because I spent, not short period of time in the Midwest, after I graduated from a university because it was the only place that I could get hired, as a job. I wasn't actually a software engineer by trade. And so I ended up there. And I was like, do I have a connection for this? Like, can I talk about, the religious values? I'm like, not really. I actually I I actually don't know a lot there. Like, I've never really been a huge into the religious scene. So I I can't say anything. And so I just was like, I'm just gonna keep my mouth shut.
David Kimura [00:03:33]:
I was trying I was thinking, like, in the second Austin Powers movie whenever doctor evil goes to therapy with his son, Scott. And they're like, tell us about yourself. And doctor evil gets up and he does, like, this long rant about how he was raised by his father. And and it goes really, really deep, but I couldn't remember enough of it to try to work it into it as a joke here. So Well,
Warren Parad [00:03:56]:
don't don't worry. You'll have plenty of opportunity to go look up that for, for next time. So today
David Kimura [00:04:02]:
might check for the next episode.
Warren Parad [00:04:04]:
Yeah. Well, I I don't spoil it now. I, I really like the idea of getting back on topic of of dev boxes, environment machines. Like, I I feel like this is a huge challenge that still exists. And, you know, it's interesting because I work personally in, like, over the years in a lot of different IDEs and a lot of different software languages, and I've always hated it. Like, there was never a time where I'm like, you know what? Working in this language, this is the best. And I I feel like, you know, if you're in the space, you've got some horror stories. You've got some, like like, how did you even get into this? You know? Why is why is this where you went?
Daniel Loreto [00:04:45]:
Yeah. So, man. So,
Warren Parad [00:04:48]:
I mean, I I guess
Daniel Loreto [00:04:48]:
a lot of these stories start with kind of scratching your own edge and trying to solve a problem that you have. So, you know, just having worked at different software companies, I I've seen those stories where, things work on your computer and they don't work on your teammate's computer, or there's a bug in production where you can't reproduce it locally. And often that's tied to the environment. And so when we started Genify, we were like, you know what? From the beginning, let's just set up some reproducible dev environments. And we decided to set up some Docker files for every kind of piece of software that we were working on. But very quickly, we ran into some pains, because, you know, Docker uses a lot of resources, on on your computer. And so when you're developing locally like, shipping to production, great. But developing locally, like, I don't know, the fan spinning in your on your laptop, you do a little change and you wanna see the results and, oh, it was the wrong line in the Docker file, so you busted the entire cache.
Daniel Loreto [00:05:52]:
And now you're, like, rebuilding the entire container. And so all of that felt really, really painful. And so, we had heard about this technology called Next. We wanted to try it. We we kinda failed two things at once. One is like, this is awesome. It it kinda creates reproducible environments. And then we were like, oh my god.
Daniel Loreto [00:06:12]:
The developer experience on UX for this thing is not what we want it to be. So, anyways, we ended up building an open source tool called Dev Box that kinda wraps next and lets you create those environments locally. And, you know, we just posted it on Hacker News. Before we knew it, it was, like, number one, on Hacker News that day. And so it kinda told us, oh, people are having this problem, that they want some solution. And that's, yeah, that's how we started with the developer environment work.
Warren Parad [00:06:42]:
I I love the story of, like, pain and suffering through development process because it's such a I mean, it's such a common strategy for innovation. Right? Like, I as a software engineer, you can, like, fix your own tools in a way. And so realistically, like, when that happens, it's like, well, I'm gonna make something better that doesn't it isn't as bad as the thing we're using. It's interesting you brought up Docker. One of the things I've seen a lot is engineering teams attempting to get the development experience on their local machine working through Docker. And this is something I've never understood and maybe this is something you could, jump into if you have some understanding about it. Like, why why why do that? Like, if I have the source code on my machine that I've cloned from a git repository somewhere, why why not just install the couple of tools I need and, you know, hit f five and run it? What does Docker really get me there?
Daniel Loreto [00:07:32]:
Yeah. So, I mean, I I think the the issue forget Docker for a second. Like, why isn't it enough you know, Docker or no Docker. Why isn't it enough just to check out the code, install the tools, and call it a day? What happens is that in many cases, you don't just need the source code. You need the specific versions of the tools that you're working with. So it's not like I need, I don't know, Go or I need Python. It's I need Python two point seven point eleven. Right?
Warren Parad [00:08:05]:
Hopefully not anymore.
Daniel Loreto [00:08:07]:
Yeah. You're right. I mean,
Warren Parad [00:08:08]:
that's that's that's an old version.
Daniel Loreto [00:08:09]:
But just well, but but see
Warren Parad [00:08:11]:
I mean, you say that,
Daniel Loreto [00:08:12]:
but, like, what happens a lot is is you're working at a large company, and there's this project that everybody depends on, but it's not staffed well enough because other other things became more important. And you gotta keep that thing running. Right?
David Kimura [00:08:27]:
Every day everybody depends on it, and nobody's willing to update.
Warren Parad [00:08:31]:
Yeah. Yeah. Exactly.
Warren Parad [00:08:32]:
There is I I I think we should just pause for a moment here because I know a nontrivial amount of our audience members are, like, suffering through some sort of PTSD withdrawal right now. Having said that, like, we're you know, you're working in some environment and, you know, some code that hasn't been touched in a while is not working. Like, I feel like that's, like, 10%, twenty % of your job, whatever it is. You know, you have no idea what's going on there and it's not working. And you didn't write the code. You may be lucky if you knew the person who did write it because, so one of one of my first jobs I had, there was an engineer who if I saw his name on a commit message, I know that there was gonna be a bug with that code that was going to cause me to lose the rest of my day. Until, like, if I'm debugging around and I see a line of code by, by by by Craig, that's it. Like, I'm sorry.
Warren Parad [00:09:25]:
Like, I know. It's gone.
Daniel Loreto [00:09:27]:
Shout out to Craig. Right there. So but but, anyway, I mean, yeah, that that's the point. Like, you often have to work with these projects, and they often need specific versions of software. And if you don't use those versions of those tools, things break. Right? And so, I guess, in a in a project where you haven't set up too much around developer environments, you might just have some read me file saying install Python. And so everybody is installing Python, but somebody installed Python, you know, a year ago. Somebody installed Python six months ago.
Daniel Loreto [00:10:03]:
Somebody installed Python today. They all got different versions of Python. Right? And you compound that by all the tools you might need in a project. Right? So it's not just Python, but maybe, let's say, you're using protocol buffers, you have a product both compiler, you know, and so on. And so, yeah, I think that it's rude. Like, that's that mismatch of versions is kinda what people are trying to solve that is not solved by just the read me saying install these tools and, start working on the on the project.
Warren Parad [00:10:36]:
Yeah. I definitely agree. I mean, there's a huge aspect here, which I think has to do with the languages themselves and their package managers are part of that. I got it working once. Let's not go back and touch it by even the language, maintainers themselves. You know, it's very limited, experience there. And so they they go off, they, you know, the package manager for whatever language you're using, like, it's sort of working. And so there are some that are definitely not as great as others, and then you just compile that on and put that usage as a dependency for literally everyone who uses that language and framework.
Warren Parad [00:11:11]:
And I still think there are languages today that no one is happy with the package manager for that language, and yet somehow they're getting over it. And
Warren Parad [00:11:19]:
I mean, there's languages where you have five package managers that you wanna know which one you're using. So so
Warren Parad [00:11:26]:
I have a feeling that you are throwing, the JavaScript, environment under the bus there a little bit. No. But I'll I'll have to I I have to say that JavaScript is one of no JS specifically is one of the best development communities compared to the other ones. Like, yes, there's n package managers, but all of them, like, work at least 99% of the way. And and and there are problems with them for sure individually. You know, they're not so totally perfectly robust. But, you know, compared to some other ones, like, I don't I don't struggle to pull out a new JavaScript package manager. I do struggle to figure out how to make Maven work even though I've gotten it to work like a hundred times before.
Daniel Loreto [00:12:11]:
And I'm not I haven't touched the Java world in such a long time. I I have no I have no comment.
Warren Parad [00:12:16]:
Yeah.
David Kimura [00:12:17]:
I just had a little bile come up in the back of my throat.
Warren Parad [00:12:22]:
You know, the thing is, like, I I remember c sharp, like, NuGet was not a good package repository. The tool wasn't great. Even dot net improved a lot of things there, but they had a diamond dependency resolution problem where, like, you depend on two dependencies who require different versions. And and Node. Js, like, figured out how to get around that. Npm solved it, yarn solved it, p m p m. They all have solutions there. So yeah.
Warren Parad [00:12:44]:
I mean, the individual tools aren't great and it's not great that there's lots of them, but at least I feel like they're not getting in my way. But, yeah, like, if I'm programming in c plus plus or something old school with for, like, a large company, then not only are there dependencies that are probably committed to a repository. Like, they're not in some package manager somewhere. They're like the the binary dependencies are, like, in the source code, and they require some Microsoft, redistributable c plus plus library from twenty years ago that I have to go and download from the Internet from some suspicious website in order to even, you know, build my project to get it to run somewhere.
Daniel Loreto [00:13:22]:
Yeah. Agreed. So yeah. Anyways, I mean, I think it's all those paper cuts that just slow you down in your development process. And all of a sudden, you are I don't know. You wanted to work on a feature. Right? You wanted to work on something related to the project that you're interested in, and you find yourself in dependency hell and just going deep into package managers when all you wanted to do was, you're working on an actual feature.
David Kimura [00:13:49]:
Well, I think that's I think another aspect of that is onboarding new developers onto your team. Right? Like, how long does it take to get them where they're actually working at on the code that you hired them to work on versus setting up an environment to actually see what they they're actually supposed to be building?
Warren Parad [00:14:11]:
That sounds like a question to air our dirty laundry on the show. So I guess I'll go first. You know, it's like a couple weeks to a month, and then I think we know that the long tail turnaround is like ninety days to get to peak performance. And then it's like six months to do effective reviews at that point. And before that, I think there is a very long time where there's still a lot of training and education that goes in. Not But it for us, it's mostly domain specific. Like, if we don't hire a domain expert for our company, we deal a lot in the, authentication authorization space. So, like, how SAML works, how OAuth two works, how OpenID works, how SCIM works, how audit trails work.
Warren Parad [00:14:50]:
Like, there's a lot of things that you don't wanna get wrong in a way, and so there are some tools. So but it's a long period of time. Will, you're smiling there. So I feel like, you know, you're gonna tell me that, you know, your new hires that I I believe you were interested in, interviewing and, getting into your team. They're, like, couple weeks max, and they're already at peak performance.
David Kimura [00:15:09]:
So I this is definitely an outlier. So I won't say that this is the state of our environment by any means. But, the last guy I hired, about a month ago, he was pushing production commits by the end of the week.
Warren Parad [00:15:25]:
Which is good or bad. Right?
David Kimura [00:15:27]:
Yeah. Oh, okay. He was pushing functional production commits that actually did beneficial things.
Warren Parad [00:15:36]:
Wow.
Warren Parad [00:15:37]:
I mean, that's amazing.
David Kimura [00:15:39]:
Yeah. But it was, like, it was a very unique situation. You know, we hired him, and he came in with a very specific set of skills that we needed. So he looked at our environment and was like, oh, yeah. Here's where it's wrong, and immediately started contributing to that. And then there's, like, the rest of the company, like, the web three space that we live in, where he still has a lot of onboarding to do before he fully understands that. So it was cool to see, and and I paraded it around the company just because that's the kind of person I am. But, you know, and and, like, the big picture of things, you know, he he couldn't have done a domain specific commit in that first week.
Warren Parad [00:16:20]:
Yeah. Are you using something, like, some sort of ephemeral development environment? Or, like, how was he able to get acclimated so quickly?
David Kimura [00:16:29]:
Yeah. It was over it was on our Kubernetes clusters. And so he had a deep Kubernetes experience and, just looked at the way that ours were configured and was like, oh, yeah. Here's here's something that's gonna gonna help you out. And then he also had a lot of OIDC experience, so he removed we were maintaining a lot of permissions and roles via pull request in a Git repository. And so he refactored that to use OIDC so that you just get it inherited when you authenticate.
Warren Parad [00:17:06]:
Yeah. I mean, I think the
Daniel Loreto [00:17:08]:
okay, you guys presented two different, like, onboarding experiences, but I think regardless, for me, you wanna spend the time creating the domain specific knowledge. You wanna spend the time kinda teaching and mentoring and and and having your your employees grow. I think you wanna take away the I'm just fighting with the tools, and this should be a solved problem. For sure. You know, I still have to deal with this now for, I don't know, two days, just to get my environment set up before I can even start, you know, doing a little coaching and and testing it.
David Kimura [00:17:46]:
Yeah. I totally agree. Like, our none of our customers benefit from the time and effort we spend into working on Kubernetes. They only benefit from the time and effort we spend building a better product that that they in that they use. And I think that's what you're alluding to. And and, Warren, to to catch on a point you made earlier there, you know, dealing with ephemeral environments, we recently refactored a bunch of ours because we had this one VM instance with I don't know. It had hundreds of gigs of RAM on it, and it was a bunch of Docker Compose files. And when you needed a new integration environment, you would copy and paste one of the existing Docker Compose directories and then go modify the stuff that you needed in there.
David Kimura [00:18:37]:
And so we refactored that to use Helm charts. So now the launching of a new environment, you just add a new helm value helm values file for the environment that you want and then let, Helm build that out for you.
Warren Parad [00:18:54]:
Yeah. So I I will say that, NIX is something that's always scared me personally. Like, I I feel like I never but wasn't a opportunity where I was like, you know what? I'm gonna reach for that new tool. So, like, you know, congratulations for, you know, seeing that as an opportunity and going at it and then building the UI on top of it to make it happen. What do you feel like the turnaround time is for getting new engineers acclimated to using the dev boxes through through NICS? So they come into a team. They have no NICS experience, no experience with maybe ephemeral dev environments. You know, they're all doing software development on their local machine. What's the turnaround time for them understanding and being able to execute effectively?
Daniel Loreto [00:19:36]:
Yeah. I mean, assuming we're talking about an environment where the tools needed are kind of available in the next package store and and kind of the versions are well represented. I think it's very, very fast because, I mean, DevBox, the tool we built, it presents an interface that's very similar to package managers from different languages that you're already familiar with. Right? So, you know, in you know, we're talking about JavaScript earlier. You have that package, the JSON file. Well, in DevOps, you have the DevOps, the JSON file, and it lists a bunch of packages that you need for your project. And you might just say, you know, Python and the version, and it might say the product of compiler and the version. And it has a command to add a new package, to search for a new package, to remove a new package.
Daniel Loreto [00:20:28]:
And so what we try to do is kind of present that next functionality, but under a UX that people are are already familiar with. And because people are are already familiar with it from other tools, I think it's a a quick learning curve.
Warren Parad [00:20:46]:
How does this affect the sort of software development cycle? So I make a change in in my code. We're talking about, impacting the Docker build cache and forcing it to rebuild. Like, what's the corollary here for, Dev Box? Do I do the software development locally? And then it uses some sort of, remote server somewhere where the Dev Box is being spin up, like, you know, using
Daniel Loreto [00:21:12]:
the The Dev Box is our open source project. It lets you declare your environment in this JSON file. And then the idea is that you can take that JSON file and reproduce that environment in many places. We we offer kind of on on our paid solutions ways to spin those environments on the cloud. But just to clarify, DevOps is just gonna be the open source, tool, and let you, kinda do it locally. And the way it works is it is like I said, I it uses next behind the scenes. In practice, there's no container, locally. I mean, you can you can choose to use DevBox inside a container if you're building a container, but you can also choose to do it outside of a container if you don't need one.
Daniel Loreto [00:21:57]:
And what's really happening is that Nix kinda installs all the versions of all the software that you need in what's known as the Nix store. Think of this as some path in your local drive where every binary or piece of software that's installed is stored under a content hash so that the path for it is unique. So even if you start installed, I don't know, five versions of Python, they will each have its own content hash and therefore will live in a different place in that store. And so, anyways, it like that, it's able to install all sorts of versions of software even if they would be technically conflicting. And then when you start an environment, it kinda sets the paths, to point to the appropriate versions of things, and and now you're working locally. Right?
Warren Parad [00:22:49]:
So is it changing the environment variables and the alternatives that are set on my operating system to
David Kimura [00:22:53]:
make the environment match
Warren Parad [00:22:54]:
what's necessary for that Yes. Necessary for
Daniel Loreto [00:22:58]:
that Yes. Yeah. So it will change for example, change your path, environment variable. And so, you know, you might you might say, okay. In this project, we need in this version of Python, this version of the protobuf compiler, or I know those live over here in the next store. Let me set up the path such that you're gonna have access to those binaries and not the other ones.
Warren Parad [00:23:21]:
Okay. Sure. That makes sense.
David Kimura [00:23:26]:
And is that done, like, system wide, or is that done on, like, a project per project basis?
Daniel Loreto [00:23:32]:
Basis. So so the analogy I make if we wanna, you know, reference existing, tools for particular languages, it's kinda like on Python, you have virtual AMP type of environment. Imagine that, but now kinda like at the system level for, the the tools and compilers that you're using for a project.
Warren Parad [00:23:53]:
So done correctly is what I'm hearing.
Daniel Loreto [00:23:58]:
We think so, but form form your own opinion.
Warren Parad [00:24:02]:
Well, I mean, I I think virtual env is, like, a quintessential example of it doesn't get much worse than that. You know, maybe maybe someone can point me in the right direction there, but I don't remember. Like, usually things automatically get committed to memory for me if they're easy to understand. You know, know, like, I think everyone has this. You know, you see something. It's like, oh, I can remember what that is. There's a couple
Daniel Loreto [00:24:22]:
of other things.
Warren Parad [00:24:23]:
Three months later, you come back and you're like, I gotta relearn this thing over again.
Warren Parad [00:24:27]:
Right. That's the thing. It's like, I don't touch it enough. So the only time I actually touch virtual env at all is when I'm generating new, cert box certificates from Let's Encrypt on, one of my remote servers. And that happens once every 90 days. And, every 90 days, I cannot remember how to do that. So, you know, there's Celsius, something is wrong with that technology for sure. So, you know, it's great when there's an easy way that makes sense to set it up and exit the the the machine.
Warren Parad [00:24:54]:
And that, of course, it has to be process specific and local so that you can potentially run multiple at the same time, so that you can run multiple pieces of, like, different services that have to communicate with each other.
Daniel Loreto [00:25:07]:
Right. Right. Makes sense.
Warren Parad [00:25:10]:
How do I how do I know that, early on in my career, it was always like, I don't need that garbage I'm just gonna do it myself and later in my career I definitely gotten to the point of I should definitely listen to those things and not think that I could do it myself so what like what's the advice here you know how do you know you're in a situation where whatever you're doing needs to be upgraded to using next through DevBox? Like, quintessential.
Warren Parad [00:25:35]:
Yeah. I mean, I think
Daniel Loreto [00:25:37]:
let me maybe start with the cases where I don't think you necessarily need it. I think those tend to be you you're working on a team with this pretty standardized stack. Probably have one particular programming language that you're focused on. And for the most part, all you need are the tools for that library, and you're not bringing into many external tools. Right? I think as soon as you start having more kind of system level dependencies, additional tools that you're bringing in, maybe multiple languages, you know, across the company, then tool you know, things to control the environment start making more sense. You know, just to grab an example, let's say you're coding in Python. If you've standardized on a version of Python, and you're building kind of pure Python libraries, I don't know. You probably don't need the tool like this.
Daniel Loreto [00:26:35]:
But, yeah, AI is the rage old these these days, and you decide to do some ML stuff. Right? And and you need some Python libraries that really depend on native libraries. And so now all those native libraries are gonna get pulled. And you're doing some image processing, and so you need, like, ImageMagick installed in the container. And and so you and it depends on a particular version of ImageMagick. And so now your environment shifting from kind of pure Python to kinda like these extra dependencies. And I think that's where this makes sense because that you can now treat those extra dependencies as something that you track and version and and pin, as opposed to, I don't know, just wild west and, you know, pull whatever is available today. And, you know, it might be different yesterday from tomorrow.
Daniel Loreto [00:27:29]:
I give you you know, some examples I gave earlier were only on the development phase, but, like, imagine you you're doing image processing in production, and the container requires a particular version of ImageMagick installed. Like, now we're talking about production environments too that if you're not careful, are you just gonna changing nilly willing?
Warren Parad [00:27:49]:
How does the conversion from using Nix through DevBox to get your local development environment? I mean, I feel like you opened the can of worms here. So, when you do that locally, how does that convert to using getting those same dependencies ready for deployment to production?
Daniel Loreto [00:28:05]:
Yes. To be fair, I I do think that's an area we're continuing to improve kinda early days on the shift from using the box from development to production. But high level, I mean, you can craft a Docker file in a container in many different ways and and really just kinda running any command you want, right, and and setting up the environment inside the container. So what we think is that, in general, you can have one definition of the next packages that you need that and that one definition can be used to set up your developer environment, but it can also be used to install those same packages in the production container. Of course, you'd have to differentiate between things that are needed in both production and development versus things that are needed just for development. Now I I said it's a work in progress. I I think something that I feel next packages haven't been optimized too much for in in the past, and and I think it is one of the challenges with what I'm saying, is that I I don't think next packages are optimized for size. And so sometimes they pull in a lot of things, into the container, and the container grows kinda kinda bigger than you want it to be.
Daniel Loreto [00:29:28]:
And so I think that's something that still needs to be improved, within, you know, in in terms of using next for creating these types of production containers. Yeah. But, I
Warren Parad [00:29:35]:
mean, I think it's a huge step in the right direction because the dependency management through Docker has always been, oh, yeah. Just install like, use sudo and then install a huge list of packages that are required to run here. And, like, what version those are, whether or not they work with the actual, Linux OS that you're actually running there compared to the one on your development machine or in your or, you know, in your dev box or whatever could be totally different or what's available in production. Because you're not actually running the exact container. There are some differences there. And then there's also your CICD pipeline, which is probably a different version of Linux on it. Right.
Daniel Loreto [00:30:09]:
Right. Right. Yes.
Warren Parad [00:30:09]:
And
Warren Parad [00:30:09]:
this is Exactly. This is just if you're using Linux. Right? I mean, then you throw in ARM and other operating systems, and it's it's for sure a huge mess. So, I mean, I definitely see that persistence is another aspect. Like, Nix doesn't you know, you're focused on using it for dev boxes in sort of a femoral way. Right? Like, you're not making permanent changes to the operating system. But in production, you almost want the permanence there. So there is this sort of disconnect and you're achieving the connection there through using Docker or plugging into persisting it in the Docker container, build file, I assume.
Daniel Loreto [00:30:41]:
Yeah. Exactly. Exactly. Right. Yeah. And, I mean, I think you touched on this, but just to emphasize it, because I think sometimes people don't think about it. A a Docker file is not necessarily reproducible. In fact, I'll argue it's probably not reproducible.
Daniel Loreto [00:30:57]:
People sometimes think of Docker as reproducible in the sense that if you previously built a particular image and you reuse that image, then you know you're getting well, you're getting what you already packaged before. Right? But if you have a Docker file with instructions and you build it six months ago and you build it today, the the output, the resulting image will likely be different because, again, I mean, it's the same scenario I was talking about with with your engineers just installing the environment. Probably your docile file just says install Python. And six months ago, the version of Python, you know, was x. Today, the version of Python is y. And so you build a container six months ago versus today, you get different, images as a result.
Warren Parad [00:31:44]:
I think code dependencies definitely fall code code dependencies fall on this, abyss here. Right? They're they're not quite infrastructure as code, which has been solved as far as deploying what we have to our cloud environment. And they're not the source code, which is stored so elegantly in in Git. There is this middle area where I need to configure the that container or, heaven forbid you're using a virtual machine directly or bare metal in a very specific way. And it's like, well, it's not IAC and it's not my source code. And then what I meant end end up with is some sort of script in bash a lot of times to explain how to actually install that thing. And so it's like, well, why aren't we doing declarative programming here like we are everywhere else? It's like the one area that still is procedural and still prone to a lot of issues. And, like, that's what I'm hearing is like, okay.
Warren Parad [00:32:30]:
This is totally solved now. You know, if you need more than one dependency, how can you not be using this solution?
Warren Parad [00:32:36]:
Right. Yeah. Exactly. I I think you said it, in
Daniel Loreto [00:32:39]:
a way that I agree. Like, it'd be great to just have your developer environment defined in a declarative way. And then that declaration, you know, we can use to build the environment correctly, and you can use it to build the environment in many different places. You can build it locally. You can build it inside a container. You can build it on the cloud. Right? And, you know, take the environment with you where you need it.
Warren Parad [00:33:06]:
Will's processing all of this, I can see.
David Kimura [00:33:10]:
I am, but, I think I failed to press the turbo button on the CPU this morning because it's processing slow. It's running slow this morning.
Warren Parad [00:33:24]:
That's okay. It happens to all of us.
David Kimura [00:33:26]:
Right. So with the, when it comes back to the production dependencies, is that declarative in the manifest file? This dependency is a production one or this one's a development dependency?
Daniel Loreto [00:33:45]:
Right now, in if you want to talk about, like, the implementation we have in the in the open source tool, you would have, like, two different dev ops, the JSON files. One for production, one for development. Okay. But we have talked about, you know, potentially unifying that in one file. Actually, we were making the the analogy to, package dot JSON before, and in a package dot JSON, you have the dependencies and Right. Dependencies that are for both. So, yeah, we're we're considering kind of unifying that. But, yeah, I mean, put aside the exact implementation we have today, the exact version.
Daniel Loreto [00:34:20]:
Like, I think the idea is can we declare our environments by specifying the packages we want and the versions that we want and then specifying which packages belong in both development and production and which ones belong only in development? And then take away that very procedural, like, bash scripts that you're doing to install every little thing, and then let a system that can reproducibly create the environment from that declaration, you know, do the work.
Warren Parad [00:34:55]:
You I think you hit on really well the scenario where you may not need to do something like this. Like, you don't need the I I hesitate to call it complexity of using, next based configuration. But it's just it's one more tool. And so No.
Daniel Loreto [00:35:07]:
I didn't.
Warren Parad [00:35:08]:
I I I would call it, like, I
Daniel Loreto [00:35:09]:
mean, it is an extra tool. Yeah. I agree with you. Like, if yeah.
Warren Parad [00:35:13]:
But, you know, compared I mean, swap that in for something else. I mean, it's pretty much a very simple bootstrap and and clear configuration files that are very understandable of of what they're doing. I'm I'm curious if you see a particular vertical or industry market segment where it becomes super critical still. Like, maybe if we say web servers, web development, maybe that's really simple. It's not too complicated here. But if if you're building a game I I'm just making stuff up now. Like, you know, this is where you can jump in and say, you know, this is exactly it. Like, game or web three, like, maybe there's a lot more value in or doing something with ML, a lot more value in next configurations.
Daniel Loreto [00:35:50]:
Yeah. I think I mean, I I think one of them is when you have a lot of when you need to use native dependencies that are not, native to the language that you're using. Right? So I think Python and ML is a good example, but, you know, that it's not just ML. Like, you know, there's several cases where you might need to install Python libraries that depend on kinda underlying c libraries. I I think those tend to not work as well if you don't have a a tool, like like that box. I think the other case is when you rely I mean, made similar, but maybe it's not a library in this scenario. But, yeah, you're you're relying on a binary that needs to be present in the environment, because you wanna call you know, exec that binary or or do something with it.
Warren Parad [00:36:41]:
How much of a mess is ML development from a dependency management today?
David Kimura [00:36:54]:
Say no more.
Daniel Loreto [00:36:59]:
I think I think it can get complicated.
Warren Parad [00:37:04]:
Yeah. Because and then yeah.
Daniel Loreto [00:37:05]:
By the way, you know, I'm I'm I'm talking about DevOps and and ML, but there's even cases where we are not able to help fully in that vertical, because then kinda drivers come in, you know, for the different GPUs and whatnot. And and there's some things that Nix can't fully control there unless you were using NixOS. Right? But we're, remember, we're we're doing this in in a scenario where you you keep your current computer, current operating system, and and just use that as this added virtual environment. But, yeah, it can it can get real messy, not just with the library dependencies, but kinda like all the related drivers and how they match the corresponding architecture that you're developing again.
Warren Parad [00:37:54]:
So you wouldn't recommend doing ML development today quite yet? Maybe we wait a couple more years, though.
Daniel Loreto [00:38:02]:
Yeah. Yeah. I mean, I I think it's an improve maybe I'm saying I think it's an improvement, but it's not all the way then. Well,
Warren Parad [00:38:11]:
I do remember, and I'll date myself here, when I was trying to, extract Bitcoins from the Bitcoin faucet, when I was in the university, and that included having to, basically getting my operating system to a state where I would just ended up with a black screen with a terminal, you know, and it's like, okay. You know, I really messed something up here. And, like, that was the state of the software at the time to just do anything related in, in the cryptocurrency space, in cryptography and in general, really. Good luck using those tools. So, like, that's definitely improved. Now you can use Python or JavaScript to interact with any of the, chains that are available out there, Ethereum and all the derivatives of it without a problem for the most part. I mean, there are some edge cases for sure if you wanna stake or mine that you're gonna have a problem. And I think that's where ML is.
Warren Parad [00:39:03]:
Right? It it used to be much worse, and I think it's slightly better, but I do feel like maybe we're going so quickly and we've monetized whatever little, amount of value we've been able to create as a society through, model development that we haven't really took the opportunity to innovate with the development pipeline.
Daniel Loreto [00:39:26]:
Yeah. And I you know, I'm sure there's I I I do think there's probably plenty of people working in in solving this given, you know, the the the AI wave and the whole interest in in MLM.
David Kimura [00:39:37]:
Optimistic of you.
Warren Parad [00:39:40]:
I mean, it's I think it's I think it's optimistic, but I also I also think that maybe a little bit realistic because with the, deep state model that had come out I mean, DeepSeek. Right? That it, I thought we were in a different podcast for today. Yeah. I mean, it's a joke that, you know, I frequently go around with some of my colleagues, about, you know, the impact of of these models. But the fact that you are there's now competition requires doing things more efficiently. You can't bring a new engineer on board and have them waste months and months not on on tool set, like, not even on the domain. So that's what drives innovation there a lot when you're in competition, to succeed. And I think, that's gonna be a really important qualifier going forward in the ML space that we continue to have sufficient competition to to drive innovation there.
David Kimura [00:40:35]:
So this is an area where I haven't spent any time or effort. But, Daniel, you have right because you have been working on some ML work to do QA end to end testing. Right? So what's the what's the barrier to entry to get into that space?
Daniel Loreto [00:40:56]:
Like, meaning, like, what's the barrier for engineers to start creating agents themselves and kind of what knowledge and tools might they need?
David Kimura [00:41:05]:
Yeah. For sure.
Daniel Loreto [00:41:07]:
Yeah. So, yeah, I mean, like like you said, we are developing, an agent that we call called test pilot. It's meant to be, an AI QA engineer to away all that boring, repetitive manual clicking on UIs that people often do. Yeah. And we we've been learning a lot. I mean, I think I think agents, are relatively like, big investments in agents are relatively new. And so I think the barrier of entry is something you can catch up on. Like like, I would actually incur if if you're interested in creating agents, I I think it's a good time.
Daniel Loreto [00:41:44]:
I think you can learn. I I think you can do it. I would say I mean, to me, one of the most important things you need to do with developing AI applications is set up a evaluation framework, for the work that you're doing. So in in in non AI, non you know, kind of traditional software development, let's just say, you can often write a function that you can reason about fairly well, and then you can write tests that are fairly deterministic to know whether you your the behavior is what you expect. Right? As as soon as you involve the LLM, well, now you have nonprobabilist like, no sorry. Non deterministic behavior. Right? And so you might call it once, and it might tell you something. You might call it again in a second, and it might tell you completely different thing.
Daniel Loreto [00:42:41]:
And often, because you're dealing with very general cases, you might be feeding it lots of different examples, and you don't fully grasp how it's behaving across all those different examples. So so to me, kinda like one of the first things you need to do is set up a bunch of examples with kinda like the expected behavior, and then be able to measure kind of the changes that you're doing and whether they're improving that evaluation that you have in mind. Alright? And then in this case, it's gonna be not purely pass fail. It's gonna be, like, okay. Today, you're at 50% of being able to handle the cases that you want. And then you do a few changes. And, okay, now you're at 55%, and then then you're at 60%, and so on. So, yeah, anyways, I don't know.
Daniel Loreto [00:43:30]:
I took took I can probably think of more, kind of best practices, but I that's the first one that comes to mind.
Warren Parad [00:43:38]:
I would even maybe take a step back before that. I mean, that that's really great insight. Like, someone who hasn't spent a lot of time in it, is there a particular site or foundational model that is where you would suggest people start going for first? I mean, probably probably I I think from my own experience, I know that building models is super expensive. So they're probably not gonna do that. But maybe there's a service out there that you say, you know, go pull this model and try to fine tune it or use it out of the box. Like, any recommendations on that?
Warren Parad [00:44:07]:
Yeah. I mean, I I I
Daniel Loreto [00:44:08]:
do think you need to start with the with the preexisting general model. I I recommend using probably a kind of multimodal provider, something like open router, that allows you to easily swap models. Because what what's happening is I mean, you guys see on the on the tech news all the time. Right? Every every other month, each AI vendor, like, outdoes each other. And it's like, oh, we have this new new model, and it performs better on this benchmark. And so you don't wanna tie yourself so strongly to one model that when those innovations are happening, you can't test the other one. Right? So I would pick some provider, like like I said, OpenRouter is one of them. I think Grok, I think CloudFlare has something these days too.
Daniel Loreto [00:44:56]:
But, anyways, a provider that lets you specify which model you wanna use, and then kind of build the agent on top of that. I think the other part of building an agent is not just the underlying model that helps you reason, but there's kinda how you prompt it, right, and kinda how you tell it to do what you want it to do. And then the last piece is kinda like the tools that you give it so that it can take action. Right? So high level in my mind, like an agent is you're grabbing the LLM for the reasoning, but you need to kind of give it hands and eyes, if you wanna think about it that way, so that it can manipulate an environment that you're giving it control of.
Warren Parad [00:45:43]:
Right on.
David Kimura [00:45:45]:
Is that something that you've got a significant portion of your team dedicated to, or is it something that you can iterate on quickly with just a few people? Like, how big of a, how big of a manpower
Daniel Loreto [00:46:02]:
Yeah. I mean, I I know most of our team is working on on test pilots. I mean, something we found is so we're we're a a Go shop. Like, we use Go as our primary, programming language. We we've been using it for for a while, and so we wanted to continue using it. And what we're seeing is that a lot of the tools that are readily available in this space are Python based. And, you know, we we can use Python for stuff that's not production, but for our production stuff, we wanna continue using Go. And so we're having to build some of them ourselves in Go.
Daniel Loreto [00:46:38]:
We do plan to open source them later on and kinda give back to the community and hopefully, kinda people that like Go, you know, let let them have some readily available tools for building AI and agents. But, yeah, the the that stuff that's getting built right now, and and often we look for it, and it's like, oh, I guess nobody has, you know, done this type of library yet in in Go for AI. So it it is early days.
Warren Parad [00:47:04]:
Yeah. I mean, I think that's a really good point, actually. Like, it's usually a mistake to rely on Python for high reliability software development in production. And so if the tools aren't there to support your production workloads in a in a different language, then that really tells you how early it is because the people that are utilizing those tools don't care about, like, sustainable code or reliable code because it doesn't offer the core features to guarantee, safety or quality of the of of your value of your service that you're building.
Daniel Loreto [00:47:38]:
Yeah. Like, I I mean, there there's many different ways of building a company, but just I I've had to deal with, like, Ruby production systems, Python production systems, all that stuff. And I'm I'm kinda like in this case, we're, you know, we're picking different language and, kinda moving forward with that.
Warren Parad [00:47:56]:
Yeah. No. I'm totally with you. I used to be, like, early on in my career, I'm just like every language is the same. Right? You know, there's you can swap between one or another if you've had a sufficient years of experience. But then over time, you start to realize the fringe benefits of individual languages or negatives that are associated with them. So things like Go or other well typed languages like c sharp. The the community or the packages that are available, to help drive forward having the the right execution of the right stuff in production is, like, super important.
Warren Parad [00:48:26]:
Not which is when you're building something quick and hacky, but when you actually care about it working continuously. And, like, if you're at a company and you're one of the core things you're building is, reliable builds, I I feel like, you know, that goes hand in hand Right. Right. Using other I mean, it's sort of like a different mindset. Right? Like, if you're hiring an engineer and they care about using, scrappy tools, then ones that potentially break from day to day, the nightly build version of that tool is broken by default. They're probably not in the right mindset to be working on something where you need high reliability or reproducible builds, or, you know, these other technologies. It's just a complete other side of the spectrum.
Daniel Loreto [00:49:06]:
Right. Yeah. Totally agree. Yeah. I mean, what what we've been doing, maybe where we kinda allow Python to come in is, you know, I mentioned we have this kinda evil process. We're able to try ideas. And so I'm like, okay. We're prototyping a direction, and there happens to be kinda like a Python library that does that off the bat, and we just wanna check if that's gonna improve our performance, you know, the performance of our agent.
Daniel Loreto [00:49:31]:
Like, you know, prototype something quickly with Python, pull that library in, run it against our eval framework. Let's see if this direction has length. Oh, it does have lengths. It is promising. Okay. How are we gonna build it now for production usage and, you know, handle the load we wanna handle and take care of all the edge cases we wanna take care of and and so on? And then we and at that point in time, then we spend the time developing the the Go, software and infrastructure.
Warren Parad [00:49:58]:
Makes a lot of sense.
David Kimura [00:50:00]:
So that's, one of the things I wanna ask you, you know, you talked about, open sourcing some of the tools that you're making and contributing back to the community. That's a huge commitment, and it's a it's a conscious decision by companies because a lot of companies are like, yeah. We invested this. It's a core part of our business. We're using it for the business. But then you're saying, I wanna I wanna give it back to the the community.
Daniel Loreto [00:50:31]:
How do you
David Kimura [00:50:34]:
how do you tie that back into your business model and and, like, justify or account for the time that and the salaries that you're spending on people working on an open source product?
Daniel Loreto [00:50:49]:
Yeah. It's a great question. And, you know, I'm sure everybody will kinda make different judgments there. But for us, the way the way we think about it is, okay, we're developing Test Pilot, this kind of QA AI QA engineer. You can think of the software that makes up Test Pilot as being in two different buckets, at least in my hand. Right? One is, like, the truly domain specific logic and and, kind of proprietary aspects of it that make it behave like a QA engineer, right, and that we're not open sourcing. But then there's the tools that we wish we had had that that we had to develop for for ourselves that are applicable to developing all sorts of AI agents and all sorts of, kinda, AI related products. And, you know, we're we're a start up.
Daniel Loreto [00:51:40]:
We are trying to get more eyeballs. We're trying to kinda become more prominent and get people to hear about us. And so I actually think of open sourcing, almost as a marketing exercise. Right. We don't have marketers. We're all you know, we're very technical teams, like, basically engineers, but sometimes I think you can look as open source as as as a way to kind of bring in that audience. And because our audience are developers. Right? I think we're in a different vertical, you know, serving different customers.
Daniel Loreto [00:52:15]:
Maybe open sourcing doesn't work that way, but our audience is developers. And if we can create a reputation of, wow, those guys at Genify really know what they're doing in terms of AI agents, and have you checked you know, have you seen the tools they've made and the libraries they've made? I think that all just kinda comes back to to the type of business that we're building.
Warren Parad [00:52:38]:
Right. I I I totally agree. I mean, we went down the same path, a while ago with some of the things that we've we've open sourced. It is, like, unfortunate to say, it is like a huge marketing story. It's a marketing story for, brand awareness. It's a marketing story for hiring engineers. So they know about your company or what you do before that, you know, they show up in the interview. They can research it and see that you have something technical rather than a website with a couple of links on it and a knowledge base.
Warren Parad [00:53:06]:
Like, that's not a lot. Right? You know, this gives them more ability to have insight in what you're doing. I mean, there is there is a pessimistic side of me that says, like, no one cares what you put on the Internet. Like, you're no one's that special. Right? Your thing doesn't matter. I remember, like, a bunch of times, like, Netflix went and open sourced a bunch of huge technologies. And I'm just like, I don't care about those things. But now some people maybe are using that in production, I guess, could be, potentially, because I think they wrote them for Kubernetes or something.
Warren Parad [00:53:35]:
And, you know, so that's great. You know, you you get that experience. I I think it the one thing we try to be really careful about, that I think is important here is that the thing that you're building needs to be really close to your your value, your actual product. Because even if it's, like, one or two hops away, then it's the people that are utilizing it. They don't care what that thing is that that you're actually doing, which is super unfortunate. But I will say that rather than a marketing tool, there's a second, step here, which is if you're gonna open source something, I feel like you feel like it's more important to do it well. And things that are done well will live longer, and people will see and start to utilize more. And that give brings, sort of like a your own usage levels it up because you actually invested in doing that thing well.
Warren Parad [00:54:22]:
And so, like, even if you weren't Dev Box wasn't sort of out there as a a cloud thing that you could utilize and pay for, the fact that you've open sourced it means that that tool is now serving you hopefully better, for your ML development for your own agent. Right?
Daniel Loreto [00:54:38]:
Yeah. Absolutely. And, actually, I it's it's important for, like, different benefits that open source might bring, I'll I'll give you a third one, at least in my view, which is, you know, when you're a start up, I think one of the most important things is be talking to users. Right? Be talking to users and learning from them. And we found that when we open source things, we often get to talk to more users, but we also take get to talk to users that normally would not have talked to us. So, yeah, take that box.
Warren Parad [00:55:11]:
I I
Daniel Loreto [00:55:11]:
don't know if I can share the specific companies, but there's some biggish companies using that box that I think if we had just kinda pure proprietary solution or this little start up, they never would have looked at it. I think what gave them confidence to look at it is like, oh, there is this open source project. Kind of, you know, worst case scenario, the start up goes nowhere. We can rely on that open source project, and we can kinda continue improving it and whatnot. And so that's given us at least a channel to communicate with, those types of potential customers and users, and and it brings in a ton of learning for us in terms of, okay, what features are important? What what is that class of user or customer asking us about, and how should that affect our product road map and and whatnot?
David Kimura [00:55:59]:
Oh, that's huge. Like, in my opinion, like, one of the like, the primary objective of every startup is to figure out the difference between what you built and what you cussed what your customers thought you were building. And so open sourcing gives you a way to have that conversation without trying to force them down through a sales pipeline.
Daniel Loreto [00:56:21]:
Right. Exactly right. Yeah.
Warren Parad [00:56:22]:
It's it's scary though because, you know, when you put it out there, it's like, well, people start will start to see my actual bad code.
David Kimura [00:56:30]:
Right. Or,
Daniel Loreto [00:56:31]:
like It is scary. Yeah.
Warren Parad [00:56:32]:
My commit messages, you know, that maybe I should have worded differently.
Daniel Loreto [00:56:36]:
And, I mean, it is tricky too in, like, you know, I gave you our kinda, like, our heuristic for figuring out what parts of test pilot are open source and which ones aren't. But sometimes you're in some gray lines, and you're kinda like, should I open source this or not? Like, is it do I get more benefit from opening it up to the community and getting that audience and maybe getting some faster adoption? Or do I get more benefit from keeping this as kinda like the secret sauce of the company? And, you know, there's all those stories like Elasticsearch and AWS, and everybody gets big and starts changing their licenses. So, like Right. So
Warren Parad [00:57:19]:
I don't know how I don't know how right I am here. We've done some initial research on on the topic, and it does actually seem like customers that fall into a particular bucket never leave that bucket and go to a different one. Like, a free user never converts to your lowest tier paid plan. And the average business plan doesn't escalate to enterprise. It just doesn't happen. People self identify with whatever bucket they're in. And so whether or not you have an open source solution really only functions as advertisement. The people who are using it aren't gonna be more likely to get to like, start to pay you.
Warren Parad [00:57:53]:
And one of the things we recognize, which is one of the reasons why, our product, Authorus, is not open source for 90% of it, is because those free users actually generate 99% of the support cases that you need to deal with. So there's, like, a huge cost investment that goes into it, and you don't get any reward coming back from that. I mean, of course, there's some gray areas where there's highly coupled technology and where it's being utilized effectively. But often, the one the customers that are gonna pay you the most money or be your best customers, like, they don't particularly care about running it themselves in most cases. Unless they're like a government entity, or
Daniel Loreto [00:58:32]:
Right. Right.
Warren Parad [00:58:33]:
You know, they have some regulations they have to deal with.
Daniel Loreto [00:58:36]:
Yeah. I mean, usually, if they really carry some compliance requirement, but that might just be pointing to, like, you just need to solve that compliance requirement as well. Right? Like
Warren Parad [00:58:45]:
Oh, yeah. I mean, compliance doesn't mean security. So, like, that's a completely different story.
Daniel Loreto [00:58:50]:
Yeah. Absolutely.
David Kimura [00:58:51]:
It's an additional tax. Right on. That was that was intense.
Daniel Loreto [00:59:06]:
No. I don't in a good way.
Warren Parad [00:59:07]:
Yeah. Yeah. Yeah. Definitely in a good way.
David Kimura [00:59:09]:
Definitely in a good way. I didn't mean otherwise.
Warren Parad [00:59:13]:
I mean, I I know we we jumped, through a lot of different topics, Daniel. I if you feel like there's something you know, one last thing you wanna share, regarding either Dev Box, or the QA copilot, test pilot, you know, have at it.
Daniel Loreto [00:59:29]:
Let let me think.
David Kimura [00:59:30]:
Famous last words. Go.
Daniel Loreto [00:59:33]:
Oh, man. A lot of pressure. I mean, I don't know. I I guess I'll just add, maybe this is less about the specific products and just kind of things I'm excited about, but I am bullish on on AI agents. I mean, obviously, I'm, you know, building one. But, I just think there's more and more parts of software development that could be taken care of through agents. And I think it can be done in a way where we take away that kind of repetitive work, and we still leave a lot of the fun and creativity to to the humans. I know there's more doomsday scenarios that people talk about, right, where it's like, oh, we're all out of jobs and whatnot, but I I'm actually hoping it's more, like, now we can all achieve more, because those less interesting parts of the cycle are taken care of automatically.
Daniel Loreto [01:00:25]:
And and now you can focus more on system level thinking, on product level thinking, and and so on. So, yeah, maybe I am an optimist like you guys were saying before. I mean,
Warren Parad [01:00:36]:
I'm just gonna say to that no comment.
David Kimura [01:00:40]:
I'm gonna agree with you, Daniel. Like, regardless regardless of what technology does, I know there's always gonna be problems, and I've I've made a career out of solving problems. So I'm I'm not too worried about it.
Warren Parad [01:00:54]:
Okay. I'm super worried about it. Should I share my opinion? Yeah. Absolutely.
Daniel Loreto [01:00:58]:
So No. You you can't be worried about it, but, like, let's make the world we want.
Warren Parad [01:01:02]:
Well, I I feel like it's not a technology that's right for being democratized for access for everyone to utilize, equivalently. The people with the most money and the most power will have the most powerful, agents at their disposal and that just entrenches their own power and prevents them from being ousted. And it doesn't leave a lot of opportunity for those that are already being disadvantaged to rise up and, fight against that oppression.
David Kimura [01:01:37]:
There's a a documentary on this, called Terminator. Check it out.
Warren Parad [01:01:42]:
I I think I've heard this.
Warren Parad [01:01:44]:
I think it has a well well known, narrator. Right?
David Kimura [01:01:48]:
Yeah. Yeah.
Warren Parad [01:01:52]:
No. I mean, that's that for me is, that's optimism, that that documentary.
Warren Parad [01:02:01]:
That that that's a good scenario.
David Kimura [01:02:06]:
Alright. Oh, man. Now that we've all made our way onto an FBI watch list, let's move on to pics.
Warren Parad [01:02:13]:
Yeah. Sure. So I'll go first. Today, I my pick is gonna be The Martian, the book by Andy Weir. He has a whole bunch of books. I don't know if I'll say this one is the best, but it's definitely really good. They're all science fiction adjacent. I mean, they're not they're reality, played out in a hypothetical future where, well, it's a new scenario.
Warren Parad [01:02:38]:
And they're they're great. I think it's really well written. One of the first things I ever read from Andy Weir was actually something called The Egg, which is a very short story, which I find also really good. And I and many, many years later, I read The Martian, and I didn't realize that it was the same author. And it's quite interesting to have read two pieces by the same author and be like, wait. There was this other really good thing that was written in a similar way. And, like, going back and and actually seeing it. It's like, I actually read that thing.
Warren Parad [01:03:06]:
So if you haven't read it, I haven't seen the movie yet, actually. But if you haven't read the book, I highly recommend it.
David Kimura [01:03:11]:
Right on. I have done both. I've read the book and seen the movie. But, I'm happy to hear you say that he, that his other books are better because that's
Warren Parad [01:03:24]:
Yeah. That's not true. I I really like the short story. He has, a drawn comic with, like, an overlap between Alice in Wonderland and, Dorothy from Wizard of Oz. And I forgot who the third character is. That's okay. The Martian's great. There's two other books.
Warren Parad [01:03:44]:
I think there's a fourth one coming out. They're not related at all. The one, Luna, I think, for the moon, it's not as good, but still interesting. And then there is one with, Deep Space, which is also really fantastic.
David Kimura [01:03:56]:
Right on. Cool. Daniel, what'd you bring for a pick?
Daniel Loreto [01:04:00]:
Awesome. So I have a book as well. It's it's Wild Robot. So I think I mentioned earlier, I have two young kids, an eight year old and a five year old. And so I I do a lot of bedtime with teen and and reading with them. And so Wild Robot is, one of the books that I'm reading with with one of my kids, by Peter Brown, there's a movie as well, kinda similar to the example you gave. But, yeah, I I I just find it really interesting to read with my kids. Yeah.
Daniel Loreto [01:04:29]:
Your high level, it's about a robot that is pounds itself in an island, doesn't know where it came from, and eventually needs to take care of, an animal. And it kind of makes you reflect on all these questions like, you know, where where where is parenthood? Is this robot taking care of this animal? You know, is that apparent? What what's the relationship between that animal and the robot? And, I don't know. I guess I get a little sentimental because I'm reading that with my kid, and I I'm just fine. I've just found it, like, really, interesting for both of us to go through the story.
David Kimura [01:05:05]:
That's cool. That's super cool. Alright. Well, we're gonna make it a three peat. I'm changing my pick, and I'm going with a book as well because, Warren, you said something about, you know, recognizing the writing style a while back. I read a book and, really enjoyed it. And I was like, wow. This guy writes so much like Stephen King.
David Kimura [01:05:30]:
It's just insane. And the author's name is Joe Hill. Turns out it's Stephen King's son. So Oh, wow.
Daniel Loreto [01:05:38]:
Oh, wow.
Warren Parad [01:05:38]:
Yeah. Yeah.
David Kimura [01:05:39]:
Yeah. The book I the book I read was Heart Shaped Box. And if if you like Stephen King's writing style, Joe Hill just seems to have picked up on that style and continued running along with it.
Warren Parad [01:05:52]:
Is it also, like, thriller suspense stuff, or is it a different genre?
David Kimura [01:05:56]:
Nope. Same same stuff. Same, like, sick and twisted thing where you're like, oh, wow. I I did not see that one coming. Yeah. Good stuff.
Warren Parad [01:06:07]:
Do you ever find it a challenge to, like, pick up a book that you know is gonna be like that and go through it and be like, I'm I don't know if I can keep going. Like, I'm really afraid of what I'm gonna read next.
David Kimura [01:06:19]:
No. I don't I don't think so. No.
Warren Parad [01:06:26]:
He's like, I love it. It's great. That's what I look forward to.
David Kimura [01:06:29]:
Right? Yeah. I mean, I I guess the rest of that conversation is for me and my therapist, but
Warren Parad [01:06:41]:
Oh, don't worry. We'll shut the camera off here in a moment and,
Daniel Loreto [01:06:43]:
we can
Warren Parad [01:06:44]:
do the conversation.
David Kimura [01:06:47]:
We're gonna be like, no. No. I promise I stopped recording. Tell us all about it.
Warren Parad [01:06:51]:
Well, you see this little red light in the top left hand corner.
David Kimura [01:06:54]:
It's just a UI glitch. Don't worry about it. Daniel, thank you so much for joining us. This has been a blast. That's fantastic.
Daniel Loreto [01:07:03]:
I've enjoyed it as well. Thank you guys for for having me.
David Kimura [01:07:06]:
Alright. And to all of our listeners, hopefully, you enjoyed it as well. Than

The Impact of Open Source on Business and Development Practices with Daniel Loreto - DevOps_233
0:00
Playback Speed: