Dave and Valentino join this week's panelist episode to talk about important points to remember in web hosting. Dave begins by discussing ways to prevent security breaches in your system or data and prevent any ransomware attacks. He shares his own experience on how he maintains his website and the resources he is currently using. 

Picks

Hey everyone, welcome back to another episode of the Ruby Rogues Podcast. I'm your host today, Valentino Stoll, and I'm joined today by Dave Kimura. Dave, you want to say hello?

 

Hey, what's going on, Valentino? It's been a while.

 

Yeah, it has been a while. How are things?

 

Things are good. Things are good. So kids are crazy. Work is good. Everything's normal.

 

Nice. Hey, normal's good.

 

Yep.

 

Well, we were chatting before the show kind of about hosting services and kind of something unusual that you've done. Do you want to talk a little bit more and set the stage here?

 

Yeah, sure. So the idea is the hybrid infrastructure. And essentially what that means is you have your application up in a cloud provider, but then you also have some hardware to support that application elsewhere. So that could be in a data center or within your office or something like that. And that can kind of give you the best of both worlds, because to mimic the infrastructure that AWS has is not only extremely expensive, but there are just so many hoops to kind of jump through. However, if you can leverage their services for the user facing part of your traffic and the very important bits of it, then that's going to generally be your best route without having a whole DevOps team infrastructure team to take care of things. where it just doesn't make sense anymore. And so you may find that you have some extremely heavy background jobs that are very heavy in calculations and you want to pull those out of the cloud to host your own hardware to be able to do those calculations and communicate back to your cloud setup. interface stuff.

 

Yeah, that makes a lot of sense to me. I'm curious to see what that looks like.

 

Yeah.

 

I mean, to me, I've never had a problem with the cloud, but I also haven't had to pay the huge bills that many big companies have to pay, especially for a lot of the compute stuff. I will say I have tried out running Minecraft servers, running a

 

Mm-hmm.

 

lot of those too many at once. So I'm curious, where do you even start something like that?

 

Yeah, so the first thing is that you always have to keep in mind is security. Because if you're talking about having your data up in the cloud, up in AWS and whatever, then that has its own, you know, levels of security that it goes through. But as soon as you start talking about bringing outside services into your infrastructure, and to really have all your eyes dotted and your teeth crossed because you don't want to be the victim of a security breach because of some choices that you've made to cut corners or to cut costs. So one thing that I do is I will have a allow list of all the IP addresses that are allowed to communicate and I also have a VPN tunnel that they will securely communicate through. So nothing is done over plain text. I have a cheap, tiny, what are they? The T2 Nano instances in AWS that I have a VPN tunnel installed on. So to give you the high level overview of what I'm doing here is on Drifting Ruby. download videos as a subscriber. And if you are subscribed, you can download a video. But it goes through a encoding queue for a few different reasons. One, it will help throttle the bad actors who have in the past because I did allow downloading videos before. And people would just script and download the entire archive crashing the servers. that, but then it also helps to be an accountability thing where as a subscriber, you do have a terms of use that you agree to. And part of that is to not share out the videos. You know, it is a subscription service. It is something that helps put food on our table. So I do take that kind of piracy or whatever you want to call it serious. You know, I'm never, you know, don't quote me on it, to court over releasing videos, unless they were just very, very nasty about like, you know, I can do whatever I want. Screw you and all that. However. It's just an accountability thing. So what I'm also doing is I'm transcoding the videos because they are right now all HLS videos, so it's broken down into ten second chunks at different bit rates to help service those in other countries. with worse broadband connections. So it can stream a 720p instead of a 1080p, depending on your connection. And so I take those chunks, I stitch them together and then allow the user to download it. But part of that transcoding process, I am also embedding in a unique, non-identifiable as far as like no one can figure out who you are from that information. But I am embedding just a token and then see if anyone leaks the video, tie it back to a particular subscription and then just kindly ask that person not to share the videos and stuff. But to do that kind of transcoding locally in the cloud, I price it out. It was going to be about fifteen hundred dollars a month, which

 

Ha!

 

is a huge price.

 

It's crazy.

 

And this isn't the primary service that I'm offering. This is just for those who like to go off grid for the weekend or something, but want to catch up on videos, I want to be able to service them. And so this is a way that they can then do that. But I can't afford to spend $1,500 a month on a service that's not the primary point of the application. So what I've done instead was tiny T2 Nano instance that acts as a VPN that I can then from a outside environment connect to and it connects to the database connects to the Redis instance and that's extremely slow but it does work because I'm doing all of this in the background so it's not user interfacing where they click a button now they have to wait you know five minutes because see connections and the transcoding instead. They just had to wait for the transcoding to happen. Then they get a notification that's ready to download. And I'm doing this with Nvidia RTX 3060 GPUs. I do have a few of them that I've put in a rack. And I'm able to do all that transcoding off of AWS's infrastructure in my own, which is pennies on the dollar. cheaper than what AWS has. I think I spent fifteen hundred on hardware and it's going to be hardware that I now own. So now I don't have to pay fifteen hundred a month for it. I think the total bill came out to be like twenty two hundred dollars after GPUs and processor and everything. But that's still pennies on the dollar compared to what AWS is having. And to the end user, it doesn't make a difference if it's at home. Center if it is in an office to have this hardware because they don't ever see that in result The end of it. They just see that okay, they're Processing a video. Okay, the video is ready each video took about 30 seconds to do the full transcoding which I think is pretty quick considering what it's actually doing

 

So I'm curious, are you keeping like kind of copies of these videos on your local system?

 

So everything is kind of being done through temp files. And then it re-uploads it to the main AWS site, where it then gets stored as an active storage attachment. So it is getting stored on S3, ultimately, in the end. And then just to make sure that I don't run out of disk space locally, all that stuff is cleaned up after the job is done.

 

Yeah, that's really interesting. I mean, I'd I'd hate to be your kids trying to use the internet when you're uploading. How does it so how does your like Internet set up? Where are there other days you could have outages if you have a power outage, as an example?

 

Yeah, so the basically couple of considerations that I did take is that, you know, security is important. So all the drives locally here are encrypted, but then also have a completely separate VLAN for any kind of quote corporate traffic. So that means like my business traffic. I keep it completely separate from the household stuff because I don't want my kids downloading something on their computer create some ransomware attack that then affects stuff that is actually bringing in money for the family. So, you know, you can't just think of security as the communication between the servers or the infrastructure environments. But you also have to think about it within because hardware access is, you know, a big security threat if someone were to get access to it. And you can do that pretty easily with a malware. just hopping across the network. But as far as bandwidth goes, I have a gigabit download, gigabit upload, speeds at home, which the kids never even tap into. And through quality of service settings, I do give priority to my, as I labeled it, the corporate network.

 

That's really interesting. I mean... I'm all for throwing compute ever since the scientific computing days where you could share your compute resources with the global network for good.

 

Mm-hmm.

 

Ever since that idea, it's made so much sense to try and use all these resources you have which are just sitting around. you know, I remember a few jobs ago, we had just like these big machines and we created thin clients out of them because they were just always connected at the office. And we were slowly weaning away from, you know, working in the office. And we'd be able to SSH into them anytime we wanted to do some work and have, you know, some kind of X screen or something like that, you know, so that we can

 

Mm-hmm.

 

see what was happening. And they were just sitting around most of the time and we ended up, you know mining for Bitcoin or something like that

 

Hahaha.

 

Just you know as a start of the whole like well, what do you do with this compute stuff, right? like

 

Yeah.

 

and then you start to see what else you could do with it, right because it is just sitting there and It makes a lot of sense to try and offload a lot of this expensive compute You know that you would try and offload on premise or off premise somewhere else, right? the same thing, except it seems you've taken on a lot more networking responsibilities.

 

Yeah, and I enjoy those responsibilities. You know, I think it's fun. I already have a bunch of servers that I use for other things, so it's not unfamiliar. It does take a certain level of knowledge to get all that interconnected, especially securely, but, you know, if that's not your cup of tea, then maybe doing a hybrid setup isn't for you. Having it just up in a cloud provider is the right route.

 

Yeah, I mean, you know, it's a kind of a double-edged sword here for you now, right? Because now you have all these extra things to

 

Yeah.

 

maintain. But I mean, you are comfortable with it. And I feel like as long as you keep those number of things small, it should be maintainable for you, right?

 

Yeah, and what is small because I do have a 42 you you rack that is like mostly filled I mean, but it is just one rack. So it is small in the amount of space it takes in the basement, you know

 

Yeah, I mean, that's a good question. I guess it depends what you're trying to do. Video processing is definitely gonna eat up some resources.

 

Yeah, absolutely.

 

Have you tried doing anything else with this kind of setup or has it been specifically just video?

 

Yeah, so at the beginning, and this is kind of like repurposed hardware. So at the beginning of COVID-19, back in 2020, I did buy a few GPUs that are dedicated to folding at home, which is not cryptocurrency and it's not mining, but it's protein folding. And essentially, you get tasks to do calculations on simulations on on protein folding that research and folding at home had released a COVID-19 protein folding that they were trying to get calculations on. So that's something that I heavily invested in back then to do these to try to help fight COVID-19 in the own way, in my own way that I could. So I repurposed some of that hardware for this particular project.

 

That's pretty cool. I know there were quite a few things like that with the shared computing.

 

Mm-hmm.

 

I think I feel like the DNA synthesis was one where you could share your resources to contribute in that way.

 

Yeah.

 

It's pretty interesting stuff. I mean, I still am curious on the kind of security implications of distributed computing Um You know, because I do trust

 

you

 

all of the nodes that are, you know, that you're kind of attached to in that way. How do you feel about it?

 

Well, I think certain projects like folding at home has at least gone the reputation where it's pretty good But again anything that you do that you're kind of weary about separate from your actual network, so you can get a Network switch that's pretty cheap that has like a five or eight port switch You know RC or our j45 connections and create VLANs. So you have your router coming in or your modem coming in to your router. Then you have that router going directly into this switch. That switch, you can then have different VLANs and you can have one VLAN for any kind of crypto or folding that you're doing. You have another VLAN for all of your wireless devices or your home network. have some employer stuff. So if I have a day job that an employer requires me to have their own hardware and stuff, then I can use a separate VLAN for that. So anything that may come through on their VPN connection or something, they don't see my devices at home and my devices at home don't see the work computer and You could either do it that way or have a physical separate network switches for each, which is just as cheap, but just as functional. So I think that's the first step of security is, you know, separate out the devices if you have a kind of a mixed bag wherever you're doing this.

 

You know, this makes me think what we're really missing is distributed sidekick.

 

Yeah,

 

Ha ha

 

and

 

ha.

 

that's essentially what I've done and how I made, you know, my hybrid setup possible is because I am using sidekick and all of the jobs that I want to happen on a outside network. It does have to connect to that redis instance and it does so through the VPN and from that that I have specified to connect to that Redis instance. And so whenever my servers pick up a job on that specific scikit-q it'll then process them here at my home versus if I have you know anyways but the nice thing about that is if my home network ever goes down or my IP address changes and so it's no longer in this allowed list, then it's going to stay in the queue. And I have monitoring set up to check my home servers that they are up and running. So I would be notified that something is down. I'm able to then resolve it and then it will pick up that queue. So there's no data loss or anything like that. And if it happens in the middle of processing a job, it would fail. Next time it connects, it just says like, hey, this job failed. It didn't go through. and then it would retry it.

 

Yeah, that's really cool. I'm trying to think of where the next steps for this start heading. What do you have planned for it?

 

So I don't want to use the term microservices because that's not what this is.

 

Ha ha!

 

I call this hybrid computing, where I'm mixing bare metal and cloud services. That's what I want to call this, not microservices, because it's not. But I mean, think of, you know, take another real life use case for this, where DHH posted a article on leave in the cloud. a lot of money for the cloud services, but could they have reduced the bill enough to a reasonable point where they would be happy with the cloud services? And if they pulled all of their background jobs happening locally, wherever they're going to be hosting these bare metal servers, and if they had done that, how much would they reduce their bill by and, you know, would it made sense?

 

Yeah, that's an interesting thought. I mean... Nobody likes paying the bills.

 

Yeah.

 

I keep coming back to whether or not the management is worth it.

 

Mm-hmm.

 

And you seem to have a pretty good grasp on how it all works. You know, it doesn't sound necessarily like you've hit any snags with it yet. Have you had to debug issues yet with this kind of setup, or has it been pretty smooth sailing?

 

It's been pretty smooth sailing mainly because I'm familiar with the hardware and that kind of stuff. I did end up having to order a different CPU cooler because the CPU cooler that I had physically would not fit into the rack server. I thought I knew it was going to be close, but it was too close and I didn't want to damage any of the heat pipes. So I did have to pick up a different CPU cooler that wasn't as efficient because it's technically smaller. But this isn't going to be a heavy CPU task that I'm doing. It's mainly GPU driven. So I was OK with that kind of compromise. And the other issue is density. As you were saying, like the physical size of things, I wanted to cram as many GPUs in there as possible. But physical cases have limitations. I also had PCI Express lane limitations. And so one of the limitations I ran into was the physical case size. And back when I was doing the folding at home, I had some PCI 1X riser cards that I was experimenting with because I'm not driving videos from these graphics cards. It's a headless machine. So the question is, could I repurpose when I was doing folding at home these PCI 1X riser for GPU transcoding. And essentially what I found is that these PCI-X 1X riser cards, they just plug into your PCI-1X. So you have PCI-1X, 4X, 8X, 16X, 16X being what you plug your graphics cards into. So you are basically going down to one lane, which on PCI-4 is still like, I don't know, gigabits per second. So pretty quick data transfer. But could I use FFmpeg and the transcoding when it's going through this PCIe 1x to this little adapter card? And the answer was no. I could not get that to work, which is a shame because I could have really increased the density of that server if that had worked. But unfortunately, I'm down to only using which a riser cable is basically a thick cable that just goes into the 16x slot and then goes out and gives you a 16x plug-in basically an extension cable for your PCIe and I had to do that because the physical case would not fit three Two bin wide cards, you know side by side by side

 

Oh, the physical limitations of your basement.

 

Ha ha ha ha!

 

I mean, that kind of leads me to, as you're starting to expand this kind of server farm that you're starting,

 

Mm-hmm.

 

do you see yourself shopping your setup out to people for a similar, cheaper AWS compute?

 

Absolutely not. And the the reason why I'm so strict on that is because I don't have the infrastructure to make this a client facing product. You know, I don't have two different Internet trunks coming in that can fail over. And to do that failover isn't cheap either. And I also don't have, you know, days worth of battery backup. If something happens, I don't have the cooling or fire needed if something happens, you know, I think those are all very important considerations to have if you are going to be doing something like this, especially if that's going to be your client facing thing. In that case, you really need to just put them in a data center. There's data centers all over the United States and world where you can take your physical rack servers and put them into their data center and you pay like one one you of Rack space or something like that

 

Yeah, I remember seeing a few services offered like that. I guess it makes sense. There is the whole on-premise offering too, where people will retrofit a cloud setup locally for you on-premise,

 

Yeah.

 

which is an interesting idea also. But it sounds like you're solving a very specific problem that a lot of companies have, I feel like. business that needs to get done and they end up because they've locked into whatever you know provider that they have they end up you know getting cutting off an arm and a leg just to use it for

 

Yeah,

 

convenience

 

which ADEP US does have transcoding services. And that's exactly what you're talking about. Where if I had gone the route of, you know, okay, let's first implement this feature in the cloud because that's where everything is hosted, then we can do that, but we need to use their tools. I don't want to pay 1500 a month for a always on GPU instance. So let's use their services. Turns out it does work, but now my bill is fluctuating quite a bit and it's a lot higher than I want it to be. Maybe my AWS monthly bill is eight hundred dollars now, in addition to the normal hosting. I'm like, man, you know, I really want to switch that out. And let's just, you know, rent some cheaper GPUVM instances on AWS. And that would cost us five hundred dollars a month. Like, you know, wow. it for almost half. Yeah, it's not as fast but it's not client-facing so that's okay. But then you realize like well I really kind of like the speed that we got from their service offerings with the transcoding service so why don't we go a hybrid setup. So now have you not only switched around the infrastructure you know two three different times but you've also had to rewrite a lot of transcoding service to a GPU VM or to a hybrid cloud setup, then you know that's going to require code changes which takes time and you can also introduce bugs and that kind of stuff.

 

Yeah, you make a good point about kind of vendor complexity right with your implementations. Like if you went with the transcoding option from AWS, you got to learn how all those APIs work and how their callback architecture works. You know how they're monitoring all those extra things where a lot of things where you run it locally, you just kind of take for granted. It's built into the system, right?

 

Yeah.

 

And I'm curious. Since you're doing a lot of encoding, maybe specifically, does FFmpeg have GPU specific optimizations you could take advantage of?

 

Yeah, and that was a bit of a learning practice that I had to go through because there's a few different hurdles that I found in that process. One of them was with the RTX 3060 graphics cards. I could have gone with a quadro, but again, I was repurposing some of these graphics cards from a previous project. card. But one of the limitations that NVIDIA has is you can only do three transcoding streams on the machine at a time. But there are ways around that that I guess are permissible with the consumer grade graphics cards. And it's a bash script that you run that will unlock the streaming from that driver so you can do more streams at a time. So I think I'm running six at a time, which pretty decent performance on the transcoding service. So that was one of the hiccups that I ran into, was just figuring that out, getting that working, and all that, but then compiling FFmpeg to use the NVIDIA NVENC encoding, as well as the CUDA cores, that was a whole different process. And luckily, that's something depending on if you find the right site. But I ended up after a bunch of testing found the right compiling configurations for my use case. And it seems to work pretty well.

 

It's funny you mentioned CUDA. That was gonna be my follow-up question. Now that you have hardware-specific implementation, have you started now considering CUDA programming as a way to speed up things? Or are you just like, oh, set it and forget it. I'll worry about the infrastructure.

 

I have thought about dabbling in some Python with these Cuda cores because I mean that's essentially you know, Pytorch and all those libraries are like driven by these Nvidia hardware. So I

 

Great.

 

have thought about it. But, you know, a lot of my development and learning comes from solving an actual problem that I have. fixed by doing some AI stuff. No, I think that's kind of been my drawback.

 

I see. Yeah, I mean, if anything, having this kind of setup makes you kind of have to think about these things more, right? Like it's a it's a physical trade off now.

 

Yeah.

 

It's it's it's not just okay, I'm going to implement it this way and then I'm good. It's like, well, you know, adding this extra stress could also like make this hardware crash.

 

Yeah. Yeah, and you know, there is the power consideration. The our price per kilowatt is nine cents, which isn't the lowest in the U.S. but it's also far from the highest. And I.

 

Have you hooked up a meter to it yet?

 

Yes, my entire server rack runs about 40 to 50 USD per month.

 

Wow, that's not bad.

 

Yeah, I mean, that's very cheap considering that hosting in a data center for a one you would be like fifty dollars a month and stuff. So.

 

and then your compute charges.

 

Yes, yeah. So I did end up running a 20 amp circuit to the server. So it's on a dedicated circuit and I do have battery backups enough to survive a brown out or a power blip for a few minutes. So it does have some some protection in place there from a grid standpoint. But we only have one Internet trunk coming in and that kind of stuff. So there are single points of failures with this that it's just not cost effective to resolve either in a hybrid cloud setup or going straight AWS transcoding service or something like that.

 

Yeah, I guess if you think of it like a sidekick cue, sure, the cue could start getting filled up, but it shouldn't matter if it's running or not. It doesn't sound like you're putting too much on the stack there.

 

Yeah. And if the site kick you gets, you know, filled with requests, then one, I am not monitoring my email or my notification systems that my servers are down, you know, so that's one thing, but it's also something that I can scale this out if it becomes very popular and people are doing it a lot, you know, or I can kind of use some analytics to see. These people normally will download the video two days after it's been released So I will preemptively transcode those so I could do something like that and You know that gives a better end-user experience or maybe all the newest videos I will Transcode slowly over a few days or something like that

 

Yeah, that's interesting. And I mean, since you have it locally too, you know, changing. So how does that, do you treat it like you're, you're deploying to production? Uh, how does that aspect of it work? Right? Like, cause I feel like, you know, you just have your machine locally. Are you just like, you know, SSH again, and then testing it out live.

 

So, um... My end goal is to put it in my CI-CD pipeline. Currently, I just do a SSH in, get pull, bundle, yarn, whatever, restart, sidekick, and it's all good. But I've been working and I've been dabbling in MSRK, a project library that is aimed to solve deployment stuff. interested in looking at and or what I've done in the past is use Docker and Portainer to handle deployments which I kick off from a CI CD pipeline so that's the other route that I'm thinking and I may end up going that route and said because one it's familiar so I know how to support it but then also I'm able to which would be really interesting, I do have to first experiment with the performance overhead of Docker and GPU pass through. So that's my current hiccup, but I wanted to get that proof of concept out there and working first.

 

So when you're thinking about expanding this, like let's say you have like a whole new idea for product or service that you're gonna be offering, like are you thinking, okay, like I gotta set up a whole new box for that specific thing? Do you keep things shared? Like how do you start reasoning about that?

 

So when you talk about shared resources on a physical machine, there are considerations that you have to take with security, especially with Docker, because Docker in and of itself can end up using shared memory and that kind of stuff. So I do like Proxmox, which is a bare metal hypervisor that VMs very similar to how you do it up in AWS or Digital Ocean or something like that. And I really like that route if I am doing something where I have multiple projects kind of sharing the same physical hardware.

 

That's interesting. So you would kind of scale sideways. to support

 

Yeah.

 

new things, yeah.

 

And with the GPU transcoding specifically, I can vertically scale that too. So I can get better GPUs that can handle more streams at a time. So it's not very CPU driven at all. It's mostly the NVIDIA cards that's doing all the leg lifting there. So I can swap out those 3060s for a 4090 if I wanted to. current going prices like fifteen hundred dollars for one of those cards but if again hands down do it not only faster but handle more simultaneous streams or transcoding services then i could get three of those cards into one box i could get a different type of box i was actually thinking about getting a bitcoin mining rack server which allows for eight to be in there and then you just have riser cards. So based on the motherboard I get I could fit like maybe five on a particular motherboard on a PCI 16x lane as long as I had enough lanes to service those which I would have to get like a AMD Threadripper or something like that to handle all that traffic on the PCI bus.

 

So

 

I

 

I'm

 

really

 

just starting

 

like

 

to

 

hardware.

 

picture, I'm starting to picture your basement now, right? Like you've got, you probably have a distinctive rack, right? Like at least one, right? Setup,

 

Yeah.

 

like do you have like lighting set up? Is it like air conditioned, right? Like how's your ventilation work? Like are your cables neatly wired? You know, like what are we looking at here?

 

Yeah, so no It's not the prettiest setup But I definitely went function over form which I can share a picture of the rack so you can see it But I Keep

 

Oh wow.

 

it clean. I do keep it clean and it is very functional, but the back of it's kind of a mess it's kind of on my bucket list here to able to tie them nicely. But as far as functionality, it's good and it's clean. And I will use my air compressor to dust it down. I'll open up some of the server tops every few months and make sure that all the heat sinks are clean and stuff.

 

Oh, that is a super cool setup.

 

Yeah, and I've even gone, I go as far as, because my servers are, essentially most of these are high availability. So if I take one down, then the servers, the services will transfer over to the others. And that's just a beauty of what Proxmox can do. So I will actually reapply thermal paste to some of them if I notice CPU temperatures are too high, because that could be one indication clogged heatsink filter or dried up thermal paste.

 

Yeah, it's kind of funny you gotta keep track of all that. I mean, I guess that's one downside is the TLC you need to give this thing.

 

But

 

But let's

 

it's

 

say,

 

therapeutic for me as well.

 

okay,

 

So.

 

yeah, I mean, if you were doing it before anyway, why not take advantage of what you can do with it?

 

Yeah.

 

So in your move to doing it this way with your hybrid setup, like what problems have you seen go away from your cloud-only usage?

 

Well, because finances are a concern, and they should be for any kind of business, do we really want to spend $1,500, $2,000 a month on this service? You remove that limitation. So it is just one of those things where you open it up to be able to do more of these kind of things, where you want to allow people videos so they can learn while they are off the grid. But you also want to hold them accountable to, you know, being true to, you know, don't share these out.

 

Yeah, that's true. I mean, is there anything in particular that you've noticed that you maybe you were surprised that you, you know, no longer had to do because you were like everything was contained nearby?

 

No, I mean the management consoles that you get from a lot of cloud providers are really good and so I mean I wouldn't say that there's any real benefit there. You know I think if I ended up having to move locations or something or if we had to leave the house then I would take the important servers that I have like the transcoder servers which is a data center nearby. And I would just throw it in there for a period of time until I'm able to pull it out again.

 

Yeah, that's pretty wild that you can just kind of, you know, plug it in somewhere else.

 

Yeah. Or, you know, I would not go take it over to a friend's house because even though, you know, I'm sure nothing bad would happen, I do take a certain level of responsibility and security for, you know, that tunnel traffic and I don't trust or want anyone to compromise it. So I do take security as a forefront, a high value importance.

 

Yeah, so I mean, do you find yourself like researching like networking security things more so now that you have the setup?

 

Not really, because before I started programming, I was doing sysadmin work. And so a lot of that networking and security knowledge I already had. So I think I'm kind of in that unique situation where I'm doing all the development, but I also know how and the implications of how it affects infrastructure and how the infrastructure can affect service availability and the code Thanks for watching!

 

I mean, all that's left here is, you know, you starting to build up your pen testing toolkit.

 

I think Metasploit

 

And experimenting.

 

has that cover

 

Yeah. That's funny. Have you gotten, have you like thought about getting, you know, a third party audit just to verify some things for you? Or do you not take it that far?

 

No, uh, I have had that done in the past, uh, on a project that I was a solo dev on And based on the results of that Um, i've not had anymore so it was a project that I worked on for several years And you know, this is like a humble brag. It's one of my great accomplishments, but it's also one of those things where you know It's hard to say without like Bragging so humble brag if you will I was a solo dev on the project. It was something that was going into production and one of their requirements was a third-party audit. So they shipped the code off and they ran the audit. It came back and there was only there was like one critical and two lows on the audit report and I was able to argue that one of those lows is invalid and the critical was a false flag. really out of the couple years of development that I did on that project, only one low issue got raised on a security check or a security audit. And it was around a third party plugin with SummerNote and it was a cross site scripting or where you could do cross site scripting. And that was the only thing. And since then, I've been actually using Tricks with ActionText. And that issue just went away completely because Tricks is awesome. And ActionText implementation and sanitizing of the HTML is really awesome as well. So I haven't had to worry about it.

 

Yeah, that's interesting. I mean, congratulations on the passing of security audit. I know those can be stressful.

 

But I do run Breakman on every commit.

 

Thanks for watching!

 

So, I mean, there is a little bit of due diligence there.

 

I mean, you have some other options too, right? Like HackerOne as an example, or they have other communities where you can kind of open yourself up to, you know, white hat security firms to test your network on occasion for you or services.

 

Yeah.

 

So I mean, if you ever needed to, you could always go that route. I mean, people do that anyway, right? Even on

 

Yeah.

 

the cloud providers. So it's definitely a really good option. really interesting

 

you

 

setup.

 

Yeah, as far as security goes, I think one of the best security things that you can have is do not have any pinholes. So in your firewall, don't have anything open. Always make outgoing requests. If you don't allow incoming requests, you've just lowered your surface attack area tremendously and making a outgoing VPN connection to then have a in or the AWS infrastructure is all initiating through an outgoing request. So you don't have to open up any port ports in your firewall or pinholes or anything like that. So that's one way if you are going with a hybrid setup where you have a big offsite computing hardware to complement what you have up in the cloud. That's one way to do it securely. No. Because then you don't have to worry about it. as well, then you really don't have to worry about it as much.

 

If people want to start setting up their own system here, like hybrid setup, like you've gotten yourself into here, where's a good place to start? And what are some of the recommendations you have to, maybe some issues you ran into that could have been averted if you had known.

 

Um the best place to start You I would say the best place to start is with yourself. Ask yourself, do you really need to do this? What is the benefits of it that you're going to get? Are you doing this just because you want to or you don't know how to do it in an infrastructure? So once you've validated the need for it to have some kind of bare metal complement to your cloud services, then you need to, hopefully you have a team that you can bounce ideas and security off of because you do want to take that stuff into consideration. You don't want this project to be the reason why you all had a data breach. So definitely take security as a first class citizen. It is something that has to stay in the forefront of your mind the entire time, both at a physical access standpoint, from a opening up your AWS or your cloud environment to a data breach. external connection to the hardening of the server itself. So you know make sure you have stuff like fail to ban installed and that kind of stuff. I don't know of any good resources that I would recommend as far as like you know here's a step-by-step how you do it but a lot of the information that you need to find on.

 

So what was the best piece of hardware you've purchased in this whole thing?

 

uh... I did go overkill and got a 7950X, which is a AMD Ryzen 9 processor, and that thing is bloody fast. It is amazing. So on the CPU benchmarks, it is ranked high up there. Oh, and I would say if you are going this route, use ECC RAM. It's error correcting memory. And it's very important, especially if you're running virtual machines and that kind of stuff. Because this is not just playing a game where if a bit gets flipped in the RAM, there's no consequences. There are actual consequences in servers. So even if you are using consumer, prosumer hardware, the CPU motherboard supports ECC RAM and definitely go that route if you can.

 

This 7950X is a beast.

 

It really is.

 

Do you notice it when you use it over other hardware?

 

I have noticed that it can because initially just in testing the connection and everything I was doing just CPU transcoding and it was fast It almost made me consider like do I really even want to add in GPUs into this mix? But then seeing like the actual wattage performance It was much more efficient to use the GPUs that had laying around and that kind of stuff I would say one other piece of advice this is something that I'll need to do is redundancy. I need to almost duplicate the setup so if one of the GPUs ever dies or something like that I can take it down and then pull it out, repair it, put it back in. So that's something that I have not done yet. That server is a single point of failure but redundancy is important.

 

So do you ever see yourself going back to the cloud, or is this your setup for the foreseeable future?

 

So anything that is user facing, I will always host in the cloud of some sort. Because again, I just don't have the infrastructure at home to do it. So, and when I say anything production, that's user interfacing, that's something that people are paying me money for, like a service. If it's not a service that people are paying me money for, then I will consider self-hosting it. But if they are paying money for, then anything that's user facing part of that, definitely for me has to be in the cloud.

 

Would you ever build these for other people?

 

No. Maybe for a friend, but... I don't want to get into hardware as a service, if that's a

 

Yeah.

 

thing. Hardware building as a service.

 

That's what Best Buy is for, right?

 

Yup. Oh, but don't use Best Buy's Geek Squad. They are

 

Ha ha

 

horrible

 

ha!

 

people. Well, not horrible people. The business is horrible.

 

So is there anything else you wanted to touch on?

 

No, I think that's it. I'm sorry I've just rambled on about all this, but

 

I've

 

it's

 

loved

 

so much

 

it.

 

fun.

 

I mean, it's such an interesting setup you got. And it seems like it's really working for you. I'm curious to see where you take it from here. We'll have to check back

 

Yeah,

 

in.

 

yeah, because it really is one of those things where it's not like you have to use cloud or no You have to go bare metal. It's like why not both? Why can't you have a bit of both? You know put user interfacing stuff that you don't have infrastructure to host up in the cloud put those heavy calculations securely on bare metal

 

Yeah, I mean, it makes a lot of sense to me. People have been doing it for ages.

 

Yep So no, I think that's it But if there is anything if anyone's interested in this and wants to chat more about it you know, I'm Cobalt's on Twitter and Very active on the Ruby on Rails slack channel. So, you know hit me up

 

Awesome. Well, I think we're safe to move to pics.

 

Yep.

 

Or actually, we've started a new segment where we're kind of just highlighting what we're working on, something interesting work-wise.

 

Yeah.

 

Do you wanna maybe throw us a little something that you're interested in lately that you've been working on?

 

This hybrid setup has been my most recent project and then I did do a episode recently It was 384 rails docked where I basically looked at that new docked project that Dhh made up on github under the rails repo or the rails group and It's really cool. You know, I was skeptical of rails docked at first actually have a steam deck that I use to install Ruby on rails on and Docker and all that stuff to turn a steam deck which is a handheld gaming unit into a development environment so that was a lot of fun too

 

Oh, that's pretty cool.

 

I like tinkering. I like computers. So this is my wheelhouse of tinkering and having fun.

 

Nice. Yeah, for me, I've been toying a lot with... voice cloning.

 

Oh man.

 

It's almost just comically fun. And they have a lot of real-time cloning techniques that you can do. And there's a lot of open source out there that lets you run a lot of these computing mechanisms in place on device. And it's just so much fun. voices and then pair it back in a new one.

 

Hmm.

 

Or just clone strangers as it talks to them.

 

Oh man, that reminds me of a horrible prank that I saw. And I don't know why I found it so funny, but it was, it's where this person calls a Chinese restaurant to make an order. But then they create a three way call and put themselves on mute with another Chinese restaurant. And just the

 

Ha!

 

language gap

 

Oh

 

and

 

no.

 

miscommunication, it was really funny, but it's just so horrible.

 

That reminds me of, what was it, the... of this like a. phone service you could use that the Jolly Roger is a Jolly Roger where

 

I don't

 

you

 

know.

 

can basically you could basically merge a call from like a marketer into this bot that would just like be it had a perfect loop about the conversations

 

Oh wow.

 

so it could waste the time of the person and then eventually they can they found a way to like call another marketer and have the two marketers talk to each other

 

Oh wow.

 

I thought that was really funny. But yeah, I think the job is called the Jolly Roger and you can basically like just like merge the call with the Jolly Roger and pick from like a list of like people that they have that will waste the time like of voice recordings.

 

That's kind of hilarious.

 

It's very hilarious. They record them too and like make them available somewhere.

 

Cool man, I had a

 

Yeah,

 

lot of fun talking about this. I know it may

 

I did

 

not

 

too.

 

be our typical fit for a podcast, but it was a fun experiment and turned into a fun project. And a fun talk. So, I'm gonna go ahead and start the video. I'm gonna go ahead and start the video. I'm gonna go ahead and start

 

Yeah,

 

the video.

 

you

 

I'm

 

know,

 

gonna go ahead and start the video.

 

deploying Rails applications has never been an easy, straightforward thing. And I'm really excited to see people kind of bending it in their own ways. And not that you're using Rails, but the whole Docker portion of deployment process has simplified things a lot. And I think where do you put it is a good question, right?

 

Yeah.

 

in the cloud, you can have some kind of hybrid setup like this. And I think it's important to highlight, you know, that there are options.

 

Yeah, and again, security is important. So

 

Of course, of course.

 

listeners, take security in mind. Make it a priority. It has to.

 

All right, let's move to pics. Dave, you wanna start us off?

 

Yeah, sure. So I'm going to say I'll pick the Steam Deck, because that was a really fun project, a very fun episode. But then it's also a very fun device. It's a four core, eight thread, 16 gigabyte RAM device. So it is a computer in your hands. And it plays games really well. And it runs Arch Linux, which makes it a fully web on if you want to connect a Bluetooth keyboard and mouse. So I've had a lot of fun with that device.

 

Nice. Um, let's see. I've got a Flipper Zero. Which for those not familiar, it's a kind of little... handheld device that lets you kind of peek into open networking streams and different wavelengths that are surrounding us and see what kind of data is being transmitted on them. You can do all kinds of wild stuff with it. I've only just set it up and it's the cutest little thing. And I look forward to having quite a lot of fun with it. what's in my home. So more to come.

 

Now, can you go war driving with it?

 

Oh yeah, it was almost built for that.

 

Really,

 

And it's funny

 

I

 

because

 

remember

 

it has,

 

doing that back in the

 

oh

 

day.

 

my gosh, it has the cutest little icons to go with things. So you're

 

Ha ha ha.

 

kind of having Tamagotchi experience with it, where you gotta feed it the wifi data. So we'll

 

That's funny.

 

see, I'll go on a walk in my neighborhood and see what my neighbors are up to.

 

Oh, wow. You can I'm looking at this flipper zero. It looks like you can duplicate or basically use it as a storage for RFID cards.

 

Oh yeah, it has so many different protocols

 

Oh wow.

 

that you could test. It's got sensors out of, like coming out of its sides, you know, like it's pretty incredible. I've only peaked a couple things. So I'm really, I'm about to have a lot of fun.

 

Wow, I did not know I needed this.

 

Yeah, I mean get it while you can. They keep trying to deny them entrance into the US. So hopefully that stays on a steady flow.

 

I mean, I can see why, because, you know, one thing that I did when I was traveling a bit is I got a new wallet that was RFID protected. So the outside of the wallet was shielded. So you couldn't have someone like bump up against you with some kind of RFID reader that could then, you know, get your credit card information with that like tap thing. Potential malicious purposes or devices like it. So Yeah, it's kind of crazy

 

Yeah, I mean, my hope is that, you know, it starts enforcing security where it should be. Right. I just think of the, you know, the car, the car keys as an example. Right. Where people were just, you know, waiting for somebody to lock their car and, you know, blocking the signal or intercepting it and then cloning the key on the fly.

 

Yeah.

 

I think they fixed that since now, but like it took somebody exposing it. Right.

 

Yeah.

 

So I hope that we all get more secure because of this thing, you know.

 

Yeah, just feel bad for the poor folks in the meantime.

 

It's always going to be the case though, right? Alright, well I had a lot of fun Dave. I I'm looking forward to see what else you come up with and kind of has your how your rack evolves. I mean it looks pretty incredible already. Kind of jealous. I'm going to have to start building one of my own now.

 

They're fun for the whole family. That's that's what you have to tell yourself.

 

Alright, until next time folks,

 

All right,

 

later!

 

talk to you all later.