Greg Molnar is a Ruby Developer and OSCP Penetration Tester. He joins the show to talk about Penetration Testing. He begins by explaining what it is, how beneficial it can be to the users, its process of securing applications, the actual testing process, and many more!

Valentino (00:01.154)

Hey everybody, welcome back to another episode of the Ruby Rogues Podcast. I'm your host today, Valentino Stoll, and I'm joined today by a very special guest, Greg Molnar. Greg, you want to introduce yourself and tell everybody why you're infamous?

 

Greg (00:17.04)

Hi, Valentino, thanks for having me. My name is Greg and I've been here before. I'm a Ruby on Rails engineer and a penetration tester. At the same time, I try to split my time between the two because I'm really interested in security. I got interested in security a few years back, but I can't just give up coding because I also love to write code. So I try to split the time and enjoy both.

 

Valentino (00:45.978)

Awesome, yeah, welcome back. I think the last time we had you on, I forget the security topic. It was also security related. That's right. Go back and watch that episode. I've already forgotten a lot of the content, but I will definitely have to revisit myself. And you know,

 

Greg (00:54.632)

with server-side request forgery.

 

Valentino (01:13.746)

We had you on to talk about penetration testing today, which is, to me, I'm more familiar with that. And for those that don't know, do you want to just give a breakdown to the audience? What is penetration testing and how is it useful for securing applications?

 

Greg (01:33.676)

Sure. Penetration testing is when a company hires someone to try to find vulnerabilities in their application and penetrate into their system. And it's a really good way to uncover unknown security issues in your app before someone malicious finds those issues. And there are basically three big categories of penetration tests.

 

There is a white box, gray box and black box one. The black box one is when the consultant doesn't know anything about the application. They just get a host. Here is this host, find everything out about it, try to hack it, which is very close to what would happen in a real world scenario. But I think that's not really the best option for any company because it's very time consuming, which means it's way more expensive.

 

Then there's the gray box one, when the company gives a lot of information to the consultant, he can ask any questions about the architecture, technologies used, certain hotspots of the application which might be prone to having vulnerabilities, but not getting access to the source code. And then there is the white box one, when you get everything what you get in the gray box and you also get access to the source code. And I think that sums it up.

 

Valentino (03:03.762)

Yeah, I mean, I'm familiar more with penetration testing just without, you know, where the company or software providers don't have information that you're testing them. And I know that this is now no longer the case for a large majority of security people and probably for a good reason where it is still a thing. Okay.

 

Greg (03:27.828)

Well, that's still a thing, but it's called bug bounties now. Yeah, it's called bug bounties. And there are platforms where you can, as a company, you can sign up and you can tell what's in scope, what hosts you want someone to test, and then you can report the vulnerabilities to you privately. Then you can resolve them and usually they pay you. Sometimes they don't pay anything.

 

but usually they pay you and then everybody wins. So, but that's called a bug bounty, not a penetration testing.

 

Valentino (04:05.398)

Gotcha. So when you're...

 

I'm curious, like, where do you even start penetration testing? Because there are so many layers to this. And I'm always interested, like, there are so many tools, we'll cover all of these topics. But, you know, what's your first, what's the first thing you do when you're like sign on with, you know, a task or a company to start penetration testing? I mean, do you have to start breaking down specifics?

 

of what you're going to start to do or do you just start gathering information?

 

Greg (04:44.032)

Well, usually every penetration test starts with a so-called scoping process where you talk to the customer and you ask a bunch of questions from them. Like what is the application doing? Which are the hotspots which I should test? And what is your biggest risk in your opinion? And then, then you end up with a, an estimate of how long the test will take.

 

and then you start the actual test. There are also like things, are you going to have a separate instance of the application, which you can do your tests again, or is it the production one, that's very not recommended, because then you need to be extra careful to not accidentally delete or access someone's information. And once all of these things are hashed out, then you get access to the target. And then I have like a playbook.

 

kind of thing which I'm going through. First thing first is usually the public pages, then the authentication, and then the authorization. Can I access things which I shouldn't with certain user levels? And then from then on it depends on the application. But it's just going through every functionality and try the usual suspects to find vulnerabilities.

 

Valentino (06:10.038)

Yeah, I always wonder, you know, there's so much automation these days and the tools are so kind of robust now. I always wonder, you know, how much of it is just, you know, how good is your toolkit?

 

Greg (06:28.872)

I never actually really used tools. I mean, I used tools, but not like, so there are two categories of tools. One, which is it helps you to do certain things. And then there is this thing called like Auto Pawn, which is you execute the tool and it tries to find vulnerabilities. And I never really use those kinds of tools. They give you back a lot of false positives. Usually they are very noisy. So it's...

 

For me, it's more of a waste of time to use those. Sometimes they can be helpful, but I have my own methodology and I use tools. Like I use a proxy in between the application and myself where I can manipulate all of the request data. So for instance, if I try to brute force something, I'm not making myself a hundred requests to verify that they have some sort of brute force protection. I have this tool for it.

 

which I just configured to send 100 various payloads to this endpoint. Or if I'm looking for a SQL injection, I'm not going to manually send all of my usual payloads. I have a list and then if I find a suspect of a field, which I think this might be vulnerable to SQL injection, I just run through all my 150 payloads and see if anything comes back with an error. And then based on that error, I can...

 

take the next step and figure it out if it's just an error or if it's a vulnerability. But there are plenty of tools which you can use and just run against an application, but they are super noisy. They can't really work after authentication. You can still configure them to log in, but finding issues where you have, let's say in your application, you have three different user levels.

 

Greg (08:23.516)

Obviously, you want to make sure that no privilege escalation is possible, so they can't access each other's account levels information. Like you have an admin user who can do certain things, but it's the same login, like the same user model with a different account level. And then you don't want a regular user to have admin privileges. With these tools, it's really difficult to test that.

 

Greg (08:53.177)

because you can't just pass the same login credentials and go through all functionalities, it might give you false positives.

 

Valentino (09:01.966)

I see, that makes a lot of sense. And it sounds like your methodology is just using what your experience has been. Finding the, like you said, the most likely sources to then pick apart. So I'm curious what specific tools you're using.

 

Valentino (09:32.422)

As an example for your SQL injection, like are there specific programs that you're running to make these requests, or do you have your own custom tools built? You know, where do you start to, what do you grab first when you're focusing on various specific tasks?

 

Greg (09:53.16)

My main tool is called burp, burp proxy, which is an interception proxy. It is basically you.

 

Greg (10:03.548)

You can open a browser inside of BERT now, it's like a really advanced tool now, it's super easy to use. And every request from that browser goes through a proxy. And you can hold the request and modify, or you can just replay any request with modifications to the data. So for instance, if you're looking for a SQL injection, and you know that, if let's say you are testing that the password field at the login is not...

 

vulnerable to SQL injection. You just make a regular login request and you send that to the tool called repeater and then you can configure it to send request with a list of payloads filled into the password field. And I have my own list of these various payloads and then I send those.

 

And then if there is an error, then at the end you get all the responses. And if it's an error, then you can look into it. What's that? What was the error about? Is it just a regular error because of some, maybe the payload was too big or too, too short because the password requirements are longer, or if it's an actual vulnerability because you were able to execute malicious SQL.

 

Valentino (11:27.774)

Yeah, the BIRP suite is great. I've definitely used it to diagnose some network, you know, specific errors or issues. It's pretty incredible. I've only scratched the surface I feel like.

 

Greg (11:42.976)

Definitely. I don't use a hundred percent of its features. So it's a very, very powerful tool.

 

Valentino (11:43.267)

So.

 

Valentino (11:46.948)

Hahaha

 

Valentino (11:51.607)

So.

 

You mentioned how a lot of these automated tools kind of give so many false positives, it's almost not worth it. Are there any cases where you would reach for this tool, maybe you have a task to just poke around and find any open things? Is there any value out of using them or should people just kind of stand clear?

 

Greg (12:20.34)

That's a good question. I think if you are learning then maybe it's good to help you to find spots where you should focus on. But I don't know, I was not... When I learned penetration testing, the course I took, they discouraged every tooling because they say that you should learn the actual underlying things to find vulnerabilities. You shouldn't just use a tool.

 

and then that will give you results because you lack the basic knowledge. And probably because I went through this whole course and they didn't encourage using those tools, that's why I'm not using them. I just got used to using low-level things. I still try to improve my process to automate where I can, but automating everything is just impossible. And the other thing is there are, there is like a thing called Nikto, which is a web.

 

security scanning tool, but the problem is if I run Nikto, it doesn't know what kind of technology that application uses. Is it a Rails app? Is it a PHP app? Is it a Laravel app? Or is it a Node.js app? Because all of these technologies, every framework has their own caveats, which I always research if I don't know that framework well enough, then I research what are these framework-specific things which I should look into.

 

Like in Rails, it has a few APIs, a few methods which people don't... It's not obvious to everybody that they can lead to security issues. The most famous one is the link to helper. Not everybody knows that the second parameter can be a JavaScript payload. So you shouldn't trust any user's supply data and just passing it to the second parameter of link to.

 

Valentino (14:11.698)

Yeah, that's a great example. Yeah, there are a lot. A lot of examples like that. I feel like that if you're just even if you're, you know, you've been in Rails for a few, few years, you can just, you know, completely ignore by accident, you know. And next thing you know, it's oops. Yeah. Yeah, I'm curious. So we could dive into that next because I did have like I was

 

Greg (14:30.112)

Yeah, it's too stupid, but it is a folder.

 

Valentino (14:40.49)

I was curious, you know, you do penetration testing a lot and, you know, how does Rails hold up? Like, do you focus specifically on Rails or are there other frameworks that you also like to focus on that you're familiar with? And like, how does it hold up?

 

Greg (14:58.672)

I don't focus only on Rails, but that's where I have the most experience because I know the framework well, I write code in Rails as well. So that's where I have the most experience. But I also do other things like PHP,.NET, I don't care about the technology. So if someone contacts me and we can get to an agreement, then I usually do the penetration test regardless of the technology. If I don't know that particular framework well enough, if I don't have...

 

experience with, I always do my research to find the framework specific things to verify. And Rails holds up pretty well, mostly because there are a lot of good defaults, but I think that's also a bit counterproductive because we have multiple categories of vulnerabilities, like the regular ones, SQL injection, XSS,

 

CSRF and whatnot. Many of these are the defaults in Rails are protecting you, but there are also the logical issues. And I think because people are, Rails devs are used to being safe and they are get caught used to not even think about security because the framework does it for me, then they don't pay much attention and they introduce more logical bugs.

 

due to these logical security issues, due to this.

 

That's also another thing which automated tools rarely find when it's a logic bug because they can't understand logic as a human can.

 

Valentino (16:30.016)

Yeah.

 

Valentino (16:39.882)

Yeah, I mean, I would assume that the logical security bugs are probably most common in any application. But yeah, I can see it being more so in Rails, where it is sort of like a business-driven tool, where you use whatever the defaults are to get whatever you need out.

 

Valentino (17:04.65)

But yeah, so I want to circle back to the tooling again. We were talking before the show, you know, do you use any like Ruby tools to do these penetration tests or are there, you know, what language do you reach for, you know, when you're trying to poke around the system, or do you lean more heavily on per proxy and, and using all, you know, more bare bones kind of approaches?

 

Greg (17:35.7)

Sometimes I skipped things and then my obvious choice is Ruby. And there are, I know a lot of security professionals are using Ruby as their scripting language, to their go-to scripting language. It's very easy to do the things which you need to do for a security check. And there are other tools like Metasploit is written in Ruby, which I don't think that many people know. There's also another tool called Beef.

 

which is the browser exploitation framework.

 

which is also written in Ruby. I rarely use that because that's more, it's very useful if you want to, if you find an XSS and you want to actually exploit it. It can like, you can create your payload, embed that onto the site. And when someone opens the page with the XSS payload, then you get access to their browser and you can run checks on their browser.

 

Is there any non-vulnerabilities in the browser? Exploit that, or if there is not, you can steal a session cookie super easily. It's like a click of a button. Or you can pop up a Google login page and try to fish them into Enter their Google credentials. And there are a lot of things which are built into BIF. That's a very powerful tool. If you want to take the next step of a vulnerability exploitation and just prove

 

to your client that this can be actually a pretty serious issue, not just an alert pop-up on someone's screen.

 

Greg (19:17.484)

And yeah, though my main tool is burp and some occasional ruby scripts to do things, but burp is the go-to for me.

 

Valentino (19:29.802)

Yeah, I'm curious, is there any like limitations that you've hit using Ruby and you're like, oh, I'm just going to go to something even lower level.

 

Greg (19:39.152)

Uh, but if, if I'm not testing a Ruby, a Ruby application, Rails app, let's say when I was testing a PHP app and I found, uh, like file upload vulnerabilities, then I have to create my web shell in PHP. I can't really run Ruby on those servers. But other than that, whenever I need a bit of automation, Ruby is usually can handle that. But sometimes you need to be language specific.

 

with the tools you need to write. Like if you need to write a little payload which is... which can be executed on that specific target, and if they don't support Ruby, then you need to choose another language.

 

Valentino (20:24.074)

Yeah, that makes sense.

 

Valentino (20:28.822)

I guess just use whatever you used to. You know, it's a, I'm curious, like.

 

Valentino (20:40.774)

At what point in the process do things like start changing how you're working on them, right? Like there's the initial kind of upfront investigation and then you finally find something that you're... It seems promising, something's not quite working as it should be and you start to target that. With penetration testing, is there like a known wrap-up or is it...

 

always end with kind of like open questions. Cause I'm always curious, like how does the workload work? Right? Like, do you just end with like, okay, I found these potential exploits or there are these exploits and do you also provide like feedback for how it should be like addressed? I'm curious how that process works.

 

Greg (21:33.408)

But the way I do it is probably different for everybody, but the way I do it, I'm always thinking from the other side of the coin as well, so I want my client to get the most out of their buck. And what I do is I have my kind of playbook, I go through all the things which I know I have to check. If I find a vulnerability that goes into my notes and then I put it into a report at the end. If I find something which looks promising, but I can't easily...

 

find a way around, let's say I find an XSS which I can't really exploit, but it looks like there is an XSS, but it might be not possible to exploit it, then I just leave it. And then I go through everything I want to check, and then if I still have time at the end, I get back to it, or I tell the client, look, I've wrapped up the project because I quote it four days or five days, I found things here, if you want, I can spend an extra day and...

 

tried to get to the bottom of it, or we can just leave it like that, it's in the report that it's a potential issue, but I couldn't verify it. And then I usually, if I have time at the end, I get back to it and try to figure out a way to exploit those sort of things. And the end result of every penetration test is a report. So you get a report with the listed vulnerabilities, the severity of them.

 

And if I can recommend a fix, then a recommended fix. Sometimes I can't recommend a proper fix because I don't know the language and the framework well enough, or I just don't know how your business logic enough to recommend the change to handle things. In a more secure way, I can just give a general advice. This is what you need to make sure doesn't happen.

 

Valentino (23:28.81)

Yeah, I've definitely read through quite a few penetration test reports and they're very useful. It makes me wonder, you know, do you do work like...

 

to integrate penetration testing or just security analysis as part of like a developer workflow? Have you ever like joined a team to help them kind of think about things more as they're developing or is it not worth it to a company? Do they think more, oh, we'll hire somebody to find all the security stuff.

 

Greg (24:05.164)

I think that would really work for any company, but I never did such a thing. I always give my, if it's a Rails app, I always give them a list of recommendations. Like run Breakman on your CI and read this documentation, the security guide, make all of your developers read it. And I think if you are, if you want to be PCI compliant, you should send your developers to a security training every year.

 

usually companies don't do that, they just have their internal training, which is usually worthless, to be honest. But the other recommendation, which I always tell to every company, have one person on your team, who is interested in security, let that person learn about security, let them be the security guy of the team. And if you have any doubts about something, a feature or whatever,

 

That guy is your go-to person.

 

before you go outsource it.

 

Valentino (25:05.534)

Yeah, I mean, I definitely agree with that mentality. Um, you know, I've joined many companies where, you know, there was somebody that was just interested in it. And so you'll do learn a lot, you know, if you, if you aren't familiar with it, which I wasn't at the time, it was like semi familiar, you know, but you do, you like learn where to read about certain things and where things are published. Right. Uh, cause there are, there are like plenty of organizations that publish.

 

you know, breaches or security, new security related incidents, right. Uh, with common tooling. And so there are just like things that you get used to, uh, the more you work with it. Uh, and I'm really surprised how like, uh, how not built up a lot of the processes are for, you know, onboarding people to security, um, which is kind of surprising, uh, they kind of just expect, oh, you should know it and do it. Um.

 

And yeah, I 100% agree with like training is definitely important. Keep people up to speed and thinking about it.

 

Greg (26:13.332)

Yeah, but companies focus more on clean code, maintainable code, than secure codes, to be honest. And just getting started.

 

Valentino (26:20.554)

So I'm curious what your thoughts are. I'm curious what your thoughts are on like, you know, there are plenty of Rails people out there, plenty of Ruby developers out there. You know, how can we stay up to date and like, keep on top of maybe not doing penetration testing specifically, but being aware of these concerns as we're developing. Because I feel like that's an ongoing concern is how do you like start?

 

thinking about this stuff or continuing to think about them as you're developing. What are some good, like best practices for that?

 

Greg (26:58.612)

That's a good question. You should build into, it's kind of a mindset thing that you should have, or maybe you can actually do like a checkbox on every pull request that I check this for every security concerns. Then you can build automated toolings like breakman into your CI flow. And if that highlights something, then someone needs to verify that that's an actual issue. And to learn about it, there is the company who's building burp, it's called Portswigger.

 

and they have a thing called the Burp University, which is a web-based, all of the web-related vulnerabilities that are really good documentation and write-ups and tutorials to go through them. So that's definitely a good learning resource.

 

And other than that, what I also like to do is we have a few good open source Rails projects like RubyGems.org, GitLab, and sometimes they have security releases and it's always interesting to check what happened, what kind of bad assumptions those guys made to introduce the vulnerabilities and just learn from it.

 

Valentino (28:16.402)

Yeah, I agree with you there. It's always interesting to read those CD's that come out from Ruby, Ruby Lang and just, you know, read through it. I mean, there's pretty wild, some of them, what people are able to do, right? To, to exploit the system or the language even. And it is really interesting reading the kind of how they untangle everything. And fix it ultimately. So that's pretty cool. I didn't know.

 

that Burt offered or the Portswigger offered the Academy or whatever they call it. That's pretty cool. I'll be checking that out for sure.

 

Greg (28:55.464)

Yeah, it's also, it's completely free. So it's a very good resource, to be honest.

 

Valentino (29:00.798)

Yeah, that's awesome. Uh, so I'm curious if you've gotten into, uh, kind of the discoverability of, uh, you know, common penetration testing, uh, as far as like, uh,

 

you know, reading through code source. Like, do you use any tools like that? Like, let's say you get a white box customer. Like, do you use any static code analysis to like read through a lot of the code to look for common patterns or, you know, what do you do for things like that?

 

Greg (29:40.76)

Yeah, I do. I used Breakman in the past, but Breakman being acquired and they change their license, you can't use it for any paid engagement, but I created my own tool, it's actually it's open source. It's called Spectre, which is it's basically the same thing as a Breakman. It's a static code analyzer. It's not going to find every vulnerabilities. It's going to find some false positives, but

 

the good thing about it is it gives you ideas of patterns in that codebase and then you can search for those and check them. But that's the only tool I run on the codebase and then it's just a manual check. But even if it's a white box test, what I do is I just check some obvious suspects in the codebase, but then I do a regular penetration test, like if it wouldn't even be a Rails app.

 

I'm just going through on each pages, I do the same request what I would do if I wouldn't have access to the code base. The better thing in that scenario is if I find something which is a suspect, I can just easily go to the source and just maybe even make some modifications to the source code to debug that issue and see if it's an actual vulnerability, if I can exploit it. Or if I just read the code, then...

 

sometimes it answers all the questions and tells me no you can't do you can't really get this through here or it becomes obvious that I can Do what I was intending to do. I just need to do a slight change to my payload or whatever

 

Valentino (31:24.734)

Yeah, I would love if more and more static analysis tools came out. I know it's hard with Ruby, it not being typed and everything makes it a little more difficult, but we've definitely caught at Doximity a ton of stuff using BrapeMan and a few other tools, just doing static code analysis and being like,

 

You know, you can't do this, here's a link why. And it is just like, it saves you, you know, especially with like a link to, you know, even if you just had an automated message and a, you know, review that's like, you know, is this user supplied, right? Like, you know, little checks like that are just like so nice and helps, they can really save you, you know. So I'm curious, like, are they're open sourced or like?

 

Greg (32:14.324)

Yep.

 

Valentino (32:22.702)

Uh, resources you use to have like these checklists. I know you probably at this point already have like your own checklists that you run through, uh, for doing your testing. But, uh, are there any resources out there? People can look through to kind of see, oh, like here are, here's an example checklist of things to check while running through.

 

Greg (32:44.092)

I have my own, but that is, I think it's from Andy Kane, you know, the guy who has a bunch of really cool gems. I think he has a, like a kind of guide in a GitHub repo, like things to check for secure Rails development. But I will Google it and we can put it into the show notes. That's probably a good one. But other than that, I can't, I don't really call any.

 

As for static code analyzer, there is also another one, I don't remember the name, it came out recently. And that one focuses more on the JavaScript side of the Rails app as well. To be honest, I haven't used it, you know, I just tried it when it came out, I haven't used it since. But that might be also useful to, to even to run it on your CI. It might overlap with Breakman, but you know, the more the merrier if...

 

Maybe it finds things which Breakman wouldn't. We can also put that into the show notes. I need to Google the name. It was... I can't remember.

 

Greg (33:53.64)

remember what it was but we can put it into the show notes.

 

Valentino (33:58.92)

Cool, yeah, I'd be interested to check that out myself.

 

Valentino (34:04.578)

So we've talked a lot about the tooling and...

 

you know, maybe some of the procedures for, for walking through a lot of this. I'm curious, like, uh, you know, what are, what are some pretty wild, you know, things that you found where, uh, maybe it seemed obvious to you. Where it was just like completely oblivious to the, you know, the people that had made the software.

 

Greg (34:33.364)

I can't bring up any specific examples, but the thing with these vulnerabilities, once you find them, they all look obvious. And before it's, so it's a bit hard to say that all these things was obvious, because of course, once you test it and find it, it becomes obvious to you as well. But it's usually these issues are, I think the bigger problem is not that they are obvious, but it's when people...

 

just make a really small bad assumption and they introduce a big security hole. And it's just super easy to do that. Also, it's just missing things. For instance, with the, I can bring up an example without giving out any names. There was a project where they overwrote the, they used Hummel and they overwrote some Hummel methods to render some specific things. I...

 

It was years ago, so I don't remember the exact details, but they overrode the humble rendering to do some specific things which they needed in the application. And they didn't realize while the way they overrode it, they totally disabled the HTML sanitization thing. So anything you put into the application, which they rendered was potential accesses.

 

because by default, Huml also like ERB escapes everything by default and you need to mark everything safe if you want to. But because they overwrote the rendering, that they disabled this default function. And it was actually when I saw it, I was like, I can't believe this because this cannot be the default in Huml. So I looked through everything and I went through the Huml source code and was like, it's not the default. So I didn't understand how it's happening. And then I...

 

I told them that I found this thing, but I don't really understand why it is full of accesses, your app. And then of course they immediately know, oh yeah, we override this thing and it's caused by that probably.

 

Valentino (36:41.302)

You know, it's funny that you bring up a custom Haml renderer. I feel like I've seen people even just like, you know, create a fork of the renderer engine to make a tiny tweak where it is even safely sanitizing, but then there's, you know, something upstream that changes that they fix. And then you've lost that, right? So many instances like that I've seen.

 

where you've like adopted and tweaked something that, you know, is in the core of the framework or language that you're using. And then, you know, something happened, you know, there's a, you know, security fix and, you know, you can't merge it in because you've made all these modifications. So don't do that. But yeah, that's

 

Greg (37:37.552)

Another interesting one, which I, it was fixed in Rails 7, I think. And in Rails 6, if you sent a, a non-existing HTTP request with an non-existing HTTP method in the request, then that results in an error page. And that error page was displayed for some reason, even when the verbose error pages were disabled.

 

And the verbose error pages include your secret key base, which is like the key to the kingdom. And I also found that one on an application once and that was an interesting one.

 

Valentino (38:23.278)

Bummer.

 

Greg (38:25.824)

Yeah.

 

Valentino (38:26.278)

I mean, I feel like the end result of a penetration test is always a bummer. We've made a mistake, you know, and it's never a... I feel like it's a bittersweet ending, you know? All right, we fixed it and we'll work toward getting it resolved, but like, how long has this been there? Is there even a way to know how long that something's been tampered with?

 

I know you can find out maybe how long that it's been exploitable, but...

 

Greg (39:03.196)

Yeah, you need to check your logs in scenarios like this is just check your logs as far back as you can to see if anyone tried this, rotate your secret keyways immediately. And then if you are like PCI compliant, then you need to check for sure that nobody, if any, any findings from the penetration test report needs to be verified that nobody exploited those before, because if anyone did, then you need to disclose that to your customers.

 

Valentino (39:16.172)

Yeah.

 

Valentino (39:35.062)

Yeah, I mean, and that is a painstaking process, sifting through logs and trying to tie all the pieces together.

 

Greg (39:38.686)

Yeah.

 

Greg (39:44.868)

Yeah, that's not really easy. There are companies actually working specifically on those investigations. Like you give access to your infrastructure to them and they dig out everything they can to check that nobody accessed anything.

 

Valentino (40:03.594)

So as a company that wants to hire somebody like you to do penetration testing, is there a suggested route that you would suggest? Is white box better than gray box, better than black box? Are there specific scenarios where one is more advantageous than the other?

 

Greg (40:26.128)

I think white box is the best for everybody because that's the lowest cost wise. You spend the less time if you do a white box test because you can verify things more easily so you save time. But sometimes they don't want you to have access to the code base because of various reasons. Even if you sign an NDA they don't want to give out the code base then a grey box is the way to go. Black box I don't recommend that to anyone. It's just a waste of money.

 

because you just, you can just, you could just share information, it doesn't cost anything and you are just losing money, throwing money out the window.

 

Valentino (41:05.482)

Yeah, when you said black box, I immediately thought bug bounty. And then why would you not do bug bounties?

 

Greg (41:10.44)

Yeah, exactly. Then you can just do bug bounty and only pay if someone finds something. But there are bigger companies who are doing this. Actually, they have a bug bounty program, but they also do the yearly penetration test with a consultancy.

 

Valentino (41:16.227)

So.

 

Valentino (41:25.634)

So for you, are Bug Bounty's worth it?

 

Greg (41:31.308)

Not really, to be honest. I know some people that are making like half a million dollar a year or even a million. It's insane. But the way I do bug bounties is if I find something on a website by accident, which I use, I go and check if they have a bounty program and then report the issue. And then I got some money like in like this way. I found an issue. For instance, it was

 

A few months ago I found an issue in something which I use. I reported it because it was like I can't believe they do this and then it turned out like what happens usually they did a change and they didn't realize that change has a side effect and it disabled something and it led to a... it wasn't a serious vulnerability but it was still a security issue. And then they gave me a bounty for that.

 

But to spend like time, the problem is sometimes if you do bug bounties, you might spend a week without finding anything. And then you maybe find something and you report it and someone found it two days before. And then you are not getting paid, that guy is getting paid. And they even, some people say that some of the companies just sometimes claim that it's a duplicate or they already knew about the issue and they just don't want to pay a bounty.

 

I don't know how much truth to that is there, but I don't know. So I think it doesn't, for me it doesn't really make sense. The guys who are making a lot of money, what they do actually is they focus on one specific security issue. They get really good at it, like let's say server-side request forgery. And they find something which is a very new and novel.

 

And then they go to a bug bounty platform, they go through all of the programs, they try that single thing and like 50% of the targets, they find that thing to be working, they report it and then they get a lot of small payouts from everywhere, which adds up to a lot of money.

 

Valentino (43:44.65)

Yeah, I could see that. I mean, that would be my approach is finding one exploit and then trying it on, you know, any that are in the programs, right? It seems like that would make the most sense. And really, it would help to specialize people, right? If you're trying to do a bunch of security, like trying to get it all, it's probably very difficult, right? I imagine even for you, you know,

 

If you wanted to do something more granular, I don't know what is in your wheelhouse or not, but I always think, you know, encryption, you know, busting and things like that are probably pretty difficult and niche. Right. And so, you know, I feel like having more niche security people is a good thing.

 

Greg (44:31.836)

Yeah, definitely. And you are spot on encryption is not my strongest point for sure. I can find logic issues with encryption, but I, if you want me to, to check your cryptographic algorithm is solid, that's I'm not your guy.

 

Valentino (44:32.227)

But I-

 

Valentino (44:51.922)

I was just a guess, I mean, because I do know it's difficult. It's difficult to get into. Yeah. Yeah, I mean, my eyes just glaze over anytime. I think about a new encryption algorithm and I'm just like, all right, somebody's going to say this is a better one and I'm going to use it, you know, and hope that is true, you know. So I wonder how much of that is that you find true.

 

Greg (44:57.16)

That's the most difficult part, I think.

 

Greg (45:13.374)

Yep.

 

Valentino (45:20.378)

even in frameworks like Rails, where it has safe by defaults kind of mentality. How much of that continues to be true as you continue to use the framework?

 

Greg (45:38.796)

With Rails it's actually just getting better and better as time passes by. Everything is just being improved security wise, slowly but surely things are getting improved. And I think...

 

Today, Steel Rails is one of your best shots if you just want to get a basic level of security in the framework. That's your best shot probably.

 

Valentino (46:09.494)

Yeah, it makes me wonder, you know, I would love like, you know, a real game where I have to, you know, pen test an example application and find, you know, vulnerabilities that have been explicitly set. I feel like that would be a lot of fun, fun way to learn. Yeah, you're kidding.

 

Greg (46:27.504)

I have good news for you. There is such a thing. It's called Rails Goat, which is a Rails app. Yeah, it's an intentionally vulnerable Rails application.

 

Valentino (46:33.044)

Real Scout.

 

Greg (46:41.104)

And they even have... I never went through it, but I just once I checked it out, and they even have, I think, these like scenarios. So you can go through a certain scenario. It's not like that... Yeah, it's from OWASP. It's not like that here is this application and find vulnerabilities, but you can, I think you can select what you want to check. Like...

 

Valentino (46:41.672)

Oh, this is great.

 

Valentino (46:56.712)

Oh, it's from OASP.

 

Nice.

 

Greg (47:10.148)

XSS and then it teaches you about XSS as far as I recall. I just had a quick glance a few years ago.

 

Valentino (47:19.858)

Oh, that is really cool. Yeah, I'm gonna play with this. Ha ha ha.

 

Greg (47:24.352)

Yeah, that's a good one to play. And also the Burp University, they also have, they have the learning material and they also have a lab, which is like, let's say they teach you about SQL injection. And then do you want to actually try to use what you learned? It is our lab, click on this link. And then it's an application where you need to find a SQL injection and you need to exploit the SQL injection and then you can test your knowledge.

 

Valentino (47:52.754)

Oh, that's really cool.

 

Greg (47:54.1)

that's also totally free.

 

Valentino (47:58.09)

Yeah, those guys are awesome. Ha ha ha. OK, well, I feel like we've gotten through a lot here, Greg. Ha ha ha. Is there any other topics you wanted to cover before we wrap things up?

 

Greg (48:16.12)

Nothing really, I think we covered pretty much everything. Oh, there is also one more thing, which is, it's a bit about, it's a bit talking against myself. So if you are a company and you are doing penetration test every year, but I always recommend to everybody, which is, as I said, a bit speaking against myself, you hire me this year, don't hire me next year, hire someone else. And just rotate, find two or three people who you can work with, or companies who you can work with.

 

and rotate them because when I do a test, I do my best to find every vulnerability, but I can, I'm a human, I can miss things. Maybe someone does the same test, does a test two weeks later and he finds something which I missed, but he might not find things I found. So it's always better to mix people because everybody has a different approach, different eyes. So it's good to mix people up and that will probably...

 

lead to a better security posture for your application at the end.

 

Valentino (49:19.87)

Yeah, I liken it to music producing, right? Like you get the same producer, no matter how many different singers you get, it's all gonna sound the same. Ha ha ha.

 

Greg (49:29.72)

Exactly. With Secret it's not exactly like that, but different people have different perspectives and they can find different issues. But everybody likes it.

 

Valentino (49:41.866)

Yeah, I mean hiring, hiring Greg every year still better than not hiring Greg every year.

 

Greg (49:48.249)

Yeah, doing a penetration test with the same company every year is better than not having a penetration test for sure. But rotating companies are even better, in my opinion.

 

Valentino (50:01.098)

Yeah, that's a great, great idea. Well, cool. If there's nothing else you wanna cover today, it was great having you on, Greg. Thanks as always. I love talking about security and you always give some great recommendations here. And it's always good to spend some time on security. If you're in any software position, it's always good.

 

even just to play with a tool or two and learn something new. Cause it's very easy to overlook, you know, these tools and, and these exploitations really, um, you know, they, they layer, they're there, they're always there and they'll get fixed over time, but you know, you just have to be on your guard, you know.

 

Greg (50:49.308)

Yes and the other thing which may be worth noting is cyber criminals these days are like professional criminals. They are in it for the money. Your system might be exploited without your knowledge and they are just trying to get information out from the system without raising any alarms and making money from it. For instance I'm getting... For some that I paid with my card and...

 

I put down my phone number and they get my last four digits of my cards and since then I don't know which website it was but I'm getting phishing texts from my... they are trying to phish in behalf of my bank and just make me call a number which doesn't belong to my bank and probably they want to get some information out of me. I'm getting a text like your payment couldn't go through, call us please.

 

And that company, I'm pretty sure wherever they got my data from, that company doesn't know about this. They don't know that someone is inside their system, getting all the orders and trying to fish their customers.

 

Valentino (51:50.764)

Wow.

 

Valentino (51:58.062)

That's crazy.

 

Greg (52:00.384)

Yeah, this is like a pretty common scenario. These are professional, these people are in it for the money. 10 years ago, 12 years ago, you know, they defaced websites and just made it for like fun. And oh yeah, look, I can hack into this site. I remember in Europe, we had a lot of like in Turkey, there were political issues and we had these Turkish freedom fighters defacing a lot of websites and just saying how terrible the Turkish government is. But...

 

Valentino (52:03.351)

Yeah.

 

Greg (52:28.416)

that thing doesn't really exist anymore. If you find a vulnerability, you are making money from it. It's not like let's make some fun of this company. So it's, it should be taken seriously, to be honest.

 

Valentino (52:45.674)

Yeah, the fun days are over. But, yeah, that's always the problem.

 

Greg (52:47.976)

Yep. Yeah, people realize there is money in it.

 

Greg (52:56.434)

Yeah.

 

Valentino (52:57.93)

All right, well, it was great talking to you again, Greg. Let's move into PICS. Is there anything you want to share with everybody today?

 

Greg (53:07.42)

Yeah, I want to do the same pick I did last time. But the reason for that is a thing which I saw on the internet. So there was this guy who posted on Twitter that, shit, I lost access to all my online accounts. And how so? Someone hacked his password manager, which is just not an issue, right? Because you have two-factor authentication. But his setup was, I'm using one password to store my passwords and I use one password.

 

for my two-factor authentication as well. So once someone got access to his one password account, they got the key to the kingdom. So my pick is a YubiKey, which is a hardware device for two-factor authentication. It's separated from your password manager, which makes it a good second factor because it's not in the same basket. The only issue is it's not cheap, but it's very convenient and it's the best way to do 2FA in my opinion.

 

Valentino (54:06.602)

I'm curious about the YubiKey. What happens if you lose YubiKey?

 

Greg (54:11.912)

You stick... there is a recovery mechanism, I think, but you can't have a backup. So you ideally you would have two ubiquis and they are like a mirror of each other. It's like, you know, it's just using the mathematical model algorithm to generate the second factor, so you can probably just store your seat somewhere. I think I have a seat somewhere stored. I don't know. I should check my recovery process.

 

but also I have backup codes. So whenever I set up two, if I set up a two factor authentication somewhere, I store a list of backup codes.

 

Valentino (54:52.094)

Yeah, I think I had a... There was... It was years ago, but there was a time where I had one specific account that I had recovery codes stored, and then the hard drive failed and I hadn't backed up that hard drive. And then I lost that account, basically.

 

Greg (55:11.904)

What I do with my backup codes, I print them and they are printed. They are somewhere in a folder because I don't carry someone breaks into house, they find the backup codes. It doesn't make any sense on its own. You still need the password. So the password is in the password manager, which is encrypted. So it's still, they can't do anything with the backup codes.

 

Valentino (55:18.606)

That's a good idea.

 

Valentino (55:27.822)

All right. Yep.

 

Valentino (55:36.215)

Right.

 

Greg (55:38.928)

And then if I lose the hard drive or if the other good thing is if someone would get onto my system and somehow they managed to get my password, let's say with a key log gap and they also find the file on my computer with my backup code, they have the access to that service because they have the password, they have the second factor.

 

Valentino (56:02.322)

Yeah, that's a great idea. I might have to get a YubiKey or two. I think you've convinced me.

 

Greg (56:08.999)

Yeah, it's very convenient. Yeah, it's just a touch. It's plugged in. I just touch it and that's it.

 

Valentino (56:19.259)

All right, great. Let's see. I've been in large language model world for probably too long here. And so my picks are very related to that. Number one being MagmaChat. Pretty awesome Rails open source application from OB Fernandez, who's from Hash Rocket, if you're not familiar. Really awesome framework. I've just been digging in.

 

Greg (56:26.163)

Hehehe

 

Valentino (56:47.014)

and generating a bunch of role specific bots. And it's been a lot of fun just running it locally. Pretty incredible. So recommend checking that out. And to go with it, he has a paid service called Olympia that has very human-like bots that you can buy into, that are legal specific or are very specific in very specific roles.

 

and act human or more conversational. It's pretty neat, pretty wild what he's put together. So I recommend checking those out. They're a lot of fun.

 

Greg (57:28.888)

I will definitely do, I'm curious about it. If it's an open source Rails step, I'm always curious to see if there are any vulnerabilities.

 

Valentino (57:36.95)

Yeah, please do, because I'm definitely playing with it a lot more and taking it a little seriously. So, I mean, you're right. Maybe I should be doing a little pentesting of my own on it.

 

Greg (57:48.904)

Yeah, at least run Brakeman on it, just to see if it can find anything.

 

Valentino (57:51.318)

Yeah.

 

Valentino (57:55.87)

Alright, well until next time folks, I'm Valentino Stoll. This has been an awesome time with Greg Moller and we'll talk to you guys soon. Bye.

 

Greg (58:06.112)

Thanks for having me.