Nir Valtman is the Co-Founder and CEO at Arnica. It is a behavior-based software supply chain security platform for DevOps. He joins Jonathan and Will to introduce his company. He talks about some of the issues and challenges they encountered and how they overcame them. Moreover, they also tackle security tools in Software Development.

What's going on, everybody? Welcome to another exciting episode of Adventures in DevOps. Joining me in the studio today with our co-host, Jonathan Hall.

 

Hello everybody.

 

And we are missing Jillian today, but I will be performing the role of host. I'm Will Button. And we also have with us Nir Valtman. Welcome.

 

Hey, thanks Will and Jonathan. Great being here.

 

Thanks for coming. It's a pleasure

 

Yeah, we're

 

to

 

excited

 

talk.

 

to have you here. So we're going to be talking about security today, right?

 

Yeah, just a little bit. Let's call it cyber security, not the physical

 

Cybersecurity,

 

security.

 

Ooh, even better.

 

right? Yeah, we'll rank better in the buzzwords with that. So for our listeners, give us a little bit about your background to set the context of this conversation.

 

Sure, I am actually the CEO and one of the three co-founders of Arnica, which is a software supply chain security company. But quickly about the previous endeavors, I was a VP security at Finastra, which is a quite big FinTech company, been running AppSec, DevSecOps, which is technically DevOps team that were developing security automation, data security and security architecture. CISO at Cabbage and beforehand I was heading the AppSec at NCR. And I'm an open source contributor.

 

Right on. Cool. So we were talking before we started recording about the challenges and ways to implement security, application security. So get us started on that topic. What are your thoughts there?

 

Yeah, you know, I've been working in smaller companies and bigger companies and application security looks a bit different in those areas. On one hand, if you look at bigger companies, where, for example, I worked at NCR and Finastra, the two biggest challenges that I had were what is important and who do I talk with? Because at the end of the day, you can hook up a lot of security tools, all of the buzzwords of SAST and DAST and software composition analysis, S-BOMB, which is a thing now. You can definitely hook all of those up. And eventually, you will get results, and you will get a ton of those results. And the question would be, how do I get what is important? And the way that I tackle that in the bigger companies... was actually going to the finance folks and the strategy folks. And, you know, I spoke with them to understand either it's a revenue generating product or like maybe that's the top revenue generating product. Maybe it's something strategic that is driven by the, let's say, you know, CEO. So eventually it helped me to get a lot of the context to understand where should I put my efforts. There are sometimes deterministic ways. to get that. For example, if you look at the code repository contributions, maybe a stale repository where you didn't make any contributions in the last six or nine months, you'll probably guess they're not that important as the others. You can also look at highly performing teams that contribute to pull requests. So if you see that there is a lot of chat around pull requests within repositories, yeah, it's probably active. But if it takes 20 days to respond to a pull request, maybe it's either not fast-paced moving team or maybe it's just not that important. So the context matters compared to the rest of the organization. So that's maybe a bit more deterministic.

 

Yeah,

 

In smaller

 

for

 

companies,

 

sure.

 

it's important.

 

You know, I want to underscore that because I think that's, like to paraphrase the meme, that's a pro gamer move right there. Because

 

Hehehehe.

 

you can easily go down this path of finding all these different things to work on. But the tactic that you use there of figuring out, does anyone care? just makes a huge difference in your career, not only in terms of like how your peers and your coworkers and the people you report to see your performance, but also in your personal stress level because it's going to be pretty stressful if you're putting in all this effort to solve all these problems, but nobody really cares if those problems are solved or not.

 

Exactly, exactly. Which is why, I mean, I would say a few years ago, I took only the top approach, the top down, let's go strategically and figure out what is important. Thing is that based on quite a lot of conversations with peers, not everyone even know how to make that conversation, not because they're not wise enough to ask those questions, it's just because there aren't enough people that can provide those answers or Maybe in smaller companies you have a handful of products, and then you know that these five products are important. But then the resources you have in a company are significantly lower. So maybe the conversation drives from what is important to what is the priority to actually get things done at all.

 

Yeah, absolutely.

 

It's different.

 

Yeah, it's very much different in the size of the company.

 

Exactly, exactly. And then when you are done with that piece, then the other challenge that I had is really figuring out who to talk with. And you know, how do you get that? I mean, you start through one person, maybe you guess, you know, you probably open the organizational tree and start looking through, oh, this is the product manager, this is the person that is the top of the hierarchy and runs that product. which is a way to do that. That's historically how I had to do that. But the people that actually fix whatever you find are not there. They are the least, you know?

 

Yeah.

 

And how do you find the right people there? Because it's very hard practice to even build when you're going through that approach. I mean, again. There are deterministic ways to do that, and there are less deterministic ways to do that. So for example, what we did at Finastra, we built something that is called the Security Champions Program. In some cases, it's also called as a satellite program. And in that case, we just took each product and we asked the engineering team to find us someone that is excited about security. And it had to be an individual. It had to be an... individual contributor, it couldn't be a manager. And once we

 

Yeah.

 

got those, we knew that, hey, this is a developer, that someone that is excited about security, we can teach him something, and he'd be willing to actually learn by himself, him or her, right? And this is how we built a program with about 100 individuals in the company that were tied to 100 products back then

 

Oh wow.

 

that... Actually, it's a bit more because some legacy products had smaller engineering teams. But that was pretty straightforward. And then you start a conversation about how you build a community within the company to actually make that program work. On the flip side, you can also go in a bit more technical approach. Think about it this way. GitHub and GitLab and others, they have good controls on branch protection. You can look at those branch protections and actually identify who should approve pull requests.

 

Mm-hmm.

 

The people that approve pull requests are the ones that have the most context in the product.

 

Ah yeah, true, true.

 

Right. So, so you can also go to those, um, and identify not only who are, let's say, like in GitHub, there is a feature called code owners, not only

 

Mm-hmm.

 

who's the owner of the code, um, and typically again, the ones that are, that approve the changes are architects, engineers, maybe principle engineers, other people that actually care about the product and care about the quality of the code and security is just a subset of that quality. So there is a way to identify those owners in a more deterministic way without working too much and talking to too many people.

 

Yeah, and getting passed around from person to person, continually striking out.

 

Yeah, eventually, I mean, you can't hit a 100% coverage on, let's identify everyone who owns everything. But for example, maybe you look at the behavior of the last 90 days only. Maybe

 

Yeah.

 

I want to see anyone that reviewed PRs in the last 90 days. These are the most important folks, right? But what happens with repos that are not there, right? They don't have contributions in the last 90 days. So obviously, there will always be missing parts. But as long as you can go to the active ones and find in a good enough guess who are the ones that are active, I think you can start with a pretty good coverage, building that program to answer who to talk to.

 

Right on, I think that's

 

So

 

one of the

 

I

 

biggest...

 

once...

 

Go ahead, Jonathan.

 

No, no, you finish first.

 

I would, but I totally forgot what I was saying. Hahaha.

 

Okay.

 

Yeah.

 

This is the part where our hero editor edits us out to make us sound smarter.

 

Right?

 

So I was gonna say, I think I held a role similar to what you're talking about. I was working at a pretty large company and they had, I think they called them ambassadors, security ambassadors. And honestly, I looked at that and I thought the way they did it seemed like theater more than useful to me. And I wonder if you can talk about that. I'll try to describe what we did, or actually more like what we didn't do. And you can tell me how this is the wrong version of what you're describing.

 

Okay.

 

So I think monthly or at some cadence, I don't know if it was monthly or every six weeks or whatever, we'd get together all the security ambassadors and we would talk about security things. Maybe somebody give a presentation, OK, here about password hashing or I don't know, whatever the thing was. And then we were just supposed to be aware in our problem domains. of security things and bring them up, or make sure that the developers we worked with thought about security things. And I don't know, my feeling was that this was incredibly ineffective, because there was no framework within which to decide if something was security related, and even if it was, how you should respond to it. It just kind of assumed that by giving you a label and having a monthly meeting about security, somehow through osmosis, our software is gonna become more secure. I don't think that's what you're talking about, but maybe you can use that as a contrast for what you are talking about and what we should do.

 

Yes, and I think this is actually a good point. We tried that out as well. I'll give you my thoughts here. Back in Finastro, what we did is that we created the channel for all of the, as you call, ambassadors. We tried to make at least quarterly education session. The thing is that we set a poll on that channel and we asked, what is the next thing you want to learn about? And based on that, we got the interest from the team on what is the most interesting thing for them. In some cases, it was a generic topic, like as you mentioned, hashing passwords. And in other cases,

 

sure.

 

it was, hey, I just want to learn from other teams what they are doing, because

 

Mm-hmm.

 

it's such a big company. When I was there, we were about 10,000 employees. So it's such a big company that. You don't know what others are working on. You don't know what practices they have, what problems they're trying to face or trying to solve right now. And that was also a subset of those calls. Now, did that make the software more secure? I don't know.

 

Mm-hmm.

 

At the end of the day, when you learn coding, what you actually need to learn is how to Google, right?

 

Yeah.

 

Or how to Google. Right? You

 

Or

 

learn

 

nowadays

 

the basics.

 

how to chat GPT.

 

Yeah.

 

Ha ha ha

 

Oh, you know, you go to get to build an e-commerce website for you. Right. But the idea is that you always need to figure out a way to put that in people's mind. It can be whatever cadence on a session with everyone. It can be newsletters and it can also be, hey, You have your tool, you have your results, let's work on those issues and help you fix those. So it can be as hands-on as possible, it can also be as, I'll just grab a coffee or we'll grab all of us a beer and just have this conversation. So I wish I could say that it's not successful, but the reality is I don't know. I just

 

Yeah.

 

know that the bottom line is that when I started, we had quite... big number of high severity vulnerabilities in our default branches. And when I left, we had, we fixed about 97 percent of those.

 

Oh, so at least you had some measurement. That's something we didn't have

 

Yes.

 

in my example.

 

Yeah,

 

Yeah

 

so

 

We

 

it's.

 

had no baseline against which to measure success or failure. It was just, we're going to try something and see what happens.

 

Yeah, so we had data behind that, but the thing is that that was also a thing I kind of wrote at the beginning. I didn't have all of the data, so I wrote a, I don't know code, if Python is a code or a script, but I wrote some Python code that actually goes to all of our tools and grabs all of the information. So I actually managed to map between a product, which is a business context, and what is that scan? that actually runs in a specific repo. And when I did that mapping, I started exposing that information on our Power BI. It was like as simple as that. And I grabbed that snapshot every day of how many vulnerabilities we had, what are the products and such, and I started building views. So I broke it down to initially lines of business that we had in the company. Then the general managers that saw that, and obviously they saw so many vulnerabilities, they started asking questions. Okay, so can I break that per product? So I built a view for their direct reports that led product teams. Now, at some point, the product teams wanted to know what to fix. They had their findings in the tools, but they wanted their individual contributors and security champions. to log in into that Power BI and actually see what are their specific items. So we built a view for them. Think about multiple tabs in the same dashboard, but just different details in each dashboard. So if a GM could care about the top number, the developer would care about the, give me the link where the vulnerability is, right?

 

Right.

 

And then at some point, we built even multiple metrics between which vulnerabilities exist and which teams and how do they do that and kind of help collaborate between the teams. We built SLAs since vulnerabilities were open till they were closed. We built a lot of that data into the same dashboard. And up to the point where I left, we started having actually a conversation about how do we expose information to the CISO teams and to the internal audit teams? Because as you can imagine, financial services company, internal audit is always looking into what we are doing and where we can improve.

 

Well, you would hope

 

So

 

so anyways.

 

instead of... Oh, come on, I'm trying to be gentle.

 

Yeah

 

But technically I just wanted to give them something. to stop wasting time on questions that I already have the answers for and I can present them. So that's kinda how I looked at that from a, you definitely need to push the data to the relevant audience, and you need to make it actionable. Each audience needs to know what is the level of detail they can access to fix a problem. One interesting thing, by the way, in the general manager's dashboard, we actually had like the, how many total vulnerabilities you have, how big is the impact that you did since the last meeting? Typically, I presented every other week to the general managers. And it was like a wall of shame of like, the GM that did the less... the least progress was always on the top. And

 

Yeah

 

it ended up being like a competition. unintentional of course, but it ended up very well because, you know, it's very hard to cut about 97% of all of the issues that you had without

 

Yeah, no,

 

that

 

to

 

type

 

make

 

of

 

that kind

 

a...

 

of progress, you have to have everyone involved in it. And I think that's actually a cool way of doing it, of creating a leaderboard, because whether we like to admit it or not, I think most of us are motivated by some level of competition.

 

Exactly, exactly. So that's kind of what we did. And it's, as far as I know, it's still maintained in the company. So it's a good view that makes sure that not only you got that sprint through, you know, mitigating risk, which that sprint was technically a bit over a year. But now you know that you can continue maintaining that state.

 

Yeah, for sure.

 

Yeah. And that also reminds me of a point, you know, and we also had that conversation, you know, right before we got into this, that coverage, the ability to see everything with a context and actually be able to shift the message to the right person is... It's a very hard problem to solve. You can't have access to old scans. You can't have access to old repos. You can do whatever you want. But at the end of the day, when you're trying to be effective in fixing problems, it needs to be right there. It needs to be in the right context, even in your mind. So think about it this way. Let's say you just wrote a piece of code. And it has a vulnerability in it, a new vulnerability that you just introduced. I like the concept that is something I just like to call zero new high severity defects, which means whatever I have in the backlog, we'll just go and fix in our own pace as needed.

 

Mm-hmm.

 

But I want to prevent a situation where there is new high severity vulnerabilities in the system. And there are a few ways to do that. You can run certain scans on the IDE, right, on the developer side, but then with the different IDs and different versions of IDEs and the inability to push extensions, like what is the chances you'll get 100% coverage?

 

Yeah, it seems pretty unlikely. Yeah.

 

Zero? Zero-ish.

 

Zero is a number.

 

But then you can also put controls in something like a check, like a GitHub check or something that runs a quick scan.

 

Mm-hmm.

 

Well, there are caveats on that, of course, it needs to be quick. And

 

Yeah.

 

the thing is that you never know whether the developers that have just pushed code from their IDE will go back to, let's say, wherever their GitHub and check whether the check passed. They will likely check it when they try to create a pull request, right?

 

Right.

 

Because that typically from, I mean, they can do it from IDE today, but it's typically on the web, you get all of the details there.

 

Great.

 

You will have your automated tests, you'll have your security tests before you can merge. And it's all great, but for example, one of the things that I've been struggling with in the past is, is that a security scan can take, you know, a day, day and a half. Depends what is like a mono repo or thing or things like that. How do you handle that? You just open a PR. And it falls because of security a day after.

 

Yeah, and then you push what you think is a change to it and then wait another day.

 

Exactly. Exactly. Which is why I get that impression that Scanning when you just start a pull request is good, but it may be too late in some cases.

 

Great.

 

Not too late in terms of, hey, you pose a risk, but maybe too late in the fact that you can deliver code slower, or maybe it will frustrate the developer that it's not, like I worked on that code in the last sprint, and now I opened the PR, I'm already working on something else, and now you bring me back that context. Why?

 

Right. Right.

 

Right? On the flip side, would you see that it's very typical for developers to commit code at the end of the day? It's like a save button.

 

Mm-hmm.

 

Or multiple times a day. And if you could, let's say, run a check the moment the developer pushes code. that would be more efficient. So this is pretty much where, it's something that I kinda came up with, but I don't want it to be the thing that Nier came up with. I want it to be more of a term that is a concept of how you can do things better. And I just call it pipeline-less security, like agent-less security, server-less, pipeline-less security. And the idea is that.

 

I like that name because it really goes along with the theme I see a lot of startups that I work with called security less security.

 

I'm sick. Hahaha.

 

Hahaha

 

Yeah, there's also a good concept. Let's talk about blameless security. That can also work.

 

That's a good one, yeah.

 

Yeah. So the thing in pipeline-less is that, as opposed to, if you look at each of those areas of where you can integrate scans and maybe feedback loops, think about it, maybe the first one. IDEs. So we know you're not going to have 100% coverage, right? But also if you do have 100% coverage from within a magic way, you run a script and everyone have it on their environment, something like Git hooks that is very typical to have like linters or anything like that that can check things like for you. Well, in that case, Git hooks can be bypassed. right? And in that case, you're also getting to the point where you can't run too heavy scans, but things that shouldn't be bypassed can be bypassed. Let me give an example. Maybe like you have one of the open source secret scanning solutions, you know, like a Git leaks. You run that, you found secret, you want to bypass it. Why? For whatever reason, I don't know. Maybe you just need to, you have the rush and you need to deliver code at the end of the week and you just push it now because they don't have the time to integrate it with your vault.

 

Great.

 

So this is like one problem that coverage and can be bypassed. If you go, let's say, to GitHub checks. On the GitHub checks, there are a few things. One. everyone can see your results, right? Which in that case also, let's say you did push a secret. Should everyone see that you just pushed a secret?

 

Right.

 

Right? Maybe if it's a vulnerability, it's fine. If someone sees that, it can be on your feature branch. But a secret is a damn secret. It can be used somewhere else. Right?

 

Right.

 

It's a different risk type. Or again, with checks, you may have a full coverage. You may not have a full coverage. Again, it depends. But at the end of the day, how can we make sure the developer actually looks at the results? And

 

Right,

 

the answer

 

yeah.

 

is, we can't. On the flip side, I'll just give you a hint about pipeline loss. Let's say that you did identify something like that. You did identify a secret. Wouldn't that be more useful if you could just send a Slack message to the developer and say, hey, well, you just pushed a secret. FYI. I'm going to revert your code right now.

 

Oh yeah.

 

As simple

 

That's

 

as that.

 

pretty slick.

 

Self-promo, we do that at Arnica. Closing.

 

Hahaha

 

No, but what's the value of reverting a secret?

 

Well, the thing is that the challenge with secrets in source code, with hard-coded secrets in source code, is that anyone that has access to that repo and actually clone that repo have that secret in the Git history.

 

Right, exactly.

 

So if you actually, what you can do in Git is, Git is all about pointers. what you can do in Git is that you can take, let's say you just pushed and we have the head commit on that push. What you can do is you can say, you know what? From now on, that branch is not at that head, it's at the head of the pre-push state, which means that you don't have a way to get to that secret. So you technically deleted the secret. You don't have a reference to it.

 

But then if anybody's pulled down your latest master with the secret in it, then they're gonna be

 

If it's in

 

in

 

master,

 

a confused

 

it's already

 

state.

 

too late. I'm talking about as early as you work on your feature brand.

 

Okay, yeah, right, okay,

 

I mean,

 

sure,

 

historical, it's

 

sure.

 

again the same difference between historical versus zero new defects. If it's historical, you need to rotate the key no matter what.

 

Right.

 

Actually,

 

That was my thought.

 

there

 

Almost,

 

is a matter.

 

no better way.

 

There is a matter. Let me correct myself. If you have a repo and you are the only one that has access to that repo, maybe it's not a big thing.

 

Although, once you push it to GitHub, you're not the only one with access anymore.

 

all

 

Technically

 

depends on

 

speaking.

 

the private or not.

 

Yeah.

 

But, yeah, I mean...

 

Well, I would say even if it's private, someone has access to it. It's not

 

Well,

 

encrypted

 

if

 

at

 

you

 

rest.

 

are the only one, right? If you are the only one with access to it, it's not that big of a deal.

 

Yeah, my point is that GitHub has access to it. Someone there

 

Yes.

 

has access

 

Yeah.

 

to it.

 

Well, if you put your code on the public GitHub, you trust GitHub,

 

Right.

 

which

 

Well,

 

is,

 

perhaps yes.

 

perhaps. They are investing

 

There's different

 

quite

 

levels

 

a lot

 

of

 

in security.

 

trust.

 

What

 

Microsoft

 

is that?

 

would never let me down.

 

Yeah, but my expectations are low enough. They can't possibly go lower than

 

Hahaha

 

that.

 

Hahahaha

 

Hopefully Microsoft's not the sponsor of this episode.

 

No, no, no, no, no, no, no.

 

This episode brought to you by GitLab, apparently.

 

Yeah.

 

Well, yeah.

 

So I have a good friend who used to work at, I won't even say the name of the company, I don't think it's a secret, but he worked at a big well-known company and they accidentally discovered they had pushed secrets to a private Git repository. I don't know if it was hosted on GitHub or even their own internal Git thing, but they sorted it around table internally and decided. What do we do? Like we don't believe anybody has access to this. It's a private repo. But if somehow this ever got known somewhere, this could be serious. So they decided to do a key rotation, which was a big deal, because I don't remember what these keys were doing. Even in that case where they had no evidence that their keys had or could be leaked. So it's not a matter per se of. Do we trust GitHub or even our internal data team that's managing this? It's just, sometimes you wanna be extra cautious anyway.

 

Correct. I mean, I agree with you. You need to be extra cautious. I think my reference was mainly around the altitude of the risk,

 

Exactly. Yeah.

 

right? If only GitHub can access your secret, maybe probability is a bit lower than, you know, the rest of their company has

 

Definitely,

 

read access to the repo

 

definitely,

 

and

 

yeah.

 

actually have it in the audit trail that they clone the repo, right? But I agree. I mean, at the end of the day, you want to have that context. that tells you whether someone has access, or at least has the key on their machines somehow. So that context matters. And therefore, based to the previous question, reverting that push, reverting that code push where you do have a secret, it's a piece of the solution. Because if you only revert and you don't... provide a solution to the developers to actually push their code, you're harming their development velocity.

 

Mm-hmm.

 

It's annoying. The thing at least that we came up with is that we do that, and depends on policy of course, but we actually revert your code to the pre-push state, but we also create a feature branch to the feature branch of the developer. And therefore, in the new feature branch that we created, we replay all of the commits except where you have the secret where we just mask it. So there is no secret.

 

Oh wow.

 

So now there's no blame to anyone. You don't have any results in the checks. And the developer, in our case, the way that we came up with this is that we actually send a message to the developer on Slack. And we say, hey, Will, you just pushed a secret. We created a new branch for you. Click here so you won't lose your changes. And that's it. And you're done. And we just merge the branches. And this is by the way, one of our patents. But

 

Okay, nice.

 

it's a very elegant solution to make sure the developers are not pissed by security controls that you put in front of them. Right?

 

Yeah, absolutely, because you're identifying the problem but giving them a path forward, which I think is one of the pitfalls that we've fallen into in the past is falling down this pattern of, oh, this is a security violation. Here's a hard stop at this wall until you resolve it.

 

Exactly. So you can resolve it for them. And it's not in 100% of the cases, but there are very few, I wouldn't call it low-hanging fruits, but maybe things that make sense to developers. Secrets

 

Yeah.

 

make sense. Maybe CVEs in your third-party libraries make sense to some extent. But if you just introduce a new whatever remote code execution, uh... vulnerability more difficult to fix i mean there are fixes but you really need to think about what would be the right solution in your context in your code

 

Right, yeah.

 

Right? See, there's no one size fits all, but if you can at least make a little bit of their life easier, you know, you're getting to the right...

 

Yeah, I agree. I think that's a pretty cool approach to it.

 

And thanks. And by the way, and this is only like the two of the things that I mentioned, right? I mentioned IDEs, I mentioned checks. What happens in pipelines? Well, a pipeline is too late, right? A pipeline

 

Yeah.

 

is already too late. If you detect a secret, it's too late. If you run vulnerability scans, again, you may have that feedback that just takes too long to get it back. So therefore, What you really want to do is to go to, what I like to call it, a pipeline-less approach. As things change, as events coming in, you can react and feed the developer with private messages instead of blaming it all over. Bye.

 

Nice.

 

private message cracks me up. I love the approach. I think that's such a cool concept. But

 

Thanks for watching!

 

the inner troll in me is like, here's the private message, respond within five minutes, we broadcast it to general.

 

Ha ha ha

Hahaha! We can

 

Alright!

 

I'm thinking how do you turn this into some sort of like a bounty system like, you know, fix this within five minutes Or maybe better send me a hundred bucks or I'm gonna tell everybody about this

 

Yeah

 

Send me one bitcoin.

 

There you go.

 

Yeah, right. Ha ha ha.

 

It's the it's the ransomware bot.

 

There you go.

 

Yes!

 

There you go. What's your revenue model? We exploit developers.

 

Right? Ha ha ha.

 

And we want ads in their chat.

 

Yeah,

 

Hahahaha

 

and the developers who get three strikes, you know, we just, we start a social media campaign against them so they'll never be hired anywhere.

 

Hahaha! I think we've got

 

Although

 

a new

 

This,

 

it can

 

business

 

ladies

 

go faster.

 

and

 

model.

 

gentlemen, are things you should never do. Adventures in Miss DevOps today.

 

Dirty. Ha ha ha.

 

Yeah.

 

So going back to talking about the amount of time it takes on the pipeline scan. So do you advocate, if you implement the pipeline list approach, do you advocate just ditching the scans altogether in the pipeline or do you have like a modified version of them for sanity?

 

So I think as long as you can build a security pipeline that is parallel to the one that is related to DevOps, there are certain things you can do in security, but not all of it. So in pipeline, examples of things that you can do is as I mentioned, it's very simple static checks. It can be secret scans, it can be linters, it can be, it can also be as... complex as identify whether the developer wrote anomalous code to that developer itself. Like, think about an account takeover. You can do these things. I don't know if you can or cannot, but what I would question whether it can be done in a pipeline less is something like a dynamic scan. Let's say that you have a pipeline that runs CI tests.

 

Mm-hmm.

 

Well, in that case, you do want that to be in the context of the CI tests.

 

Yeah.

 

Right? So, so as long as it's static and it's quick, that's the good fit for, for pipeline less. Um, dynamic stuff or dynamic scans, it's something that I haven't rubbed my head around that yet.

 

Right, yeah, because in that instance, you actually, many times, you need the compiled, running version of the product in order to execute that.

 

Exactly, exactly. And I mean, hypothetically, you can build by yourself. You can run a new pipeline that builds things by yourself. But you need to make sure, and this is where the devil's in the details is, you need to make sure that you don't deploy twice.

 

Yeah. Yeah.

 

And so there's a lot of details on how would you not do that.

 

Right.

 

I don't know how would you do that yet. Yeah, it's an interesting problem to solve from a dynamic standpoint. But reality is that if you look at, I just read one of the reports that Gartner just wrote about the adoption of security tools. And static code analysis is the most common tool that companies use. And that is something that you can technically take into a pipeline-less approach. You need certain capabilities with a tool like that. For example, you need the ability to scan individual files or only diffs instead of scanning the entire project because you need quick feedback.

 

Right, yeah.

 

Or maybe you should be able to execute a scan without a build because the build takes time. So there's a lot of considerations that come into this, but think about a product like, maybe something like a SemGrab. It's an open source, you can run your rules and identify vulnerabilities based on very basic grep rules. And that is a great example for something that can come up in a pipeline-less approach.

 

Turn on.

 

Same with third party libraries. Let's say that every time that you push code, then you change your package JSON file, run a scan on that and tool like a trivium, you'll find the vulnerabilities.

 

It's pretty much guaranteed if you change package.json, you're going to find new vulnerabilities.

 

Uh, likely. Likely.

 

job security

 

Even if...

 

right there.

 

Exactly. What are your thoughts about where security is typically integrated and whether there are other areas where it can be improved?

 

Well, I have one sort of general, I guess, thought about that. We have a lot of good tools that do scans, you know, scanning vulnerable libraries, secret detection, all these things are fairly easy to detect and automate. And that's great. But I'm sure you'll agree that's not enough. There's always security holes that can be implemented without that a tool can't detect, or at least can't detect before we've thought about how to detect them, right? And so, I still think that there's a big gap in knowledge and awareness. And unfortunately, it's one of those things that I don't think is taught well in software development schools. You know, I guess there's a branch of computer science that is all about security. But it's not at the forefront of most people's education. So if you go to a software bootcamp or something like that, you go to a Ruby bootcamp or whatever, you're probably gonna get almost nothing about that. I mean, you might learn that you should hash passwords and store credit cards encrypted, or don't store them at all. But that's kind of the end of the security chapter. in most of those lectures. And people aren't thinking about things like, I mean, some of the really common things I see frequently are people not considering sequential IDs as a potential security risk, right? I mean, that's gotten some more attention in the last several years. You know, a lot of databases give you UUIDs now if you want to use them. And some people now abuse that. They think, oh, you have to use the UUID for everything now, which just... kills performance unnecessarily many times. So I see some of these things get some attention, but in general, I've had countless conversations with developers who've been doing development for five years or 10 years about some of these security concerns. Like, oh, I never thought of that before. And I believe you, I understand that. I'm not blaming you, but this is what security is. And... I don't know the answer other than education, and I don't know how to educate billions of people, millions at least, with this stuff when we're still trying to teach them how to write unit tests.

 

Hahaha

 

Oh man, that too. Okay.

 

Now, I would agree with Jonathan. I think education, and even for myself, I've been doing this for a long time, but still consider myself largely uneducated in the world of security because you have

 

Yeah, me too.

 

the static testing and the dynamic testing, and then there's all the different exploits. There's all these different attack vectors that you have to consider that I feel. really confident about it if I remember what the acronym stands for, but then my knowledge sort of cascades exponentially from there. So it's definitely an education problem. I think one thing that might help with that is we all have pretty much standardized on Git as our repository. So it seems like there's a path forward in doing a template where you when you type get new that it stubs out some kind of security framework in there so that you at least have an idea of what you should be looking for.

 

That's a good idea, actually. I mean, there are some guides that you can find about how to write secure code. Actually, there's plenty of those, but no one that opens up a new project on Git will say, you know what, that's the first thing I do after Git new. Right?

 

Yeah, and context is hard on that too, because you can search for it and find some stuff, but then it's like, wait, who is this person and what's their relevant experience? Is this person I should really be trusting for security advice or is this someone who did a Google search on trending keywords and found that security was one of them and wrote an article that matched the SEO requirements?

 

Yeah. You know what also...

 

You found my secret.

 

Yeah, right? Well, no, that's the voice of experience. That's how I got

 

Ha

 

here.

 

ha ha. Ha ha ha ha ha ha ha.

 

So interestingly enough, we are using Copilot in the company. All of our developers absolutely love it. It saves time. It's a productivity tool for everyone. Challenges that in a few times that they tried to implement their code, it takes it based on models.

 

Mm-hmm.

 

It's very much that you can do. to build the right model, but then you can't really ensure that the model that you built to suggest that code is actually secure, right? You need different models for that.

 

Yeah.

 

And in many cases, we got a code that the developers look at that and they say, oh, you know what? This is, it doesn't seem right. It doesn't seem like this is a secure way to do that. And... It's really interesting that there are tools that actually help you to develop code. But it would be amazing if those tools could also, while you write code, could just suggest to you a secure way to do that.

 

Yeah.

 

That's great, although I still think there are times when that's not perhaps not possible, certainly not feasible. And I'll just give a simple example from a client I was working with not long ago. Logging. Logging is a huge security vulnerability in many cases, and you need to do it. And it's not obvious to a static analysis tool what fields should be masked or encrypted or whatever. So, you know, when you're logging something, I mean, so in this particular case, we were logging sensitive customer data in many cases, or dumping entire REST requests, you know, and that's great for debugging. You know, you try to debug something,

 

I'm sorry.

 

of course you want to see the entire request. Oh, but that includes customer name and data and who knows what else, customer credit card numbers and, you know, their blood type and their children's birthdays and everything.

 

You know, so interestingly enough, I had the same problem in the past, and one of the things that we looked at was sending that through a sanitation service, because developers, again, they don't know what will hit there, right? It's just a logging, right? So we thought back then about Google DLP that have that service that you can actually send a stream to it, and it will return to you a sanitized stream. and that is the thing that you would log. So that is kind of it.

 

I've seen those tools. I don't think they're foolproof. I mean, I've yet

 

We're

 

to

 

not.

 

see one that is. You either are gonna have, I mean, you probably have both false positives and false negatives in almost all cases. It's better than nothing, no doubt. I mean, I'd rather have that than nothing as a customer of this company that might be logging my personal data. But I don't think it's gonna protect you legally if you ever find yourself in legal trouble. And it's certainly not technically the ideal solution.

 

Yeah, I don't know if it would protect you legally. The reason why I'm thinking about whether it will or will not is because at the end of the day, a breach is a breach. Period. That's a fact. On the flip side, when you are being investigated in a case like that, there's also a question of what did you actually implement to minimize that risk?

 

Mm-hmm.

 

And

 

Sure.

 

if you do have something that you did implement to minimize that risk, and it's maybe just a mistake of a false positive, the outcome is different than the one that you weren't careless enough or careful enough. Well, careless enough is a bad problem, but

 

Ha ha ha ha!

 

careful enough about minimizing that risk. All

 

Sure.

 

right. So it's a... This is why I don't know how much would that impact from that standpoint. But if it's, let's say you log something that is false positive every time and you end up with a million of customer records and the log and someone actually accessed it, yeah, it's pretty bad. It's pretty bad. Yeah.

 

Yep.

 

Anyways, I'm really curious to see where all of that help developers write secure code goes. I'm just really intrigued by initiatives that are driven by AI to help developers. Security is just one of them.

 

Yeah, right. Awesome.

 

Cool, well we are coming up on an hour here. Is there any other topics we've left uncovered?

 

Um, I don't know. I can, we can talk about Bob Sponge and security.

 

Right on.

 

I... I...

 

Cool. Jonathan, any other questions?

 

No, I don't think so. It's been a good talk. I guess I could ask one more question and I don't know if there's a good answer to this, but I'm asking for a silver bullet almost, but how can a company, because I mentioned earlier that I work with a lot of companies that don't have security or almost none. I tend to work with small companies and often their attitude is, by the time we're big enough to care about this, we'll have more resources and we'll have time to. to do this, which is a kind of, I mean, I understand, but it's also foolish, right? What can small companies, especially early stage startups, what can they do that isn't gonna drive them insane to start to improve their security? Where do you get started? So, I'm gonna start with a question. I'm gonna ask you this, and I'm gonna ask you this,

 

So there's a challenge with the balance of, hey, let's use freemium tools versus the effort to actually integrate them and extract the results and actually act upon them. So open source is great for the reputation of the company that actually releases the open source, but smaller companies typically don't have the resources to actually run it at the need, the... they have, right? Which is, by the way, the reason why I'm a big believer in what we do at Arnica. We have secret scanning and permission analysis and anomaly detection and scanning your vulnerabilities and code and SBOM and essentially all of the basics that you need to have, plus plus, when you just kick off a company. And the thing is that I'm a big believer in... visibility that must be free. If you want to see stuff, you don't need to pay for that. If you want to fix stuff, that's how you need to start investing your own resources into fixing. And therefore, what we did, for example, with Arnica is that we said, you know what? Hook Arnica to your GitHub, get all of that visibility for free. Your secrets, your permissions, and S-bomb, and static code analysis, all free. for unlimited users forever. You want to start fixing stuff when it becomes a problem for you. Then you'll be able to pay when it becomes a big problem, because it's a good problem for you to have to actually pay for that.

 

Right.

 

And therefore, this is how I see that balance. So I'd say, taking open source is great. If you don't have the resources to take an open source, take a product that has a valuable freemium. not necessarily ironica, like for any need that you have. Take any product that has a valuable freemium for unlimited time preferably, if you can, and use that as much as you can. And as you grow and as you need to start automating, you will figure out to pay for a product that will help you to do that. Or even. pay for an internal resource to take those inputs and develop automation by himself or herself. That's the way that I would probably tackle that.

 

Awesome. Great answer. Thanks.

 

Right on. So let's move on to picks. You got a pick for us this week, Jonathan?

 

I do. I just started listening this morning on my morning walk to a new book, and I'm enjoying it so far. And I enjoy historical books. They're especially on audio book. I like the narratives, but I don't, I feel like I'm wasting my time when I listen to fiction. I know that's a controversial view, but I feel like there's so much non-fiction I could be learning about. Why would I waste my time listening? to fiction when I could just watch that on TV and waste even more time.

 

Right?

 

So I know it's not really logical, but anyway, I'm listening to the book called George Marshall Defender of the Republic by David L. Roll, R-O-L-L. And as you can tell from the title, it's about George Marshall, who was a famous general and... I guess he was a general. I haven't gotten that part of the book yet. He's still a lieutenant

 

Hahahaha

 

or something.

 

Ha ha ha!

 

In the US Army, he was instrumental in World Wars I and II and the one other, the Korean War maybe. So I was turned on to him actually while reading my pick from a few weeks ago, A World Without Email by Cal Newport. and he uses George Marshall as an example of a manager who doesn't use a manager's schedule. We've probably all heard the manager schedule, maker schedule dichotomy, and he uses George Marshall as an example of a manager who uses a maker's schedule. He spends a lot more time thinking about what he's doing than living by calendar. So I thought that's interesting. I want to read more about George Marshall. So that's why I'm reading the book. So far it seems good, entertaining. Yeah, so that's my pick for the week.

 

right on. Nir, you got any picks for us this week?

 

Yeah, I can give you a pick about an audio book as well. I listen to audio books typically when I run, jog in the morning. One of the books that actually changed a bit of my mindset is actually a business book, it's called Good to Great. And it's really a corporate book, but given that I'm a CEO, I wasn't a CEO in the past, I'm kind of trying to get into places where I learn a bit more about leadership and maybe there are other areas that I can learn about. And it's really interesting to see how companies convert from being whatever brand they were into whatever they are today. I'll give you an example of something that I saw in the book. Did you know that Walgreens had a chain of restaurants?

 

I had no idea.

 

Exactly, exactly.

 

They used to do like soda bars and stuff, right?

 

Yeah, it's bars, restaurants. They had

 

OK.

 

a few hundreds of those. They weren't generating revenue as they wanted. And what the CEO had to do is really to make that hard decision. You either die or you cut your arm. That's pretty much what you need to do. So he had to cut an arm. And they took a different strategy. They said, you know what? Let's double down on pharmacy. And what they did, they said, you know what? We'll put a pharmacy in a. in the cross between multiple streets. So if you look at Walgreens, think about any branch that you were in, most of them are in the cross of two streets, right? So they took a strategy that says, you know, we'll attract more people, we'll double down in the right way, because that is something that we see that is working. And then, I mean, even in San Francisco, you can find, you know, a few branches in the radius of a mile. Right. And

 

Yeah.

 

that's because you build a brand that is familiarized with something that clicks for everyone, things that you did in the past, maybe last matter, you need to just separate from them. And when you do the things, you do them the best way you can, not only in the product and that experience, but in the brand that you build and in cross between streets, everyone sees you. So that's kind of really interesting strategies that it was good to read about.

 

Right on. Cool, so my pick for the week. I've actually picked this one before. I'm repicking it again. And, Near, I think you'll like it, given what you were just talking about. It's a book from Jaco Willink, Leadership Strategy and Tactics. And if you're not familiar with Jaco, he was a commander in the Navy SEALs. And his book, This book on leadership strategy and tactics has just been really cool and really insightful for me because of the different ways he gives you to think about and solve problems. And then if you're into the audiobook thing, he actually narrates the audiobook himself. And his voice is one of those voices that is just unquestioning. So whatever he says in the audiobook, you're like, oh, you're damn right. I'm going to do that. I'm going to do it right now. So that's been entertaining from that perspective.

 

So he has my wife's voice?

 

Alright! For sure. And on that note, I think we've got ourselves an episode.

 

I'm

 

Wait,

 

gonna go.

 

wrap it up, Jonathan.

 

always there to help.

 

Yeah. All right, cool. Well, Nir, thank you so much for coming on the show. It's been a great conversation. And it's actually been pretty insightful for me. I've got a whole page of notes here that I wrote down from our chat today.

 

Thank you. Thanks guys. Thanks Jonathan. Well, it was really pleasure. I had fun.

 

Thanks for coming on.

 

All right, cool. And we will see everyone next week.

 

Adios.

 

And still says it's stopping.