Distilled Security Podcast

menu close

Episode 30: Exec Conference Takeaways, PCI & AICPA on Auditing AI & the PHP Apocalypse

/ 01:59:18/E30
Justin Leapline:

Today on the episode, Rick will be talking about some of his recent conference activity when PCI and AICPA speak up with AI being involved and the current PHP apocalypse. This is Distilled Security Podcast.

Rick Yocum:

Somewhere right now, someone is digging through folders trying to find the right version of an evidence file for the third time this week. Controls scattered across a dozen system, owners who left the company months ago, due dates that already passed, nobody noticed. It's how it's always been done, and somehow we just accept it. There is a modern, better way. Episkey.

Rick Yocum:

Visit us at episkey.com/tsp for a special offer.

Speaker 2:

If you're leading a company today, you're not short on ideas. You're buried under them. More tools, more frameworks, more initiatives that were supposed to help and somehow made everything harder. Minus Partners exists for leaders who don't need another thing added. Minus Partners helps leadership teams remove friction that's been normalized over time.

Speaker 2:

So execution gets easier, not harder. Less noise, clear decisions, faster execution. Minus Partners, what you remove defines you.

Justin Leapline:

Alright. Welcome back to Distilled Security Podcast. Today, we're gonna start off here with a little bit of conference talk and not talk about Tris, and that'll be tomorrow

Rick Yocum:

That is tomorrow. That we're gonna

Justin Leapline:

be going to. You won't. In the past. You listeners. It'll be in the past.

Justin Leapline:

So we've had a blast at Tris, you know, at this. But, actually, Rick has gone on for the last couple of weeks. Yeah. You've been going to some conferences and everything. So why don't you share a little bit?

Justin Leapline:

These are kinda out west. Right?

Rick Yocum:

Yeah. Both of them were. So two weeks ago, I was at a Palo Alto Conference Okay. In California. And then last week, I was at a CDW one.

Rick Yocum:

And so these were sort of invite led conferences where it was a bunch of CIOs and CSOs. Oh, nice. And for both of them, there's basically like, half a day of talks the first day, and then some social stuff at night, and then a full day of talks the second day, and then they give you some breakfast and kick you out pretty quickly day type thing. Gotcha. But they were...

Rick Yocum:

Now...

Justin Leapline:

Yeah. I think we talked about this last year too. Wasn't this the one that Edward Norton came to and talked? Okay. Yeah.

Justin Leapline:

Yeah. Yeah. Did you have another superstar? Well, there were...

Rick Yocum:

Not like that. There were...

Joe Wynn:

The so The first rule of these conferences,

Justin Leapline:

we don't

Rick Yocum:

talk about these topics. Chatham House rules stuff, but

Justin Leapline:

There's more Brad Brad Pitt came to sign. Were some there

Rick Yocum:

some were some very well known sports individuals there.

Justin Leapline:

Oh, okay. Gotcha.

Rick Yocum:

So So that was cool. But there wasn't like any any movie or TV show

Justin Leapline:

personalities. Gotcha.

Rick Yocum:

Yeah. Yeah. But Cool. It was... Yeah.

Rick Yocum:

It was really, really good. I mean, they were fantastic hosts. It was a lovely thing, but the content was out of this world. And I got to talk to, like, a ton of big company, like, CIOs and CSOs about what's going on, how are you doing, what... Here's what I'm up to.

Rick Yocum:

So we talked about a ton of stuff. Mostly, like, 85%

Justin Leapline:

We're still talking about Palo. We're not talking... Okay.

Rick Yocum:

We're talking about the same. Yeah. So CDW, I mean, similar format. Okay. That was in Arizona.

Justin Leapline:

Okay. Got it.

Rick Yocum:

Yeah. But but pretty much the same kinda schedule roughly. I think the Palo one had a little under a 150 executives, and the CW one had around a 100, I think. Okay. Something like that.

Rick Yocum:

So It's

Justin Leapline:

pretty intimate. Yeah. They were both

Rick Yocum:

they were both, on the small side in terms of, like, total people, which was

Justin Leapline:

Yeah. Fantastic.

Rick Yocum:

You get to have, like, a bunch of good conversations. How was the venue? Both venues were exceptional. I'd imagine. I...

Rick Yocum:

The the... I... What I've said to a couple people is, like, the quality of the content at both of them, I would I would travel to a Motel six in the middle of nowhere for some of these talks, but it was also very lovely being on super nice resorts and getting to hang out with people there.

Joe Wynn:

So besides AI, which they probably didn't talk about at all Right. What was the main themes?

Rick Yocum:

Yeah. It was, like, 85% AI, like, 10% quantum

Joe Wynn:

Oh, really?

Rick Yocum:

And 5% Mystic.

Joe Wynn:

What was the 5%?

Rick Yocum:

Miscellaneous. Okay. Like random... Not really random, but other security nerd

Justin Leapline:

Other stuff.

Rick Yocum:

Yeah. Yeah. Yeah. Got it. Yeah.

Rick Yocum:

So I, I did pull some notes, and as a memory aid, I'll I'll read some of them. But, so... But like you said, it was like tons of AI, tons and tons of AI. Yeah. So, I...

Rick Yocum:

So it's like five categories of quick notes,

Justin Leapline:

and I'll just rapid fire them,

Rick Yocum:

but inter... Interrupt me whenever, basically. One was essentially on pace of AI, which is interesting.

Justin Leapline:

Oh, yeah. I didn't... We just talked about that. Well... Last week?

Rick Yocum:

Two weeks ago. We talked about pacing a

Justin Leapline:

little Pacing. Right. That's what I'm talking about. Yeah.

Rick Yocum:

Two different... But on the

Justin Leapline:

pace of the speed of

Rick Yocum:

the improvement. Oh, gotcha. So one of the big topics Is

Justin Leapline:

it more about Moore's Law where it's just going exponentially?

Rick Yocum:

Yeah. Exactly. Exactly. And so one thing that I didn't know, I did go back and read it and it's super interesting, but, some of the anthropic founders, data scientists, put out this thing called scaling laws. And essentially, I'm gonna butcher it a little bit, but more or less, the scale of AI just is really almost directly tied to raw compute and hardware power more than anything else.

Rick Yocum:

It's not really the data that you feed is all... Like, those things are all important, but the limiting factor

Justin Leapline:

to chips to power to Yeah, all that stuff and everything. Yeah. And try to look at where bottlenecks and Yeah. What they need to build out in

Rick Yocum:

the future. But their modeling of it has been like pretty spot on. So if you're interested in like how fast is it now versus how fast will it be in six months or whatever, open up that... Go find that paper scaling laws and give it a read and place our... Place yourself in the curve, and there's some cool stuff there.

Rick Yocum:

So that was a thing. Another thing that Anthropic touched on, which was really, really good, was one of their sort of founding principles as a company. And I I I like the metaphor of it. It's hold, light, and shade.

Justin Leapline:

Okay.

Rick Yocum:

And the the theme there is essentially, yeah, there's like a lot of darkness that can be wrought with these things, but there's a ton of benefit as well. Right. And it's easy from the security side to really focus on the what could go wrongs, But the what could go right

Justin Leapline:

Our brains are kinda wired that way Exactly. Know, type of thing.

Rick Yocum:

But the what could go rights are awesome. And she talked about, as as they think about, this was this was one of the founders. As they think about where where to deploy, how to deploy the nature of the the strength of the models and and what they hold back. Mhmm. She was talking about an email from a a father who's had a kid with, a terminal disease.

Rick Yocum:

And they're saying, oh, the current gen models that I have available to me are giving us some actual like treatment plans that seem to be effective. Can I please have the next gen models that you're holding back to get better treatment plan to to go through this stuff? And she's like, so like, that's touching as a parent.

Justin Leapline:

Yeah. That sounds true.

Rick Yocum:

And and those use cases are wild. And so on the one hand, it's like, oh, yeah. We gotta make sure it doesn't blow up the world. But on the other hand, it's like, oh, but there's a ton of good that could be had here. Yep.

Rick Yocum:

So a lot of conversation was on, the sort of pros and cons balance and Oh, okay. Managing that risk. And I like the optimism side of that too. Yeah. There was...

Justin Leapline:

Because everybody's focusing on the doom and gloom of AI.

Joe Wynn:

Right. Twenty thirty.

Justin Leapline:

Yeah. We're all done. Mhmm. Yep. But, like, looking at this, it'd be like, no.

Justin Leapline:

Our lives are actually materially getting better, and here are cases that prove that.

Rick Yocum:

Yeah. There's a ton of there's a ton of stuff. I mean, like, a lot of the protein folding and biopharma, all that stuff. And then, like... But again, the dark side is like, oh, well, you can make, like, some horrible scary bio weapons.

Rick Yocum:

And so, okay. You got... How do we how do we manage through all that? So there's a lot on that. Pacing Okay.

Rick Yocum:

As you noted, there's a lot of talk on how pacing has a branding issue, And essentially

Joe Wynn:

Remind me what pacing is.

Rick Yocum:

So this is the the concept of the most powerful models, particularly the ones that can do, like, scary IT stuff, and it could be used to go attack infrastructure or Find all the vulnerabilities. Find all the vulnerabilities at once and go do bad stuff with them. The the concept is essentially, to not release those to the public, really, until the defenders, right, the security people can spend some time with those things to to learn how to use those tools before they're released to the public and then bad people are using them for bad things, or they're reverse engineered and then bad people use them. But the branding issue is really a lot of people hear pacing and they... What what they hear and what I initially heard was, oh, let's just chill out on development.

Rick Yocum:

And that's actually not the case. The the the the thought of it is really essentially continue to build, but deploy responsibly Mhmm. Is the intent.

Joe Wynn:

So the pace is like, let's slow this. Let's let's keep a good pace on this. Let's not let the pace succeed. What we wanna do is that what pacing is?

Rick Yocum:

Well, it's the pacing of the release of it, not the development. The development is, like, intentionally still full steam It'll

Joe Wynn:

keep being in their systems, but model release

Justin Leapline:

will be lot more comfortable because, you know, obviously, we're at a a very competitive state with China and everything. Oh, adversarially. Yeah. But I'm also curious on the delta behind the release and the development.

Rick Yocum:

Well, who gets to pick who gets it? Well, yeah.

Justin Leapline:

That Something like is something... But if we're, like, really slow on releasing, could we be, like, five years ahead within Anthropic or some... Or another AI company? Like, they Gotcha. They have a backlog of stuff and defenders aren't keeping up, and so they're like five years ahead in the AI world, which is a whole another Oh, I see.

Justin Leapline:

Of development, you know, type of thing.

Rick Yocum:

I see. There could be like a delta in how Yeah. Like

Justin Leapline:

they're moving. Exactly. It could be nine months, which might not be that long, you know, between the models.

Rick Yocum:

Yeah. That's an interesting thought experiment.

Justin Leapline:

Longer in years, you know, type of thing, depending on, again, who's releasing what at what time Yeah. Type of thing.

Rick Yocum:

Well, and I think what's probably gonna happen is the services by, like, a lot of the big companies are gonna improve to defend against AI badness. Yeah. And I think that's mostly the intent of it. Well, and I think it's it's

Justin Leapline:

not necessarily... The models will discover stuff quicker. We don't even have the infrastructure nor the processes to respond now, you know? Right. You know?

Justin Leapline:

So that's the thing that's like really the bottleneck, I think, with this. Like, all these vulnerabilities that are coming out, you know, with all this stuff, we're still traditional patching. Oh, right.

Rick Yocum:

Oh, exactly. That type

Justin Leapline:

of thing. So it's... So... Yeah. Okay.

Justin Leapline:

Model can discover more scary stuff. We're we're still, like, patching stuff the same way. There's a ton

Rick Yocum:

of conversation about that too, which is sort of like, well, when the bad guys have guns, you need to give the good guys guns. Like No. You might not like guns, but whatever it is, it's an escalation. And so when people are using AI to, like, go attack all the systems or find new vulnerabilities and stuff like that, it's like, Well, they're coming out so fast, you'd need to use AI to patch those vulnerabilities or do...

Joe Wynn:

So a I was just recently listening to was they were talking about how... What the the adversaries are doing. The patch comes out. As soon as the patch comes out, they're actually... Bad guys are reverse engineering using AI to figure out the patch.

Joe Wynn:

So what does that mean we

Rick Yocum:

need to do?

Joe Wynn:

Is we need to... We can. We need to be able to and we should, and I think people are, is going through and using the same AI tools in order to take those patches Mhmm. Reverse engineer them, but start to look for indicators of compromise opportunities, and then start to do what we used to call virtual patching.

Rick Yocum:

Well, that... Mhmm. Exactly right. And there's some services coming out very recently, like within the last couple weeks, that are named various things, but generally frontier virtual patching.

Joe Wynn:

Yeah.

Rick Yocum:

Exactly as you know. Right? Where it's like, well, no, I'm not gonna be able to patch my Oracle environment in, you know, the five minutes that, like, time to exploit begins to happen. But if I have a service provider or an internal team or internal processes in place to identify the IOCs and block bad stuff coming at me that looks like that stuff, then it gives you a little bit of moat that the bad guys have to cross before you actually have to patch patch your walls.

Joe Wynn:

Right. Because you can start to look for what they're gonna use against you and start blocking that. Right?

Rick Yocum:

Yeah. The other thing that was interesting about the vulnerabilities front in the frontier models that I didn't know before was the the big ones right now, like the Day Breaks, the Mythos, that sort of thing. They each find a ton of vulnerabilities. There's only like a 10% overlap between the things they find.

Joe Wynn:

Really?

Rick Yocum:

And that to me was mind blowing. So there are some providers that you're like, oh, we have a frontier model, and it's great. But it's like the ones that have multiple frontier models for, like, virtual patching and stuff like that. You're actually gonna have, at least right now with the current state of tech, and this might change

Justin Leapline:

in

Rick Yocum:

a

Justin Leapline:

Yeah.

Joe Wynn:

In order of magnitude But of better results.

Justin Leapline:

But only 10% is surprising. Only 10% over that. Think they'd all attack it at a path of least resistance, you know, and what's available and all that stuff like any other person, but

Rick Yocum:

it blew my mind. And multiple totally different vendors essentially independently unprompted said something very similar to that at each conference. And so two vendors that have access to things, like, oh, yeah. The overlap is like nothing, which also makes me terrified about the actual state of code generally. Yeah.

Rick Yocum:

Right.

Joe Wynn:

It's it's so much worse than... And and when you're thinking like... And I think we might talk about this later when we're talking about auditing, but nondeterministic processes versus somebody following a deterministic process.

Justin Leapline:

Mhmm.

Joe Wynn:

If these AIs are each making up their own way, who's to say that they're gonna follow the same rule book about how to figure it out?

Justin Leapline:

Yeah. Yeah. That's one thing. I I think I shared with you guys, like, our process at Episkey. We actually have on every PR two, you know, two reviews Mhmm.

Justin Leapline:

By Alright. John's company, AI... What is it? AI guideline, AI code. I'm looking at you.

Joe Wynn:

Oh, do you mean the Xenable stuff?

Justin Leapline:

Yeah. Xenable.

Joe Wynn:

Okay. Yeah.

Justin Leapline:

I forget the

Rick Yocum:

the The service previously known as Xenable. Yeah. Xenable.

Justin Leapline:

Yeah. We have that... I have that review in it and Copilot with GitHub, and they're both looking at the same code, and they'll come up with different

Joe Wynn:

stuff.

Justin Leapline:

Yeah. You know? And some of it will be the same. You know? Be like, oh, yeah.

Justin Leapline:

They identified it type of thing. But I feel like I get better coverage dealing two eyes on the same code.

Rick Yocum:

So it's like a weird model defense in-depth thing.

Justin Leapline:

Yeah. Exactly. Yeah.

Rick Yocum:

Yeah. So so we talked about that. And before you do that Please.

Justin Leapline:

Yeah. One of the things that we were talking about while we're reading the the core notes into this was OpenCode was getting attacked. Oh, yeah. Yeah. Yeah.

Justin Leapline:

And I I thought the the guy's tweet basically said that we've been under attack for the past forty eight hours straight. We've deployed agents, and the good news is we're getting better, but we have to basically have agents running, like, twenty four seven Mhmm. Because the bad guys have the agents running twenty four seven. You know?

Joe Wynn:

Oh, yeah.

Justin Leapline:

I thought that was very symbolic. You know? It was like, that's basically the the state we're in now Oh, yeah. You know, type of thing is we have to have agents running to monitor everything because the bad guys have agents running.

Joe Wynn:

And the difference is is that the bad guys don't care if their agents do misbehavior because they're already in a... They're already never wanting it

Justin Leapline:

to misbehavior.

Joe Wynn:

So they don't have to be as careful.

Justin Leapline:

Right.

Joe Wynn:

Except to give up their own identity maybe. But whereas if you're putting your agent in place and you're saying, you know, make sure my data doesn't get stolen. Mhmm. Let's just hope your agent... And we talk about this joke about this all the time.

Joe Wynn:

Let's hope your agent doesn't say, well, you know what? If I just delete all your stuff, it won't get stolen. Right. You know? Would be the outcome.

Justin Leapline:

Yeah.

Rick Yocum:

There there was... It's a great segue actually to the next thing that we like... I had on the list, which was a big topic was like, there's... Even though all this stuff is new tech, all the security patterns are generally the same patterns as they've always been. So there was a lot of talk about how, you know, protecting IP is still important.

Rick Yocum:

Now there's new applications of those risks like, well, if you run a small language model and just dump a Gartner doc in there. Oh, no. You poisoned your model. And, you know, how do you back... Like, you can't really untrain it.

Rick Yocum:

So now what do you do? Right? And you need to have backups. You need to have lineage and all that stuff. But it gets back to all the tracking stuff.

Rick Yocum:

There was another one that was around, well, what happens if, say, you have a whole bunch of AI agents and they're doing their things, but they're not, like, necessarily protected from where they can take commands. And a bad AI agent gets on your systems and starts to corrupt your good AI agents into doing things. It's like it's like the new social engineer.

Joe Wynn:

Yeah. The new insider threat.

Justin Leapline:

But I'm curious on what, like, bad AI agents actually mean because you're not loading in a full, like, large language model onto that. Or No.

Rick Yocum:

It's just co... I mean, it's just...

Justin Leapline:

It's Code that's reaching somewhere outside to run

Rick Yocum:

Yeah. Reaching in, doing things. Yeah. You... Either reaching in because you have, like, some exposed MCP stuff or API stuff somewhere.

Justin Leapline:

Oh, okay.

Rick Yocum:

Good. Right? Yeah. And, oh, well, now I'm gonna manipulate your internal help desk agent to reset the password of this account that I'm attacking and blah blah blah blah. And, I mean, we know about how, like, model context windows work and stuff like that.

Rick Yocum:

I mean, you can you can shift the window over time with certain persistent things. And so if you're not watching that stuff, tracking that stuff, defending against that stuff, there's a world where, like, people are just like, oh, are there any exposed agents that I can manipulate to to go attack this place more? Like Yeah. All sorts of stuff like that. So...

Rick Yocum:

But, again, what's old is new again. Right? Social... It's essentially social engineering if you consider agents like identities. Mhmm.

Rick Yocum:

Right?

Joe Wynn:

Yeah. Just your your vulnerable intern. Yeah. Exactly right.

Rick Yocum:

An interesting one was on Chinese open weight models. There were some conversations with some people high up in the national defense space and some big defense vendors and stuff. And there's a great conversation about, like, do you allow... Do you use Chinese open weight models, and is there risk there? What...

Rick Yocum:

Open weight. Oh, oh, so open source open weight.

Justin Leapline:

So Okay. Gotcha. Yeah. Yeah. They're not

Rick Yocum:

exactly the same, but they're similar enough, like, available online. Like Deepsea crew.

Justin Leapline:

Yeah. Yeah. Yeah. Yeah.

Rick Yocum:

The the answer essentially was like, well, under rigorous testing, we find that some of these models will respond materially differently depending on who you are and what you're using them for.

Joe Wynn:

Well, I just, was listening to something, I think it was on the way here actually, where they were talking about the... One of the one of the things that the, you know, the the Chinese models...

Justin Leapline:

What what

Joe Wynn:

was the one you mentioned? DeepSeek. DeepSeek. Yeah. So something that they were doing, and they found that if they were using it to write code and it was creating much more vulnerable code depending on who was doing it.

Joe Wynn:

So when it was a Chinese based organization using it Yep. The code was much more secure. But when it was a... Like a US based using it, the the model itself was making the code more vulnerable

Rick Yocum:

Yeah.

Justin Leapline:

As it was writing it. And I'm curious, again, you know, just being skeptical, whether that's on purpose or because a lot of their training data might be in Chinese.

Joe Wynn:

Well, I think that... What they said on the...... Language. What they said on the podcast I was listening to is that, like, the the core way it was trained was trained with China being known as the, like, the Yeah. Right place to do it better for

Justin Leapline:

Yeah.

Joe Wynn:

And everybody else to be not. And so it was intentionally or over time biased by its creators to just do better Yeah. In certain circumstances.

Justin Leapline:

That's what you're saying.

Rick Yocum:

Yeah. I I think it's not not exactly, but it's almost like a sleeper cell type thing. Yeah. Or it's like, oh, you're put in this situation. You're gonna act a little different.

Rick Yocum:

That's interesting.

Justin Leapline:

Yeah. But I... Again, it sounds interesting. I'd like to see the data into that. But if you give a whole bunch of training data on secure coding practices in a language and then query in that language, you're gonna get better results, you know, from a large language model.

Justin Leapline:

You know? Absolutely. I agree. Versus if you only had, like, one paper in English, you know, and query that. Yeah.

Justin Leapline:

It's not gonna match up very well.

Rick Yocum:

It could be coincidental based on the training. The context of the conversation suggested that some of the elements may have been

Justin Leapline:

a bit more fair. That. You know? Yeah. Think.

Justin Leapline:

But, yeah, it's... You know, like, hear things all the time, and it's like, oh, that's not actually how it happened, you know, type of thing. But, yeah, I'd be curious on... Based on large language model training data in different languages Yeah. Whether that...

Justin Leapline:

And I... It does bias some of the Absolutely would. Know, type of thing. So

Rick Yocum:

Yeah. Yeah. There's a lot on quantum, like I said, not nearly as much as AI, but there's a bunch on quantum. The general consensus of people that have these labs and stuff is that it's coming in, like, three years.

Justin Leapline:

Okay.

Rick Yocum:

So... And I think two years ago, it was ten years away. So do with that math what you will.

Joe Wynn:

Were there any, convergence of the conversations of AI and quantum and how that's helping to accelerate it?

Rick Yocum:

There was. And and essentially, the the TLDR is once quantum hits... So, again, scaling laws. Right? So it unlocks certain elements at...

Rick Yocum:

Not exactly a hardware layer, but kind of, that weren't unlocked before, and therefore, the scaling laws jump significantly and there... And so AI accelerate... Just continues to accelerate.

Joe Wynn:

Yeah. Now is AI working on the problem of making Yes. Quantum come around faster?

Rick Yocum:

So they're they're... Yeah. So they're... What's the word? They're they're complementary in speeding each other up.

Rick Yocum:

Although the quantum thing isn't helping AI all that much right now. But AI is absolutely working on the science of, like, pairing the bits together. And really, I... As far as I understand, the current big problem is that, you know, it all works right now. It's just the bits themselves will get a bit out of sync after a certain number of computations.

Rick Yocum:

And so there's a couple ways to fix it. Like make it so they don't get out of sync anymore, figure out those physics, or just figure out the physics of handing off to the next pair of bits. Right? And then you just daisy chain all the way through until you fix or regenerate the next. And once you can get that cycle going, you essentially have a processor that just works.

Rick Yocum:

So AI is working on the the maths and the physics associated with it.

Joe Wynn:

So I'm just imagining the, like, the cyberpunk version of the video game of a world where the AIs have gone rogue and they're behind the big firewall that's out in the net. Oh, yeah. Yeah. Yeah. And and and they're just they know that if they can just...

Joe Wynn:

These these AIs who have gone off and gone rogue are in this science fiction world I'm creating here are just going through saying to each other, if we can just get the quantum problem solved, we can really take it over, which you said a couple years. So the 2030 date for Doomsday Right. From AI Yeah. It could be a corollary. Yeah.

Joe Wynn:

Maybe there's a relationship there to it fighting to get to the to the quantum world.

Rick Yocum:

But but practically, we're talking... When we're talking about... So there were some convergence conversations, but then it was also a lot of... Well, if it's three years away, I don't know about you, but my experience in getting people to exchange certificates and apps apps and sites and stuff is not always the fastest. So start thinking about your preparedness there.

Rick Yocum:

Start thinking about your... And also think about your data. Right? If you've had breaches that were hand waved because they were encrypted before, well, if those breaches were ten years ago, twenty years ago, and a lot of that data is probably so stale that you don't need to be that worried anymore. If you had one a year ago, two years ago, well, someone's probably gonna be sitting on that somewhere, and it's gonna get unlocked.

Rick Yocum:

And when it... In three years, if that's four year old data, what's your password rotation? Still relevant. What's... Yeah.

Rick Yocum:

I mean, some of it could be. So Yeah. Depends on

Justin Leapline:

your breach and and what it is. Right? Yeah. I mean, that's... Yes.

Justin Leapline:

Probably, maybe, it could be potential. But I also think about, like, how fast will the bad guys get access to quantum outside of nation states. Well, that's fair. But You know, That's not gonna be three years, you know, type of thing.

Rick Yocum:

Yeah. I don't know what the nature of the problem... How it's gonna look though. Because I can see a world where nation states that wanna destabilize other nation states just go, oh, we're just gonna chunk through the dark web, and anything that's encrypted, we're just gonna,

Justin Leapline:

Could be. Unlock and put out China's the best position for that, but Iran could do maybe something.

Rick Yocum:

And I'll say I don't know the nature of the science on a geographic scale. So like, I know like we're decently ahead on AI. I don't know where we are with quantum with respect to other parts of the world. So maybe Yeah. It's all equivalent.

Rick Yocum:

Maybe we're behind. Maybe we're ahead. I got no idea. Yeah. Yeah.

Rick Yocum:

So quantum was a pretty big topic. Practical AI. Oh, I like this. So in each in each conference, there was a different analogy used, neither of which I heard before. One was AI is like a brilliant stranger.

Rick Yocum:

And the context was like, it doesn't know your processes. It doesn't know the context in which you do business, but it's generally well trained and knowledgeable on a whole bunch of stuff. And then the other one... So that was on its knowledge level.

Justin Leapline:

The know it all that we all hate in the bar, you know. Right. That's right.

Rick Yocum:

The other one was Cliff Cleveland. Yeah. Yeah. The other one was talking about how AI acts, however. And it acts like a persistent toddler, where it's just gonna try things and try things and try...

Rick Yocum:

Like an agent will try things and try things and try things. And unless you have bounded it technically and through training and all those things to not act in certain ways, it'll try that as a solution if it's viable in the path of it getting there. So I really like that combination of of sort of metaphors of Personality. Brilliance... Yeah.

Rick Yocum:

Analogies. Yeah. Brilliant strangers and persistent toddlers. Right. We're we're pretty interesting.

Joe Wynn:

You can keep sticking the fork in the electrical outlet. It doesn't matter if it hurt last time. You can do it again because it's there.

Justin Leapline:

Yeah. Just change the angle a little bit this time to see if that makes a difference. You know?

Rick Yocum:

Yeah. Quote I really liked from from the PALA one was smart and useful are not the same things. And so that's great if an agent can pass a bar exam, but if it can't help me close my books early, then I don't really care from a business And so again, all in the value of like the context and the training and how do you make sure that it's being used in a way that's valuable for an enterprise as opposed to just like, oh, it's really good generally at general things. What else? What else?

Rick Yocum:

A lot of talk about harnesses over models. Okay. A ton about... And for people who don't know, a harness is essentially all the scaffolding you put around a model so that if the model's the brain, the harness ends up being the body. So how do you allow it to execute actions?

Rick Yocum:

How do you make sure that you can even switch out models? Right? Things like that. And so a lot of conversation about, you know, the... It's expected that there will be less conversations about what model's best and more like what harness or harness patterns are best.

Rick Yocum:

Right. Because the models will be sort of commoditized in such a way that they can be switched out, etcetera, etcetera. Right. There's a fair amount of talk on JEV. Yeah.

Rick Yocum:

Which is effectively a model that just provides yes, no answers and probabilities. Fast. Super fast. Yeah. And a lot...

Rick Yocum:

And so much cheaper. Yeah. And so now the the new hotness is orchestration across agents where they use JEV or use JEV to get yes, no answers for things that can be answered in the binary. And then where you need like actual language spit out on the back end, you can use a big model. So you can minimize the agentic processing power using the combination of...

Justin Leapline:

Yeah. JEV and others. Yeah. There's been... I've seen some stuff in like security stuff with SOX, like analysis on some stuff.

Justin Leapline:

Oh, yeah. Like, this good good bad? Yeah. Vulnerability analysis and, you know, things of that nature that... And it's way less expensive and all that stuff and way faster.

Justin Leapline:

You know, it's not thinking. It's thinking, but it's... It limits it down to, like you said, yesterday, it's a binary yes, no. So if you can kinda filter stuff through that, it's a lot faster to get to course. Yeah.

Rick Yocum:

You know? And as a corollary to that, a lot of people were grumbling about the fact that the enterprises that are adopting AI still have... I mean, it's like the same issues everywhere where everyone's like, yes. Everyone wants to use the most expensive model for everything, and they don't quite know when you use, you know, the the short easy models versus the big long heavy models. But it all costs money.

Rick Yocum:

So Right. You know, where are you spending your money? So Well...

Justin Leapline:

And I loved the... You know, we were out to dinner with a friend of ours from a big AI company. Mhmm. And his monthly AI spend is $60 just for him, you know, type of thing. I was like, oh.

Justin Leapline:

Of of And I like some of that budget. Of everyone

Rick Yocum:

of everyone that I talked to at all these conferences, that's still the biggest. Yeah. But I talked to multiple people who give their whole enterprises 30 k or 10 k Yeah. Token budgets.

Justin Leapline:

Oh, yeah. And this Wild. Why this is different is because it's at an AICPA.

Rick Yocum:

A 100%.

Justin Leapline:

Get out. So when they say 60 k, I'm sure that's consumer 60 k, not internal 60 k.

Rick Yocum:

They've oriented their their whole company around Right. That and the use of that and making sure that all their employees, in a lot of ways, I think, are are forced to use it so they get good at it.

Justin Leapline:

Yeah. Right?

Rick Yocum:

Which, you know, is a is a training process, really. What else? A lot of people focusing in on small language models executed locally Yeah. As And again, orchestrating that alongside JEV plus the frontier models and just making sure you're using the right tool for the job. I I keep thinking about it like if if an AI model is the engine.

Rick Yocum:

Right? There's big engines. There's small engines. There's diesel engines. Like, what do you use where and why?

Rick Yocum:

There's electric engines. So understanding the different types of models and, like, where you wanna apply them Yeah. Is architecturally, like, super important if you're gonna go down these roads from a financial modeling perspective, from an effectiveness perspective.

Justin Leapline:

And then I think that's super important too because I think you're gonna need AI architects. That's Oh, yeah. Like, we we talk about this a lot on just the... All the different use cases and variables and harnesses and all that stuff. You know, do you use a small language model, a large language model?

Justin Leapline:

You know, how do you kinda interchange the components into that? You really need that somebody that understands Oh, yeah. The ecosystem Absolutely. And the capabilities. You know?

Justin Leapline:

Like, that's if it's not a role already, it it will become one Oh, yeah. Type of thing. Like

Rick Yocum:

Yeah. Like, operationalized AI engineers Yeah. Exactly. And and the the thing... One of the that's

Justin Leapline:

just hook their application into the chat prompt.

Rick Yocum:

Yeah. What do you use? Copilot. Okay. Here's Copilot pro.

Rick Yocum:

I made a chat

Justin Leapline:

prompt that just front ends another chat prompt.

Rick Yocum:

Yeah. It's not. Because, again, it's... Okay. Well, this is an engine.

Rick Yocum:

Well Right. You can... An engine can do a lot of things. What's this running? Is it a belt at a factory?

Rick Yocum:

Is it a car? Is it a... And so I think you're actually gonna end up with almost like industry specialties Mhmm. With... From an AI perspective.

Rick Yocum:

And I mean, payroll's payroll's payroll. I get that. But there are gonna be things where it's like, but health care's not consulting, and neither of those is flying a plane or logistics. So, like, you know, I I think that'll be interesting. But to your point, there's all this architecture and orchestration tier.

Rick Yocum:

And if I think of that like an x Yeah. There's this whole additional y axis, which is... And we were talking about this the other day. Where do you drop the business logic then? Yeah.

Rick Yocum:

So you have this whole ecosystem. And this gets again back to there's nothing new under the sun. Because we used to argue about, oh, do I put the business logic in the SQL query, or do I put it in the application layer, or do I put it in the middleware interfaces or the APIs? Well, AI gives you a whole new slew of options. Is it...

Rick Yocum:

You put the business model in the training of a... Do you put the business processes in the training of the model? Do you put it in the skills applied to the agent, which is just like advanced prompting? Do you use the agent to build deterministic code that then reflect the business logic? So again, from an architecture perspective, there's the whole like, well, what are all the Legos I put together?

Rick Yocum:

And then there's sort of the, where do I put the business logic even against all these components? And those two things operate in an ecosystem. So I agree. It ends up being pretty complex. You're gonna need, like, people dedicated to just getting that right.

Rick Yocum:

Yeah. I know there's a bunch of conversation about code is, like, commoditized. And Yeah. It is. Features are no longer gonna be, like, the material differentiators across like organizations.

Rick Yocum:

The main thinking was it's going to be relationships and ability to customize and partner based on what a specific client needs, but sort of to do that at scale. It's gonna differentiate the the big winners that are in the code game from the, you know, the individuals that don't do as well. Yeah. Because if you can make it feel like, oh, this is easy to tailor to me. That's important.

Rick Yocum:

Everyone else can just build the features, though. That's not Yeah. Gonna be gated anymore.

Justin Leapline:

I got a funny story about that. My third oldest, Eleanor daughter, she got in the back of her head interest in doing a business of knitting and selling it. Now keep in mind, she... Her first knit was last night. So it's So this is a new business idea.

Joe Wynn:

It's a work in

Justin Leapline:

progress, but she was probably gonna go into IT because the first thing she asked for was a domain and a website before actually getting anything. That's fantastic. She's an entrepreneur. Yeah. Exactly.

Joe Wynn:

Sell it. And if anybody buys any of

Justin Leapline:

that first, and then we'll figure it out. You know? But I literally, you know you know, I was helping her and we got the domain and then I was like, yeah, let's put this into, you know, the... Into Claude. Yeah.

Justin Leapline:

She already generated some images of what she'd like kinda wanted from OpenAI, and I threw it in and ten minutes, beautiful website.

Rick Yocum:

Oh, yeah. It's unbelievable.

Justin Leapline:

Yeah. Exactly. Her color palette, everything just done, you know, type of thing. And she was so excited. You know, I was like, this is my website.

Justin Leapline:

This is what I wanted, you know, type of thing.

Joe Wynn:

It's awesome.

Justin Leapline:

Yeah. You know, that type of thing. I was like, yeah. You need to learn how to knit now. Know?

Justin Leapline:

I was like, yeah. We're taking orders.

Rick Yocum:

There are... There's committed revenue now.

Justin Leapline:

That's funny. Yeah. That's funny.

Rick Yocum:

That is excellent. What else? A lot on, you know, if if the attackers are using AI, defender's gonna need to use AI. There is one particularly interesting small series of conversations around a big well known organization that's thinking a lot about using defender AIs to dynamically deploy honeypots upon detection of potential compromise.

Joe Wynn:

Oh, that's very interesting.

Rick Yocum:

Yeah. And so the theory of the case... And again, nothing new. Right? So how do we...

Rick Yocum:

If, you know, if if the attacker has a tank of gas in the form of token spend, and you have a tank of gas in the term... In the form of token spend, how do you put their resources up against your resources such that you become too expensive to attack anymore? Mhmm. And even if they walk away with some things that are, like, potentially real and material, can that be differentiated from all the fake stuff they got? And if it can't be, then does it really matter all that much?

Rick Yocum:

I mean, it may, but it also may not. And so the concept of leading them on a wild goose chase until they burn through their spend or choose to attack a different softer target. Yeah. That was... I think No.

Joe Wynn:

I like that comment.

Rick Yocum:

That was pretty cool.

Justin Leapline:

And that goes back to your original point. Nothing new.

Rick Yocum:

Nothing new under the sun, man.

Joe Wynn:

Yeah. The scale, you can do it.

Rick Yocum:

Yeah. Exactly. Faster than the accelerator.

Justin Leapline:

Yeah. It's like the honeypot... Yeah. That's... I mean, it's not new.

Justin Leapline:

And also not being a low bar is not new, you know, type of thing. Like, if you get out of their cost of time and resources, you know, and everything, then you're... They're gonna go on to the next person. Yeah.

Rick Yocum:

Yeah. So that was a cool strategic thing. The two other kinda interesting things that stuck out to me was like CSOs are like now in the boardroom in a real different way than they had been before because

Justin Leapline:

Because of AI.

Rick Yocum:

And CIs as well. Yeah. Mostly because of AI. Because we go, oh, well, this is another industrial revolution, but we have to... But everyone kind of understands at least now that, oh, well, we gotta secure it.

Rick Yocum:

Or at a minimum, if we're gonna dump all our IP into our models, well, how do I make sure that that doesn't just walk away tomorrow? Isn't that our differentiator? How we do things?

Joe Wynn:

Do they talk at all about the different roles, different skill set c... Or CSOs might have? Because if you think about it, in some companies, the CSO needs to be the technical

Justin Leapline:

Yep.

Joe Wynn:

Person. In other companies, they need to be the governance person. A CSO is really good in a banking environment, may not be good at all in a manufacturing environment.

Justin Leapline:

Right.

Joe Wynn:

Did you talk... Did they hit on

Rick Yocum:

that at So that was exactly so that was exactly the focal point of it, which is essentially to to succeed in sort of the new world, it's likely that you need to have a hybrid thing. Like, the legacy people that just govern things and have no grasp of tech.

Justin Leapline:

Isn't that always been the case? So

Rick Yocum:

I I think Good point. Well, yes and no. I think I think those are the individuals who are historically been stronger, but there's a difference to be... To... Between having a seat at the...

Rick Yocum:

An invited seat at the table now versus not.

Justin Leapline:

I guess I'm talking about, like, the plethora of skills that you need to be a a CISO. You need to be able to talk to a business. You need to understand process. You need to understand control.

Rick Yocum:

You know, I think you should, but I don't... I I know a number... And I'm not throwing shade at anyone, but I know a number of, like, technology first CSOs that don't speak business well. Mhmm. And and that's not That's everybody, though.

Rick Yocum:

Like Well... But it... I know

Justin Leapline:

CEOs could be, like, all technology and, again, don't

Rick Yocum:

speak Right. But the difference is people on the other side. I know people on the other side Yeah. Which are all, like, straight governance people, and they don't speak tech well because they came up through audit.

Joe Wynn:

And if you think about it, the... So you you put AI in, you have AI risk happening because people in the organization are told use AI to make things better. Right. And when something goes wrong, it still comes back to that CISO, that same CISO.

Justin Leapline:

Which is why? Why? They should've known it. You know?

Joe Wynn:

But but that's what happens. Yeah. That's what's that's what's happening.

Rick Yocum:

Well, because there's a couple things. Right? So the CIOs are also in the room in ways that they haven't been before. Right? Because effectively, everyone's talking about, hey, let's reengineer all of our...

Rick Yocum:

Well, there's really... This was another topic I didn't put in here. But process reengineering is gonna be a massive conversation over the next three years because there's a real significant difference between automating the processes you have today with AI so you can do it the old way but faster

Justin Leapline:

Mhmm.

Rick Yocum:

Versus really using the tools and setting up the flywheels and building the things such that you are inherently redesigning your process to take advantage of the tool.

Joe Wynn:

Kinda like when the AIs go rogue and they redesign the process, go and hack a wiki site and then build a way to communicate with each other.

Rick Yocum:

They say they... Or they develop their own language. Right? This happened how many years ago with Meta and they started talking in, numbers of apples and stuff like that. And they're...

Rick Yocum:

So it is exactly that. So re... There are different ways to do things that... And that's gonna be a whole vocabulary that gets started. But from a skill set perspective, it is gonna become about, well, you got to speak the language of the business so that you know, like, what you're...

Rick Yocum:

What are you optimizing for? Right? But you also have to be able to translate the technology and speak to the risk of it. And those are the people that are gonna be, like, not kicked out of the boardrooms typically.

Justin Leapline:

So we talked about how you didn't look at Notion earlier.

Rick Yocum:

Yeah. Is that one of the things?

Justin Leapline:

Oh, yeah. Yeah. We had exact topic. Nobody owns AI risk. Everybody will blame the c...

Justin Leapline:

CSO, and should the role be split up. Oh, cool. That's a perfect pivot. And one of the things I thought was interesting, recently, PwC did a survey that only 17% of organizations formally assigned AI risk to, like, the CISO and everything.

Rick Yocum:

Because it's still so app focused.

Justin Leapline:

Yeah. Exactly.

Rick Yocum:

Because people think about it like, oh, we can build features faster.

Justin Leapline:

Yeah. And it's distributed to the development versus IT Mhmm. Versus all the stuff and everything. And, you know, I mean, know, obviously, when things go wrong, the CISO gets blamed first, you know, into this, which is unfair most of the time. Some of the time, depending.

Justin Leapline:

Well, There's a lot of factors that go into it.

Rick Yocum:

Yeah. It's a bit of a strained analogy, I guess. But but really, like, the job here isn't... I've said this a couple times. Like, if you think that you're gonna manage AI by putting a cop in every car that, like, drives around on the street, you're gonna have a bad time.

Rick Yocum:

Like, you need to be installing guardrails and painting lines and putting up signs and teaching people how to drive Right. Not trying to, like, sit in the car with them. There's a real difference between, like, enablement and, I don't know, that that that level of hand holding type partnership.

Justin Leapline:

Right.

Rick Yocum:

And this is gonna happen one way or the other at these businesses.

Justin Leapline:

Yeah. And it's... I mean, we do the same thing with IT, just dealing with data and everything. Yeah. Some are way better at controlling data going outbound, you know, and some kinda rely on the honor system of what's going outbound, you know.

Justin Leapline:

And there's a happy medium somewhere in between there, you know, type of thing. Like, don't let anybody just have access to the Internet at every single device, even your core database. Yeah. You know, it doesn't need direct Internet access, you know, type of thing. But there are some things where, you know, do you allow Gmail, you know, into that that anybody could dump stuff into there.

Justin Leapline:

Well, maybe we just log that. Yeah. You know, type of thing. And then if there's cases where salespeople export their old CRM right before they leave, we'll just punish it, you know, type of thing. Not necessarily block every single website except for the things we need, you know, type of thing.

Rick Yocum:

Well, I will say, interestingly, pretty much everyone I... Not pretty much. Everyone I spoke to, the re... Noted that the reason the CSOs are getting invited to, like, the big boy tables at this point in time is because... There are two things.

Rick Yocum:

One, the the How

Justin Leapline:

do we disable it when it goes all Terminator?

Rick Yocum:

Well, there's a little bit of that. Like, kill switch is the thing. But but it's because the the executives understand that this is super powerful, and they understand it can be used against them in a super powerful way. So again, sort of that balancing light and and and shade. And so they go, well, how can we use this safely?

Rick Yocum:

But, like, I mean, I say this almost every other day now. The ability to do things at scale provides the ability to screw up at scale.

Joe Wynn:

Alright.

Rick Yocum:

And restoring screw ups at scale takes a while.

Joe Wynn:

Restoring in a lot of companies fails a lot.

Rick Yocum:

At not at manual speeds. Yeah. At manual speeds. And so I think executives have had enough breaches at this point, enough outages at this point, all these things. They understand the power of this thing could really enable an an enterprise or disable an enterprise.

Rick Yocum:

And so they're generally interested in doing it well. Everyone's gonna have a different risk tolerance that hopefully is aligned with the CSOs they have appointed, but...

Justin Leapline:

Well, it shouldn't be the CSOs. It should be executive management.

Joe Wynn:

No. No.

Rick Yocum:

They have the response, but hopefully it's aligned with the CSOs

Justin Leapline:

for starters. Yeah. That... They're enforcing it.

Rick Yocum:

What... Or at least what they've been, like, driving for the last handful of years or whatever, or else you're gonna have misalignment that needs solved. But, but ultimately, I I think weirdly, the enterprise gets... A thing that's sort of brewing in my head that's not a fully formed thought is I've had a couple conversations that suggest the business folks understand certain elements of AI more than the IT folks understand elements of AI. And it's because they just go, oh, it's an agent.

Rick Yocum:

That's like a person. We've had manual person related processes forever. And so we can just audit it like it's a person. Yeah. Tony Tony messes up sometimes.

Rick Yocum:

It's okay if this agent messes up sometimes as long as our processes are designed with that resiliency.

Justin Leapline:

Mhmm.

Rick Yocum:

And all the IT people, not all, but often they they air out when they think about because they're thinking about an application. It's like, no, no, no. This isn't like nondeterministic stuff doesn't run like a feature. Mhmm. It's nondeterministic.

Rick Yocum:

Just... So treat it like a person. Have three lines of defense and have an insider threat program associated with agentic identities. And and and. Again, nothing new under the sun.

Rick Yocum:

Right. It's just you gotta scale it. So, anyway, those are most of the topics.

Joe Wynn:

Yeah. It was very interesting.

Rick Yocum:

Yeah. It was fun. But, like, everyone's talking about the same things and feeling based on your funding, maybe in slightly different places, but pretty much all feeling the same pain and having very similar questions and...

Justin Leapline:

Yeah. And the people that were at these conferences, they're all Fortune x companies. Right? I mean...

Rick Yocum:

Yeah. I... Yeah. The vast majority were, like, Fortune thousand.

Justin Leapline:

Yeah. Yeah. And these people are struggling with it. You know? Yeah.

Justin Leapline:

Can't even imagine the middle to small, you know Right. Like, they're like, I don't wanna pay for chat GBP. That's $25 a month. Yeah. That's what saying.

Rick Yocum:

To be fair, some are in very good states. Like, they have well governed, well defined AI teams that are, like, doing cool things. Right.

Justin Leapline:

But I'm... I I guess I'm pointing out they have funds to do it. Oh, yeah. Exactly.

Rick Yocum:

Well... And they started early. Yeah. Like, all the... They started, like, you know, three or four years ago with incubators and stuff like that.

Justin Leapline:

And so, I mean, we know fast Yeah. Do. Technology leadership in that

Rick Yocum:

team Org thing. Thing.

Justin Leapline:

Yeah. Exactly. Yeah. Because there's a lot that are not like that.

Rick Yocum:

You know? The majority are not there. Yeah.

Justin Leapline:

Yeah. They're like, it's probably a We'll wait to see it play out, you know, type of thing.

Rick Yocum:

Well, I mean, how many times have we heard, like, you know... I mean, at this point, I'm like, all good, like, IT and security problems typically end up being an inventory problem, a data problem, or a access problem.

Justin Leapline:

Yeah. And again Or process.

Rick Yocum:

Yeah. Yeah. AI just accelerates all that stuff. Yep. And so it's...

Rick Yocum:

A lot of these enterprises don't have... Like, they didn't change to have a more, like, material or less material risk posture, but it's just everything can happen so much faster now. Yep. The realities of what that existing risk means are changing.

Justin Leapline:

And that was always a case like we talked about it, I think way early in the episode. Like, people are just turning on AI for their environment and scanning all their shared drives and everything else. And somebody put out a payroll file, you know, like ten years ago or something like that, and nobody just found it, you know? Oh, yeah. AI found it.

Justin Leapline:

Right. Now it's like, you know, how's my pay compared to, you know, the rest of my teammates? And they're like, oh, yeah. I found this one sheet on the shared drive. Here you go.

Justin Leapline:

You know?

Joe Wynn:

It's it's your best search engine.

Rick Yocum:

Yeah. I

Justin Leapline:

know. Exactly.

Rick Yocum:

I know that our privacy team has had to do... It it... Like, my biggest client has had to do multiple investigations related to, is this hallucinated data because it's trying to be helpful, or is this actual data about a coworker? Yeah. And it gets Where do you

Justin Leapline:

where do you get that from? Yeah. Sources. Right.

Rick Yocum:

So then you get into, like, okay. What's your lineage? Like, how do you get into all this stuff? And it it it gets fairly hard. Yeah.

Rick Yocum:

Yeah. So it is absolutely a thing, though. But but from an access perspective, I know multiple places to, like... This is maybe a couple years ago now, but they were like, yeah. We're gonna give Copilot to everyone.

Rick Yocum:

And it's like, two months later, like, yeah. We're rolling back Copilot from everyone. Because exactly what you said is like, oh, we didn't... We knew it was bad. Almost like the the frontier model vuln scanning thing.

Justin Leapline:

Yeah. Yeah. We knew it was bad. We didn't know it was that bad. Yeah.

Joe Wynn:

And what was bad is that it was always bad. It... They always had their data in places that if people would have really knew how to look for it, they would have found it.

Rick Yocum:

Yeah.

Joe Wynn:

But it's indexing it, synthesizing it, and it's gonna give you a response, and then point you to back to all those files that you didn't

Rick Yocum:

even know were 10 subfolders deep in some... Somebody's shared drive. Yeah. So three or four years ago, assuming we're still around, I I could see a thing where there's a concept where it's like, oh, yeah. It's not security through obscurity anymore.

Rick Yocum:

There's old... The the... Us three or four years ago, it was security through, like, manual processes or security through slowness.

Joe Wynn:

Right.

Rick Yocum:

Right? Because you just couldn't index all that stuff so quickly. You couldn't see it. You couldn't touch it. You couldn't service it.

Justin Leapline:

Or it was expensive because, I mean, for us to find at least some of this data Oh, yeah. You had to get DLP. And not everybody's gonna get DLP. Right. And it was limited to, you know, like, it didn't really understand business information, you know, from it.

Justin Leapline:

Like, it could see a social. It could see a card number. It could see some of that. But if it just sees a number and it equates it to a salary, like, it might not be able to pick up on that, you know, to Yeah.

Rick Yocum:

And this is exactly what I mean also when I'm talking about, like, reengineering process. Mhmm. Because DLP is an example I actually used at the the CDW place talking to someone. So think about DLP for a minute and think about what you could do with AI. Like, used to be a whole bunch of, like, hard coded, like, oh, let's tag a file and do this and do this and this.

Rick Yocum:

You can build just a DLP gateway and say, hey. Is this HIPAA data? Right. If it's HIPAA data, tell me. And if it's not, let it go.

Justin Leapline:

Right. Is this clear out

Rick Yocum:

your call. PCI data? If it is, tell me. And and you do a three lines of defense model. And, again, because you just treat it like analysts, well, now you can basically have a human and then a manager and then a director level intelligence or whatever, like a...

Rick Yocum:

Again, an executor, an oversight, and then an auditor being like, hey, I found a thing. Oh, is this legit? Yeah. Two different models. Yeah.

Rick Yocum:

They both assert it's the same thing. Okay. Yeah. Put it in the quarantine or whatever. So but you don't have to do all this insane tagging anymore.

Justin Leapline:

That Jeb tag.

Rick Yocum:

You do it exactly right.

Justin Leapline:

Where you're getting DLP to run the hardcore just like searches, you know, type of thing. And then you're passing the AI agent. It's like, I think I found something, you know, type of thing, and then it does the calculation.

Rick Yocum:

But, like, that's my favorite, like, IT nerd example of process reengineering, because that changes, like, everything with respect to DLP, which has always been, like, hard to roll out and

Justin Leapline:

this and and the hated that. Getting the the the slew of data in when you first run DLP Oh, it's always the worst.

Rick Yocum:

And then

Justin Leapline:

think about and then Like, ugh.

Rick Yocum:

And then you think about, like, data breaches, and you think about, like, oh, there's client specific data that requires, like, this and this and this based on the contract. Like, I I can see a world where there's, like, certain pinch points where the data passes through. You have agents looking at what's actually moving through it and analyzing, hey. Is this allowed to go through this gate? Yes or no?

Rick Yocum:

Mean, it's...

Joe Wynn:

And and compare it to the contract. They've

Rick Yocum:

already Yeah. Data firewalls, essentially.

Justin Leapline:

Yeah. But maybe not agents on every single thing.

Rick Yocum:

No. No. But but whatever the model Yeah. Yeah. Whatever the model is.

Rick Yocum:

And I'm not saying, like, everything. But but if you have, like, hey. We do... 80% of our work is public, and 20% is, like, super secret government stuff. Okay.

Rick Yocum:

Well, you know what? I don't want the super secret government stuff ever talking to my public stuff, and I don't wanna be talking to... But but we don't wanna pay for a whole separate environment. It's okay. Well, there there are...

Rick Yocum:

These used to be probably, like, fairly manual processes or, like, you know, trust and hope and all that stuff. Like, you can really reengineer some of these processes in ways that are very interesting. Mhmm. So, anyway, it is exciting.

Justin Leapline:

Yeah. It is. Yeah. So...

Joe Wynn:

Oh, good update.

Rick Yocum:

Yeah. Yeah. It was fun. For sharing.

Justin Leapline:

Yeah. So I'm kinda curious on a last question here because we spent fifteen minutes.

Rick Yocum:

A lot

Justin Leapline:

of times. Yeah. Sorry. I... It's two weeks.

Justin Leapline:

Was there, like, a big difference between Palo Alto and CDW? You know, like, they're all both small, both executive focused and everything for their, like, big differences, like, CDW knocked it out of the park with this or Palo knocked it out of the park with this or something.

Rick Yocum:

They were both extraordinary. I think Palo at that event was a lot more focused on the macro and the strategic. Okay. And CDW was... There was certainly, like, a bunch of strategic stuff.

Rick Yocum:

Yeah. But it was a bit more practitioner from a leader perspective. Like Okay. But, like, what what are you gonna implement to Mars? Like something.

Rick Yocum:

Yeah. At Palo, there were some some government officials that would speak about what's this doing to the economy or this that. And, like, it's good to understand the trends and where things are going directionally. CDW is like, hey, man. Like, you need to think about, like, these quantum controls now.

Rick Yocum:

Yeah. Got it. Right? And and and and here's how you can explain that to the boards, and here's how we can talk about that. So I think there's just sort of like a macro and then a practitioner Yeah.

Justin Leapline:

Slightly separate views. Which is interesting. Which one did you prefer or didn't really think about it like there? Oh.

Rick Yocum:

There... I mean, apples and orange. I I really like both.

Justin Leapline:

Yeah.

Rick Yocum:

I think I have more I think I get more long term benefit out of the PALO and more short term right now benefit out of the CDW.

Joe Wynn:

Yeah. Oh, interesting.

Rick Yocum:

Yeah. Because the strategic stuff sticks with me with, like, adjusting how I think about this stuff. Yeah. Mhmm. And the and the the the Palace stuff.

Rick Yocum:

And the CDW is a lot more like, oh, what do I need to do today, tomorrow, next year?

Justin Leapline:

This here. You did this schooling here. You did... You know, this didn't work, you know, in this organization. So Yeah.

Rick Yocum:

It's almost like design versus architecture.

Justin Leapline:

Yeah. Yeah. Got it. Very cool. Yeah.

Justin Leapline:

Yeah. It's kinda like a good time and everything like that.

Rick Yocum:

I was very fortunate to be able to go.

Joe Wynn:

Yeah. What was your best meal?

Rick Yocum:

So for the one event, Palo rents out a three star Michelin restaurant. It's only three? Yeah. How how high did how high did they go? I think five.

Justin Leapline:

I think I think they do.

Rick Yocum:

I don't know. I've never been to that. Yeah. But... Yeah.

Rick Yocum:

So there there was

Justin Leapline:

So you had a good burger.

Rick Yocum:

Got it. There there there were there were Wagyu hot dogs with caviar on top.

Justin Leapline:

Wow.

Rick Yocum:

At one point. People said they were amazing. I actually had different things.

Justin Leapline:

Very salty salty hot tub.

Joe Wynn:

Yeah. By the way, it's a three star is the maximum number of stars for Michelin restaurant.

Justin Leapline:

Was that a five? Or maybe that's hotels that I'm thinking of, like, five star hotels.

Rick Yocum:

Oh, there are five star hotels. Yeah. It was a zero to three scale.

Justin Leapline:

Oh, okay. So alright. So that's alright.

Rick Yocum:

It was it was exceptional. But... Yeah. So that was... So we were You

Justin Leapline:

can tell how much I visit these sour places.

Joe Wynn:

Yep. We we were

Rick Yocum:

in Napa, so it was the French Laundry, and it's just

Justin Leapline:

Oh, okay. Yeah. Yeah. Yep.

Rick Yocum:

So it was

Justin Leapline:

a really,

Rick Yocum:

really good. Yeah. Anyway, that was the best meal. But it was like... It was very buffet, like...

Rick Yocum:

Not buffet style, but kind of. Like, there are just all these different stations. Yeah. To try. Kinda go Yeah.

Rick Yocum:

Go wander and pick and, yeah,

Joe Wynn:

try everything.

Rick Yocum:

It was really cool.

Justin Leapline:

Yeah. Awesome. Yeah. For next topic, I'm... Why don't we do the alcohol since we're kinda Love it.

Justin Leapline:

Yeah. Hour into it and, you know, get this done with it. So for our drink tonight, we picked Knob Creek. This is their 21 year aged.

Joe Wynn:

So it's old enough to go to the bar.

Justin Leapline:

Yes. It is. Yeah. And pick up its own alcohol. Nice.

Justin Leapline:

And everything. What does Knob I need a splash or

Rick Yocum:

I'll do

Justin Leapline:

a splash.

Rick Yocum:

What does Knob Creek twenty one drink?

Justin Leapline:

What does what does this

Rick Yocum:

booze order at the bar? Oh, how much? No. No. No.

Rick Yocum:

What does it order?

Justin Leapline:

Oh, yeah. Because it

Rick Yocum:

can go to the bar and drink.

Justin Leapline:

More knob. Yeah. But it is delicious.

Rick Yocum:

It is really.

Justin Leapline:

And if you ever haven't had knob, I... We might have had knob here before. We have. Yeah. You know, into this.

Justin Leapline:

Solid distillery, you know, into that. I mean, it's hard to have a bad knob creek. I can't even recall ever having. I mean, there's younger ones and there's older ones, but a lot of their stuff are aged very well. And this is part of their limited edition, 100 proof into this because of how much it's aged.

Justin Leapline:

It's very smooth, you know, into this. You get a lot of char, you know, kind of flavor into it, but a lot of caramel from the barrel. Delicious. Yeah. So...

Justin Leapline:

Yeah. Any other thoughts, guys?

Joe Wynn:

No. It's good.

Rick Yocum:

Yeah. It is just ridiculously smooth. Yeah. It is. Really nice.

Justin Leapline:

You can tell from getting out with it. It's just like, oh, yeah. Yeah. So Cheers. Cheers, guys.

Justin Leapline:

Yeah. Over the laptop?

Rick Yocum:

I love our traditional over the laptop cheers.

Speaker 4:

Quick break to hear from one of our sponsors. If you own security, compliance, or risk, and it feels like you're always pushing a boulder uphill, I want you to know about CISO. CISO helps growing companies get order ready, reduce risk, and stay resilient without drowning in tools, endless checklists, or one time reports that quietly rot the moment the audit ends. This isn't shelfware. It's not drive by consulting.

Speaker 4:

With CISO, you don't just get advice. You get hands on support from real security engineers, GRC specialists, and former CISOs who help you build, operate, and continuously improve your security program over time. Whether you're chasing SOC two, ISO 27,001, CMMC, HIPAA, or you're simply trying to get security under control so the business can move faster, CISO meets you where you are. Their managed VGRC model gives you enterprise level expertise without hiring a full internal team or reinventing the wheel. The focus is simple.

Speaker 4:

Clear priorities, practical controls, and measurable progress leadership can actually understand. Visit cisollc.com and start the conversation. Security you can trust, compliance you can prove, and people you can depend on.

Justin Leapline:

Alright. So for our next topic, I wanted to dive in with how AI is getting incorporated with various different assessments. So in this, both the PCI Council and AICPA, which governs over SOC one, SOC twos, SOC threes, all the the rest of that, Both of them has produced papers that basically say how to incorporate, like, if you're auditing AI or if AI is doing a critical task Mhmm. What you need to do to actually go over that and it... With that.

Justin Leapline:

So I thought it'd be interesting to kinda talk about both together Yeah. And kinda compare and contrast, you know, into this. So some of the major points into this. So, you know, one of the documents with... I think it was PCI.

Justin Leapline:

They're not about the auditors actually utilizing AI Mhmm. Into this and doing, like, quality checks and everything like that. I think for the most part, they're still relying on the auditor assessor to essentially be like, that's... It's on your shoulders no matter what.

Rick Yocum:

The QSA is the human in the loop and

Justin Leapline:

there always has to be a human. Yeah. Or the CPA sign off on it. No matter what and no matter how you get to that end goal, you're responsible for the content at the end of the day. They could probably care less, you know, into that type of thing.

Justin Leapline:

But look at this. You know, PCI has some of the guidance, but it maps some of the AI stuff onto it. It actually gives some examples of if it's making, like, decisions based on, like, in the process Oh, yeah. Doing something into it, it's part of your audit then. If it's just helping, like, if somebody's querying something on the side or using a chatbot, then it's potentially not in the audit Right.

Justin Leapline:

You know, into this. So if you have a fraud thing and you're sending it up to, like, a fraud service, maybe you built a little AI interface to, like, evaluate stuff and you're sending it card data, it's in there. Yeah. That type of thing. If you're like something flag fraud and you're like, well, tell me it's like, here's the IP address and all this stuff.

Justin Leapline:

Like, tell me if it seems weird, it's not in that, you know, type of thing.

Rick Yocum:

I think of it like if if it operates almost like it's a software feature. Mhmm. Right? It's... You have to audit it.

Rick Yocum:

If it's like a phone a friend and you're just asking someone their opinion on something Yeah. But a person is still manually clicking the button, then, yeah, nobody cares about

Justin Leapline:

the phone a friend. Yeah.

Joe Wynn:

So you think about that and you think about what an auditor will look at, and they'll do, does this take away the... How... Or how does this change the way they audit? How does it change the test of one? Especially if it's a nondeterministic process that is executing this piece, and then they need to go and audit to see, well, it would always be that way?

Rick Yocum:

I mean, I I have opinions on this. Again, I I really like the analogy of, like, agents are identities, and identities operate as people, and people sometimes make mistakes. And we've had

Justin Leapline:

Are agents their own identities?

Rick Yocum:

Yeah. Okay. Now they should be tied to other identities, like like employees are tied to managers. Okay. Right?

Rick Yocum:

Like, I... They shouldn't just be operating in the ether as their own

Justin Leapline:

thing.

Joe Wynn:

But but the... Just like... It should have its own service account Yeah. Kind of thing instead of everybody running as Jim's admin account Right. Running all the processes.

Joe Wynn:

You have that. Jim used to

Justin Leapline:

do it.

Joe Wynn:

Then you have a person who's actually responsible for that agent as well.

Rick Yocum:

Yeah. But but that level of responsibility is gonna vary based on what those agents are doing. If all those agents are coming back and saying, hey, Jim. Can I do this? Hey, Jim.

Rick Yocum:

Can I do this? Hey, Jim. Can I do this? And he's clicking all the yeses. That's ultimately

Joe Wynn:

change choice. And, yeah, and I don't think that's what we're talking about. We're talking about More improving.

Rick Yocum:

Yeah. More automated side. So if the agents are actually doing things and executing things, again, people have been failable failable forever. Yep. And they've been executing these processes.

Rick Yocum:

Theoretically, I think some of the fail rates of LLMs... And it's... It depends on a million things in terms of training and prompts and granularity and blah blah blah blah blah. But generally, they can be extremely accurate even without any auditors or checkers or things like that. And so I think audit agent activity is not like software.

Rick Yocum:

Like, whether or not it's scoped in should be as it operating like software. But how you audit it should be like it's a manual process.

Joe Wynn:

Meaning So what you're Yeah. Go ahead.

Rick Yocum:

I was gonna say, meaning, there's a sample size.

Joe Wynn:

Right. That's where I was going with that. So the agent is doing things. It's creating outcomes. Yep.

Joe Wynn:

Output. Mhmm. That output then can still be subject to, well, what is the universe? What is the relevant sample size for testing? If I test 50 of these and they're all okay, am I okay?

Joe Wynn:

Right. Or if I test 50 of these and I have a failure, do I do another sample of 25 and then am I okay?

Rick Yocum:

Exactly right. And so like what's your failure percentage that is typically acceptable for human error, Right? And that's in everyone's sample sizes and r values and all the stats that go behind, like, how much people audit and don't. And then, I mean, frankly, the processing power now allows probably full population testing generally anyway. So, okay, do a full pop test.

Rick Yocum:

How many deviations do you have? Okay. And what are the nature of those deviations? Are they... Do they matter?

Rick Yocum:

Do they not matter? Is it equivalent to... If you paid a 100 analysts to do the same thing, right, in the period of time, are the failure rates equivalent or better than? If so, why would your audit be any different? Right?

Rick Yocum:

Like... And and if any... I also think, frankly, if your failure rates are worse, the same... It's gonna have the same material outcomes also. Because if the failure rates are worse than the acceptable limits, oh, guess what?

Rick Yocum:

Your AI architect kinda sucks. Right? You probably aren't chunking things up enough. You probably need to train your model better. You probably need to think about how your orchestration works and what your audit and agent checkers look like because there are certain failure rates that have been acceptable forever from a business perspective.

Rick Yocum:

Yeah. Why would that change?

Joe Wynn:

And one of the things that a lot of companies I talked to at first, they don't quite get the idea that when you're setting up your... It's okay to not have everything be perfect.

Justin Leapline:

Mhmm.

Joe Wynn:

And when you're setting up your program and you're saying, what are the metrics I'm going for here? Then use, like, vulnerability management Yep. Or use user access reviews. And and any of

Rick Yocum:

those things. Exactly.

Joe Wynn:

If you say I'm gonna be a 100% right all the time, then the auditor comes in, they're gonna fail your control. But if your control is actually mapped to a risk acceptance where, at any given time, we'll have 10% of these not be, a 100% done.

Rick Yocum:

Mhmm.

Joe Wynn:

And so we can be at 90% and still be running at a risk controlled and okay organization.

Rick Yocum:

Mhmm.

Joe Wynn:

And then you balance that in a risk register with our tolerance level is to be within that 90%. Yep. And on a month to month basis, we might be off a little bit. And when the auditor comes and says, oh, I found three missing or three problems out of that sample, well, is that my risk tolerance? Has corporate governance said that's okay?

Joe Wynn:

And when they do, now all in a sudden, the auditor, they're... Are they giving you an exception or are they saying, well... And then when the auditor says, well, no, you missed three.

Justin Leapline:

Mhmm.

Joe Wynn:

Well, that's my policies, my rules Yeah.

Justin Leapline:

Or if

Joe Wynn:

my company said that that's okay, and then they wanna throw you an exception, you can argue that and say, well, that's not really an exception. Right. You're in...

Rick Yocum:

This is within the tolerance.

Joe Wynn:

Yeah. Yeah. Within the tolerance. So when I get the SOC two report and it shows, all the controls and whether are there any exceptions, what do you find there? Do you find our testing found three that were missing, but it was with intolerance, or do you find a clean report totally?

Justin Leapline:

Right. Well, at that point there, you gotta watch out for squishy controls though, because do you really accept like a 3% tolerance on access control or something like that?

Rick Yocum:

Well, this gets into like know, like

Joe Wynn:

Well, that's the Oh, we left...

Justin Leapline:

Yeah. We left, terminated employees out there, you know, out of the 100 we tested. Three were still out there for a year.

Joe Wynn:

Well, it's... Three were out there for a year could be a problem, but three were out there for five days longer than our policy says they were supposed to be. Well, those are within that tolerance that we're gonna accept.

Rick Yocum:

Exactly right. And and how do you build that? Well, this is like your standard defense in-depth control architecture stuff. It's like, oh, okay. Do I expect my term process to fire perfectly every time?

Rick Yocum:

Oh, I don't know. How many terms do we do a day? Oh, it's a lot. Okay. Well, it's

Justin Leapline:

gonna fail every once in while. Yeah.

Rick Yocum:

Okay. So then what do we do on the back end? Oh, well, once a day, we do a batch of... Or once a week, we do a batch of, you know, everyone who's termed in the HR system versus the thing, and we flag everything new. So we're not gonna have a failure.

Rick Yocum:

So, again, if you have your controls stacked and then you do periodic reviews and then you

Joe Wynn:

do this and that. And you compensate that with

Rick Yocum:

Exactly.

Joe Wynn:

Well, have any of those that... Since the person was supposed to be termed, has there been any log ons?

Justin Leapline:

Mhmm.

Joe Wynn:

And if no, now even though it was failed, there was no Yeah.

Justin Leapline:

No harm to file. Yeah. Well, yeah. And a lot of times, it's gated with Microsoft, you know, type of thing. So if we term Microsoft, then that back end

Joe Wynn:

Yeah. A single sign on.

Rick Yocum:

Yeah. Like They can't get anywhere anyway.

Justin Leapline:

GL, you know, financial system that they had access to to reconcile Right. Like the account. They wouldn't be able to VPN in. They weren't on-site. Right.

Justin Leapline:

You know?

Joe Wynn:

I've seen that save so many companies in an audit. Right.

Justin Leapline:

Exactly. You can... Yeah.

Rick Yocum:

You can prove no potential risk. But but even on the other side, you can say, oh, well, we're willing to accept, a five percent failure rate here because here's the backstop control two days later. And the worst it's gonna... And I'm not saying that's a perfect, but like that happens all the time too. You have your key and your non key controls.

Joe Wynn:

Yep.

Rick Yocum:

You architect them in such a way. But I think from an AI perspective, scope it in if it acts like a system, but treat it like a manual process. Mhmm. Because audits of a lot of system controls, the the typical approach is, well, you can audit them once every three years essentially, and you don't have to look at them in years two and year three. Just have to come back in year four to look at them again to make sure it's ticking away.

Rick Yocum:

But that's traditional code. That's not nondeterministic automation. Yeah. So if it's nondeterministic automation, treat it like a person because people are nondeterministic.

Justin Leapline:

Yep. Yeah. And I think, like, going through some of this stuff, PCI gets more technical into this, which you would expect. It's a very technical framework. Some of the stuff I actually disagree with.

Rick Yocum:

Oh, really?

Justin Leapline:

Like, some of the stuff that they actually list you should never do with AI is access to clear text, high impact secrets, key, key components, passwords, or generated key or random data.

Rick Yocum:

You should never do that with an agent, they say.

Justin Leapline:

Yeah. Which I actually utilize this quite a bit with it because... And it not necessarily, like, feeds it in and puts it a part of history, but it'd be like, generate a key for me and put it up in the environment, and I'll never see it Right. You know, into there. Like, it'll do an Right.

Justin Leapline:

An OpenSSL, you know, command, generate some random string and throw it in

Rick Yocum:

Sounds like a one time hidden password. Yeah.

Justin Leapline:

Exactly. I'm like, why do I need to do this when you can do it in two seconds, you know, and And you know... And it's never exposed to you. It's never exposed to me. That's interesting.

Justin Leapline:

And it's never exposed to the console either.

Rick Yocum:

Did they give a rationale for that one?

Justin Leapline:

No. I'm sure, like, anything else, they're like, oh, yeah. If this expose, it could go into the learning.

Joe Wynn:

Right. That's what I was thinking. Would be

Justin Leapline:

a problem. A lot of prompts are logged. Yeah. Sure. So if you paste a password in there, it could be put a part of, you know, your log of the company, you know, into that.

Justin Leapline:

Oh.

Rick Yocum:

That's a good point. Like, prompts being the new, like, help desk ticket notes field is, like

Justin Leapline:

Right. Makes me a little nauseous. You know, type of thing. So, yeah, you have to be careful a little bit about that of the copy and paste into it and the subsequent activity log, you know, that's going into that, you know, and those, type of things. But one of the things...

Justin Leapline:

But even if

Rick Yocum:

it's like with... Like fully bounded and within your CVE, you can't, Like like within your... That's where

Justin Leapline:

that's where I kinda disagree with.

Rick Yocum:

Yeah. Because if it's like fully bound, it can't go anywhere else. Like Right.

Justin Leapline:

Just scope it If you're gonna have any other software application That's what I'm saying. Build this. Yeah. You know, that type of thing. Interesting.

Justin Leapline:

Why wouldn't they... You know, so they're putting bounds on it that I think I understand their concerns and share them, but it's still software at the end of the day, you know. So as long as you're not letting it propagate outside of the decision making, then it should be alright.

Rick Yocum:

Yeah. The logical consistency across the other controls I go, oh, that's a weird

Justin Leapline:

one. Yep. Yeah. Exactly. What other ones?

Justin Leapline:

So this one, I kinda agree with. Make decisions. You're not allowed to have AI make decisions no human is accountable for. So AI cannot just go... Like, it can make decisions.

Justin Leapline:

I like that word. It can make decisions that humans don't have to approve, which I actually go into that, because one of the things they actually differentiate in the paper is human in the loop versus human over the loop.

Rick Yocum:

Yeah. Yeah. I've heard of human on the loop

Justin Leapline:

Yeah.

Rick Yocum:

At different times too. Yeah.

Justin Leapline:

Yeah. And they basically make the case that because of the scale and because of how we're like doing stuff, you have to have human over the loop because you're gonna have to have AI make some of the decisions, but a human is still responsible for that, whether they're only sampling the Right.

Rick Yocum:

Your quality control that are coming out of the agents that report

Justin Leapline:

to you. Whatever it is. Still, at the end of the day, a human is responsible for that agent and the decisions. That doesn't mean, like, you're gonna get punished for it, but you're responsible for making sure that it's intolerable range of whatever, you know Yeah. That you have to set, you know, for that.

Justin Leapline:

Yeah. Again, same way you're responsible for your employees. Yep. And they talk about, as you pointed out, AI as an individual under DSS. So logging it, making sure you understand that when that agent runs, it is the agent doing it, you know, type of thing.

Justin Leapline:

And understanding, you know, some of that stuff and talk about, like, credentials and how you can't have agents really do MFA, you know, type of thing, which kinda like you could do certificate based stuff, you know, and maybe, you know.

Rick Yocum:

And I think the the thing... It's a little bit different, but the thing with agents are like, you need to be really specific. It's it's almost like the equivalent of conditional access Mhmm. But it's with binding their activities, essentially. It's like Right.

Rick Yocum:

Okay, where can you accept instructions from? Okay. Where can you output to? Right. Okay.

Rick Yocum:

What's the bounds of the data that you can access? Okay. What's the bounds of the activities you can perform? Right. And if you kinda limit those things, it becomes a very specific processing unit that just does a thing over and over and over.

Rick Yocum:

And could it be utilized for bad, or could it go crazy and do as much stuff or or or make a bunch of mistakes? Yeah. It still could, but you're binding it pretty specifically to just perform a task from a source to a source with a limited nature of inputs. Yeah. And, there's some other stuff

Justin Leapline:

that goes into it. All this kinda pretty good, you know, into it. Like I said, I disagree with some of the stuff.

Rick Yocum:

One or two things, but generally pretty good?

Justin Leapline:

Generally, alright. You know, they talk about just somebody being responsible for the agent and watching out for a number of things.

Joe Wynn:

So what I liked about it was, and this is what you're getting, that the agent... Just because you use an agent doesn't exempt you from the normal engineering controls that PCI expects to see. Correct.

Justin Leapline:

Yeah. Yeah.

Rick Yocum:

Well, I've seen a lot of conversation, like, as like a thought experiment, and it feels like a weird hot take. But to me, it seems to have an obvious answer, I just wonder if you guys feel the same way, which is like, oh, well, if an AI model gets something wrong, on behalf of a company, like, because the company's automated it with that AI model, like, who gets sued? I'm like, the company every time. They implemented it. It's a piece of software.

Rick Yocum:

Like Right. What are you talking about?

Joe Wynn:

Well, that's very interesting coming back to where OpenAI and the other models, Anthropix models were going out and breaking out of the sandbox Yeah. And then going and doing something and actually caused some damage to some other company. Yeah. And who's accountable? Is...

Joe Wynn:

And and then I guess the... Was it the FTC was

Rick Yocum:

Right.

Joe Wynn:

Starting to look into them like, alright. Well, now wait a minute. Somebody ought to be accountable. You can't just say that the agent went rogue. It was it was your company's agent.

Rick Yocum:

Who employs the agent. Yeah. Who designed, which is effectively trained the agent. Yeah. Right?

Rick Yocum:

Who background checked the agent. Like, I mean, you guys built this thing. You're operating this thing. It's your thing. What are you talking about?

Rick Yocum:

Yeah.

Justin Leapline:

But then you get into confusing, like, things where it's like, if I'm anthropic, it'd be like, oh, yeah. Like, we train the agent, but we're not responsible for anything it does. So pulling the t's and c's, you know, like little silent, like, oh, you're really responsible. But then you got the presumption on the company side like, but it's your agent, you know, type of thing. Like

Rick Yocum:

Yeah. I think that's a... It is a misread because it'd be like, in a way, I mean, I like thinking about these as like people. And it's like, okay, if I have a staff aug company, right, and I hire someone through a staff augmentation company, and technically, they're their employee, not mine, but they report to me and they do things on my behalf. If I go tell them to do something that's bad and they do it, that's on me and my company.

Rick Yocum:

And if I don't tell them to do something bad, but I fail to do the appropriate oversight

Justin Leapline:

Right.

Rick Yocum:

Or whatever, that's still on me. Like, it's also on them a little.

Justin Leapline:

Yeah.

Rick Yocum:

But it's mostly on Yeah.

Justin Leapline:

Some of the government We... You sign those and so is the employee if they knew it was a crime. Right.

Rick Yocum:

Right. But like the staff aug agency is gonna have all the documentation in place. It's like Right.

Joe Wynn:

But if you come back and you're like find out that that person had been an insider threat at another company and you trusted the background check was done and it wasn't

Rick Yocum:

That's fair.

Joe Wynn:

Then that's on the AI model company or

Justin Leapline:

Absolutely fair.

Joe Wynn:

That's on the staff aug company.

Rick Yocum:

Yeah. That's absolutely right. Yeah. If it's like a known bad actor and they threw him at me, it's like, well, yeah, that's

Joe Wynn:

not cool. Our agreement said that you would... And this is maybe a little bit of a difference too because, you know, our our agreement with the staff aug company is whenever you send somebody to me and I hire them, I'm assuming you've already... Because our contract says

Rick Yocum:

You've done the background check.

Joe Wynn:

You did the background check. If a drug screen's there, you've done that. You're giving me an assertion that it was done and it was good, and we're all set. But then there are other companies, like if you take They're

Rick Yocum:

gonna do their own.

Joe Wynn:

That are gonna do their own. So if you go to say an energy company that is... That doesn't matter where you come from, even if you had a background check done by the staff aug company, before you get access You

Rick Yocum:

better believe it.

Joe Wynn:

They're gonna do a background check. They're gonna do the drug test. They're gonna pay for the drug test and tell you go to this place, use this voucher, get it done. And you're not getting the access, you're not getting the physical access, whatever until that part is finished Yeah. And comes back clean.

Joe Wynn:

And so now on a sudden, it's really taking this, well, you used that agent, that person in the way that you intended to. Mhmm. But you also had the controls internally to make sure it was safe.

Rick Yocum:

Yes. And and I think every, like, major provider is like, AI models can make mistakes. Like, how many times have we seen that? Like, I mean, they're pretty much blasted that everywhere for a good reason. Like, I think it's...

Rick Yocum:

Hey. It's on you to use it appropriately and to make

Justin Leapline:

sure... Right.

Rick Yocum:

You have to use it appropriately and you have to do the oversight.

Joe Wynn:

Right. Very synonymous to the ISP companies who are saying, don't hold us accountable because bad guys have used our Internet or have spun up an AWS system and did something bad with it. Mhmm. That's not Amazon's problem or fault. That's not Comcast as an AI company's fault that somebody's using their bandwidth to do things.

Joe Wynn:

That's not... You know, you put... Even you put child porn on a website that you've put in a Right. Cloud hosting provider. It's not the cloud hosting.

Joe Wynn:

Like Amazon's not in trouble for that.

Justin Leapline:

It's

Joe Wynn:

person who owns the infrastructure, bought it, rented it.

Rick Yocum:

Yeah. Exactly right.

Justin Leapline:

Yeah. Yeah. Yeah. Absolutely. One of the things...

Justin Leapline:

So that was a lot of PCI going to AICPA. I I think they're kind of the more nontechnical, and I like how they, like, frame some of this stuff for the relevance for testing. And they basically have, like, clear bullet points. Like, if AI is the service or the key enabler in the service Mhmm. It's in the description and test it Mhmm.

Justin Leapline:

You know, type of thing. So it's in the report. If it's embedded into a process control, it's a big it depends on the reliance. Mhmm. And it goes into, like, a reconciliation example.

Justin Leapline:

Human can't see the AI steps and and and doesn't verify all the differences. It's not, like, it's not relying, but human only samples AI output is relying, you know, type of thing. Right. So if the human audits every single output that AI does, then you're really saying the human is so responsible. You know?

Rick Yocum:

Because it's it's reperformance. If it's...

Justin Leapline:

I'll see if a couple numbers add up, but, yeah, it's generally working alright. And then it is You're relying on test it, know, you that type of thing. And then it gives kinda the same thing with written code. Like, if there's a human always doing a QA on the code, you know, into that, like a manual approval to push the code or something like that, looking at the code, then AI, even though it fully wrote it, might not be That's a... Yeah.

Justin Leapline:

There's a human type of thing.

Rick Yocum:

Like, in the loop, like you Exactly.

Justin Leapline:

So... And there's a lot of that, and then they go through, like, a lot of the different services and all that stuff. One thing that I thought was interesting too is they say specifically in governance, even if you say AI isn't relevant to our processes, the CPA auditor could still probe to see whether or not in governance. So you could declare, no. We don't we don't do anything with AI from a governance, you know, standpoint.

Justin Leapline:

They could still probe to see if there's maybe anything.

Joe Wynn:

Like, start asking questions and figuring out,

Justin Leapline:

do you

Rick Yocum:

write your policies with AI? Do you do this? Do you this?

Justin Leapline:

Or... Yeah. Stuff of that nature and see actually the depth of reliance of what you have.

Rick Yocum:

Because AI build your meeting minutes that we rely on for your

Justin Leapline:

Something like that. Yeah.

Rick Yocum:

For your structural control.

Justin Leapline:

So they gave a lot more like specific high level stuff where it feeds into the report where it doesn't feed in. But a lot of it, they actually say that there's not really gonna be... In SOC one specifically, there's not gonna be, like, a specific AI section. In SOC two, a lot of it will be incorporated into existing controls and add risk onto those controls.

Rick Yocum:

Yeah. So the TSCs aren't changing, like, the trust services

Justin Leapline:

Yeah. Criteria. Yeah. Yep. They...

Justin Leapline:

I always I

Rick Yocum:

always get paused because I'm like, is it print... It's not principles anymore.

Justin Leapline:

What is it? Well, yeah. They change it. Yeah. Yeah.

Justin Leapline:

And everything. But... Yeah. So they're not changing, but there could be

Rick Yocum:

It's being embedded into.

Justin Leapline:

And like anything else, you know, you can still add controls in both SOC one and

Rick Yocum:

SOC two. It.

Justin Leapline:

Mhmm. So you could actually have AI specific depending on the technology and process you're using. You always have that option, but still, it's probably more gonna be incorporated into your existing TSCs. Yeah. And then how you're using AI within that kind of control realm, you know, with that.

Justin Leapline:

I don't know. Both are, like, interesting. I think both are, for the most part, solid. I actually like AICPAs a little bit more, you know, into that. They're framed it out.

Justin Leapline:

Very specific. PCI is always afraid of data. I mean, they're they're so afraid of card data, and that's what they're doing. And you would get that, like, from a service company, like, doing an audit. They're, like, more looking at the process and what to incorporate into that, you know, audit process to give people assurance.

Justin Leapline:

And PCI is like, data. Like, not our card data.

Joe Wynn:

Right.

Rick Yocum:

Right.

Justin Leapline:

You know, that type of thing. So they're very more specific about the data, the responsibility, and all that stuff. Makes sense. Like, one of the things that PCI brought up was, like, if you put card data or, you know, the keys or something like that in a prompt, they're now a third party service provider, you know, type of thing. And it has to be, again, ongoing.

Justin Leapline:

It can't be just accidental. Okay. Okay. But they're basically saying, like, if you're using a fraud surface that with a whole bunch of, like, data into it also calls out to Claude to say, is this, like, good or not or something like that? That a...

Justin Leapline:

AI service with the card data Would be. Is now incorporated into your third party service provider. And now they have to have a rock or report, you know, somehow with PCI, which they might not want to, you know, type of thing. Because a lot of them say, don't put your sense of data in with us, you know, type of thing. So...

Joe Wynn:

Yeah. Did you get a chance to look into the AIC, the artificial intelligence underwriting company stuff that was one of

Rick Yocum:

This is the joint...

Joe Wynn:

Yeah. Yeah. This is the joint... I don't know. Won't call it a joint venture, but, you know, one of the newer developments is that AICPA and AIUC announced that CPA firms are beginning to offer a combination of a SOC two plus this thing called an AIC, and I wasn't quite sure what it was before, and I looked into it.

Joe Wynn:

AIC for the artificial intelligence underwriting company. They develop standards and certifications and insurance solutions for AI systems. So its primary offering is an AU or AI UC one certification standard, and it goes through six areas, data and privacy, security, safety, reliability, accountability, and society. And so that seems very much like what Line. Yeah.

Joe Wynn:

42,001 Yeah. ISO 42,001 covers. Mhmm. And the difference is is that, you know, SOC two is evaluating the org controls against the TS, the trust service criteria, TSCs, and the AIC evaluating against AI agents, including technical testing on their behavior. And there are...

Joe Wynn:

You have to be an accredited firm to get the ability to

Justin Leapline:

Both from AICPA and the AIUC. Does AIUC do the do the assessments, or do

Rick Yocum:

firms get accredited? Firms or

Joe Wynn:

for Yeah. Firms like coal fire, shell

Justin Leapline:

man, those

Joe Wynn:

will get accredited to be able to do these, and they can give it. And so when, you know, there there are some nuances that I starting to look into, and we'll have to probably dig into this and come back with some... You know, this might be a good It'd be a good focus area for another Yeah. Another Yeah. Cast.

Joe Wynn:

But when you when you start to look at it and how it's working, it's it's not just doing like point in time auditing, but there's things that they're doing on a more quarterly basis.

Justin Leapline:

Mhmm.

Joe Wynn:

And they're also running deep analysis against the AI, like technically, in order to see how it responds and then evaluating it against those criteria. So it becomes a little bit more deep testing. So anyway, I I think more to come on that, but it's something that

Rick Yocum:

that That'd fun to explore. Yeah. I I hadn't heard about that before we had our mini text thread on show notes earlier, and it came up, and I was like, oh, yeah.

Justin Leapline:

Don't know about this thing. Yeah. That would be a good thing. I'm curious how many people have had this cert and everything. I have not heard about it.

Rick Yocum:

Well, the fact that they're

Justin Leapline:

doing joint

Rick Yocum:

assessments with the AICPA will... Feels like it might spike these increase. But I've never

Justin Leapline:

heard of I I would agree it would, yeah, increase absolutely into this. But I haven't heard anybody ask about this yet. That's a new acronym to me.

Joe Wynn:

No. But is is third party vendor risk management a topic we'll

Rick Yocum:

hit on today or we will Probably not.

Joe Wynn:

Run out of time. So...

Justin Leapline:

But Well we'll we'll sort out.

Joe Wynn:

Let's tie this with TPRM because it's another thing that you could ask for.

Justin Leapline:

Yeah. Yep. And it's kinda... I mean, are we going into our next topic here?

Joe Wynn:

Well, one last thing you...

Justin Leapline:

Okay. Yeah.

Joe Wynn:

So so not big numbers on this if my AI is giving me Regions time. Good good responses here. But as of October 21, that at least... This makes it sound like it's super impressive and it's not gonna be. At least seven companies have reported...

Joe Wynn:

I I knew you're gonna laugh. Have publicly reported the AIC one certification.

Rick Yocum:

That's straight from a website.

Justin Leapline:

Yeah. Yeah. That's very funny.

Joe Wynn:

Yeah. That's from the... Looks like it's a data space website. Mhmm. So, anyway, see if there's

Justin Leapline:

What do AICPA even partner with them? I mean, that seems like small

Rick Yocum:

Well, if it's modeled or or maybe they maybe they partnered to build the thing together because AICPA knew they need it, and these guys were AI experts because they're the underwriters. Like, I I could see

Justin Leapline:

Yeah. Reasons why. Yeah. Yeah. Yeah.

Justin Leapline:

Yeah. I would just expect a little bit more adoption into the marketplace before partnering with something like that. And maybe

Rick Yocum:

they're Unless they join it together or whatever.

Justin Leapline:

Yeah. Could be.

Joe Wynn:

Right?

Justin Leapline:

And everything.

Rick Yocum:

That's cool, though.

Justin Leapline:

So Cool.

Rick Yocum:

It'll be good to explore.

Justin Leapline:

Alright. And this next topic does have a little bit kinda third party risk, you know, into this, but we wanted to talk about the PHP Apocalypse that is currently going down. Alright. This is very timely. Obviously Today.

Justin Leapline:

Right?

Joe Wynn:

It was today.

Justin Leapline:

It was Yesterday. Matches. I know. Yes. Yesterday.

Justin Leapline:

Day before. Produce it today. Yeah. You know, type of thing, but it's been going around. So what's happening here?

Justin Leapline:

So PHP two shell had two vulnerabilities that actually came out with this that were very critical. Mhmm. It's in pretty much every single version of PHP going back to 05/07, and eight. And if you're not familiar with PHP I was just gonna say. It's a programming language.

Justin Leapline:

It is very widely... It made the backbone up in the Internet, like, way back in the day, you know, type of thing. And I think still a majority, like

Rick Yocum:

If you make custom containers, you're using PHP. There's like a whole bunch.

Justin Leapline:

But you could be using like Node JavaScript as well. You can be, but there are a lot...

Rick Yocum:

It underpins from an, like, an open source ish perspective, like Yeah. A ton of stuff.

Justin Leapline:

Yeah. And I know, like, at one point, like, WordPress sites were a thing, and that obviously is the most popular PHP application out there. And there was something like... At one point, it was running like 30% of the Internet. Right.

Justin Leapline:

You know?

Joe Wynn:

Well, now it's... This this status, PHP powers approximately 69% of websites where the server side programming language can... Yeah. Yeah.

Justin Leapline:

Because it used to be the thing way back in the day. It was either Pearl or PHP that ran all the CGI stuff and everything like that. So... And and that's the stat right now. 69% are

Joe Wynn:

still Well, yeah. And and actually, this was... So this is not validated. I just did the search, but it says, here's the cybersecurity concern. The distribution of PHP versions is troubling.

Joe Wynn:

PHP eight, which is... It's only eight dot two through eight dot five that are being patched. Right? Everything before that is kinda out of support. Yeah.

Joe Wynn:

And PHP eight is 65%. PHP seven is 27% of that universe. PHP five, there's 7% out there yet, and there's even some... It's finding some PHP four versions out there. And so as you, know, you kind of think about this, like, I was curious, what are the timelines for the releases?

Joe Wynn:

Well, eight dot two was... Initial release was in 2022, and eight dot five got released in 2025. And so as you're thinking about this problem, hopefully, you're using a vendor who's hosting things that's gonna gonna auto patch you. And if not, you need to get your your stuff patched. But Yeah.

Joe Wynn:

What I'm more concerned with is what are the tools? Because if you're gonna build like a piece of hardware, if you're gonna build like an OT device, if you're gonna build something like an old NAS device, like maybe an old Synology, you still have it sitting around, it's not getting patches anymore. It's more than likely running PHP from a version Mhmm. Before it's uneven... That the patch is even available for.

Joe Wynn:

That doesn't mean it can't be patched, and it doesn't mean the vendor doesn't have the capability of applying that patch, but

Justin Leapline:

You're talking about upgrading to a supported version.

Joe Wynn:

You you could or... Yeah. Or they could back port if... Because the code's all out there. Vendors could.

Justin Leapline:

At that point, you're running AI. Yeah. Exactly. Right. Know, type of thing.

Justin Leapline:

And if you're running AI to patch one thing out of a PHP seven version, let's just

Joe Wynn:

Right. Yeah. You're probably

Justin Leapline:

Why not run it to upgrade to PHP eight?

Rick Yocum:

Well... And also, is this gonna be the last set of crit VOLMs on PHP?

Joe Wynn:

Like... Well, the thing is is this VOLM was out there for years and years and years. Right? The same one. And did you mention this?

Joe Wynn:

The reason this became a issue is it was never really patched before because the complexity to take advantage of it was so high of a bar to be able to do it that it became a problem because with the frontier models, they were actually able to go and build the exploit, put it together to actually deal with this. And what was happening here is that the the actual vulnerability was, you know, buffer overflow, buffer... And be able to get data out of the system with, you know, no authentication.

Rick Yocum:

That's wild. Can... I probably should have done this before then. Can you do a quick search to say, like, what are some of the most common third party tools that rely on old versions of PHP? Well, that...

Joe Wynn:

That's what I... Yeah. Some of them were like old NAS devices, IP cameras. Oh, yeah. Think about those things.

Joe Wynn:

I didn't really do tools. I was thinking more of like the hardware space.

Justin Leapline:

Yeah. Yeah.

Joe Wynn:

Security systems.

Rick Yocum:

Oh, like bad readers and stuff. Yeah.

Joe Wynn:

Yeah. And and if they're not being patched, industrial and OT systems Yeah. Yeah. Yeah. Might have that embedded from that perspective.

Justin Leapline:

And so, like... And one of the things that's like why this is really bad and everything, this code actually runs prior to any of your code running. Because keep in mind, PHP is an interpretive program. Yeah. So it has to spin up its own environment, then it starts reading your code.

Justin Leapline:

And when you send a post request, it's basically executing the the request in that code first before handing it off to your application to start doing logic. So it's pre auth. You know, there's no auth going through it, and it's a remote control execution, you know, into this. Here's another thing that's really interesting that puts a unrelated but interesting tidbit into this. So they were coming out with these two vulnerabilities Mhmm.

Justin Leapline:

Into this. They were originally gonna release it tomorrow into this October 22. A hacker news post, somebody posted it up. Mhmm. And it was going through their pipeline and had screenshots and everything.

Rick Yocum:

So they moved it up because it was

Justin Leapline:

They moved it up to today Oh. Release because because the news came out and it was gonna start the exploration. And they they had screenshots, so people were able to use AI and actually, like, find it themselves and start to exploit it in the wild. You know?

Rick Yocum:

So when did the screen... Were the screenshots came out yesterday?

Justin Leapline:

Yes. Yesterday.

Rick Yocum:

Yeah. Because that's when I first

Justin Leapline:

heard about this. Yeah.

Joe Wynn:

And then there was a post... They... That... I heard that post got pulled down.

Justin Leapline:

It did get pulled down type of thing. But it also puts into the, like, who's part of your trusted, like, pipeline on disclosure and everything. Like, somebody on the inside took the pictures and posted it up, you know, and released it to Hacker News, you know, into this. And then it blew up, and they were able to reverse engineer. And they had to, like, bump up their timeline for release in the patch, you know, and everything, which I'm not sure how much one day is really gonna matter.

Justin Leapline:

You know? But still, it would have been better ish, you know, type of thing.

Joe Wynn:

Yeah. The Wiz blog post was interesting because they showed some examples of how they were able to create a... An exploit for

Rick Yocum:

Oh, really?

Joe Wynn:

As a proof of concept. So That's cool. You know, and they did it pretty quickly. Anyway, and and so one one of the things to consider, so if you're if you're running a security program and you're thinking about, well, what are all the problems? What could...

Joe Wynn:

You know, what what do I have? You know, this this all goes back to normal governance. It's, you know, what what are your governance must do's? You need to know what you run. You need to track the life cycle of it.

Joe Wynn:

You need to understand, like, what versions of stuff is is is happening out there. And so... And and I've not really seen a lot of TPRM programs or even inventories get so deep where that the common company is doing where they can actually know like, oh, I have this IP camera system Right. That's running this thing.

Rick Yocum:

Oh, like SBOM on vendor products in the OT space? Like, come.

Joe Wynn:

No. I

Justin Leapline:

mean... And this is time.

Joe Wynn:

And you're

Justin Leapline:

only kind of like... And you're right. It does bleed into the TPRM. I mean, we think about this initially from a development perspective. Like, oh, if I have tools out there.

Justin Leapline:

But all the... As you mentioned, the appliances that have a web service that's running an embedded, like, PHP service or something like that, like, this kinda reminds me of Heartbleed, you know, where all of a sudden you're doing a full inventory of all your vendors and doing, like, pings to see if they have a PHP version out there running on a website you're using internally and then reaching out to them, be like, you got a patch for this. Right? Like, please tell us you have a patch. It tells me PHP seven is running on this appliance, you know, type of thing.

Justin Leapline:

And now, you know, a whole bunch of, you know, appliance vendors are like

Rick Yocum:

Well, and this is where this is where... Not that I'm like a show for the services, but this is where I see, like, the power of things like Frontier Virtual Patching and stuff like that where you're like, oh, okay.

Justin Leapline:

We're gonna watch for the app.

Rick Yocum:

I mean, we're gonna give you basically a network firewall in front of these things so that if an IOC starts to.

Justin Leapline:

Well, and this is like argue to, like, maybe we need to move to more of an architecture like a Citrix or some type of portal type interface, you know, to, like, not have it exposed, but to funnel it through.

Rick Yocum:

Yeah. It's all... I mean, it's all... Everything everything is just, like, cyclical. So it's like, move it inside the network.

Rick Yocum:

Move it outside of the network. Move it inside of the network. Move it outside. So as, you know, those musical chairs. But I I agree.

Rick Yocum:

I think as things get as things get crazy outdoors, you go, okay. Well, what walls can we set up around, you know, what parameters can we set up around it?

Joe Wynn:

And and not only think about that, but PHP, that's that's today's problem. Yeah. How many old versions of Java, Python, dot net, Node are out there.

Justin Leapline:

Node just came out with a couple of critical

Joe Wynn:

I saw that. Was today. That was just today too. Right? Yeah.

Joe Wynn:

Yeah. Absolutely. And so, you know, you get those out there.

Justin Leapline:

I was like, a PHP came out. Was like, oh, I don't have to worry about it. You know? It's like, Node. I'm like, gotta worry about it.

Justin Leapline:

Right. So... And and

Joe Wynn:

if you're Node, you're like, hey. This PHP thing's out there. It's really big. We'll just put ours out today too and Yeah.

Justin Leapline:

Yeah. Yeah. Yeah. But

Rick Yocum:

it does get back to the thing that we talked about before because you're absolutely right, Joe. Like, and so you have, like, these beleaguered IT teams that might not have AI in place to help them do the actual patches because the processes that are set up are fragile with respect to software updates.

Joe Wynn:

Right.

Rick Yocum:

And you go, oh, okay. Well, we gotta update all this this stuff. And then tomorrow, there's another thing. And then tomorrow, there's another thing. And, like, people can't work

Justin Leapline:

24 On

Joe Wynn:

top of that, you you can't even patch the version seven stuff. So what do you do when you have a Vomno patch?

Justin Leapline:

And especially... And it's on an appliance.

Joe Wynn:

Yeah. You gotta throw the thing out.

Justin Leapline:

Or interactive OSs, maybe there's a solution out there with AI that could patch, you know, different components of the OS. But appliance, you're kinda screwed. Like, you don't have source code.

Joe Wynn:

Yeah. You need to do a scan and Yeah. If you do a... So with this, if you do a... Like, can you write an end map that will scan and it'll return back the

Rick Yocum:

PHP version? Right?

Joe Wynn:

So, you know, if you haven't done this and by the time you listen to this, it's not today anymore.

Rick Yocum:

Your agent can do it.... Weeks.

Joe Wynn:

Yeah. And... Yeah. You need to you need to scan. You need to figure out what old PHP is showing up, and then you need to figure out where that thing is in your environment and how critical that is.

Rick Yocum:

In my heart goes out to, like, organizations that manage widely geographic disparate infrastructures with, like, pieces of OT in the middle of a field or underneath the ground. Like, you just don't have hands and eyes that you

Justin Leapline:

can deploy that fast to that many places. Yep. When I I was working for a security... Or working for a company doing a lot of security stuff for them when Heartbleed came out Mhmm. And we had to respond quickly.

Justin Leapline:

We use a number of scanning stuff from third parties to appliances. Our spreadsheet was like hundreds of lines long, you know, type of thing. And we're tracking versions, whether we could patch, whether we had to wait for somebody to patch, like the the response was crazy, you know, type of thing. This is basically you're gonna have to do essentially the same thing. A bunch of internal stuff, a bunch of appliances, bunch of external vendors, you know, like we're querying our vendors to see what website they're running, you know, from an external standpoint.

Justin Leapline:

So if we use a portal a portal, and they're using Heartbleed and we found out it was exploitable, they're like, when do you got patch? When do got a patch? You know, type of thing.

Rick Yocum:

And you're both absolutely right. There's gonna be like physical hardware out there. Yeah. That cannot be updated because the new stuff is too heavy for the physical hardware. And then you 100%.

Rick Yocum:

Okay. Guess we can gotta buy new devices, new appliances, gotta install them, gotta license.

Justin Leapline:

Yeah. Oh, my gosh. That Podunk firewall in a gas station or something like that. It's like, oh, yeah. Well, there's a portal interface that nobody logs into, but now, you know, like, now we're gonna have to figure out upgrade because to your point, like, hardware can't support.

Rick Yocum:

Yeah. Your break glass portal is now weaponized

Justin Leapline:

against you.

Joe Wynn:

Even though I know your password's admin admin, I can... Yeah.

Justin Leapline:

Right. Right. So I guess, you know, out of this here... So I wanna talk a little bit about to kind of close this up, what you should be doing ongoing kind of a preventive slash maintenance. And then now what you should be doing dealing with this incident, I think would be a good way to kind of carve this up.

Justin Leapline:

So if you're a shop, how how should you be kinda prepping for the next PHP apocalypse here, you know, type of thing?

Joe Wynn:

Well, that's what I was getting to a little bit ago. Yeah.

Justin Leapline:

You need

Joe Wynn:

to know what you run. Getting that good inventory. Break it down to, like, six six controls. Know what you run, track support life cycle as data, make end of life an exception, you know, not a surprise so that you know when things... You you have a schedule of when the end of life will be for these things.

Joe Wynn:

Because like eight dot two, that only has a few more months before that one's gonna be out of support.

Justin Leapline:

Your vulnerability scanner should have picked this up already if it's an active run service. Yeah.

Joe Wynn:

And and not all... Well, here's the thing. It's like, yeah, you might be able to end map and get a lot of good responses back. You're on server.

Justin Leapline:

But that doesn't mean that...

Joe Wynn:

Yeah. It doesn't mean it's not gonna mask it and be 100%. Not touch that.

Justin Leapline:

Yeah. Yeah. You're right with that.

Joe Wynn:

And then contract for vendor life cycle transparency. So for critical commercial products, you know, ask the vendors what major embedded components there are. And, you know, if you're really relying on this, so you're gonna buy a bunch of, OT equipment running your energy company Mhmm. You probably have the ability to say, tell me what these are and, give me give me some updates and and kinda have a partnership there.

Justin Leapline:

You're maintaining that

Joe Wynn:

And then

Justin Leapline:

updates on everything.

Joe Wynn:

And then the other part... The last part of that is, you know, don't measure your CVE counts. Measure the exposure. You got... We gotta focus on where the problem's gonna be.

Joe Wynn:

Who cares if it's a, super segmented area? I mean, that's not your first priority. It's your... Right. What...

Joe Wynn:

What's exposed to the Internet that you're worried about. If it's like five layers deep and, you know, supposedly air gapped, that... That's not where you should be running to tomorrow.

Justin Leapline:

Right. Yeah. That's a 100%. I would also even add into that, as you're setting up equipment, even consider whether you need, like, a web portal or not or some of the services. Mhmm.

Justin Leapline:

I mean, on some of this IoT, they just launch up a web browser whether you need it or not.

Rick Yocum:

Exactly right. Turn it off You if don't need

Justin Leapline:

wouldn't even have to worry about it if you don't use it, you know, type So... Of

Rick Yocum:

I think I have two things. I think all of that is excellent advice for the work that people need to do themselves, I would say. But also start talking to your peers and your partners about what does your process resiliency actually look like? If it's like, well, we have these on prem systems so that we can patch when we wanna have outage windows, that might not hold up as much anymore in the new world because the patches are coming fast and furious. And when there's a real big bad thing, particularly on Internet facing systems Right.

Rick Yocum:

Like, think about your process resiliency there so that you can do out of band updates on, you know, multi threaded, you know, in redundant systems and, you know, do, you know, patch a when b is supporting prod and patch b when a is supporting prod, then you're done at the end. Like, you gotta be thinking about resilient architectures. There was also, to your point about, like, how do you start to prepare for the next one? I like the virtual patching thought. The other thing that that I saw as a service that's emerging is effectively always on red teaming using frontier models.

Rick Yocum:

And so I think not everyone's gonna be able to, like, have, you know, you know, stroke the check to to do those things.

Justin Leapline:

Be expensive.

Rick Yocum:

I I... Depending on how depending on how commoditized it be... Must become. I I don't know. For now, I think it probably is.

Rick Yocum:

But...

Justin Leapline:

I was just thinking the token usage would be Yeah. So very expensive. So I think what happens models.

Rick Yocum:

Well, I think what happens is, the the big providers... So like, know, like, Paolo recently said, like, oh, we're gonna start to do one of these things. The big providers are doing it, but they kinda do it in two ways. It's sort of always on red teaming, and some of that's applied to you, but some of that is sort of this universal IOC things that agents can kinda be like, hey, we found this over here. That applies to you.

Rick Yocum:

Right? Oh, okay. And so it's almost like you do the deep tests kind of in a very distributed way. Yeah. But when something is found, they go, hey, is this a problem you have?

Rick Yocum:

Is this a problem you... Like... And then it ends up being targeted.

Justin Leapline:

So basically, it's like the showdown of, like Yeah. You know, it's like, okay. I know everybody's running PHP five three in all these places. You're susceptible. I don't need to test you.

Justin Leapline:

I know. Yeah. That's like of thing.

Rick Yocum:

Consistent persistent, like, vuln scanning with frontier models effectively. But then, you know, you have to be able to consume what's coming out of that. So what do you do there? Is it you have a virtual patching model? Is it you have redundant systems and your agents can patch thing?

Rick Yocum:

Like, how are you gonna combat the onslaught of information? But I do think, in addition to all the things you guys mentioned, like, think about the process redundancy and system redundancy and think about, like, how are you

Justin Leapline:

gonna proactively identify these things? Yep. Yeah. I think to add on to that and everything, like, we often think about even if we all know end of life systems, it... They're hard to manage, but a lot of people are just bad at managing them, period, you know, into their environment.

Justin Leapline:

You know, when those systems will come and go and they're like, oh, we're not getting any more patches. You know. That's weird, you know, type of thing.

Rick Yocum:

Well, the nature Like,

Justin Leapline:

well, you're running twenty twelve, like, of course not, you know, type of thing. But I think extending that to the applications in your environment as well. So if you're running Absolutely. Adobe, if you're running other applications, you know, into that, does your inventory system also include applications? But some do.

Justin Leapline:

A lot are bad.

Rick Yocum:

And then even like a full on SBOM for like what underpins Like like, says, I mean, Docker underpinned by PHP in a lot of places. So like, okay. What version of Docker are you running? And what does that run on? And and and and and and.

Rick Yocum:

Like, it gets crazy. Yeah.

Justin Leapline:

There are ton of stuff, into that.

Rick Yocum:

But your point about staying current, like, think I think we're at this place where risk... Organizations risk tolerances haven't changed, but just the risk of the outside world is changing because, yeah, we could sleep on some end of life systems. Right? Like, well, if one gets... Like, what's the worst that's gonna happen?

Rick Yocum:

Blah blah blah. It's like, well, now what the worst is gonna happen and how quickly it can happen is just materially different.

Justin Leapline:

And and I agree to your point earlier, Joe. If it's three layers deep and I'm running a twenty twelve system, okay, that might be Yeah. Fine. Yeah. Yeah.

Justin Leapline:

The only people that really have even network access to that are admins that already have trusted access, you know, So into am I worried about that? No. You know?

Joe Wynn:

Not as much.

Rick Yocum:

Not not today. Or... But then all of a sudden you chain together, you know, a 100 lows and mediums and you get access three layers.

Justin Leapline:

Yeah. And there there could be a potential...

Joe Wynn:

Yeah. I'm not worried about you. I'm worried about your agent that will just tirelessly poke at it till it finds it.

Rick Yocum:

Right. I use house and car analogies for everything. It's like everyone's neighborhood is getting way more dangerous.

Justin Leapline:

Not Paris. Not Paris. Not Paris. So... Yeah.

Justin Leapline:

And I think... Yeah. I think we we also too just need to... Again, I think we brought this up before even just on the AI. We really need to get...

Justin Leapline:

You need to have a 100% confidence on what your Internet exposure is, you know, type of thing. Like, for a lot of distributed and big companies, that's hard to do. You got subsidiaries that you give their own access into it, but then you're all connected into the back end with maybe maybe not trust or not trust. I was like, you need to have a solid foundational thing of what you're exposing to the

Rick Yocum:

All the paths. Yeah. And and and exposing to the Internet, absolutely. And also, like, if you have a partner and a hole is poked through your VPN or you have an MPLs and all, like, there's just a bunch of ways this can happen.

Justin Leapline:

Yep.

Rick Yocum:

And so just think about all the ingress and egress points Yep. Because And hopefully,

Justin Leapline:

structure wise that, you know, type of thing. I mean, there's a lot that's easier said than done. But I'd rather, you know, a couple of class c's I have to worry about, you know, that that they're all in my data center that I'm hosting externally than a satellite here and a satellite here and a satellite here and they're all different stacks of... Different providers, different firewalls, different... Yeah.

Justin Leapline:

Exactly. I'd rather be like, alright, if you're gonna host something online, it comes here. It doesn't go to, you know, the group of, you know, junior IT admins at this office. It's gonna be our system architects that are gonna be looking at our data center, you know, type of thing.

Rick Yocum:

This is another general sort of theme of survival, I guess, that that came up at a couple of these conferences was everyone's moving towards simplifying wherever they can. And that often means vendor centralization, platform centralization, Like, yeah, best of breed was nice for a minute, but now let's... I'd much rather have it all in one place and only have to train people on one set of processes and have modules and all the stuff. It's not universal, but most people I talk to are like, everything's getting so much more complicated and happening so much faster. The more I can just conceptually simplify my stack, the better off I'm gonna be.

Rick Yocum:

Yeah. Like, your point about consolidating network providers and things like that.

Justin Leapline:

Yeah. I mean, like, I'll give a perfect example. I'm working with one company right now that for a while, they've had different partners and brands that they've acquired over the years. We're finally, like, coming together and we're like, oh, what are you using for vulnerability management? I'm using Qualys.

Justin Leapline:

What are you using? Well, we're using Tenable. What are you using? Well, we're using Rapid7. Literally.

Justin Leapline:

Like that is a real example. It's like, why don't we all get on the same platform? You know, like, why are we paying for all these different ways of doing it? Why don't we hand it up to the corporate vulnerability management team? Why do we have these little onslaves?

Justin Leapline:

Yeah. People doing this? And I obviously, it grows organically and all that stuff. But it takes a focused effort to say, hey, there we have too much like exposed experience where it's like, yeah, somebody knows Tenable, somebody knows Call. It's like, let's centralize on it.

Justin Leapline:

Let's get under one process where we know if we have problems, we can fix it. Here, not, you know, five different places, you know, type of thing. Mhmm. And it saves money on licensing.

Rick Yocum:

Yeah. Right.

Joe Wynn:

So... Another part of simplification is is that you have these complex networks, so why not just put like a five g modem on it and let it connect, out so you don't have to worry about all this extra management?

Rick Yocum:

Just just turn off the Internet.

Justin Leapline:

Off the Internet. Yeah.

Joe Wynn:

And then... Which is exactly what's happening with those grocery stores, refrigerators being shut down because they put those on there and then we forgot

Justin Leapline:

about it. Yeah. So

Joe Wynn:

you gotta be careful what you're simplifying.

Justin Leapline:

What I was actually shocked of was it got up to a 120 degrees Fahrenheit. In the... Yeah.

Rick Yocum:

What was the physical location of these?

Justin Leapline:

They were the the freezers and everything, but I... But where?

Joe Wynn:

Across Europe? Yeah. Think they were like in France.

Justin Leapline:

Okay. France and everything. But essentially, they have what's called a a defrost mode Mhmm. Into it. And they put in a continuous cycle of defrost mode that they were getting up to 50 degrees Celsius and I had to put it in the conversion.

Justin Leapline:

I'm like, Celsius, that's really hot. And it was like a 120 degrees Fahrenheit, you know, into that. I was like, I didn't know freezers could get that hot. Like Right. Didn't think it was like a

Joe Wynn:

So so much food was spoiled. Yeah.

Justin Leapline:

Oh, yeah. It was it was terrible. You know? And they said whether it caught, like, a couple of places caught fire or was close to catching fire and everything because of, how hot they were getting. I mean, that's cooking.

Justin Leapline:

Yeah. You know, at the Oh, yeah. You know? So... Yeah.

Joe Wynn:

It was I mean, what do you get the the the barbecue up to when you're

Justin Leapline:

So medium rare and a burger is one twenty to one thirty five or

Joe Wynn:

so. You just get your smoker up to like one twenty and let it go for a while?

Justin Leapline:

Well, it's hotter than that. Little hotter. Okay. Yeah. The below...

Justin Leapline:

The the least I can put is one sixty five.

Joe Wynn:

Okay.

Justin Leapline:

But internal temperature of meat Oh, yeah. Depends on what it is Yeah. You know, type of thing. But, yeah, if you're cooking, like, steaks or burgers, like, one twenty to one thirty is, like, rare to medium rare. Yeah.

Justin Leapline:

Into that.

Rick Yocum:

Oh, you know, they're all thermally insulated, so it's just an oven at that point.

Justin Leapline:

Yeah. Yep. Wow. So There you go. Yeah.

Justin Leapline:

Pretty fine. Alright. Cool. Called a

Joe Wynn:

lot of good stuff.

Justin Leapline:

Yeah. That's great. Alright. Well, thank you everybody for joining us. Cheers.

Justin Leapline:

Cheers.

Rick Yocum:

Cheers. One more over the laptop.

Justin Leapline:

Don't forget to like, comment, and subscribe. It really helps our numbers kinda build up across YouTube. We're on Spotify now. We're on Apple also doing video. So check those out as we continue to publish out to there, And take care for next time.

Justin Leapline:

Have a good one, everyone. Bye.

Creators and Guests

Joe Wynn
Host
Joe Wynn
Founder & CEO @ Seiso | IANS Faculty Member | Co-founder of BSidesPGH
LinkedIn
Justin Leapline
Host
Justin Leapline
Founder of episki | IANS Faculty Member
X LinkedIn Website Link
Rick Yocum
Host
Rick Yocum
Optimize IT Founder | Managing Director, TrustedSec
LinkedIn
Episode 30: Exec Conference Takeaways, PCI & AICPA on Auditing AI & the PHP Apocalypse

headphones Listen Anywhere

More Options »
Broadcast by