Speaker 1: 0:00

Welcome to the Audit, your podcast on all things cybersecurity. We want to have a shout out to our marketing manager, Kent. It's his birthday today. Happy birthday, mr Kent Weber. We're recording this. On March 22nd, the White House EPA warned water sector of cybersecurity threats. This comes from CyberScoop. The EPA is also convening a task force to take on some of the challenges facing the sector around cybersecurity efforts. The White House sent a stark warning to the US governors on Monday that disabling cyber attacks targeting water systems are occurring throughout the United States. In what is the Biden administration's latest plea to state authorities to direct more resources and attention to protecting water utilities, this stood out to me. We're hearing a lot in the news about nation states and bad actors and cybersecurity threats. What's going on here? Eric Brown and Nick Mellum.

Speaker 2: 0:54

Essentially what's happening is. The concern is nation state actors are positioning themselves to be able to disrupt the water utility. So either the distribution of water or wastewater, handling of wastewater and wastewater treatment that could be quite disruptive to society if that were to happen. So kind of that laying and waiting is a concern. Where they might be in the environment not really doing anything. But then should tensions become hostile with the US, then those nation states could potentially leverage that disruption of service to create more chaos within the nation. So that's one of the things.

Speaker 2: 1:48

But in general water treatment facilities will operate with an OT network or a function of their technology that's air gapped from the rest of the corporate network, and that OT network will have its own contained ability to do patching, to do maintenance. And sometimes those OT networks are configured in a way that they shouldn't be, where they may be able to talk out to the internet. They may not go through an intermediary to get out to, say, a patching service or something that they need to do. Typically you would set it up such that inside that network you couldn't get out anywhere but to a specific computer. So it's like if I wanted to talk to Josh I would have to reach out to and call up Nick, talk to Nick first, and then Nick would relay that message to Josh, josh back to Nick, back to me, and that would prevent me from interfacing directly with anything outside. I have to go through a specific channel to do that.

Speaker 2: 3:12

So this warning is certainly out to all of the municipalities, all the state agencies to pay special attention and to bring best practices in attention and to bring best practices in. And there's lots of great federal programs going on right now coming out of Homeland Security and CISA. So CISA is the cybersecurity and infrastructure portion of Homeland Security and that organization is focused on bringing out those guidelines and those mandates but actually helping organizations with security assessments and walkthroughs and tabletop exercises to make sure that not only do they know how to implement best practices, but that they're testing themselves. And it is a requirement for these water treatment facilities and water distribution facilities to do annual testing so they know where they are, are they up to speed on all of the patching, do they have any vulnerabilities that have been exposed, things like that.

Speaker 1: 4:19

Eric, could you give us some insight on what is actually at risk here? Is it that the water is going to be contaminated, is it that there's not going to be potable water for the population, or what are the risks really going on here?

Speaker 2: 4:35

Yeah, so you could interrupt the distribution side of water. You could interrupt essentially the cleansing of water. You could interrupt the essentially the cleansing of water. So typically what happens with most water treatment facilities is they'll bring in sewage and they'll treat that water and that treated water will then go back out into a river or stream. What have you? Out into a river or a stream, what have you? So you can imagine what would happen if raw sewage at a large scale was being dumped into bodies of water. That could potentially cause substantial damage. And then on the other side of the coin, there's the distribution of water and clean water. And causing any sort of disruption in either the intake and cleansing or the outtake, as you can imagine, would be quite disruptive to society.

Speaker 1: 5:38

Are these like gates and systems that are automated, that move water from place to place and kind of direct these processes that are treating the water?

Speaker 2: 5:50

then yeah, absolutely. There's an intricate series of lifting houses and controls, gates, as you mentioned, to move and contain and treat that water. So those systems are controlled by computers and technology and are just as susceptible to vulnerabilities as computers that might be on the corporate network. The idea is you isolate these machines and these technologies vulnerabilities as computers that might be on the corporate network. The idea is you isolate these machines and these technologies so that they can't have any interruption from the outside. And maybe the most famous of these examples is what happened in Iran with the nuclear centrifuge or the facility that was refining plutonium and they were using these centrifuges to do it and the US government and not report the spinning of that rapid spinning. And then that took those centrifuges. It essentially broke them, broke quite a lot of them and set back their program for a number of years and I think it took a little while for them to realize what happened, but the damage at that point was already done.

Speaker 1: 8:00

Given recent concerns about cybersecurity threats to a water supply, it seems like the water supply is particularly vulnerable, or at least it just hasn't been shored up the way some other systems might have been, according to the article. Can you give us an overview, and maybe, Nick, you could jump in on this too? What are some of the challenges when safeguarding the US water supply? You'd mentioned, Eric, that each water treatment facility has different mechanisms and different processes in place and different equipment, I'm assuming. What are some of the other challenges in guarding the water supply?

Speaker 3: 8:36

I think that it's interesting to have this conversation because we have all different types and sizes, like Eric was saying, with the municipalities, right, you're going to have bigger ones, you're going to have smaller ones, and the smaller ones are probably they're getting this message and they're wondering what are we going to do next? Because they might not necessarily have the bandwidth to you know or understanding how to work on this or better their systems. And I think that's when you really want to take hold and use the different programs that are out there that Eric mentioned to make sure you can share up your operations, but then also reaching out to a third party for help, because a lot of the areas that are going to be impacted, which is basically everybody, they just don't have the know-how or what the next steps might be. And when we're having these conversations it's really no different to me than a risk management we need to understand what we're dealing with and how we best want to attack it, because now that you know we've seen these attacks happen right, and they're kind of inevitable.

Speaker 3: 9:33

It seems like you know this is going to be one of the easiest ways for especially like a large nation state to, you know, inflict mass panic right, you cut off the water. That's going to be one of the biggest things that they're looking for and in my eyes, right, it says that in the article they talk about that it's obviously kind of what they're going after. But yeah, that's real quickly. I thought it was interesting to read this and then think about what are the smaller players going to do that might not have an understanding or the bandwidth slash manpower, you know, to get up to speed quickly.

Speaker 1: 10:10

So it sounds like one of the key problems with this is that they're laying in wait some of the bad actors they could be infiltrating, infiltrating these systems and then kind of waiting. You know the article had pointed out that they might be laying in wait in case something breaks out in Taiwan, for example with China, or you know there's some more things start to heat up with Iran. How do you detect those breaches that have already happened but they're not, you know, showing any signs of an exploit or showing any signs of disruption, but they're just sitting there waiting. What kind of tools do you guys use to detect those things and kind of clean things up?

Speaker 2: 10:45

It really starts with understanding that whole environment, josh, and like I was talking about that OT side of the network, having that physically separate from the Internet and any other network is one of the best things that you can do, because there's always a vulnerability for a particular operating system and if you can separate that operating system from everything else, you reduce your chances of risk. Then the only way for a vulnerability to get into that environment is to bridge that air gap and that's what I was referring to, what happened with Stuxnet. In the case of water treatment, the plants by and large can operate independently and on their own, so they don't need to talk to other plants or other systems. They certainly do in their normal day-to-day operation, but in the event of an emergency they can operate on their own. The challenge becomes and I think what you were getting to with your question is if the environment is compromised, then how do you know it's compromised programmable logic board or PLC, which the board might be reporting.

Speaker 2: 12:25

In the case of Stuxnet that maybe the RPM was 14,000 and that's a normal range, but really it's spinning at 20,000 RPMs and that's in a dangerous range and it caused the machine to break apart.

Speaker 2: 12:41

But the, because that PLC board was reporting 14,000, uh, the, the operators and everyone looking at it wouldn't know that there was something wrong with it. So that's really the the true danger of not having great control over critical infrastructure and water treatment, over critical infrastructure and water treatment, just like electricity, is critical infrastructure, and the purpose of the letter was to bring awareness that we need to pay attention to all areas of critical infrastructure and maybe not just focus on ones like power generation or transit, but that water treatment and water distribution also needs to have the same amount of rigor. And I know the people operating in the industry know that. But I think this was bringing that awareness to elected officials so that money could be allocated. The federal government distributed a lot of infrastructure money a few years ago and then the states are figuring out what to do with that infrastructure money and some of that money is allocated and mandated to help areas improve security around critical infrastructure.

Speaker 1: 14:07

It'll be interesting to see how much this narrative plays into the next election. You know you guys are behind the scenes. I don't know if you're preppers or not but, on a scale of you know one to 10, one not worrying about it at all, sticking your head in the sand to you know ten. You know going to Fleet Farm for Nick and stocking up on a lifetime supply of cat food for all his cats.

Speaker 1: 14:34

Where do you guys fall on the preparedness scale? I have some water I like to keep on hand. It's a little old. Right now I think I'm due for a trip to Fleet Farm. But just curious are you guys preppers? Do you think about this outside of work?

Speaker 3: 14:50

I would not consider myself a prepper. I am prepared, though I'll say I do have plenty of cat food. I have one of the wise company. They make the buckets of food that lasts for I don't know. It's like 30 years, uh, in a basement or whatever so I have I have that, have some water and, the most important, you gotta have some firearms. So I'm covered covered there.

Speaker 3: 15:20

we know you're covered there, just in case anybody tries to come steal all my toilet paper. But yeah, I wouldn't consider myself a prepper, but you know staying as prepared as I can. But in the news, especially down in the Texas region, right now, we're getting ready for the solar eclipse. I'm sure everybody's heard about it. We've got people flying in from all over the country to come see it.

Speaker 3: 15:43

So they on the news the other night they were warning about, you know, filling up your cars with gas. Get extra food, water, toilet paper, all that stuff. Because there's so many people coming into the state to see the eclipse, because I think down especially kind of by Austin, houston and Dallas, kind of in that triangle. The eclipse is going to be totally black us out. So people are coming in from all over. So just a little tidbit but it kind of went into the prepping so I stocked up on a couple of years worth of cat food and all that to go for Mr Miyagi.

Speaker 2: 16:18

Oh, boy Josh, I am a one on that scale. I probably have about four days worth of food in the house.

Speaker 1: 16:29

You have a dryer though, an air dryer, a dehydrator, correct, yeah, a freeze dryer you should be locked and loaded my man.

Speaker 3: 16:39

Well, you know, I'll go down to nick's house if uh we're good yeah I didn't give a number, but I I think I I maybe I'm a healthy five or six right, so I'm kind of a happy medium I like to think of myself as like a six or seven, but I did get this survival handbook just to have on hand.

Speaker 1: 16:58

You know it goes through all of the things, all the skills, briefly, that you would need to know.

Speaker 2: 17:03

Whatever somebody says they are, you add to. So that means Josh has a buried container in his basement, buried shipping container, and he's ready to go. He probably has a small arsenal down there.

Speaker 1: 17:17

Extra guitar strings. And yeah, all right, this is coming from the Verge. We're one step closer to a global cybersecurity standard for smart home devices. You know this is something that's come up from time to time. I think it's interesting. Internet of Things announced this week the CSA's IoT device security specification is a baseline cybersecurity standard and certification program that aims to provide a single, globally recognized security certification for consumer IoT devices. In the past, we had talked about the Roombas and some of the security ramifications of being able to hack into that camera and peek around someone's house. We've talked about rings and the data that's being collected there on all of our devices. Curious to see what you guys think about this. Is it going to really make a difference? Is this kind of just a little too late or helping provide an awareness to the masses, the consumers?

Speaker 3: 18:22

It's a good article, josh. I don't think it's too late by by any stretch. I think we're just evolving in this space, you know, as we go. So being get these standards and policies up is just a natural progression. You know right, it's going parallel with the technology. I, overall, I'd say I feel like it's a good thing right, although in my mind I'm thinking, oh goodness, it's not something else we might need to comply with or need to do compliance audits for, which is a space I'm happy to live in.

Speaker 3: 18:50

But I think overall it's good, just because it's going to keep maybe the smaller players that are making maybe more inexpensive IoT devices. It'll keep them honest, it'll keep them having to comply with this checklist that that is in here with a couple other items. So I think, from a from a high level, it's a really good thing just to make these, these items, standardized, just to make sure you know everybody is on the up and up. But, uh, yeah, I mean, if I feel like you know, almost every day now in the news you can read something about iot devices, you mentioned the Roomba one where I think there was some woman that was using the restroom and there was pictures on the web of that.

Speaker 3: 19:29

You know child or children's baby monitors, we see all that kind of stuff which is scary to everybody. But yeah, I think, strengthening this, you know we see other countries are already doing- it. It seems like a necessary standard that you know we should probably be putting more resources into, and then having you know just common standards that can evolve is, I think, a good step.

Speaker 1: 19:56

When I think about Eric's residence, I'd like to think about Tony Stark. You know, here at Airline Enthusiast or Airplane Enthusiast, and you know, judging by some of the things at our office, I often imagine that you have some pretty cool tech stuff at your house, eric, and maybe you too, nick, that I'm not even aware of. But do you guys have some favorite IoT devices around the house, stuff that you like to use, and do you take precautions when using those things?

Speaker 2: 20:25

Are you going?

Speaker 1: 20:26

No, you go.

Speaker 3: 20:28

I was honestly waiting on the build-up. Because you said, because he was saying they got the Tony Stark house, I was like no, let's see what he's got. What do you got, Eric?

Speaker 2: 20:39

You know. So on the protection side of those things and it's similar at home, depending on you know you can you can go down a rabbit hole with any of these things. On the corporate side, I recommend, and we recommend, putting IOT devices on a separate network. So keeping those off of the corporate network is just a rule of thumb. There are some great technologies out there that will help you identify what IoT devices are on your network and that can help with management of those devices. But just putting them off on a different network is important.

Speaker 2: 21:19

A couple other things on the IoT side is a lot of the IoT devices are manufactured by multiple vendors and this is why there's a problem. Behind it or behind them is you have, say, the overall manufacturer whose name's on the outside of it, but then inside of it you have maybe 20 different controllers, a couple of different cameras. They could all be made by different suppliers and having those individual suppliers may have security issues on that particular device. But the company who is assembling it might be a small device on a camera that's doing some form of CMOS processing or what have you that could have that vulnerability that's leading to a larger exploit. It could be the wireless controller on it, lots of different ways that that device could potentially be or have vulnerabilities. So figuring a way out in which to secure these devices is difficult.

Speaker 2: 22:54

And it's the CEO of Ford I think it was a couple of years back talked about this not from a security standpoint, but talked about it from competition with Tesla. You look at Ford, the Ford Lightning, and what they're doing with their electric vehicles, the Mustang, and the trouble that they've had getting to market, the trouble that they've had with recalls, post-launch and just in general the trouble they've had with electric vehicles, non-tesla vehicles. Not that Tesla hasn't had its own slew of problems, but the manufacturers like Ford or GM, are going out to third parties to procure their parts for their electric vehicles, and all of those different parts may not speak the same language. They may not all be able to easily be updated with BIOS and firmware updates, and the CEO was saying that with Tesla they did it all in-house and Tesla can push firmware updates to all of its components over the wire. That's why Tesla can do digital recalls of their products.

Speaker 2: 24:13

They can add or take away features of the car all remotely, because they control it all the way down to the individual device that may be part of a much larger unit.

Speaker 2: 24:28

So when you compare that to what's going on in the IoT space, you have that same thing just operating at a much different scale of economy, where the devices in a camera, you know there could be 300 different pieces of technology in that camera, anywhere ranging from a dollar down to 10 cents, and they could be all made by different manufacturers. So how do you control that ecosystem? It's going back to the previous article we were talking about with air gapping. Put that IoT device on a separate network at home or in your business and that will start to help you secure that device. So if you have a Ring doorbell and Alexa and all of those sorts of things, yes, it's convenient to have those devices on the same network and be able to talk to each other and work in harmony. If you can do that while keeping them separate from your home network where you have your work machine is a good idea.

Speaker 3: 25:42

I was waiting for Eric to talk about him on a wait list for the Ring flying smart camera.

Speaker 2: 25:48

I am I know me too.

Speaker 3: 25:51

I am too Still waiting for that I think it's going to be a long wait. We'll see if it ever happens.

Speaker 1: 25:57

But I think, Amazon has the car now.

Speaker 2: 25:59

I think you can get it.

Speaker 3: 26:00

Unless I'm mistaken, you can get the car that drives around, but Ring has the, like we were saying, the flying versions.

Speaker 2: 26:08

You mean the car that drives around in your house with the camera on it.

Speaker 1: 26:11

Yeah.

Speaker 2: 26:12

And the Ring is supposed to be coming out with this flying camera that. I know we want to get it to take it apart and see if we can find some vulnerabilities in it, but it does sound like a pretty cool device to be able to go and check on something remotely in your house.

Speaker 1: 26:28

Maybe we check on if there's any bug bounties available for something like that. We have a little office party.

Speaker 3: 26:34

Call for papers. That's a good one.

Speaker 1: 26:39

Nick, you sound like you've leaned into the IoT. You's a good one, nick. You sound like you've leaned into the IoT. You're a tech guy. What are some of your favorite IoT devices around your home, and do you take any precautions when using them?

Speaker 3: 26:50

Well, I think at the very least, yeah, I have a separate network per se, like a guest network that they live on through the URL system. But yeah, I think probably 80% of all our light bulbs in the house are the Philips Hue lights. Those are nice Yep they work really conveniently. Got the cool lights behind the TV, you know, so you can change colors for watching a movie or whatnot. But yeah it's fun.

Speaker 3: 27:16

We have our nephew in town right now and he's been changing the colors on the lights um, far too often to make me think about taking them off, but anyways, it's really fun. So, yeah, I do like the light bulbs and, uh, it's a super convenient because you you know my wife and I, you know you lay down in bed and there's no longer a fight who's gonna go turn off the kitchen lights that were forgot. So that's super convenient. We do have the smart vacuum, but honestly it is the best one, I think, or the one that gets the most use is the keypad, you know, similar to like one on your garage door. We have the one on our deadbolt, you know, for the front door. So we can, we can set codes. If somebody's coming over to to let the dogs out or feed our 50 cats, we can give them a special code to come in but then we can turn it off right away.

Speaker 3: 28:05

So if it's like somebody, if you use a paid subscription or somebody to walk your dogs, you can give them a special code and then only have it be good through a certain amount of time or a specific time, and then if you are no longer using that, service you can retract that entry access so.

Speaker 3: 28:23

I think out of all of them that's probably my favorite is just the front door. I can check if it's locked or unlocked and then you can see who's entering when, if you have kids, but for me it's a small one, but I think that might be my favorite.

Speaker 1: 28:39

I'm going to ask a real basic question here. Don't tease me for my lack of knowledge, but how do you set up a separate network in your home For someone like me? I'm just running on my Wi-Fi. What's the fastest, quickest way to set up a separate network without having another big bill every month?

Speaker 2: 28:59

Yeah, so it starts real, simply as you need a device, an access point, a modem, what have you? The device that is going to talk to your router and also talk to all of the devices on your network. So Nick mentioned one called Eero, e-e-r-o. That is a mesh network, mesh wireless network home device. So you plug in your internet, pick your favorite internet company, whoever it is, doesn't matter. You plug that connection into the Eero device and then the Eero device essentially creates one or more wireless networks for your house. And if you have a large house, you can move those additional pucks into different rooms in the house. So it will then talk to the puck that's say in the garage or the one downstairs in the basement, and you don't have to worry about, you know, are you placing your central router, your central Eero hub, in the right spot in your home? You just add these small wireless access points throughout your house. So that's, you need a device that's going to be able to allow you to configure networks. So if you just take the one that you get from your internet service provider, it's unlikely that that one has the capability. They might have the capability to create separate wireless networks. So the gist of it is you need to create more than one wireless network.

Speaker 2: 30:36

So the gist of it is you need to create more than one wireless network so you could have a Josh network and that's for all of your say your, your family's devices. I wouldn't use my first name or the name of my house or anything identifying in the name of the network. I would create something fun pick your favorite movie, whatever you want to do, but create a network. And then you could have a guest network. So when your friends come over or you have people over, let them get onto that guest network and you keep that guest network separate from your internal private network that your computers and your musical instruments are on. And then you can further subdivide that out and say, okay, I'm going to create an IoT network and I'm going to put all of my IoT devices on this network. They're going to be able to connect to the SSID that's, you know, josh Music, and they're not going to be able to talk to the network that's Josh or Josh Guest. So you just use the technology to create those separate SSIDs and then you can create you know it's all graphical of how you do this. You can say, well, I only want the guest network to be on between 10 am and midnight because there's probably no reason for it to be on after hours. That way you prevent people sitting outside your house jumping on that guest network Not that that would happen.

Speaker 2: 32:12

But if you live in, say, an apartment complex, then you run into sharing of networks so you can create these time periods when the network is on or off and you can create where that network goes. So you could say well, the Josh Guest network only goes out to the Internet and you can get more specific. The better the gateway technology is, the more geeky things you can do. So you could say well, if someone on the Josh network is going out to the internet, they're going to watch Netflix or whatever it is. They can have full access to the 50 meg connection or the gig connection, whatever your internet connection is, but I'm only going to let the guests have 10% of that.

Speaker 2: 33:02

Let's just say and people are also doing these to control how their kids access the internet, so you can have quiet time where you hit a button and the network is off. You could filter down. Well, they can only go to these sites, right, they can't go to Reddit or YouTube or whatever it is you want to have those restrictions be. You can set that up. But to answer your question, it all starts with a device that's capable of doing that level of configuration. The devices aren't really expensive. I think you can start getting into them for a little over $100 to have that level of configurability.

Speaker 1: 33:50

That's great. Yeah, I always think it's funny to see some of the creative names people come up for their networks. In my old neighborhood there used to be one called FBI Surveillance Van. Have you guys seen any funny network?

Speaker 2: 34:04

names. I have seen that one.

Speaker 3: 34:07

I used to have a good buddy of mine that his was like To Catch a Predator or something like that, or CIA Safe Van or something like that. But yeah, there's a lot of good ones out there safe van or something like that.

Speaker 2: 34:20

But yeah, there's a lot of good ones out there. We do a, when we do a roadshow with some of our customers to talk about wireless security and device security, we'll bring along a device called the Pineapple, and the Pineapple is essentially a way to attack wireless networks or, as we like to say, research wireless networks. The interesting thing about wireless networks is any of your devices, so your mobile phone. If you allow it to connect to wireless networks, it's going to retain the network that it connected to and most people don't clear that information out in their phone. So they'll have Starbucks, they'll have airport, you know, they'll just have dozens of different networks stored in the phone that the phone had previously talked to. So when we do these educational sessions, we'll bring in a pineapple, turn the pineapple on and just have it sitting in the corner, and then, halfway through the presentation or whatever, we'll go over to the pineapple, do a screen share and show all of the different wireless networks.

Speaker 2: 35:36

And inevitably somebody has a funny name to their home wireless, right, like it.

Speaker 2: 35:43

You know it could be Mr Meowgi or whatever it is right and you see them pop up on the screen and then you kind of hear, you know, laughter in the room and whatnot, and as people are seeing, not only the Starbucks and all of those networks show up, but they're also seeing the General Meow or the Mr Meowgi whatever Nick you've called the networks, because their mobile devices are playing back those networks, because the pineapple device is essentially announcing itself as those networks, because what happens with wireless devices is they are always trying to associate with a network. So if you disconnect them from the network, they're going to reach out and they're going to say Starbucks, are you there? And if they're not in an area where there's a Starbucks, they're not going to get a response. So they're just going to go on to the next thing and they're going to say you know, caribou, are you there? Nope, and then you know they'll get to all of the networks in their list until they get one and then they'll stay connected.

Speaker 2: 36:54

What the Pineapple does is it will announce itself as any of the networks that we're calling out. So if a phone is calling out to Starbucks and saying Starbucks, are you there, the Pineapple device will say yes, I am, and it will be able to get that SSID. And that's what we're showing back as a light man in the middle attack. But really it's kind of fun for awareness because most people haven't seen something like that happen.

Speaker 1: 37:23

Yeah, that'd be fun to do on the show. I think we're going to explore looking at some hack five tools, and I've seen the pineapple come up in that package. Maybe we get Cameron on for that one. Hey guys, it's been a great conversation today. We've touched on some topics that I've been meaning to get to for a while. I'd love to go more in depth in the future on IoT and even the water management stuff, but I think we'll leave it there for today. Appreciate your time and thanks for joining us on the audit.