Speaker 2: 0:04

You're listening to the Audit presented by IT Audit Labs. I'm Joshua Schmidt, your co-host and producer Today. We're joined by Nick Mellum and Eric Brown, and today our guest is Thomas Rogers from MetaCTF. Thomas, how are you doing? Doing good? Thanks for having me. Absolutely Well. We're going to jump right in. I'd love to get a little background on you, metactf, what it is, what you're working on, and then we'd love to talk more about hiring, and then employee training, cybersecurity, all those good things. So, without further ado, I'll let you take the mic.

Speaker 3: 0:32

Sweet Thanks. Yeah, so I'm Thomas. I'm the co-founder of MetaCTF, so we're a cyber skills platform. We got started basically building, managing, hosting, running CTF competitions. Our background is all about learning by doing and making cyber more accessible. I think that's why we ran the first CTF competition about 10 years ago and it was really just meant to be kind of this fun thing. And then we did it for a couple of conferences, went well, and then a couple of large companies reached out and asked us to do it for them and we sort of have grown and evolved since then. But yeah, really our whole mantra is just around helping companies run better cyber skills programs, manage it, measure it and all of the above.

Speaker 2: 1:22

Absolutely Well. We ran into your partner, Roman, the co-founder, at the recent Secure360 conference and I believe Nick and Eric have something to work out here or something to share about how that all went.

Speaker 1: 1:36

Well, yeah, I'm really excited because Eric said if I won he was going to double my salary for the next couple of years. And we tripled it. There we go.

Speaker 2: 1:45

You heard it here first. I was trying to psych the guys up. I'm like we're here to win boys.

Speaker 3: 1:48

Let's go, make me proud.

Speaker 2: 1:50

This is my first cybersecurity conference, so just to give our listeners a little background. Thomas, for those who don't know what a CTF capture the flag event is, can you give us an overview and maybe how MetaCTF likes to run theirs or create theirs?

Speaker 3: 2:04

Yeah, so there's a bunch of different types of Capture the Flyer competitions. The one that we typically do is a Jeopardy style competition, so we provide a host on our platform website. We provide a list of what we call challenges. They're really just questions, they're isolated, they're generally bite-sized and contained to a specific topic or subject or technology or something like that and, yeah, users basically just have to solve a problem in exchange for points.

Speaker 3: 2:33

It's time boxed, so I think the secure 361 was like a few hours. That's pretty. You know that's a pretty good amount of time. We do a lot of them for big companies, where they'll do it as a part of you. You know, security awareness month or you know whatever sort of security week or month they have, and we have some companies that will run it for like a week at a time. That's pretty long.

Speaker 3: 2:53

But the good thing about Jeopardy style is you can jump in and out, like if you're working on it and you got to like get lunch or run to the bathroom or something you're not worried about. You know, maybe lose a little time, but you can really kind of choose your own adventure with these. You can also like accomplish a lot in a short period of time so you can start with the easy challenges and get through a few and then be like, hey, I just, you know, got you know a few hundred points for my team. But yeah, the idea is that you as your team are collaborating, trying to score as many points as you can and beat the other teams, but yeah, in doing so, hopefully also learning some new things and getting exposed to new concepts. It's all very hands-on.

Speaker 2: 3:41

It was fun to see the whole IT Audit Labs crew there. Eric graciously put in the bill for breakfast. We had the whole crew there and I could tell Eric was jazzed up. It was like a kid in the candy shop Christmas morning. It was really cool to see Eric out of the office. Eric, can you tell us why you like doing these things and bringing the whole team? It felt like a really cool vibe having the whole crew there. It was a camaraderie and a lot of fun, absolutely.

Speaker 4: 4:04

Yeah it's. You know these sort of things that are the kind of tease the brain. It's almost like an escape room type of thing, but you're virtual and you can bring together a team of diverse engineering skill sets and people don't even have to be programmers or you know really deep security engineers to solve some of the problems. A lot of it is really just critical thinking. And there was one particular problem that I recall in the CTF. That was the question essentially prompted you to go to a X feed and it was a picture of Zuckerberg in front of a building. And the CTF question, the meta CTF question, was essentially where was this taken? And you had to give the GPS coordinates of the exact location that Zuckerberg was in when he took the photo. So you know it's like well, wow, how do we, how do we break this problem down to get the answer. And you could be in a, an English major or you know a musician and still have the critical thinking to be able to break the problem down into bite-sized chunks to get to the answer.

Speaker 4: 5:27

All the technology is freely available. You don't have to program anything, you just have to figure out well, how do we figure out where he is. You could research to see that day where he made the post. Well, where was he? Oh, he's at the headquarters.

Speaker 4: 5:45

Okay, let's go into Google Maps, zoom in on the headquarters. What's the background of the building? Look like that. He's in All right. He's outside. There's plants, there's a boardwalk, and you just keep getting it narrower and narrower and then you've got a few tries to put the correct coordinates into the question and then that's how we narrow it down and got it. But there are really technical questions too of cross-site scripting that you do have to know some coding or some scripting capabilities and how websites work in order to solve the problem. But the great thing about bringing a diverse team is you can have some people working on some of those really technical things and then other people working on some more of the general things. And as long as you're communicating and able to break down problems, you know you're going to do well and you know I think part of the reason Nick's team did so well is because they were sitting near us and we probably weren't real quiet as we're discussing it.

Speaker 1: 6:46

That's definitely the problem, definitely the problem. We have our secrets, but I can't give them out here.

Speaker 2: 6:53

That's great. So, thomas, tell us more about MetaCTF. What got you into this?

Speaker 3: 6:57

space focused on providing that high quality experience and, you know, giving people the confidence and the exposure to these concepts. I think it's a. I think it is a good way to step out of your comfort zone as you're, as you're learning new things in technology, not just cyber, but technology, so you're learning how you, you know different ways that specific technologies, programming, languages, computers, like how do they work, how are they connected, so that sort of being like the baseline problem that we want to solve, and just like providing good experiences for people who want to learn new things. I think our mission is sort of evolved into actually helping these security teams.

Speaker 3: 7:46

Generally, skills development is kind of a nebulous.

Speaker 3: 7:51

It feels like not super clear, like how do we do this, how do we maintain it? I think setting it up, you know, setting up training or setting up skills development at first, at first maybe you can do that, but how do you maintain it? And then, how do you measure the effectiveness of it? And so those are some of the business problems that we're interested in helping managers, especially security teams, deal with. And then obviously, the individuals, especially as you move to big companies, for sure, but even smaller companies, especially as you move to big companies, for sure, but even smaller companies. People are hired in a lot of ways as they're going to get to do a specific set of things in their job, and so how do you learn new things if you're doing the same 10 to 15 tasks over and over again? I mean you can reach for tools like ours, ideally, where you can get exposure to new things, to kind of upskill. So, yeah, helping managers to run and maintain and measure these programs and then helping the individual to get exposure and try new things.

Speaker 1: 8:59

We've noticed too, thomas, all the reasons you said before. But for us it's also become a really big like camaraderie event and we do them at one of our accounts regularly. We have a block on a day of the week and everybody usually jumps in and does that challenge either together or separately, and comes in, you know, later to the meeting or we discuss it later. But it's been really fun. We've been doing this for a while now and it's really fun just to come together and figure out if either somebody couldn't solve it and then they learn from somebody that did, or we'll do it together and or they'll do it together and you figure it out. And it's been really fun to see them learn that way. But then it's also done. The two things brought them together. You get that collaborative item and then you get that camaraderie piece, but then they're learning new ways to solve problems and in critical thinking.

Speaker 3: 9:49

I was in New York earlier this week it's funny you say that because I was I had coffee with a woman who runs detection response for a big fintech company and she said the way she put it. I was like we need to talk about that more often. She was like my current company does not have a CTF culture and like companies I've been at in the past have a CTF culture and it just builds like yeah, it's not even like, it definitely is partially. Like you know, they encourage professional development and growth and that kind of thing and the company like actually prioritizes it. But it's yeah, yeah, it's like the collaboration like do we work together as a team? Do we do we do things well together? But yeah, uh, funny you mentioned that because I was thinking a lot about that this week yeah, it's like the.

Speaker 1: 10:34

it's like, instead of just being a collection of team members like everybody, can you know, we want to be one team that people can solve the same problem, work together and be collaborative, and I think a lot, a lot of these you know questions or the CTFs that you go to you know, solve that problem. And one thing that I have noticed going to other cybersecurity conferences is, you know, you get a lot of great talks and tracks and classes, but a lot of people, I think, from my point of view at least, are going to the CTFs at the conferences because they want to play the games or they want to do the badge challenges or whatever is the side, the parallel pieces to a conference. But most notably to me, it's what are the CTFs? Or that's what I'm interested in. What do they have for the CTF, what's going on, what's the game, et cetera. So it's cool to see everything growing in this space.

Speaker 2: 11:21

How do you see that really benefiting a company as a whole? I'm sure you get to work with a lot of different people and see a lot of different company cultures, right? Yeah, I think.

Speaker 3: 11:31

I mean there are kind of obvious ways. I think maybe I would assume, eric, you've got a pretty strong retention rate, employer retention rate, happiness, I think in general People enjoy working there. I think in general, you know people enjoy working there and I think one of the challenges probably with recruiting you know for you all is like you need talented people and talented technical people and you know if you're competing against it, probably in some cases, like the, you know Google and you know large tech companies, you know what. How do you differentiate? And I think that's that's how you differentiate. So so yeah, I mean, I think I think the ability to recruit well and just like get talented people, that maybe are, you know, going to be, maybe they'd be less happy there, but they can, who knows, maybe pay better. I would assume maybe Google pays a little bit more, but yeah, you get people in the door and then you keep them, and there's just so much to be said for continuity, I think, on Teams.

Speaker 2: 12:46

We have a cat room. Google has a nap room, but we have a cat room, yeah.

Speaker 3: 12:51

How do you beat that? Google has a nap room, but we have a cat room.

Speaker 4: 12:54

Yeah, how do you beat that? I was talking to a friend of mine and he'll get mad if I name the exact company, so I'll say it's either Amazon, microsoft or Google. He works at one of those three and he was kind of giving the analogy that working for that company is almost like bringing the really attractive person with you to say you know a prom or a party or you know an event where you know people are like wow, you know high five or whatever for bringing that really attractive person. And you know not everybody's going to do that. But you know you might get that cliche of like wow, right, that's cool. But he said what they don't know is the craziness behind the scenes. Like that really attractive person might be crazy, kind of outward appearances are attractive, but all the behind the scenes nonsense is really not attractive. But nobody knows that. And he equated that to working at one of those three companies. Like yeah, it's really cool, pays good, but it is a mess behind the scenes.

Speaker 2: 14:08

Really bad credit score. I've seen this affect hiring. A lot of people are using chat GPT now, so we've been talking about Cluely around the IT Audit Lab's office. How are you differentiating between people that can actually move that mattress out of the highway and people that are just using tools or maybe answering things using some of these AI tools that are coming online?

Speaker 3: 14:36

I saw I think it was a tweet last week that was like the cheating during hiring has gotten so bad that I'm almost ready to ask candidates to close their eyes when they answer questions uh uh, we've heard that too, I mean.

Speaker 3: 14:53

So I I think I think it goes back to what eric was saying about the escape room. It's like it's all about process, thought process, like can this person problem solve? And then how do they talk about solving the problem? You can't cheat your way through that like you're gonna, and and I mean I guess the best way would be like let's go back to like the old school, like interview in person deal. I don't remember ever having a virtual interview before 2019 or 2020.

Speaker 3: 15:25

But but yeah, I mean I think I think it's people use CTF challenges and our platform. We have kind of a candidate assessment portal that we provide to help interview and it's basically like case you know case interviews. It's like here's a situation like talk to me about how you would solve it, and using that on a live interview I think is is a way to. It's not just like here's a you know assessment portal. It's open for two hours. Like you know assessment portal. It's open for two hours. Like you know, do it when you can.

Speaker 3: 15:56

And I still think, with CTF challenges, like you can use LLMs and you know cheating tools, but it's only going to get you so far for most challenges. But yeah, I mean, I think it's all about thought process and like understand, like how does this person communicate? How do they talk through? You know solving a problem and you know stuff like Clueless will help you. You can't you can't fake your way through like true knowledge. You can, I think you can fake your way through like knowing bits and pieces, but it's just not going to get you all the way there.

Speaker 1: 16:29

I'm the way there I'm. I think I'm most curious about this whole thing is, you know, I guess, one good on the candidate for being so tech savvy, I'll give them that. But two, let's say you get through the right, you, you get through. You know, you don't. They don't find out that you're cheating. What happens on day one? Exactly what happens when you get there and start doing the work, right? I mean, most places have a 90 day, or you, whatever it is, grace period or whatever that they can let you go right. Probation period is what I was looking for A lot of times. If this is the lengths you're going to, you're probably going to get found out within the 90-day period or whatever it is. So I guess I'd like to interview one of these people that have come through there and asked like, what was your plan when you actually got there, right? Were you going to rely?

Speaker 2: 17:15

on.

Speaker 1: 17:16

AI so much and hope you were remote the whole time that you could. Just, I'm very interested in what the thought process is, but to me it also is really cool that this technology is here, right, that we're able to see this stuff and how people are using it. Obviously, not always in the best light, right, we don't want people to do this. But I think, going all the way back to the beginning of this question, I think one thing that we've done or noticed when we're talking to people is asking more cultural questions. Not, you know, obviously we want a strong technical basis, but what do the soft skills look like? And I think then we can branch into the technical. But first off, especially at IT Outlives, for me, we look for those soft skills first, right, how are you, like we talked before, like a service desk, right? What are those soft touch points? How would you handle this situation? Not just you know what do you?

Speaker 1: 18:08

do in this case, or what are you doing in an outage, or whatever it is. So I think the searching for the soft and tech first before you go to technical a lot of times can help just to see what they're thinking, how they're thinking about things and what their cultural thought process is.

Speaker 2: 18:24

So, eric, I know you have some juicy stories about AI and hiring and some things that have happened recently, maybe around Cluely or ChatGPT.

Speaker 4: 18:36

How have you seen that manifest when you're in the hiring process? So we were just going through a process where we're recruiting for three technical roles and working with several different recruiting firms. I think for one of the roles in particular it was a technical security role there were maybe 11 vendors that were responding, each with a maximum of three candidates. So it was a ton of resumes that were coming in and the thing that struck me off the bat was how thorough and similar all of the resumes were. Each resume four or five pages. That was just bullet level detail of everything that the person did and it was very well matched to the job description of the role that we had. So it was like, okay, there's some AI going on here and there was one recruiting firm or two recruiting firms that each submitted the same candidate and the resumes were different but just as thorough. So it was like, okay, very clearly, here they're taking AI and creating these resumes, probably different AI platforms that are doing it. But then when we went and we were interviewing the candidates, we narrowed down all of those candidates to I think we maybe interviewed through video interviewed, maybe it was, I don't know eight people, something like that, and some of them were not native English speakers, like English was not their first language, and the questions were all standardized. So the way in which we asked the questions was all very structured and some of the candidates that were clearly using a tool to assist with the interview process.

Speaker 4: 20:45

There were these really long pauses, like you'd ask a question and then there's like a long pause and then the candidate would ask if you could repeat the question and the question was not one that you know really needed to be repeated. It was a pretty straightforward question, but every time it was I'm sorry. Can you repeat the question? And but every time it was, I'm sorry. Can you repeat the question? And then the the answer. They're just staring at the screen and normally in conversation when you're talking with somebody, you know they may look up or to the side, or you know they could be could be a little more animated.

Speaker 4: 21:33

But some of the candidates, when they were responding, were just fixated, looking straight ahead, and it was. It was pretty clear that they were. They were the old presidential teleprompter was going on. Uh, so you know it was we. We, after the first couple, the, the, the interview team has like a side chat going on and it's like okay, clearly ai is involved here. Let's try to change the questions a bit so that they're harder to answer with AI.

Speaker 2: 21:58

How are you talking to recruiters and things about how to navigate this situation, or do you have any kind of tips for them to maybe deal with this sort of a thing?

Speaker 3: 22:07

Yeah, I mean, I think you just have to build in processes and checks and balances around, you know, getting multiple people to sort of verify and then and then also you got to. I think you probably kind of got to put people on the spot, especially if you are suspicious of of you know them, using some of those tools to help them answer questions help them answer questions, Other than being inexperienced in the hiring process.

Speaker 2: 22:35

what are some other pitfalls you're seeing that recruiters are bringing to the table when looking for talented?

Speaker 3: 22:41

people. I think recruiters are pretty good at getting a large. They're good at sort of matching, like, hey, this is what the company's looking for and this is what the you know the people have, and there's there's some really great recruiters out there, I think. I think the main thing is, like what we've talked about a lot. It's like the soft skills like how do you, how do you measure, for you know, problem solving, adaptability, cyber is intense. You know it can be a very intense and stressful field. So how do you measure for that and how people are going to respond to these crazy and intense situations. And then also, how are they going to work together as a team?

Speaker 3: 23:29

The candidate assessments are not, you know. They make it out as if, like, they hire like the top five scores from, like the candidate assessment. Like that's not how interviewing works. It's like they either use it at the beginning of the process because they have way too many applicants and they're trying to narrow that down, or, you know, they use it as like kind of a benchmark, like, hey, we want to make sure that we can. You know, this person has some level of technical skill. Like no, I don't know of any companies that are just like sending these out as like tests and like hiring the tops. You know it's not. You're not just hiring like the A plus students, otherwise you just hire people with the highest GPA. So that whole that whole thing is kind of silly to me.

Speaker 2: 24:11

How have you seen that type of investment show up in the day-to-day security practices and fortifying organizations? And where the rubber hits the road, how have you really seen that pay off?

Speaker 1: 24:21

Yeah, it's huge and we touched on it already a bunch and I think Thomas brought it up right. You could go work for Microsoft or Google or whoever it is, crowdstrike Palo, these big great jobs. But if you can go work for a smaller organization, sure, like Tom said, you might not make that huge paycheck, but you might be way happier way, way happier.

Speaker 1: 24:43

That company is going to invest in you more, so they're probably going to send you to training. You're going to get hands-on with a lot of different things. You probably never had an opportunity at these other organizations. You might get to work with a lot of different things. You probably never had an opportunity at these other organizations.

Speaker 2: 24:56

You might get to work with a lot of different organizations on a lot of different tasks Great. Let's wrap up with a couple of tips maybe Eric and Thomas, about people breaking into the space and how they can use tools to break into cybersecurity. Any quick tips Get?

Speaker 1: 25:08

a help desk job.

Speaker 4: 25:11

Get a help desk job. I was at Secret con yesterday here in town in Minneapolis, which is a really cool conference my first time going there and it's a really approachable conference. But I think, in general, josh, to answer your question, if you're trying to break into the industry or even just you know you're in the industry and you want to look at other opportunities or you're happy in your current role awesome. But you got to get out from behind your desk, from out of your basement or wherever your home office is. You got to get to these conferences, you got to get to the meetups and you got to socialize in person.

Speaker 4: 25:48

I mean, the online stuff is great, it's cool, but there's just so much that you miss in the hallway conversations interacting with people. I mean, we're social animals, it's just you know, still you know, but that's just the way it is. And you got to get to these conferences. Us, there's likely within 100 miles, some sort of a conference that's happening in your area at some point in time throughout the year. There's meetups of all different security types that are approachable and I would say that's the best way to get out and start this as a career.

Speaker 3: 26:32

And there's.

Speaker 4: 26:33

CTFs of those conferences there was.

Speaker 2: 26:37

Thomas, maybe you could tell us if people are interested in MetaCTF, where can they get started and where can they find you?

Speaker 3: 26:43

Yeah, so we're pretty active on LinkedIn Just trying to get as much information out there as we can. We have a lot of free resources so we run a free to enter monthly. We call it a flash CTF, the third Thursday of every month. So it's next Thursday Actually, the winner of that. We have some pretty good prizes. Usually with those, this month is especially good we have a DEF CON ticket available to the winner, and then we have some other like cash prizes and stuff like that, but we do that every month and then we have, with that flash CTF, we put all those challenges in a practice environment. That's free, it's available. All of the write ups are on our blog, so that's a really good place to learn because you can go try them and then read about how to do it. And then on-demand labs is out and that's uh. You know we're giving away free trials right now, so if anyone wants that, um, I'm on linkedin and you can reach out anytime, thomas, that that monthly ctf is no joke.

Speaker 4: 27:45

We were doing it two months ago to prep for the, the, the one that nick and I were in, and we were the, the, the team that I was working with. Some of us about half the team jumped into to kind of practice. We must've been 30 minutes in no answers to any of the questions. Looked on the leaderboard and it was all like zeros, like nobody had answered anything. It's like, wow, this is going to be tough. And then at the time, unfortunately there was an incident that had started so some of us kind of got distracted. But I think maybe we ended with I don't know, like 200 points or something like that. But it was a tough, much tougher than the regional CTF that we were in at Secure360.

Speaker 3: 28:35

Good reminder for us. Yeah, we always joke with our head of content because she's always like oh, these are pretty easy challenges and we're like no, Samantha, easier please.

Speaker 2: 28:47

Well, great. Unless you guys have any follow-up questions, we can wrap it up here today Cool. Thanks so much, thomas. You've been listening to Thomas Rogers from MetaCTF, our guest today. Check out MetaCTF on LinkedIn and online. You've been listening to the Audit presented by IT Audit Labs. I'm your host, co-host and producer, joshua Schmidt. We've been joined by Eric Brown, managing director, and Nick Mellum, security engineer. Thanks so much for listening. Thanks so much for listening. Please like, share and subscribe. You can find us video on Spotify now and please leave us a review or comment on YouTube and see you in the next one.

Speaker 4: 29:20

You have been listening to the Audit presented by IT Audit Labs. We are experts at assessing risk and compliance, while providing administrative and technical controls to improve our clients' data security. Our threat assessments find the soft spots before the bad guys do, identifying likelihood and impact or all. Our security control assessments rank the level of maturity relative to the size of your organization. Thanks to our devoted listeners and followers, as well as our producer, Joshua J Schmidt, and our audio video editor, Cameron Hill, you can stay up to date on the latest cybersecurity topics by giving us a like and a follow on our socials and subscribing to this podcast on Apple, Spotify or wherever you source your security content.