Is it a pen test, an audit, or a vulnerability assessment?

Blog Post created by kevinbeaver on Jun 7, 2013

Hi everyone, I'm glad to be here writing on SecurityStreet. Many thanks to Nate Crampton and Patrick Hellen at Rapid7 for helping me to get rolling! A while back I wrote a piece for TechTarget on the difference between pen tests, security audits and vulnerability assessment and I figured that this may be very relevant to Rapid7's community given Nexpose and Metasploit.


Way back in the mid-1990s, at my first full-time job out of college, I went to work as a Technology Coordinator for a K-12 school system. I remember being concerned with my job title at the time. The role was more of an IT “manager” rather than a “coordinator” and I wasn’t sure how people would perceive me and how it’d look on my resume down the road. Well, my boss (and now long-time friend) Terry, responded with some of the best career advice I’ve ever received. Terry said “Who cares what they call you as long as they pay you what you’re worth?”


Wow, okay. I couldn’t fully comprehend Terry’s advice at the time but now I get it. Having worked for myself for the past 12 years, I’ve learned that you can't get caught up in semantics like job titles. It’s more productive and fruitful to focus on the value you bring to the table instead. So, what does this have to do with information security? A lot.


It’s the long-time debate, argument, confusion over penetration tests versus security audits versus vulnerability assessments. Are they the same? Many people use them interchangeably so you'd think they all refer to the same type of security testing. The bottom line is they are different...arguably worlds apart, at least in our world of information security. Penetration tests, audits, and vulnerability assessments have different objectives, different deliverables, and so on. I won’t go into the details of how I see things since I’ve already done that in my TechTarget piece. What I will say is that you’d better darn well know what value the security testing you’re currently performing is providing to the business. Is penetration testing enough? Are audits helping or creating a false sense of security? Perhaps you (and management) believe that basic vulnerability scans are good enough. Your mileage will certainly vary based on your approach.


You have to be careful. I cannot tell you how many times I’ve seen people perform high-level IT audits only to miss numerous (and glaring) security holes because they didn’t look deeply enough often using the wrong tools and techniques. Documented policies, procedures, and technical controls look nice but they don’t make you secure. On the other hand, I often see people get caught up in the minutiae of IT (and pen tests) that they miss the big picture of what’s really going on and what’s really needed. They’re enamored with their technical skills but fail to do what’s best for their business. Big mistake. You have to strike a balance.

We don't tell home inspectors how to inspect how our homes. Nor do we tell radiologists how to read our CT or MRI scans. In the same spirit, people shouldn't dictate the specific methodology for uncovering the real issues that matter on their networks. We can debate semantics or we can get to work finding and fixing information security risks. As much as it bugs me (similar to the year 2000/millennium debate years ago), in the end it doesn’t really matter what you call your security testing. If the meaningful security flaws are uncovered based on your specific business needs that’s all that counts. Just make sure you're looking in all the right areas.

You can check out my TechTarget piece here.