The path to a false sense of security: Leave your security controls enabled during testing

Blog Post created by kevinbeaver on Sep 11, 2015

In my work performing vulnerability assessments and penetration tests, I’m often confronted with the dilemma of dealing with a pesky intrusion prevention system (IPS) or web application firewall (WAF). Sometimes we know they’re there. Other times, they rear their ugly heads and force a days-long change management process for a whitelist request. Or, when testing web sites/applications, it’s not uncommon to find out that I can just connect via SSL/TLS and carry out my tests that would’ve otherwise been blocked over HTTP.


When performing your scans and tests, I think it pays to consider the following – well in advance – to save yourself or others some hassle and (especially) false sense of security:

  • Which technical control(s) might end up creating challenges? Perhaps it’s an IPS or a WAF. Maybe it’s a firewall or even an endpoint security technology running on the system(s) being tested. Maybe it’s a third-party monitoring system that places suspect behavior on a blacklist. Strangely enough, many people aren’t familiar with which controls they have in place, often because a third party is “handling everything”. These are considerations that need to be mastered rather than assumed.
  • What happens when scans are blocked? Do you stop there? Some people do, assuming that all’s well with security...penetration thwarted! That’s what the bad guys would encounter, no? Not really. The thing is, not every exploit originates from an automated scan. The savvy hacker (and security professional) knows that a lot can be done on the down-low. Such manual analysis/exploitation often flies under the radar of security controls. How are you going to address that? If you’re doing this work in terms of PCI DSS compliance, the Security Standards Council addresses the very issue of blocked scans/test and the importance of whitelisting when it’s needed. It's good to see that this approach is becoming an industry-wide recommended practice.
  • How do you document your findings? If you uncover weaknesses only by manipulating the traditional, scoped assessment approach, is it still reality? Is a blacklisted, whitelisted, or “skirt the situation altogether” approach a true reflection of the security posture of the systems being tested? My take is that if it’s out there for others to play with, it should be fair game for all types of testing. If not, perhaps your online footprint needs to be reduced. Still, I see people requiring that the testing only be done in certain ways to, perhaps, have a more favorable outcome for the auditors and management to see. That’s a slippery slope, but it happens all the time.


In the end, only you will know what’s required, what’s best for your organization, and what your lawyers are willing to defend. This issue underscores the importance of properly scoping your security assessments. Leaving your security defenses up all the time doesn’t mean that something cannot fail and get you into a bind. Don’t let finely-tuned testing parameters end up creating a false sense of security that facilitates a breach or is otherwise held against you in the future.