The Big Easy

Blog Post created by rapid7-admin on Dec 1, 2010

Originally Posted by Sean Taylor



People don't like to hire blackhats. It's great because it speaks to so many levels of assumptions and interests me immensely because of it. Arguably, the mentality speaks to a much lower level issue with the pervasive American ideal of perfectionism-- but if I wanted to wax wasteful poetic on the irritating low-level sociological tendencies of our culture, I'd start a LiveJournal. I've already got this blog, so let's just stick to the context of the greater security community. 

We all know the classic hat distinction: keyboard cowboys sporting single-shade shirts doing digital duals at subnet sunsets. The white hats are the sherrifs, the black hats are the bandits. The white hats defend, the black hats attack. What the duel definitions eventually come down to is the classic and simplistic bifurcation of "good vs. evil." 

There's a natural human tendency to make things easy. It's a survival trick: if you can get up a hill using the least amount of energy possible, then you can get up the hill more effectively and likely outrun any potential predators. Going up a long, steep hill while using minimal energy output yields your survival for one more day from being consumed by the circle of life. With a lack of predators to deal with-- or at least predators that will quite literally kill and consume us-- our brains are allowed to drift off into other areas. The drifting eventually leads to solving social or technical problems. (Whether or not they're
actual problems, hey, that's something for those "intellectual" types to figure out.) 

The problem is that we try to make
everything easy. Everything! Things that are hard we try to make easier-- even when making it easier makes the process more difficult or arduous. We also make easy things even easier. Apparently there are people so clumsy they need a tool to crack eggs. Why the hell did that problem need to be solved? Was there a massive uprising of clumsy American consumers who were sick and tired of their eggs shattering in the bowl and getting egg shell mixed in with their yolk? Cover your children's eyes, it's about to get ribald: is it that hard to crack an egg?! (Contrariwise, that Slap Chop sure makes chopping look like an unnecessary chore. It also makes a great tune.) 

The traditional American hiring process is, I think, one of the greatest examples of this natural attempt to make things easier that can't necessarily be made easy. Finding someone that you want to come work for you starts out as an almost impossible process-- there's roughly 6 billion people in the world, are you just going to hope that all of them come your way and one of them will be a good candidate? That's ridiculous! You'll go out of business before you hire your first employee. So in order to hire someone you need to put out an ad and establish a series of requirements that you feel fit the job. And right there, really, is where the "ease" process should end. You can only have so many relevant requirements for the job. After that, all you can really do is interview someone and get more specific questions that the resume can't answer. 

But for a lot of people, that's just not easy enough. Over a span of a few decades, the hiring process for a majority of companies evolved toward almost entirely catering to a preconceived notion of competence and keywords-- in other words, image. Image is much easier to grok than having to think about the logical links of all the various attributes on a piece of paper. In order to be a voracious potential employee, one must play the game of expectations and verbal wizardry. Resumes are frequently turned down because they don't have the right phrases on them-- regardless of the actual content that exists on, underneath or outside them. A great example of one of these missing keywords is a college degree. Someone I know who works in HR even once told me "I don't even
look at resumes without a college degree on it." 

The necessity of a college degree for any given job is purely contextual. There are certain fields where any sort of training is just flat-out untrusted if it doesn't come from a place of higher education-- the medical profession is a great example of this. I would never trust a doctor without a medical degree-- at least from the outset. For all I know, he could be a field-trained physician without certification. But for me to trust his medical expertise, I would need to see one of two things: experience or certification. Obviously, the certification verification is much easier than twiddling my thumbs about and waiting for some poor schmuck to get hit by a bus nearby for the medic to work on. Additionally, an uncertified but well-trained physician may be flat-out dangerous when trying to cure certain ailments and a savant at curing others-- you don't know
what he's trained himself on without the certification to tell you unless you see, flat-out, that he's got the experience to cure X, Y or Z. 

Interestingly, though, the mistake people make about college degrees-- and any certification, really-- is the degree as a verification of absorbed knowledge of everything covered under a specific umbrella. Certifications and college degrees are seen as a verification of "knowing stuff." If our field-tech had a certificate of completion of a general practitioner, we would assume that he would have knowledge to cure X, Y, Z and others. As a rule of thumb, making this assumption is A-OK with the medical field-- the intensely fierce competition involved with the race to the degree forces memorization and ultimately great medical training. But the certificate of completion is still, ultimately, simply that-- a certificate of
completion. You can make assumptions around the degree all you like-- it doesn't make them realistic. Our certified field tech might be awesome at sewing stitches on a recently-torn-open wound, but how good is he at diagnosing rashes? Maybe he thinks everyone has psoriasis when they turn a little red. The certificate doesn't tell you that. That's what the interview is for. 

Certifications are intended to make selecting people easier because they create assumptions of retained knowledge, thus alleviating the need to put too much thought into the determination of one's knowledge and putting more focus on the experience of the candidate. Unfortunately, there are a plethora of outfits that assume a lack of certification is a lack of retained knowledge, and thusly-- as the smug example a few paragraphs before-- reject purely on the basis of no certification. A lack of certification makes it hard to really verify from the outset whether or not someone has actually retained the knowledge they say they have on their resume. And, gosh, it sure would be a waste of time if we found out the candidate we wanted to interview didn't
actually know anything he wrote on his resume. (Of course, the same problem occurs even with people with degrees and certifications... but don't tell them that.) 

Our pervasive two-toned hat analogy is one in the same with the E-Z-Fallacy. It's attempting to apply morality to a blatantly non-moral system because that makes people easier to segregate. (If you can deterministically say who's good and who's bad, obviously you want all the good people and none of the bad.) Many people illogically tie legality to morality, thusly tying criminality with black-hattery and labelling it "bad." If illegally breaking into a system is bad and considered the thing only blackhats do, then it follows that Stuxnet's intended purpose-- to disrupt or corrupt the centrifuge process of enriching uranium for (most likely) nuclear weapons-- is bad, solely on the basis that it infected a machine it had no authorization to be on. In contrast, a whitehat can be working for an oppressive government regime, searching and hunting for vulnerabilities in software to allow for the expeditious and anonymous tapping of citizens' data. Neither of these examples meet the assumed criteria for blackhat or whitehat-- the blackhat in our scenario is fighting for the greater good by preventing nuclear weapons from being created with an insanely complex and wide-spreading virus, thus preventing the potential slaughter of thousands of victims. The whitehat, while acting within legal and purported moral boundaries to find vulnerabilities-- you know, not doing anything "bad"-- is ultimately creating an oppressive environment for the society of underlings their leaders loom over. "Lawful evil" exists on the D&D spectrum for a reason, you know. 

The traditional definition that seems to constitute the hat color of a given hacker is dependent almost entirely on the techniques they employ-- exactly where it doesn't belong. Computers don't "get" morality. The techniques employed by hackers when attacking systems are almost always purely based in the exploitation of flawed logic. A system vulnerable to MS08-067 will gladly accept a payload that pops up a message-box with "YUO = PWNED" inasmuch as it will also gladly accept a botnet payload. One action is arguably more moral than the other but both results are completely independent of the system: logic. The morality of black and whitehat hackers should be determined by the results of their findings rather than whether or not they exploit the flaw, as the flaw was (usually) not their fault. 

By refusing to hire "blackhats," people with potentially useful real-world experience are shunned at the starting line. Because computers are completely non-moral systems, the only people with the most intimate experience with breaking into systems wind up being blackhats-by-definition due to the morality-play forced onto the computing realm. Of course, the security community isn't stupid-- it realized there was an experience gap with actively rejecting people who might have done some illegal things. But instead of accepting the moral void that exists within code, they went the way of the E-Z-Cracker-- certification programs. How many companies can you name that refuse to hire people if they
don't have a CISSP? Seeing "CISSP" on a resume makes things easy: it means the candidate has at least 5 years of security experience and is-- supposedly-- "one of the good guys." 

In effect, many certification programs are trying to have their cake and eat it too by providing pseudo-experience coupled with unrealistic moral boundaries in a logical realm. While it's wonderful that they're providing a legal environment for people to learn about computer security, it's simply training. By definition, the training exercise is modeled around supposed real-world scenarios, but due to that ever-pervasive need to make things easier, simulating real-world scenarios transforms from "how do we make this as realistic as possible" to "how do we make sure things work as we want them to." Creating a realistic box to be attacked is tough. 

There's a reason
the black hats are always ahead of the game-- they're digging in and learning all about the systems at their fingertips because they can. They don't anthropomorphize the system in place by applying a set of unnecessary moral guidelines to it, they learn how the systems are at the given moment-- of which the internet is a vast resource for. The training programs are always in the wake of the blackhats as a result because they quite literally leech off the research that eventually trickles out from their splintered communities. This isn't necessarily a bad thing-- it's just an unfortunate game of Catch-Up. 

To refuse to hire blackhats because of the preconceived notion of what constitutes "good" and "evil" in the world of networking is to shoot one's self in the foot. Blackhats just have more experience by the very nature of the game: logic. They have their own sets of moral codes that dictate what they do with their findings-- do they report the holes they've broken into (for trolling, for protecting their backdoored boxes), do they deface the site with "HACKED BY TURKISH EMPIRE," or do they steal credit cards? All of these actions are different shades of different blackhats. If one cares so much about legality of past actions as to how they relate to experience, you don't hire this hacker at all because-- god forbid-- he probably did something illegal. If one
doesn't care about morality, it really doesn't matter so long as you get a hacker among your ranks-- but, man, are you going to have egg on your face if you hired the one who does identity theft for fun and has no respect for humanity. 

The morality measure should therefore not be applied to whether or not someone has broken into a system illegally or without authorization, but rather
why they broke into the system and how. The legality doesn't matter. If they weren't caught and no charges were pressed, it doesn't factor in at all to what this hacker is going to do with your company-- all it tells you is that the hacker is good at what they do and what they're capable of. 

One should look for experience, intelligence, ingenuity, quick-thinking and honesty. Ultimately, it comes down to a factor of moral character. Instead of looking at the legality of the situations culminating to a life-long experience, one should look at the scruples of the individual. Did they attack that website due to a vendetta of some kind? Was revenge a factor? Did they attack the company to prove a point and make things more generally secure? Did they exploit the systems for knowledge or did they exploit the systems for profit? Did they use a vulnerability scanner to find that SQL injection, were they hunting on instinct or did they write that scanner themselves? What drives them to do the things they do? Are they vindictive? Nice? Patient? Logical? Quick to anger? It's ultimately these fractionally-different decisions that should ultimately define what makes a hacker worth hiring to your company-- and good
lord are they hard to contend with. Won't hiring someone who has a potentially shady or illegal history make you look bad as a result? What if you read him wrong when you hired him and he winds up running out the front door with all your money? 

A certification or a college degree should be a negligible factor-- in fact, searching for one of those two requirements may actually force you to overlook the right candidate for the job. You'll probably get lied to, have experience exaggerated on resumes or get inflated egotistical accounts of compromised systems, but with patience the right candidate will come around. 

Ignoring the legality of one's experience is an extremely hard sell-- especially in a highly authoritative and legal society such as the United States. Anyone who is seen as potentially subverting authority is considered an enemy of our society. And rightly so, to a certain degree-- the legal system was put in place mostly as a way to legislate morality, so in a roundabout way anyone who's willing to subvert legality may very well be willing to subvert a good chunk of natural human morality as well. This is a complex situation to contend with when knowing that legality and morality are actually two logically disjoint-- but not necessarily dissimilar-- virtues. But if you have the scruples to sniff out the type of person someone is while simultaneously ignoring the societal fragments of legality which surround a candidate's experience, I guarantee you'll hire the best hacker for the job. Who better to protect your network than the guy you trust who just so happens to know all about the 0day exploit your network is being attacked by? 

Bruce Schneier and Marcus Ranum wrote an excellent
point-counterpoint on this topic earlier this year, I suggest you read them as well.