Thoughts on Penetration Testing

I’ve watched our industry slowly mature itself over the last fifteen years, and I am still somewhat bemused regarding penetration testing.  It seems like every standard now requires a penetration test to be done, why?  I struggle to see the value.  I will admit that penetration testing has a lot of sexy attached to it, but there are several reason I believe it is over valued, and under delivers.  I have come to this position because of my views on engagement rules, scoping, and the wildly variable skills differential.

When starting a penetration testing engagement, the first things to establish is the rules, and the scope of the engagement.  You now have just failed.  By establishing a set of rules and scope, artificial limitations have just been created that no longer models the real world.  A penetration test is supposed to model a real world scenario, so that the effectiveness of implemented controls can be validated.  In all my years of managing information security incidents, I have never come across a situation where an adversary has requested the rules of engagement, or a scope before attacking.  To a real adversary, nothing is off the table, and nothing is out of bounds.  A motivated adversary will use every tactic, every technique, every dirty trick available to them to achieve their goal.  They don’t play by the rules, nor do they care what your scope is.  Setting rules and a scope breaks the first rule of warfare, which is, there are no rules.

The second issue I have regarding penetration tests is that an assumption is made that the penetration tester you have hired has the same skills as your adversaries.  I have worked with some rather large, and well known players in the penetration testing industry.  They have very smart and motivated people working for them, but I have never come across anyone I would be truly worried about.  The engagements I have been involved in were basically an information security management vetting process.  The penetration testers had some tools that someone else wrote, that exploit known vulnerabilities that already have patches available.  In my opinion, the service they provided was simulating script kiddies who could write fancy reports.  Information security management 101 would thwart this type of adversary.  If the basics are not being taken care of, then the outcome of a penetration test will already be known.  You will be compromised because a patch wasn’t installed, or your systems were poorly configured, or you were using weak passwords, or you’re using a flawed architecture.   Any security professional worth their salt should already be aware of these risks in their environment, and actively working towards mitigating them.  Spend the monies  on mitigating risks instead of on a third party that will tell you what you already should know.

If you really want to get some value from penetration testing, then use the process pen testers use to develop a inventory of risks to mitigate.  Specifically, footprint your organization to find all ingress and egress points to the internet.  Analyze these areas for missing patches, poor configurations, and weak architectures; a.k.a. use a vulnerability scanner.  Prioritize your findings and get to work.   If you really want to raise the bar, have any custom code exposed to the Internet tested for vulnerabilities, and then fix them.

On a side note, why are the majority of images of hackers have them wearing face masks?  If these hackers are so elite, why can’t they disable their webcams?

Another opinion on the matter – The Myth of Ethical Hacking