innismir.net

There has been some hub-bub lately about Fortify saying that “Penetration is Dead! .. Oh yeah, and by ‘Dead’, we mean, not dead, but just different.”  This was following a similar, but completely unrelated post by Jack Daniel stating that “Penetration testing is a farce and largely a waste of time and money.” While I am inclined to agree with Jack’s basic tennants regarding the two possible outcomes of pentration tests, and I do have a disdain for the term “ethical hacking”, I don’t think that the current model is going away, nor that it is useless.

There are two types of penetration testing that should exist: The kind of penetration test that is worked into the QA process, and the “How screwed are we?” audit-type penetration test. The former should be worked within the application development process, testing the codebase as the project moves forward and giving the application one last assessment before it moves into production. The latter is one where you have a no-holds-barred scan on your network. Both of these accomplish two similar, but different goals: Within the QA process, it gives you and the developers ideas on how secure a certain application is and if there are any show-stopping security bugs. As an audit, it gives you a better idea as to where the weak spots are on your network.

Both of these need to be accomplished by an independent party who do not hold an interest within the project. If you have an independent security team, they can usually handle the tests within the QA process. However, for audits, more often then not, it is a good idea to call in the consultants and let them go to to town. Now, I loathe consultants and feel that they often aren’t worth half of what they charge, but, there needs to be an air of impartiality to upper management. Also, by not putting the security group in charge, it gives them equal time within the crosshairs, something that may be glossed over if they are the ones running it.

More often then not, companies don’t have an independent security team. This has given rise to numerous “penetration testing” companies that specialize in shining a flashlight into all of the dusty corners of your applications and network. This is great and fills a vacuum for a lot of small businesses who just has a “computer guy” who realizes that security is an issue, but does not have enough cycles to address it. However the major issue is, as Jack correctly points out, that we don’t have a common criteria to judge what kind of “penetration test” we’re getting. Are we getting some ninja dropped into our environment to wreak havoc for a week or are we having someone show up with Nessus, scan, and drop off a report later that day? Also, what happens afterward? Does a report get dropped off and the auditor washes their hands of it or will they assist within the remediation phase of the problem? Does the report even get read by upper management? If management and IT is relatively clueless about what a good “penetration test” is, there potential for abuse is very high. When dealing with security that is a very dangerous game to play.

I don’t have a solution to this, besides suggesting that outreach and education is key. The issue is who should be doing the outreach and do companies really want to be reached out to. There is no quick and easy solution to this, just like a “penetration test” is not a silver bullet for solving security issues.