Security Testing: The Last Stage of the SDL

When most people think about Application Security, they don’t think about a Security Development Lifecycle. What they think about is Testing Applications after they’ve been written to see if they’ve been written securely. Nice idea but, if you’ve been keeping up with this blog, you understand the need for the full SDL. That said, it’s now time to actually talk about Application Security Testing.

There are two types of Application Testing that you need to keep in mind when it comes to Security; Automated and Manual. As you can probably guess, Automated Testing involves using tools that are out in the market place where as Manual Testing involves someone actually sitting down and actually going over the code.

I can just hear you saying “Why would I do manual reviews of code when an Automated tool is much more efficient?” My answer to that is this – Just because something is easier doesn’t make it better. If you talk to the different Tools Vendors, they will say that their tools will catch all of the vulnerabilities that they find. But if you talk to the major Application Vendors (eg. Microsoft, Oracle, etc.) you will get a different answer. The rule of thumb that I go by is that Automated Tools will catch in the area of 30% of the vulnerabilities that are in the code. Scary, huh?!?

Let’s go back to the Impact Assessment that should be done at the beginning of your SDL and remember that it is supposed to drive the level of security that you carry with you through your development process. If the application you’re dealing with has a low criticality (eg. A small dll that doesn’t affect Information in any way), you may not even want to do any testing. But if you have Applications that need testing, then you want to determine what level of security testing you want to go with. If that rule of thumb of 30% is okay for the criticality of the Application you are working with, then go ahead and just use the Automated Tools out there. But if you need a higher level of assurance, I would recommend that you use both the Automated Tools and a manual review.

So, the next obvious question is “How long does it take to do a manual review?”. I once had a conversation with a gentleman that has a lot of knowledge in this area and he said that any of his code that he had written 10 years ago would take a lot longer to review than his code of today. In other words, it depends on how good your Developers are at writing secure code. Just goes to show you how important training your Developers is – you can get a ROI based on the length of time it takes to manually review the code.

So what type of Application Security testing do you do? Well, there are two primary types of testing. The most common one is what is commonly called Penetration Testing. This is when the Application has been compiled and a “Black Box” test is done on the Application. The problem with referring to this as Penetration Testing is that Security professionals use the term Penetration Testing for any type of SECURITY testing of an entire environment, not just the Application. The results include things like Server configuration, network configuration, and Application vulnerabilities. As a result, for the Application Development team, this type of testing reveals information that they really shouldn’t care about specific to the writing of the Application. I would like to promote using the term “Runtime Testing” with regards to Application Security Testing so that the focus is solely on the Application.

The second type of testing is Static Code testing. This is done on the actual code prior to compilation and reads the actual code. This type of testing is also commonly referred to as “White Box” testing since the testing has full access to the inner workings of the Application itself. This is also the area where you can most effectively combine Automated testing with Manual review of code.

If you combine these two types of testing (Black Box & White Box), you should have a pretty good picture of how secure the written Application has been written. I would recommend that you create a standard process for both the automated and the manual testing so that you can have repeatable processes to follow and try to improve upon. And try to document any improvements on a regular basis.

Anyway, remember that Application Security Testing is NOT the only thing that has to occur to ensure that Applications are written in a secure manner. Testing and Manual Review occur at the end of the SDL and is the last chance you have to catch vulnerabilities in the Application before it moves to Production. Using Testing as a component of a properly run SDL means that the number of vulnerabilities found at this stage of the SDL should be much lower (and thus have a much lower cost to fix) than if you only do Application Security Testing alone.

Neil R.

Posted at 03:40 AM  Permalink