The Web Application Security Consortium (WASC) recently posted their Web Application Security Statistics Project results for 2008. I found this an interesting read because of the unique approach they took in collecting and analyzing the data where the results are separated by how the information was gathered: Automated Vulnerability Scanning, Black Box Testing, or White Box Testing. Not only are the results themselves interesting, but there were some interesting conclusions that were drawn about how certain types of vulnerabilities are best identified.
The statistics includes data about 12,186 web applications with 97,554 detected vulnerabilities of different risk levels identified. The WASC site provides their own conclusions, but the one that jumped out to me was that “the probability to detect a urgent or critical error in dynamic web application is about 49% by automatic scanning and 96% by comprehensive expert analysis”. This truly is the belief of Clear Skies Security and why we have developed our assessment services to be focused on manual assessment techniques over the simpler automated scanning. Customer’s are always questioning why our services seem to take so much longer (and therefore cost more), so it is nice to finally have some numbers to back up our approach – quite simply, expert analysis is going to find almost twice as many issues as an automated scanner. Tools simply can not replace the thought processes a potential malicious user would attempt to identify vulnerabilities.
Bottom line, when it comes to security there is never a silver bullet. Given that 13% of the sites were exploited with just automated scanning, it should be a given that all business that have custom web applications should perform automated scanning at a minimum. Then depending on risk level, more analysis should be done through manual exploitation attempts, or whitebox testing for the most sensitive of sites. And as the applications grow and change, security testing should be continually validating the sites for potential new vulnerabilities. By leveraging a true security life cycle model, starting in the development process, and maintained through regular assessments is the best process to keeping your data safe.