Re: Web Application Source Vulnerability Scanners

I think all the tools mentioned in this thread are pretty good at what they do, and getting better all the time. I think they can automate a small part of the problem very nicely.

In my experience, most of the big problems in web applications are logic errors -- frequently flaws that result from poorly designed or completely missing security mechanisms. These flaws are difficult to scan for because every web application has different mistakes. It is not like scanning for vulnerabilities at the network layer where everyone is running one of a handful of operating systems.

A perfect scanning tool would be able to find ALL possible variants of the OWASP top ten web application vulnerabilities. I'd be willing to bet that none of the existing tools can find all possible variants of ANY of the top ten. Again, no disrespect to the folks who have built these tools. The problem of finding security vulnerabilities in arbitrary custom code is extremely difficult, especially from an external perspective.

In my opinion, if you want to find flaws in custom web applications, look at the code. All the flaws are there in black and white. If you can't read the code, find someone who can. A security code review is the fastest way to find the most serious holes in your web application.

--Jeff

Jeff Williams
Aspect Security, Inc.
http://www.aspectsecurity.com

• Original Message ----- From: Kevin Spett To: webappsec@securityfocus.com ; ory.segal@sanctuminc.com ; securityarchitect@hush.com Sent: Tuesday, March 04, 2003 2:22 PM Subject: Re: Web Application Source Vulnerability Scanners

Moderator: As SA stated, this is delicate as it involves the discussion of
commercial software produced by the companies that I (and Ory) work for. However, I think that my points are valid and discussion-worthy.

Do you need help?

In addition to our free technical library, Pantek provides professional IT services to companies who utilize Open Source software: Learn more about Pantek's Technical Support offerings Place an order for 24/7/365 Technical Support here Learn more about Pantek, who we are, and what we do Chat with a Live Operator Related securityfocus.com Links bugtraq certification focus-bsd focus-ids focus-ih focus-linux focus-ms focus-sun focus-unix-other focus-virus forensics incidents libnet linux-secnews ms-secnews pen-test secpapers secprog sectools secureshell security-basics securityjobs sf-news vuln-dev webappsec Request more information about Pantek

First, there are other tools besides AppScan that know how to keep state correctly. WebInspect does.

Second, on to the aforementioned article. The info security magazine test
report is less than scientific, as it doesn't detail any of the exact testing procedures, server configuration, give source code for the applications, etc. It does not allow anyone to duplicate or evaluate its
findings. Also, the test was financed and performed by a company that makes
money by performing services that tools such as AppScan and WebInspect are
designed to test. Finally, the two authors of the article are not well-known in the area of web application security.

Keep in mind that these are not accusations. I am not alleging that the test results were incorrect. I am not saying that the authors are unqualified. I'm just saying that the test really doesn't really provide
enough information for real technical discussion. Thus, its findings cannot
be "proved" either way. I distrust that which cannot be proved.

I simply recommend that people who are interested in appraising the quality
of web security tools, both free or commercial, make their own tests and judgements, so that they can control every variable of the analysis. This
will have to be the case until there are truly "open" evaluations that are
not lacking in steps for reproduction.

Kevin Spett
SPI Labs
http://www.spidynamics.com/

• Original Message ----- From: <securityarchitect@hush.com> To: <webappsec@securityfocus.com>; <ory.segal@sanctuminc.com> Sent: Tuesday, March 04, 2003 11:48 AM Subject: RE: Web Application Source Vulnerability Scanners

>
> I know this list doesn't cater for commercial tool discussions
specifically so choosing words carefully moderator ;-)
>
> To counter that you should look at the latest review of commercial
tools.
All failed pretty miserably and the general recomendation was to wait until
the next generation of tools come out.
>
> http://www.infosecuritymag.com/2003/jan/cover.shtml
<ory.segal@sanctuminc.com>
wrote:
> >Hi,
> >
> >The problem with most open source tools is that they are very strong
AppScan
> >
> >is the only scanner to perform this action.
> >5) Good performance
> >6) Contstant updates.
> >7) Logging of raw HTTP traffic
> >8) The ability to easily implement new tests.
> >
> >-Ory Segal.
> >
> >
> >
> >
> >
> >
> >
>
>
>
> Concerned about your privacy? Follow this link to get
> FREE encrypted email: https://www.hushmail.com/?l=2
>
> Big $$$ to be made with the HushMail Affiliate Program:
> https://www.hushmail.com/about.php?subloc=affiliate&l=427
>
Received on Tue Mar 4 22:48:40 2003