Re: False positives, negatives and don't cares

--On Monday, August 11, 2003 00:12:46 -0400 Martin Roesch <roesch@sourcefire.com> wrote:

>

Herein lies IDSes' true weakness. They're only as good as the rules that are written for them. And BTW, it's my considered opinion that rules are going to get harder and harder to maintain as the "body" of rules grows ever larger. (For an analogous explanation, see my two articles - "Past Its Prime: Is Anti-Virus Scanning Obsolete?[0] and "Life After AV: If Anti-Virus is Obsolete, What Comes Next?[1])

I'll give you a real world example from snort:

sid 1002.
alert tcp $EXTERNAL_NET any -> $HTTP_SERVERS $HTTP_PORTS (msg:"WEB-IIS cmd.exe access"; flow:to_server,established; content:"cmd.exe"; nocase; classtype:web-application-attack; sid:1002; rev:5;)

(I see that in rev 5 someone has finally limited the target to $HTTP_SERVERS rather than any, which should help greatly in reducing the FPs, but will result in FNs when someone installs an IIS server that you don't know about. Previously the rule read $EXTERNAL_NET any -> any $HTTP_PORTS.) Previously this rule looked for "cmd.exe" in the content of any packet, regardless of the purpose of the computer to which the packet was traveling. The problem is that "cmd.exe" shows up in a *lot* of packets that are coming from the windowsupdate site, because MS is polling for that information. So, you get flooded with FPs and you begin to ignore the rule.

When I pointed this out on the rules list and suggested that the content section should look for "cmd.exe?" instead, I was told that "this might miss some legitimate infections". I'm not sure how, because I have yet to see an exploit that doesn't supply the "?" immediately after the "cmd.exe". It's pretty much necessary. (I suppose you could put padding between the "cmd.exe" and the "?", just to bypass this rule, but no one has yet.)

Rather than swim upstream, I created a local rule that looks for *outgoing* traffic only (because I only care about infections coming *from* my network) and the content is "cmd.exe?", not "cmd.exe". I still get some FPs, but not nearly as many. I've even asked if anyone on the list can explain the FPs, because they appear to be just random crap stuck in memory, but I've never gotten an answer.

Do you need help?

In addition to our free technical library, Pantek provides professional IT services to companies who utilize Open Source software: Learn more about Pantek's Technical Support offerings Place an order for 24/7/365 Technical Support here Learn more about Pantek, who we are, and what we do Chat with a Live Operator Related securityfocus.com Links bugtraq certification focus-bsd focus-ids focus-ih focus-linux focus-ms focus-sun focus-unix-other focus-virus forensics incidents libnet linux-secnews ms-secnews pen-test secpapers secprog sectools secureshell security-basics securityjobs sf-news vuln-dev webappsec Request more information about Pantek

I'm not complaining, mind you, I think snort is great. It's the IDS we use. The idea that there are no FPs is simply not accurate, however. There are.

Now, I think your idea of vulnerability correlation holds great promise to make IDS much more useful than it already is. Pouring through 1000's of alerts trying to figure out if the boxes they're hitting are vulnerable to the attack detected is not my idea of fun, nor is a productive use of my always limited time.

Paul Schmehl (pauls@utdallas.edu)
Adjunct Information Security Officer
The University of Texas at Dallas
AVIEN Founding Member
http://www.utdallas.edu

[0]<http://www.securityfocus.com/infocus/1562> [1]<http://www.securityfocus.com/infocus/1604>

• Instantly Stop DoS/DDoS Attacks, Worms & Port Scans
• Automatically Control P2P, IM and Spam Traffic
• Ensure Reliable Performance of Mission Critical Applications Precisely Define and Implement Network Security and Performance Policies **FREE Vulnerability Assessment Toolkit - WhitePapers - Live Demo Visit us at: http://www.captusnetworks.com/ads/31.htm

  ---