RE: Protocol Anomaly Detection IDS

As you have rightly guessed, marketing buzzwards mean whatever you want them to mean.

One way of doing IDS is to read the RFCs and trigger an event whenever the network traffic doesn't conform to it. I first did something like the back in 1992 -- but more as a debug/interoperability tool in order to figure out why two applications couldn't talk to each other.

I call this "protocol-validation"; I don't do it much because, as you've noticed, vendors suck at reading RFCs. If I triggered on every little protocol violation, then I'd drown in a torrent of events. This is indeed what customers are telling me when they try out competing products that are based mostly on "protocol-validation" in real-world networks.

One of the (marketing) advantages of protocol-validation is that you get to claim 100% "accuracy". Because they follow the RFC and the network traffic doesn't, then its the network traffic that is wrong. It's still a false-positive in my book, and a lot of our protocol-validation signatures have anti-signatures to filter out popular products that produce invalid traffic.

Protocol-validation has a problem that most attacks are "legal" protocols, or the attack is against a vuln that isn't defined in an RFC. For example, there is no RFC for the port 1434/udp Microsoft protocol attacked by SQL-slammer. Protocol-validation vendors claim that they can detect attacks without dirtying their hands with security research -- I disagree.

What ISS calls "protocol-anomaly" is somewhat different than other vendors. We refer to an anomaly as something that is legal (as far as the RFC is concerned), but otherwise odd. For example, a 100-character password is legal as far as the FTP RFC is concerned, but we still trigger on it as an anomaly. Note that RealSecure/1.0 and Wheelgroup/1.0 (now Cisco IDS) have been doing this sort of anomaly detection since 1996, and it has also been a big part of NFR and Intrusion.com. It's also worked well as a "post-vuln pre-exploit" way of developing signatures. Something like Snort waits until exploits are developed, then develops signatures for that exploit. E.g. the Snort sigs for SQL-slammer were for that worm, but could miss variants. The NFR and RealSecure signatures would trigger on the VULN not the EXPLOIT, so they catch the exploit. Protocol-anomaly detection in RealSecure has also been pretty successful at catching things before even than the vuln is known, but this doesn't happen nearly as often as the post-vuln/pre-exploit period. Note that sometimes vuln announcements are imcomplete; I've gotten burned twice in reviews because hackers found an alternate way of exploiting something, and if I had just done a signature-match I would have caught the variant -- but this is very rare.

Note that none of the reviewers test the post-vuln/pre-exploit issue. They just run known exploits. This irritates me a bit.

In answer to your original question, all vendors support some sort of protocol-anomaly detection. Even Sort has some signatures along those lines. Protocol-anomaly (or validation) is the preferred way that most vendors now do signatures. I think we have more anomaly sigs and more application protocols supported than anybody else, but then, everyone has their spin.

Do you need help?

In addition to our free technical library, Pantek provides professional IT services to companies who utilize Open Source software: Learn more about Pantek's Technical Support offerings Place an order for 24/7/365 Technical Support here Learn more about Pantek, who we are, and what we do Chat with a Live Operator Related securityfocus.com Links bugtraq certification focus-bsd focus-ids focus-ih focus-linux focus-ms focus-sun focus-unix-other focus-virus forensics incidents libnet linux-secnews ms-secnews pen-test secpapers secprog sectools secureshell security-basics securityjobs sf-news vuln-dev webappsec Request more information about Pantek

BTW, Lancope does statistical anomaly, not protocol anomaly. That's a different beast altogether.

-----Original Message-----
From: Michael L. Artz [mailto:dragon@october29.net] Sent: Tuesday, February 04, 2003 11:07 PM To: focus-ids@securityfocus.com
Subject: Protocol Anomaly Detection IDS

I am trying to supplement our existing signature based IDS (Snort, gotta

love open source) with a protocol anomaly based one in a fairly large enterprise network. I am in the fairly early stages of research, so I guess that the first question would be, is it worth it?

I hear the anomaly detection buzzword thrown around a lot these days, and can't quite get past all the marketing hype. From what I can tell, protocol anomaly detection seems to be the more promising than the statistical for detecting new or IDS-cloaked attacks. However the notion of "conforming to RFCs" leaves a lot of leeway for the vendors to

play with. How well do these types of systems actually work?

Does anyone have any recommendations as to which systems to look into/stay away from? Below is a list of some of the ones that looked like they might support protocol anomaly detection from their marketing hype, let me know if I left any out/incorrectly added any:

Lancope Stealthwatch
Tipping Point/UnityOne
ISS RealSecure Guard
Cisco IDS 4250
CA/eTrust IDS
Intruvert Intrushield
NFR Network Intrusion Detection System
Netscreen/Onesecure IDP
Symantec ManHunt

Do you need more help?

In addition to our free technical library, Pantek provides professional IT services to companies who utilize Open Source software: Learn more about Pantek's Technical Support offerings Place an order for 24/7/365 Technical Support here Learn more about Pantek, who we are, and what we do Chat with a Live Operator Related securityfocus.com Links bugtraq certification focus-bsd focus-ids focus-ih focus-linux focus-ms focus-sun focus-unix-other focus-virus forensics incidents libnet linux-secnews ms-secnews pen-test secpapers secprog sectools secureshell security-basics securityjobs sf-news vuln-dev webappsec Request more information about Pantek

Any clues or headstarts to get me pointed in the right direction would be great.

Thanks
-Mike Received on Thu Feb 6 12:37:45 2003