Re: IDS thoughts

From: Mike Frantzen <frantzen(at)nfr.net>
Date: Tue May 20 2003 - 14:25:23 EDT

> You are joking, right ? There's a whole lot of research still open in the

I don't think anyone has forgotten anomaly-based detection. Most players are taking a hybrid approach.  

> In the next few years, while established IDS products will strive to keep up

Keeping up isn't as hard as you would think. Even for pure pattern matching, most algorithms scale roughly on the order O(log n) w.r.t performance and number of signatures. Some of the poorer algorithms start getting ugly cache/tlb/bp behavior but there are algorithms that fix any two out of those three secondary affects.

> When it comes to firewalling, we all agree: you just shut down everything

Ok. I do both firewall development (OpenBSD) and IDS development (NFR). And they are totally different, dare I use a buzz word, paradigms. To give a fairly deterministic measure of the difference, the size of NFR's regression tests alone are four times larger than the entire OpenBSD PF firewall. With firewalls, you have the luxury of being able to assume port == protocol (yes yes, you can do very rudimentary protocol validation with things like inspect scripting; and there are a few proxying firewalls that checkpoint hasn't killed off yet)

That naivety from the security device just doesn't cut it in the IDS space. Sit down and stare at several captures of HTTP transactions. Ones from IE, Netscape, Konq, Opera.... They all look different and this is where theory diverges from implementation. An anomaly in one is perfectly normal in the other. It gets worse, the transactions start looking differently depending on the server they talk to. Sure, so we can leave the client protocol models agnostic of the server type. But now you start to factor in that HTTP clients lie about what program they are. And certain venders are notorious for protocol variations between minor patch levels...

Yes, pure anomaly detection is a very complex process and *VERY* difficult to create something that doesn't have a propensity towards false positives. Subtle changes in an environments usage patterns often occur suddenly and tend to come across as anomalies -- now the IDS/IPS admin gets inundated after acclimating to relatively few alerts.

Thus you see many venders transitioning (have already done so) to doing anomaly detection where feasible, and "bad thing" detection when not.

I'll make a standing offer, I will buy anyone a cookie that can describe their enterprise network usage adequately enough that would allow pure anomaly detction. Hint, you control all the clients versions and all their connections either terminate at your servers or your proxies.  

.mike
frantzen@(nfr.com | cvs.openbsd.org | w4g.org) PGP: CC A4 E2 E8 0C F8 42 F0 BC 26 85 5B 6F 9E ED 28

---

INTRUSION PREVENTION: READY FOR PRIME TIME? IntruShield now offers unprecedented Intrusion IntelligenceTM capabilities - including intrusion identification, relevancy, direction, impact and analysis - enabling a path to prevention.

Download the latest white paper "Intrusion Prevention: Myths, Challenges, and Requirements" at: http://www.securityfocus.com/IntruVert-focus-ids2

---

Received on Tue May 20 14:36:16 2003