Re: Crossover Error Rate (WAS "Intrusion Prevention")

> Just as with an IDS, you can reduce

That's not indicative, really.

In evaluating a system with that metric, you are supposing that both kind of errors are equally costly. They could not be (for example, in a biomedic system it is FAR better to have a false alarm than a false negative !).

In addition it is not known, a priori, if the cost linearly scales. Having 10 false positives a day can be acceptable, 100 false positives may be a bit more harassing (but not, necessarily, 10 times more), while of thousands of false positives are completely unmanageable (they have an "infinite" cost: we don't absolutely want to have that). At the same time, 1 false negative may be bad, and 100 false negatives are probably in the scale of "better to launch this crap out of the window".

Please note that all the figures are totally subjective, and here only for the sake of an example, do not flame me on the figures :P

What you really want to build is an ROC, Receiver Operating Curve, which is a diagram with a measure of the false positives on X axis, and a measure of the detection rate on the other. They are in some kind of 1/x - like relationship (the more false positives you accept, the better you find attacks, and vice versa). A "higher" graph (A larger area under it) means a "better" system, on the whole. But more accurately, you can match this graph with your own "cost function" for false detections and misses, by using really simple operational research techniques (you build the gradient on the graph, and find the tangent with the ROC curve).

It's all theory with 40 years of background.

Do you need help?

In addition to our free technical library, Pantek provides professional IT services to companies who utilize Open Source software: Learn more about Pantek's Technical Support offerings Place an order for 24/7/365 Technical Support here Learn more about Pantek, who we are, and what we do Chat with a Live Operator Related securityfocus.com Links bugtraq certification focus-bsd focus-ids focus-ih focus-linux focus-ms focus-sun focus-unix-other focus-virus forensics incidents libnet linux-secnews ms-secnews pen-test secpapers secprog sectools secureshell security-basics securityjobs sf-news vuln-dev webappsec Request more information about Pantek

Stefano Received on Wed Dec 11 15:48:51 2002