RE: Help in evaluating Inline IDS/IPS solution

From: Brian Laing <brian.laing(at)blade-software.com>
Date: Thu Jun 05 2003 - 17:14:49 EDT

Ravi,

        Having come from an IDS vendor and now offering products around IDS and firewall auditing, I would be happy to answer some of your questions as well as point you in our direction.

Do IDS vendors really test the signature against the vulnerable applications, hardware platform of the application and version of application before
releasing the signature?
[Brian] This really depends on the vendor and the signature. Some
signaturers are written without an exploit existing. For those that have an exploit seom vendors do this some do not, some are using our products since our product can put this sort of traffic on the wire.

Do the IDS vendors claim this?
[brian] I have not seen andy vendor claims on this doesn't mean its not
there just I have not seen it

If so, what is it I need to look for?
[brian]What I would look for is frequency of updates. Also if you can
extend your evaluation to vover several updates of the product you will be better off. I have seen many times in the field IDS doesn't detect attack, make update it does detect, apply update it goes back to not detecting attack. I have seen signatures change severity, or drop off all together. I have seen the packet reassebly work in one version and be broken in another. Only testing across multeiple releases can you see this. That is why we recommend testing EACH update.

 From sensor technology perspective, I find that all the vendors seems to be having similar capabilities. But, I am trying to see the continued
support on new attacks and vulnerabilities found.

[Brian] Yes I would agree many of the vendors SENSORS are very similar. I
think customers now need to focus a lot more on management of the IDS. This was less true several years ago when the Sensor was the main thing, but now most sensors are VERY close to each other in performance, detection, and other features. Managing those features etc. is now the biggest differenatiator I am seeing customers ask for. Followed by speed!

      One vendor claims that they have 5 dedicated analysts looking at the vulnerabilities and updating signatures (if needed). Another vendors claims that
they have more than 20 analysts doing this job. Can this be considered in my

eval?
[brian] I would not consider the number of analysts doing the work but the
frequency of updates and the quality of updates. If one vendor has 10000 people working on the problem but updates are sporadic, difficult to implement etc. then those 10000 people are useless. If however one vendor has 5 people and is regular as clock work on updates that is the route to go

 Is it that other vendor exaggerating the number of resources they have for this job.
[brian] creative counting has always been part of this market, just look
back at the way signatures were counted both for early IDS and vulnerability assessment. One vendor counts a single teardrop (but checks for 15 iterations) while another vendor counts each iteration as a different signature.

  Performance:

      What is the best metric to look for? I feel HTTP1.0/1.1, SMTP, IMAP, NNTP, TELNET, POP3 connection rate and UDP throughput for different sizes is good metric. Is there anything should I look for?
[Brian] I see this as really being several thigns that need to be tested
for
1. speed how much raw bandwidth can the sensor handle without dropping stuff. This is especially improtant in an inline IDS as dropped packets don't make it regardless of attack detection. Knowing the protocols on your segment can help immensly in running your test as a 100% http traffic segment is a lot different then a network with a variety of protocols to assemble and analyze.
2. attack detection what do I detect at close to 0% network utilization once you know this then you can step up to
3. Attack detection under load Various network loads to see when it looses attacks vs just dropping packets. Dropping packts and missing attacks are two different beasts all together.
4. management, it can be the best sensor in the world but if you can not manage the number of sensors you have and the number alerts you receive then the sensor is useless.

      Are there any labs, which provide testing facilities for testing IDS/IPS with latest vulnerabilities and with real vulnerable applications? I am really
looking for lab which provides facilities and allows us to test the IDS/IPS solution on regular basis.
[Brian] I am not aware of labs that let you walk in and test what ever you
want with this sort of test. We do have that sort of facility but we only have it open to a few people and its not available to the general public. However you can use our software to simulate 100% accurate attacks between two points using our IDS and Firewall informer products. If you have any questions about them please don't hesitate to drop me an email

Cheers,
Brian

---

Brian Laing
CTO
Blade Software
Cellphone: +1 650.280.2389
Telephone: +1 650.367.9376
eFax: +1 650.249.3443
Blade Software - Because Real Attacks Hurt http://www.Blade-Software.com

---

-----Original Message-----
From: Ravi [mailto:ravivsn@roc.co.in]
Sent: Wednesday, June 04, 2003 9:41 PM
To: focus-ids@securityfocus.com
Subject: Help in evaluating Inline IDS/IPS solution

Hi,

      My company plans to resell the Network Inline IDS/IPS solution to our customers and support

      customer. I was given task of evaluation of different solutions in the market. There are some

      questions asked by our customers and I would like to keep these in mind while

      evaluating the IDS solutions.

      Do IDS vendors really test the signature against the vulnerable applications, hardware

      platform of the application and version of application before releasing the

      signature? Do the IDS vendors claim this? If so, what is it I need to look for?

      From sensor technology perspective, I find that all the vendors seems to be having

      similar capabilities. But, I am trying to see the continued support on new attacks

      and vulnerabilities found.
      One vendor claims that they have 5 dedicated analysts looking at 
the vulnerabilities
      and updating signatures (if needed). Another vendors claims that 
they have more
      than 20 analysts doing this job. Can this be considered in my 
eval? Is it that other
      vendor exaggerating the number of resources they have for this job.

      Performance:
      What is the best metric to look for? I feel HTTP1.0/1.1, SMTP, 
IMAP, NNTP,
      TELNET, POP3 connection rate and UDP throughput for different 
sizes is good
       metric. Is there anything should I look for?

      Are there any labs, which provide testing facilities for testing 
IDS/IPS with latest
      vulnerabilities and with real vulnerable applications? I am really 
looking for lab
      which provides facilities and allows us to test the IDS/IPS 

solution on regular basis.

      Thanks
       Ravi

-- 
The views presented in this mail are completely mine. The company is not
responsible for whatsoever.
------------------------------------------------------------------------
Ravi Kumar CH
Rendezvous On Chip (i) Pvt Ltd
Hyderabad, India
Ph: +91-40-2335 1214 / 1175 / 1184

ROC home page <http://www.roc.co.in>




----------------------------------------------------------------------------
---
INTRUSION PREVENTION: READY FOR PRIME TIME?

IntruShield now offers unprecedented Intrusion IntelligenceTM capabilities 
- including intrusion identification, relevancy, direction, impact and
analysis 
- enabling a path to prevention.

Download the latest white paper "Intrusion Prevention: Myths, Challenges,
and Requirements" at: 
http://www.securityfocus.com/IntruVert-focus-ids2
----------------------------------------------------------------------------
---


-------------------------------------------------------------------------------
INTRUSION PREVENTION: READY FOR PRIME TIME?

IntruShield now offers unprecedented Intrusion IntelligenceTM capabilities 
- including intrusion identification, relevancy, direction, impact and analysis 
- enabling a path to prevention.
Download the latest white paper "Intrusion Prevention: Myths, Challenges, and Requirements" at: 
http://www.securityfocus.com/IntruVert-focus-ids2
-------------------------------------------------------------------------------

Received on Thu Jun 5 19:48:41 2003