RE: common criteria draft

From: Brewis, Mark <mark.brewis(at)eds.com>
Date: Thu Jan 09 2003 - 06:00:39 EST

Ron,

Thanks for your comments. You've raised some very interesting points, which I will try to cover. I only have a superficial understanding of the ICSA process, so please correct me on any incorrect assumptions.

>>We were discussing the merits of Common Criteria versus ICSA especially
for
>>firewalls. The discussion ended with a basic understanding that both were
>>needed.

I agree. This is a perfectly valid conclusion, and reflects the fundamental differences in the rationale behind CC and ICSA evaluation. Without wanting to get into detailed descriptions of CC Evaluation (because I'm not sure this is the correct forum for that discussion, and I'm certainly not the best qualified to do so,) CC is best understood as a detailed assessment of a product at a milestone. To this end, detailed analysis of the development environment, code, documentation and product can be undertaken.

>>The reason was that the Common Criteria evaluation materials didn't allow
>>for constant real world penetration tests against known vulnerabilities.
>>While the product was under "contract" for evaluation by ICSA labs, any
new
>>vulnerabilities would be also tested. Common Criteria evaluations don't
>>have a mechanism for this type of continuous testing.

This isn't quite correct, although it is substantively so. The scope always existed for Ad Hoc testing, although the trigger for an ad hoc test had to come out of the testing undertaken. The evaluation process comes out of a requirement for stability - hacking boxes wasn't seen as meeting core requirements on thoroughness or on the reproduction of results. Flexibility and reactivity were sacrificed to meet these aims. There is far more to it than that, but it is a good starting point.

In addition, it is perfectly acceptable to write a test script (by which I mean a paper record that details what you do, how you do it, the expected result and any deviation from the expected result. None of that perl stuff here, thank you very much) that uses an automated tool - Nessus or ISS for example. Load the most recent signatures on the day you test, and your records detail the tool, version, signature date and time. At that point, you should be fairly up to date. The problem lies in the testing window, which may be a considerable time before the product is Certified.

>>This is important from a point of cost. Many of the tests for a product
to
>>get certified under CC and ICSA are the same. For most areas CC goes much
>>further than ICSA.

Without knowing ICSA very well, I can't comment too much here, but the CC can be extremely detailed, and does, in general, go much further. The point is that CC evaluations try to identify which vulnerabilities are relevant to a particular product before testing, so many of the ICSA tests may not in fact be carried out in CC if the in depth analysis that goes into CC shows that the vulnerabilities being tested for under ICSA are not relevant to the product. Also, the CC tests will include some that are as a direct result of the in depth analysis of the design and source code and so will test for vulnerabilities that are not known to anyone else not directly involved in the CC work so there is an element of 'future proofing' in CC.

>>The problem is that a vendor has to put out real money
>>for both certifications. I would much rather see that the vendor only
need
>>to certify their products under one set of rules.

The schemes are complementary, but I think they serve different purposes. CC is as much about the process of development as it is about the end product. It looks to see if the product has been created in a methodical, documented, detailed way. The important thing is to look at what was evaluated in a product, to what level, and then to see what wasn't evaluated. The Security Target is the critical element.

>>To that end Mark, is it possible that this draft method will cover the
>>missing testing from CC that ICSA delivers?

I'm not sure that it can close the gap between the testing phase of a CC evaluation and the Certification of the product. Evaluation is a monolithic process. Set it in motion and it trundles along in a straight line until it stops. ICSA, I'm sure, also has a window between the end of testing and certification, but I would assume it is much shorter. A more reactive process.

Hope this is of some use to you,

Mark

Mark Brewis

Security Consultant
EDS
Information Assurance Group
Wavendon Tower
Milton Keynes
Buckinghamshire
MK17 8LX.

Tel:	+44 (0)1908 28 4234/4013
Fax:	+44 (0)1908 28 4393
E@:	mark.brewis@eds.com

PGP Key ID: C36D 770F 49F7 CC91 2E5A A2BE FE6E CD43 E6CD 9184

---

This list is provided by the SecurityFocus Security Intelligence Alert (SIA) Service. For more information on SecurityFocus' SIA service which automatically alerts you to the latest security vulnerabilities please see: https://alerts.securityfocus.com/ Received on Fri Jan 10 14:51:52 2003