My $0.02 on PCI DSS 6.6.

The PCI Security Council on April 15, released clarification on DSS requirement 6.6.

Requirement 6.6 states that all web facing applications are protected against known attacks by having a code review or installing an application-layer firewall (WAP) in front of the web application.

I am of the opinion that the clarification document for requirement 6.6 still does not address the issue adequately, and leaves mis-interpretations about code review and WAP.

Let’s start off with observation #1.

From the Information Supplement: Requirement 6.6 Code Reviews and Application Firewalls Clarified document:

“Manual reviews/assessments may be performed by a qualified internal resource or aqualified third party. In all cases, the individual(s) must have the proper skills andexperience to understand the source code and/or web application, know how to evaluate each for vulnerabilities, and understand the findings. Similarly, individuals using automated tools must have the skills and knowledge to properly configure the tool andtest environment, use the tool, and evaluate the results. If internal resources are being used, they should be organizationally separate from the management of the application being tested. For example, the team writing the software should not perform the final review or assessment and verify the code is secure.”

What qualifies a qualified internal resource? Does the QSA qualify this internal resource?

There currently is no standard certification in our industry for code review, and in my experience, very few organizations have any staff that could perform adequate code review if the focus is on identifying security relevant issues.

Scenario 1: Development team uses someone from the IT/QA group to run a web application vulnerability scanner against their web application.

Does this meet Requirement 6.6?
Absolutely not.

Web application vulnerability scanners do not find all vulnerabilities, in addition, they throw out a lot of false positives.. Here at Neohapsis we have seen that and so have others, Rolling review: Web Application Scanners.

“Ultimately, you can’t automate your way to secure software–any combination of analysis tools is only as effective as the security knowledge of the user. As Michael Howard, co-author of Writing Secure Code (Microsoft Press, 2002) and one of the lead architects of Microsoft’s current security posture, put it in his blog: “Not-so-knowledgeable-developer + great tools = marginally more secure code.”

We recommend taking advantage of documented processes, including Microsoft’s SDL (Security Development Lifecycle), The Open Web Application Security Project’s CLASP (Comprehensive Lightweight Application Security Process) and general techniques available at the National Cyber Security Division’s “Build Security In” portal (Automated Code Scanners.” Network Computing Magazine, April 16, 2006).

Web application vulnerability scanners should be used as a tool in conjunction with a full code review.

Observation #2: Option #2 in Requirement 6.6.

I find this to be the band-aid approach to passing 6.6. Web application firewalls, WAF, should be used as an additional layer of security, not a band-aid and avoiding code reviews. Would we consider an IDS or a firewall to be the resolution to running unnecessary services on a system, or a solution to avoid us from hardening or configuring systems to best practice of vendor recommended guidelines? Similarly, WAF’s are another step in the principle of Defense-in-Depth (DiD), but are in no means a solution to securing an application.

There needs to be a balance found with Requirement 6.6 to include source code review, using a web application vulnerability scanner as a tool, and a WAP for it to be taken seriously. There are ways to do this. If an organization implements a solid SDL process, you only need to do sample source code review during the development phase, since a lot of the initial threats were identified during the threat analysis phase. Also, you have secure coding practices and modules that already address a lot of issues such as poor input validation etc. If one is looking to spend less time on finding security issues after a production / environment has to be deployed, one has to look at security right from the start. This holds true when you are building out a security network or DMZ, and also holds true when it comes to the design and development of an application.

The bottom line is that the clarification does not really help out overall.

PCI or not, a proper SDLC implementation and process, developers going through secure code training, and having a proper set of tools will lead to a more secure application.

4 thoughts on “My $0.02 on PCI DSS 6.6.

  1. @ Tom:

    The list of issues can go on forever. I am eagerly awaiting the “PCI SSC smokes crack live” Youtube video.

    While I think that the Microsoft SDL, OWASP CLASP, and the BSI Portal are great recommendations — a lot of “code review” or “application scanning” activity can be averted by having the development team switch to using a rigorous requirements/design process that takes security into account (e.g. see the SQUARE and Misuse case development on the BSI website) — combined with a Test-Driven Development mindset where unit and functional testing are done before construction of code.

    In the case of test-first development, it can be combined with the Dependency Injection pattern to functionally test all code before the code is even written. In other words, almost every security property (besides possibly system/infrastructure operational/architectural issues such as encrypted storage and session management) can be tested before integration. By integration, I mean “before the code is built”.

    These DI functional tests could work exactly like unit testing. Even unit testing itself is incredibly useful for input validation verification, as well as input/output sanitation (e.g. blacklisting, special characters, and output encoding). Unit testing doesn’t even require DI/IoC pattern support.

    A lot of times, unit testing is completely free – e.g. TestDriven.NET (NUnit, NUnitAddin, NUnitASP, and the longer list here), JUnit, or TestNG. There are free, open-source unit testing frameworks for every language used in web applications. There is often good code coverage tools to support quality improvements in unit testing as well.

    In the case of test-first development, code review can be almost completely avoided in a Fagan inspection process. If the moderator knows the test cases (or some of them) beforehand, can see the unit tests themselves (even before the code is written), can see the results of a build, and can view the metrics attached (in the form of code coverage and several others) — then he or she’s work is basically cut out for them. There isn’t a lot of inspection to do once code has been thoroughly tested before it’s even constructed.

    In these types of scenarios, Requirement 6.6 is basically a curse. It will weaken the security. Compliance isn’t always good. I am looking into compensating controls for Requirement 6.6 — there is nothing like finding a loophole that prevents companies from wasting dollars and time on nonsense such as this.

  2. Reply to Andre:

    Smells Like SDL … Tastes Like SDL … It is SDL. Haha, what you just described in your post is SDL. I mean I feel bad for this short response but I think your post just reiterated/defined many aspects of a SDL.

    Agreed, SDL is a good suggestion Tom, in conjunction with a targeted code review.

  3. Unit Testing != Application Security Testing.

    1.) Unit tests are generally written to verify (a input) + (b computation) = (c output). In essence, unit tests most often written to verify expected input leads to expected/correct output. Unit tests are not written to find (d backdoor) or (e vulnerability). The goal of an application security test is to identify these weaknesses.

    2.) Unit tests are only as good as the developers writing them. If a developer does not know about or understand a vulnerability, it is not expected they will write a unit test looking for that weakness (or all the possible variations). I don’t expect an architect to build a house, nor do I expect my contractor to design one. Development and vulnerability research are much the same way. Although not mutually exclusive, they both require a different skill set and are not necessarily inter-changeable.

    Unit testing can offer great value to software development and is recognized as a part of many secure development methodologies for that reason. It however is just one piece of a larger process (i.e. SDL). Therefore, adding vulnerability checks to your unit testing does not replace the need for application security assessments.

  4. The choice of methods to meet 6.6 is a very interesting debate. On the one hand, you want to encourage the sound SDLC practices that come with code review. On the other, you want to achieve compliance as simply as possible. I recently wrote about the reasons you may wish to consider choosing code review as your solution.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s