The new PCI ASV Program Guide is out, and the updates are much more significant than they appear …

I had the pleasure of working on the ASV Task Force last year, pulled together by the PCI SSC to revamp the rules of engagement for ASV Services. The experience was fantastic, working directly with the PCI SSC Technical Working Group and a stellar cast of representatives from the ASV Community. The most rewarding part of the entire experience was setting aside competitive differences to evolve this program. This was the most collaborative effort among competing Vulnerability Management vendors that I’ve seen since the Conficker Working Group brought a number of us together to combat the downadup outbreak in early 2009. There was lively debate among the task force members, and in the end I feel that we moved the ball forward. There is much more to be done of course, and this is the first of what I believe will be many improvements.

There were significant contributions from Rapid7 (Chad Loder – VP of Engineering and myself were involved), ControlScan, NetVigilance, Trustwave, nCircle, Critical Watch, Solutionary, and others. (Apologies to those I’ve missed – definitely not intentional). Thanks to contributors from all of the participating organizations … this was a fantastic team effort.

A number of improvements came out of this initiative and it brings about some significant changes that should be on your radar.

First and foremost, there is much more clarification around the roles and responsibilities for ASV’s and “Scan Customers” (a term intended to capture the folks who require quarterly scans to demonstrate compliance with the external scan component of 11.2). In particular, there are now clear requirements for attestation on both sides, with scan customers ultimately responsible for things like scoping and ASV’s formally attesting to compliance status following quarterly scans. This is bolstered by mandatory text for scan customer and ASV attestation:

SCAN CUSTOMER ATTESTATION:
(Scan customer name) attests that: This scan includes all components which should be in scope for PCI DSS, any component considered out-of-scope for this scan is properly segmented from my cardholder data environment, and any evidence submitted to the ASV to resolve scan exceptions is accurate and complete. (Scan customer name) also acknowledges the following: 1) proper scoping of this external scan is my responsibility, and 2) this scan result only indicates whether or not my scanned systems are compliant with the external vulnerability scan requirement of the PCI DSS; this scan result does not represent (Scan customer name)’s my overall compliance status with PCI DSS or provide any indication of compliance with other PCI DSS requirements.

ASV ATTESTATION:
(ASV name) attests that the PCI DSS scan process was followed, including a manual or automated Quality Assurance process with customer boarding and scoping practices, review of results for anomalies, and review and correction of 1) disputed or incomplete results, 2) false positives, and 3) active interference. This report and any exceptions were reviewed by (name).

In line with the attestation requirements, a great deal of progress was made on standardized reporting. There is a new reporting requirement, with an Attestation of Compliance Cover Sheet added to the mix. There is a mandatory template published as an appendix in the new Program Guide that must be adopted by all ASV’s, providing one standard report format to ensure consistency and comprehensiveness. The ASV Scan Executive Summary report also has a mandatory template that must be used by all ASV’s. Again, I believe this enables consistency and clarity regardless of the selected ASV. Finally, there is a recommended template for the ASV Scan Vulnerability Details report. ASV’s have the option to create their own report format, provided it includes all of the required information captured in the PCI SSC-provided template. This kind of standardization can only help the program and affords customers more autonomy if they are not satisfied with services from their ASV … a move that will certainly stimulate competition and consequently force service providers to step aside or deliver more value and higher quality to customers. That is what we’re here for.

Through painstaking debate among task force members, there is now much more clarity around the required components for PCI DSS Vulnerability Scanning coverage. The new Program Guide outlines the scan components, with clear information for scan customers as to why the component must be scanned, and for ASV’s as to what they must be capable of.

The new ASV program guide includes updated guidance on web application scanning, which is now listed as a Required Component for ASV scanning. With this new standard, the PCI Council has clarified once and for all that true web application scanning (with crawling) is a required part of any ASV scan solution. For a long time, many ASVs were hiding behind gaps in the previous version of the standard, which was dangerous for ASV scan customers. There was decent progress on web application coverage to move beyond “XSS + SQL Injection”, including the addition of Directory Traversal and HTTP response splitting/header injection as automatic failure conditions.

Web Application scanning still has room for improvement in the standard, and I expect to see the requirements expand in future iterations of the standard and Program Guide. Specifically, we were not yet able to have web 2.0 technologies such as java script pulled into the explicit requirements. When this expansion happens, there will be no change requirements for NeXpose … Rapid7 already has these capabilities.

Clarification was added to risk scoring and categorization, with CVSS v.2 base scores mapped to high, medium, and low categories. Under the new guidelines, both score and corresponding category must be shown. There is a process for exceptions to risk scoring that is extremely practical.

The last change that I’m going to talk about on this post is a significant change with regard to dispute workflow and tracking. Under the new Program Guide, there is additional rigour added to Scan Customer and ASV responsibilities in processing and tracking disputed findings. Much of the new rigour is explicitly calling out procedures that *should* be in place already. Call it Best Practice, Common Sense, whatever … mandatory review of False Positives, etc. should be a meets-minimum procedure.

The key clarifications in the new Program Guide are that disputes are not to be removed from scan reports and that dispute findings are not to be carried forward from one quarterly scan to the next. That means that disputed detection such as False Positives must remain in the scan report and that they must be disputed, confirmed, and documented every quarter … even if they existed and went through the dispute process in previous quarters. Disputes must be evaluated exclusively by ASV Security Engineers qualified by PCI SSC as per Section 3.2, “ASV Staff – Skills and Experience” in the PCI DSS Validation Requirements for Approved Scanning Vendors document. This is potentially the biggest set of clarifications in the new Program Guide. From a cost perspective, this absolutely highlights the need for accuracy among ASV’s. The costs associated with Scan Customer and ASV efforts to process and track disputes around False Positives have just increased dramatically, putting much greater pressure on ASV’s to be accurate in their detection capabilities. As “Potential” Vulnerability findings must be treated in the same manner as Confirmed Vulnerability findings, this is a huge difference between service providers who depend heavily on Potential checks and those who have invested in Security Research and Engineering resources to provide superior accuracy.

For Scan Customers, this changes the game from shopping for the lowest per IP price to managing vendor costs *and* operational costs associated with quarterly scanning. Using a service that is highly inaccurate and/or has a high dependency on Potential checks can result in much higher operating costs than the associated vendor costs themselves. While the initial reaction may be that this change to the standard raises the cost for customers to demonstrate compliance, I believe that there will be little-to-no change for customers using a service that already provides accurate detection. With standardized reporting and other standardization components within the new Program Guide, customers now have the choice and clear decision-making criteria to reject lower quality solutions in lieu of a more accurate service that provides cost-effective operations. In short,
it just got a lot more expensive to use a sub-standard Vulnerability Scanning solution.

For ASV’s, the same dynamic is at work. Highly accurate solutions will yield very little difference in ASV Scan Service operations and costs. For ASV’s who produce less accurate solutions or have licensed sub-standard technology for their Scan Services, their operational costs are likely to skyrocket as a result of this clarification. What once was only a problem for a first scan can no longer be swept under the rug … poor detection quality will result in high operational and support costs quarter after quarter after quarter. Given that seasoned Security Engineers must review and handle exceptions, this need cannot be accommodated with low-cost generalists, driving the costs even higher.

For Managed Security Service Providers who license other vendors’ technologies for their services, this means staffing up with more senior resources and simultaneously pressuring their technology providers to clean up their act with regard to accuracy. The former is expensive and the latter does not and cannot happen overnight, if at all. We expect that some existing Scanning Vendors will exit the ASV market as a result … it may be cost-prohibitive to maintain an ASV service if poor detection quality drives up support costs. For the sake of Cardholder data security, we believe this is simply the cost of providing assurance. Other service providers will switch to superior technologies … it’s been a busy week of prospective partners contacting us.

Fortunately, we enjoy an extremely accurate Vulnerability Management product in NeXpose, so we see very little change in costs within Rapid7, for our ASV partners, and for our Scan Customers. Accuracy is a principal driver for Rapid7, and we feel that these changes are long overdue. In line with our message, and the message that I’ve personally sent on this blog since we launched it, customers deserve better and this change helps to deliver on that promise. Hats off to the PCI SSC for fostering this level of cooperation, and to those competitors who invested time in effort in contributing to the new Program Guide. It really does take a group effort to make meaningful change … thank you for your efforts.

Also, congratulations to Troy Leach on his appointment as CTO of the Security Standards Council. Troy was instrumental in bringing this task force together and we appreciate his efforts to bring competitors together in cooperation. The outcome is a Program Guide that, combined with the Quality Assurance program, raises the bar for the program itself and all of the ASV’s serving our clients.

For customers, we’ll leave it to your discretion decide if ASV certification is an important factor in evaluating and selecting a Vulnerability Management solution for your organization. In the meantime, we’ll keep doing our part to challenge our industry to do better.

The new Program Guide requirements are in effect now and have a transition period through to September 1st, 2010. As of September 1st, all ASV’s must deliver their services under these guidelines without exception (no pun intended).

For a copy of the new PCI ASV Program Guideline, visit the PCI Security Standards Council website at: https://www.pcisecuritystandards.org/pdfs/asv_program_guide_v1.0.pdf

To learn more about how Rapid7 can help with your PCI Compliance and ASV Scanning needs, visit us online at: http://www.rapid7.com/services/pci-compliance-testing.jsp or send us an e-mail at pci@rapid7.com