PCI 30 seconds newsletter #17 – Why are my scan reports so thick? - Impact of "potential" vulnerabilities

Blog Post created by dgodart on Jan 16, 2012


"My PCI scan report has more pages than the NASA report related to the crash of the space shuttle Columbia".



This acerbic statement was made by a merchant complaining about the size of his external scan reports.


Verse #11.2 of the PCI data security bible requires organizations subjected to PCI compliance to run internal and external network vulnerability scans at least quarterly on their CDE (card data environment). The PCIco regards risk relating to the internal and external sides of the CDE differently. This translates in the subdivision of verse 11.2 into §11.2.1 (related to internal scanning) and §11.2.2 (related to external scanning). While the level of flexibility and initiative allocated to companies is indecently large in terms of internal scanning – organizations may basically do whatever they want – external scanning is subjected to more demanding, structured and explicit rules starting with the mandated use of an approved scanning vendor (ASV) submitted to annual certification. When running scans, ASVs must strictly comply with specific rules published by the PCIco in a document called: ASV Program Guide Reference.




Why are external scanning reports so thick?



The major causes of the thickness of a PCI scan report are:



1. The extent of the scan fields.

I wouldn't surprise anyone by stating that the more targets you have to scan the more thick your report would be.



2. The structure of the scan report

As part of the ASV Program Guide, the PCIco requires ASVs to report scan results in a specific way. The report must consist in threefold:

a) a short attestation of compliance

b) an executive summary

c) a detailed vulnerability reports.



In terms of structure, the pain part is the executive summary that is far away from being what one would expect from an executive summary: short and straight to the point. The PCIco requires the executive summary to list all detected vulnerabilities for each target, including the severity level, the CVSS score, the compliance status (PASS or FAIL) as well as any associated exception, false positive, or compensating control. Additionally PCIco requires that the long executive summary be amended with a consolidated solution/correction plan for each target.

Based on the above, one could easily understand why an executive summary would be long. Hopefully, one could expect in 2012 a new version of the ASV Program guide wherein the executive summary would be split into a real executive summary and a technical summary limited to the list of detected vulnerabilities that do not PASS the compliance threshold in terms of severity and excluding out informational results. Those are the ASV's recommendations to PCIco.



3. Potential vulnerabilities

Many merchants ignore what the hell a "potential vulnerability" is and show suspicion when being told by their ASV about it and for good reason as this term is absent from the PCI glossary and PCI DSS bible. However, potential vulnerabilities weight highly into the causes of the report thickness as well as into the causes of scan failure and workload associated with the vulnerability management.



The PCIco introduces this term into the ASV Program Guide as a "vulnerability from which the presence cannot be determined with certainty". Unfortunately nothing is said about what the PCIco considers as "certainty".


Abstract - ASV Program Guide V1 pg 14:

In addition to confirmed vulnerabilities, ASVs must report all occurrences of vulnerabilities that have a reasonable level of identification certainty. When the presence of a vulnerability cannot be determined with certainty, the potential vulnerability must be reported as such. Potential vulnerabilities must be scored the same as confirmed vulnerabilities and must have the same effects on compliance determination.




Confirmed versus potential?



For a neophyte it could sound strange to say that an ASV could be "uncertain" of the presence of a vulnerability. Why is that?


The root of the answer lies in the "blind" nature of an external scan. Scans are performed remotely through the Internet with no privileged (admin) access to the targets as this would require transmitting admin credentials through the Internet, which is not a recommended practice!  Furthermore, the signature (name and version) of the targets could have been voluntarily modified or obfuscated for the sake of security.






In the same way SONAR recognizes boats based on the noise pattern generated by their engines, vulnerabilities could be determined with a high level of certitude based on the response patterns received from targets. Another bunch of vulnerabilities are reported only because such service name, such version number and patch levels have been (maybe erroneously) determined. Those latest ones are what the PCIco considers potential vulnerabilities. However, for the readers of a scan report there is no difference between confirmed vulnerabilities and potential ones. They are all falling into the long list of reported vulnerabilities.



Besides impacting the size of the scan report, potential vulnerabilities are causing a high rate of false positives and therefore impacting the reliability of the scan results and the workload of the individuals in charge of verifying the level of certainty of each vulnerability. They are not considered to add value by many ASVs. In fact, some ASVs go so far as to play against the rules by completely ignoring them from their scan reports in order to increase the level of accuracy of their reports and customer satisfaction.


Despite their evident low added value and poor impact on the accuracy of the scans, the thickness of the reports and the workload, the PCIco persists in requiring ASVs to include these potential vulnerabilities into their reports

Now I see.jpg


I was blind but now I see!


As a suggestion to decrease the level of blindness of external scan reports and decrease the workload, companies should consider scanning external targets from the inside as part of their mandatory quarterly internal scans. This secured scan source allows for authenticated scans with full access to the targets and is therefore more reliable. Use the internal scan reports to quickly spot false positives in the external scan reports.


A second, but less practical option is to establish an encrypted tunnel between the scan source and the scanned network allowing for the use of authenticated checks.







1. Did you find this article interesting? YES/NO

2. Was it useful? YES/NO

3. What is the average number of pages of your external scan reports?

4. Are potential vulnerabilities included in your ASV scan reports?  YES/NO/NOT SURE

5. Would you vote for removing potential vulnerabilities from the scan reports? YES/NO


Didn't read our previous newsletter? Is your organization acting like a fashion victim or a clown?