You may have noticed that there has been a slight dip in the frequency of blogging during the early part of this week. As it happens, I've been participating in a MasterCard scanning recertification effort. Since the topic was on the brain, I thought I'd discuss that as well as PCI in general in today's entry.
So, as part of the gigantic machine that is PCI, Visa/MasterCard require that non-members (i.e. firms that aren't banks) undergo a quarterly external scan of their environment. In order to be a firm in the business of providing that scan, you have to pass an annual certification test that proves that you have the chops in order to do the scanning at a certain level of expertise. Basically, you're given a bogus environment and you have to go about performing scanning/investigatory activities on a mock-up environment in order to demonstrate your level of expertise. And then, once you successfully complete the challenge, you are given a certificate that says you're "all good" from MasterCard's point of view. That's basically the exercise in a nutshell.
Basically, it's bull. Why is it bull? Here's the low-down: the economics of doing this type of scanning are not such that a firm can deliver anything close to the kind of testing that is done for the certification. Some firms are offering the scanning "free" with other services (meaning that the costs are hidden in remediation or other projects), other firms are offering the service at "Crazy Eddie" discount rates and performing equally shoddy work. A vendor might, for example, run nessus in its default configuration against a client environment, export the results as an HTML report, and "call it day" - grand total, 20 minutes of work. What Visa/MasterCard really want is for vendors to take the time and energy to do it the right way, which of course is impossible when "discount" vendors are doing 1/20th the amount of work. So it's unprofitable to actually do what Visa/MasterCard intended for quarterly scanning; ergo, nobody does.
Of course, nobody over at Visa or MasterCard is complaining about it. Which makes me wonder what their interest in the certification really is. After all, since the certification activity is completely unrelated to what's actually being done in the field, I'm curious why they want vendors to get certified at all. I'm wondering if the fact that they require this has to do with the fees required for certification, recertification, and fees to take the test. 100k per firm per year in recertification fees and testing fees seems like a good revenue stream to me - especially when the work involved is just setting up a few default installs of various operating systems for the test procedure.
Posted by Ed at August 23, 2006 09:05 AM | TrackBackEd,
Are you dealing with the guys in Belgium? I worked for a company that was PCI certified. I can tell you that to get in, our test was handled in a manner that made us think that we were "supposed" to fail the first time(to pay another $5k entrance fee, maybe?). Our first attempt was rejected mainly because of the title on the report cover. No kidding.
Once we were in, there was a definite rush among vendors to provide bargain basement pricing. We had inquiries from all sorts of organizations, 30 unit health clubs, Fortune 500's, E-Commerce sites, etc... but in the end they all opted for a $5 an IP vanilla Nessus scan (ours was a a little more comprehensive, and something like $25 per IP plus set up IIRC) because it got the minimum requirement done.
At the end of the day, our CEO made the right decision - we gave up on the market because he'd rather do security right than try to make $10 Nessus scanning Ronnie's Pet Stores of the greater Tri-State Area.
The Card Vendors really need some sort of audit process - and the teeth to take certification away from vendors who default to poor services (and making their clients pay again to do the service right).
Alex,
You're dead-on. We're working with the same team over there.
It's interesting the experience that you had with failing the first time around; I've heard of another company that failed the first time around for not explicitly testing DoS conditions (because it said no DoS in the ground-rules), but then got dinged for not testing this. So, guess what? Another fee for MC.
In terms of the reality of PCI, the way I see it is this: every once in a while you (as a non-member) have to pay a MC-approved company to scan you using an approach that provides no value to you, then you have to pay someone once a year to show up, waste your time, and submit a document to Visa that nobody will ever read.
Not such a great way to protect your environment, but probably a good way for Visa to make money. But maybe I'm just cynical. :-)
Posted by: Ed at August 25, 2006 08:49 AMI have been doing PCI assessments for about 2 years now. I did them for a very large "company" that basically got permission to allow us to do the internal assessment and use that to meet the PCI requirements. I have heard other stories as you guys are describing and I agree that this is a shame. There is a REAL need to ensure that credit card transactions are secure and I feel that the intentions of PCI were good, but like most programs, once a good thought but later determined that the feasibility from a financial and time perspective were not so good. I hope that this turns around sometime in the future. I dunno about you guys, but I use those virtual cc numbers whenever I can!
Posted by: Scott at August 25, 2006 12:11 PMHmmm, this *is* interesting. We went through the accrediation process and we thought our experience was unique.
After evaluating our report, we were told that we had not reported some vulnerabilities that were present, and they gave us examples.
But 5 out of the 6 examples they quoted to us were actually present in the report. Even if they didn't read our report fully, they would still have found it if they did a "Find" for those keywords. Makes one wonder what kind of evaluation they did.
Posted by: David at August 29, 2006 07:20 AMDavid, we faced the same problem as you faced. They did not go through the report completely when we went through the accreditation process.
They interestingly said, "But for these few vulnerabilities that we didn't find in your report, you would have made it". As if we were among the privileged few and still missed the bus by a small margin.
Posted by: Niel at August 30, 2006 01:36 AM