The difference between compliant and not is how hard you look

The other day, while researching the thing about the PED devices, I came across some chatter about folks making the statement that a PCI-compliant entity has not been successfully hacked.  I recall hearing this particular line in QSA training many years ago (2005 maybe?) and apparently, folks are still saying it today.  From the article I cited in the prior post re: the PED devices:

Perhaps even more intriguing is what this disclosure will do to Visa’s oft-stated position that no PCI-compliant retailer has ever been breached. One of the key elements to being a PCI-compliant chain is housing only PCI-compliant equipment. Will Visa and PCI be issuing a blanket get-out-of-PCI-jail-free card for any merchants who were breached because they relied on a PCI-sanctioned device?

It’s interesting that this is still out there.  I had thought that the criticisms of this line of reasoning had knocked it out of current use.  But apparently this is something currently still in use.  In light of Heartland and others, can this still be true?  I think it’s a useful line of reasoning because it preserves faith in the standard, but it’s useful to unpack because of the questions that naturally surface in light of breached entities that hold “current” compliance certification.

The folks over at InfoSec Island allude to this when they quote Bob Russo in reference to breaches and compliance status:

Victims may have attained compliance certification at some point…  but none has been in compliance at the time of a breach.

My reading of this is that, even though a given breached environment was certified as being compliant, it really wasn’t.  The certification doesn’t really count because evidence of noncompliance eventually surfaced.  The conclusion: the determination of compliance was reached erroneously; if the assessment had gone farther and the scope was increased, the assessment team would have come to the (right) conclusion that the entity was non-complaint.

Interesting…  I don’t argue with this.  But using that same measure, I would argue that something else is true.  Namely, that no organization is compliant… ever.  Sure, we might mistakenly conclude that some organizations are compliant due to real-world constraints like scope or limits in audit budget… but if we really look hard, we’ll find an area in the organization of non-compliance.

The organization may not know about it… and auditors may not include it in scope… but it’s there just the same.  My opinion:  the difference between a compliant environment and a non-compliant one is how hard you look.

So if you have a breach and the hindsight crowd come through looking at the whole kit and kaboodle, they’ll find that problematic area.  Bet on it.  And at that point, those folks move into the “may have been certified at one point but aren’t compliant at the time of the breach” group.