Security Curve Weblog

More trouble with statistics...

The trouble with statistics is that they can sometimes be deceptive. For example, my RSS reader and inbox has been flooded for the past few days with news about the recent Mitre report citing cross-site scripting as the number one attack vector; everybody's writing about it. The short story around this is that the 2006 Mitre CVE statistics point to the fact that Cross-Site Scripting accounts for the largest share of reported vulnerabilities (21.5%) followed closely by SQL Injection (14%). Now, no doubt this is interesting. No doubt that Mitre's doing a service to the community by publishing these statistics. And no doubt this tells us something about the state of vulnerability research in 2006. But what it tells us is not exactly the same as what's being represented in the industry press.

Here's what I mean. Clearly, the data does tell us that more application vulnerabilities were located this year than in prior years. Based on those statistics, we can say without question that web vulnerabilities are a more popular target for research. However, the temptation is to draw broader conclusions from the numbers that aren't necessarily in scope. For example, Network World calls Cross-Site Scripting the "top security risk"; the Inquirer says that "hackers are looking to cross site scripting bugs as the best way to bring down a system." LinuxWorld tells us that Cross Site Scripting is "now the most preferred hacking techniques used by hackers since these vulnerabilities allow access to such data as credit card details." The implication is that XSS is in active use by blackhats to commit fraud and that it's being used as a vehicle to bring systems to a halt; not only can we not know that based on what was released, but they also go against things that we know to be true (such as, for example, that XSS can't bring a system down.) Moreover, the data from Mitre doesn't account for usage of vulnerabilities - it's just their appearance that gets tracked.

So "caveat lector." All we can say for sure based on the data is that researchers are finding more XSS vulnerabilities; there could be a number of reasons for this not having to do with attackers using it more. Maybe cross-site scripting is easier to find (it is) or maybe web-based products that might be impacted by xss are more popular now than three years ago (they are). I don't think we can draw conclusions about anything other than the fact that XSS is more popular with researchers - we can't/don't know anything about the popularity with attackers or level of risk associated with a XSS vs. a buffer overflow.