One of my posts from earlier in 2012 discussed the idea that CEOs are to blame for all of our problems in security. The idea that we “must have reliable actuarial data on incidents to stay relevant” is another of the information security holy sacred cows that rears its head every so often, and this post explains covers three angles on the incidents database idea. First I look at the impracticalities of gathering incidents data. Second, even if we have accurate data, exactly how useful is the data to us in the formulation of risk management decisions, and third, even if the data is accurate and useful, did we even need it in the first place?
Shostack and Stewart’s New School book was from the mid-2000s and a chapter is devoted to the absolute necessity of a global, ubiquitous incidents database. Such a thing is proclaimed by many as carrying do or die importance, as if we need it to prove the existence of a threat, and moreover to prove our right to exist as security professionals. Do we really need a global database to prove to the C-levels that spending on security is necessary?
There are of course some real blockers with regard the gathering of incidents data, not least the “what the heck just happened” factor, where post-incident, logging is turned on (it was off until the incident occurred) and $300k invested in a SIEM solution – one that really doesn’t help the business to respond effectively to incidents, it just gives operations a nice vehicle with which they can use to diagnose non-security related problems, and of course the vendor/box pusher consultants knew what they were doing when they configured the thing.
Anyway it really isn’t terribly constructive to talk reality with regard this subject. We need to talk about a theoretical world, somewhere over the rainbow – one where logging is enabled, internal detection controls are well configured, and internal IT ops and sec know how to respond to incidents and they understand log messages. Okay, most businesses don’t even know they were hacked until a botnet command and control box is owned by some supposed good guys somewhere, but all talk of security is null and void if we acknowledge reality here. So let’s not talk reality.
So incidents data simply won’t be available in many cases – that base is now covered. But what are we trying to achieve here? The proposal is to use past incidents data to help us prove the existence of a threat in formulating decisions on risk. We receive a penetration test report that tells us that vulnerability X was compromised 4 years ago in Timbuctoo. The other vulnerabilities mentioned in the report – according to our accurate database there is no mention of them ever having been compromised with resulting financial damage. So we fix vulnerability X and ignore the rest – a wise strategy, especially given that our incidents database hosts accurate information with check-sums and integrity built-in (well – I did say let’s not talk reality here).
But wait a minute – vulnerability X was exploited 4 years ago in a company Y in an industry sector Z. What if company Y’s network was wide open? Where is this leading? All networks are different. All businesses face different challenges. Even those in the same industry sector as each other are completely different. How ludicrous this is getting? Business X can even have the exact same network architecture as another business Y in the same industry sector, and yet the exploitation of the same vulnerability X in business X can lead to $100K damages for business X, but $100 damages in business Y. So are we really going to use these records of financial damages to formulate our risk acceptance, transferal, or mitigation decision? One would hope not.
As if this all isn’t bad enough: even if we ignore the hard reality of the limitations covered here, how long would it take before we have gathered enough information to call the database useful? 10 years? 20 years? One million incidents or…? Products are past their shelf life in less than a decade generally. I did come across an IBM AIX 4.1 (we’re talking mid-to-late 90s here) box at a customer site not so long ago, but thankfully that was an isolated incident.
After all the deliberation, it emerges that security is too complex – it cannot be reduced down to a simple picture a la “vulnerability X was exploited resulting in damages Y in company Z, and therefore we in Botnetz R Us need to address vulnerability X”.
All this is not to say that there is no benefit in knowing what’s going off out there in the wild. We can pick up on on-going bigger picture attack trends to enable us to prepare for what might be down the road for us. But do we really need details of past incidents to convince businesses to part with cash in risk mitigation, or moreover validate our existence?
What we’re really concerned with here is trust. The proponents of a big data repository of incident big data would have it that we need such a thing because the powers that be don’t trust us. When we propose a mitigation of a particular risk, they don’t trust our advice. Unfortunately though, we might get one success story where we consult the oracle of all incidents and we do magically find an incident related to the problem we are trying to fix, and we use this “evidence” to convince the bosses. What about the next problem though? We can’t use the same card trick next time to address the next risk issue if no such problem was ever encountered (and reported) before by another company somewhere else in the world. By looking into the history of all incidents we’re setting a dangerous precedent, and rather than enabling trust, we’re making the situation even worse.
We don’t need an incidents history record. What we need is for our customers to trust us. Our customers are C-levels, other business units, home users, in-house reps, and so on. How do we get them to trust us? What we need is a single accreditation path for security professionals, one that ties Analysts to past experience as an IT admin or developer, and one that ties Security Managers with past Analyst experience. With this, security managers can confidently deliver risk-related advisories to their superiors, safe in the knowledge that their message is backed up by Analysts with solid tech experience, and in whose advice they are comfortable in placing their trust.