The Ugly Truth About Risk Scores and Threat Intelligence

Whether it’s vendors that provide an external grade/score of a company’s risk, threat intelligence platforms or vulnerability management vendors, everyone is looking for the ultimate “risk score”.

A Risk Score is like a home inspection report. It makes us feel like we are making good decisions in buying a new home by telling us what could go wrong next, but the inspection doesn’t really find everything that is wrong or prepare us for the dirty truths already present in the home. It could tell us that there is a mouse hole that needs to be covered up, but not whether a mouse is actually inside. While we have a lot of promises around machine learning and AI to piece together seemingly random indicators of compromise (IOC), the burden still falls on the analyst to make sense of a discovery of “mouse droppings”. That is for another blog, but the point is, does a risk score provide you with true insights into whether an infestation is already present?

Exposing Risk Scores and Threat Intelligence: A Game of Guessing

So, what’s the point of the risk score? It can vary depending on what your objectives are:

1. Assessing 3^rd-party risk, aka Third-Party Risk Management: In its simplified form this is all based on assessing the perimeter or external “attack surface” of an organization and combining that with scanning news on the internet, Darkweb, etc., and coming up with a risk-of-doing-business-with-that-vendor score. One major problem with this type of scoring is that the quality of the data used to calculate the score varies for every organization as a lot of it is based on multiple point-in-time questionnaires that are subjective and thus the risk score isn’t necessarily consistent or even applicable after a short period.
2. Making sense of threat intelligence:  Many cyber threat intelligence sources provide threat severity scores that rate the potential impact of each threat. While this is nice for comparing one threat to another, it isn’t very useful unless it can be applied to your infrastructure leaving you guessing as to whether an attacker has already bypassed your security controls.
3. Vulnerability risk: Vulnerability vendors, even when combined with threat intelligence, only on rare occasions can map known vulnerabilities active in the wild against your current assets. That is great for patching a potentially vulnerable asset, and is a critical layer of protection, but does not do anything for detection of a successful compromise that has circumvented these layers.

A False Sense of Security

So, all these risk scores are based on the “potential” for attackers to exploit known gaps in your security. They certainly make you feel like you are doing all you can to prevent a breach, but have they? Basically, the promise of these risk scores is to help make you prepare to be “less attractive” or even more difficult to attack by threat actors, but how are these risk scores even calculated?  Is the telemetry consistent from organization to organization or asset to asset? Does working with a vendor with a “high score” mean you won’t get compromised? Is one vendor’s score superior to another? And if so, why?

The reality is, it’s a leap of faith to trust in any of these scores without knowing how they were even formulated. Do we want that kind of uncertainty when dealing with sophisticated attackers and the well-documented impacts to our businesses?

Regardless of the risk score’s “accuracy,” why is there so much focus on “potentials” when most of our money is spent on detection and prevention? Even an indicator is just an indicator. I need to find droppings whenever they are dropped, determine the age or changes in a mouse hole, and other indicators, but what are the odds of me being up at 3am when the house is quiet to possibly catch a hungry mouse in action? I need to find a better way than simply relying on a combination of risk scoring and indicators. I need current and accurate evidence.  What if you had damning evidence that a rodent had made its way inside and could pinpoint where it was within 24 hours?

Bottomline

Personally, I’d rather have evidence of an active and current compromise and whether I have a mouse or worse, a RAT, where it is and how it comes and goes. Risk scoring can certainly help provide better awareness and get you to improve your defense, but we put too much emphasis on them providing true resistance against a breach. Threat Intelligence requires resources to filter through them and certainly XDR platforms can make some use of them to shrink dwell time, but that improvement still means attackers spend on average 60 days within your environment before detonating the primary objective of the campaign. The reality is that as we come up with more automation and smarter systems, attackers are relying even simpler techniques to circumvent them.  We need something different that shifts detection far to the left. We just need to KNOW, not keeping guessing.

To learn how compromise intelligence literally takes the guesswork out of the current state of your security through hard evidence of compromise, please visit Prevailion to learn more.