The Ugly Truth About Risk Scores and Threat Intelligence

Point out Risk Scores are useless
24 March 2021

Whether it’s vendors that provide an external grade/score of a company’s risk, threat intelligence platforms or vulnerability management vendors, everyone is looking for the ultimate “risk score”.

A Risk Score is like a home inspection report. It makes us feel like we are making good decisions in buying a new home by telling us what could go wrong next, but the inspection doesn’t really find everything that is wrong or prepare us for the dirty truths already present in the home. It could tell us that there is a mouse hole that needs to be covered up, but not whether a mouse is actually inside. While we have a lot of promises around machine learning and AI to piece together seemingly random indicators of compromise (IOC), the burden still falls on the analyst to make sense of a discovery of “mouse droppings”. That is for another blog, but the point is, does a risk score provide you with true insights into whether an infestation is already present?

Exposing Risk Scores and Threat Intelligence: A Game of Guessing

So, what’s the point of the risk score? It can vary depending on what your objectives are:

  1. Assessing 3rd-party risk, aka Third-Party Risk Management: In its simplified form this is all based on assessing the perimeter or external “attack surface” of an organization and combining that with scanning news on the internet, Darkweb, etc., and coming up with a risk-of-doing-business-with-that-vendor score. One major problem with this type of scoring is that the quality of the data used to calculate the score varies for every organization as a lot of it is based on multiple point-in-time questionnaires that are subjective and thus the risk score isn’t necessarily consistent or even applicable after a short period.
  2. Making sense of threat intelligence:  Many cyber threat intelligence sources provide threat severity scores that rate the potential impact of each threat. While this is nice for comparing one threat to another, it isn’t very useful unless it can be applied to your infrastructure leaving you guessing as to whether an attacker has already bypassed your security controls.
  3. Vulnerability risk: Vulnerability vendors, even when combined with threat intelligence, only on rare occasions can map known vulnerabilities active in the wild against your current assets. That is great for patching a potentially vulnerable asset, and is a critical layer of protection, but does not do anything for detection of a successful compromise that has circumvented these layers.

A False Sense of Security

So, all these risk scores are based on the “potential” for attackers to exploit known gaps in your security. They certainly make you feel like you are doing all you can to prevent a breach, but have they? Basically, the promise of these risk scores is to help make you prepare to be “less attractive” or even more difficult to attack by threat actors, but how are these risk scores even calculated?  Is the telemetry consistent from organization to organization or asset to asset? Does working with a vendor with a “high score” mean you won’t get compromised? Is one vendor’s score superior to another? And if so, why?

The reality is, it’s a leap of faith to trust in any of these scores without knowing how they were even formulated. Do we want that kind of uncertainty when dealing with sophisticated attackers and the well-documented impacts to our businesses?

Regardless of the risk score’s “accuracy,” why is there so much focus on “potentials” when most of our money is spent on detection and prevention? Even an indicator is just an indicator. I need to find droppings whenever they are dropped, determine the age or changes in a mouse hole, and other indicators, but what are the odds of me being up at 3am when the house is quiet to possibly catch a hungry mouse in action? I need to find a better way than simply relying on a combination of risk scoring and indicators. I need current and accurate evidence.  What if you had damning evidence that a rodent had made its way inside and could pinpoint where it was within 24 hours?

Bottomline

Personally, I’d rather have evidence of an active and current compromise and whether I have a mouse or worse, a RAT, where it is and how it comes and goes. Risk scoring can certainly help provide better awareness and get you to improve your defense, but we put too much emphasis on them providing true resistance against a breach. Threat Intelligence requires resources to filter through them and certainly XDR platforms can make some use of them to shrink dwell time, but that improvement still means attackers spend on average 60 days within your environment before detonating the primary objective of the campaign. The reality is that as we come up with more automation and smarter systems, attackers are relying even simpler techniques to circumvent them.  We need something different that shifts detection far to the left. We just need to KNOW, not keeping guessing.

To learn how compromise intelligence literally takes the guesswork out of the current state of your security through hard evidence of compromise, please visit Prevailion to learn more.

The Latest

Diving Deep into UNC1151’s Infrastructure: Ghostwriter and beyond

Introduction: Prevailion’s Adversarial Counterintelligence Team (PACT) is using advanced infrastructure hunting techniques and Prevailion’s unparalleled visibility into threat actor infrastructure creation to uncover previously unknown domains associated with UNC1151 and the “Ghostwriter” influence campaign.  UNC1151 is likely a state-backed threat actor [1] waging an ongoing and far-reaching influence campaign that has targeted numerous countries across […]

Prevailion CEO, Karim Hijazi- Biden’s Cybersecurity Strategy

Prevailion CEO, Karim Hijazi, comments on lacking White House cybersecurity efforts Karim Hijazi lays out why Biden’s cybersecurity strategy lacks innovation and effectiveness to deal with modern adversaries already inside companies around the globe.    

Prevailion CEO, Karim Hijazi- Tmobile Hack

Prevailion CEO, Karim Hijazi, talks about the T-Mobile hack and cloned SIM cards Karim Hijazi says T-Mobile’s breach is the largest in carrier history and discusses SIM swapping and other forms of identity theft.    

Copyright 2021 Prevailion, Inc. All rights reserved.    

Disclaimer: Gartner “Cool Vendors in Security Operations and Threat Intelligence,” Mitchell Schneider, Ruggero Contu, John Watts, Craig Lawson, October 13, 2020. GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and is used herein with permission. All rights reserved. Gartner Disclaimer: The GARTNER COOL VENDOR badge is a trademark and service mark of Gartner, Inc. and/or its affiliates and is used herein with permission. All rights reserved. Gartner does not endorse any vendor, product or service depicted in its research publications and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s Research & Advisory organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.