Scorecarding in the Age of Threat

There’s no lack of cybersecurity scorecards, but before choosing one, consider the following.

Cybersecurity scorecarding has been an aspect of the security scene for five years as IT admins looked for a simple and fast answer to a critical question: how safe is my network? The answer was to monitor critical aspects that would serve as key indicators on this health check.

Today, there’s no shortage of third party vendors offering cybersecurity scorecarding as a service with the end game to declare both the network safe but as well evaluate the threats coming from the web and other threat surfaces. The general premise of these companies is that they’ll undertake a wide (and widening) range of passive and active probing techniques to map out a target organizations’ online assets, crawl associated sites and hidden crevasses to look for leaks and unintended disclosures, evaluate current security settings against recommended best practices, and even dig up social media dirt that could be useful to an attacker; all as contributors to a dynamic report and ultimate “scorecard” that is effectively sold to interested buyers or service subscribers.  

This is all well and good but when considering a cybersecurity scorecarding service or function, consider the following.  

Any One Can Start Up a Scorecard… But That Doesn’t Mean They Have Security Expertise

It sounds trite to say but almost anyone can get into the scorecarding business… they claim “proprietary” techniques and “specialist” data sources and voila, a service is born. If for some reason third-party scorecarding becomes popular and financially lucrative, there’ll be a move by popular managed security services providers (MSSP) or automated vulnerability (VA) assessment providers to launch their competitive service with as little as a month’s notice and only a couple of engineers. This is surely the wrong reason to create a cyberscorecard feature and service. The intent should be to help customers have quicker insights to improve their security and protect their business continuity.  

At some point in the future, if there ever were to be standardization of scorecarding scores and evaluation criteria, that’s when the large MSSP’s and VA’s would likely add such a service. The problem for the all the new start-ups and longer-toothed start-ups is that these MSSP’s and VA’s would have no need to acquire the technology or clientele. And critically, they’d lack the security experience that would give them the credibility to back their solution up as something to be trusted. No, the far better solution will be a cybersecurity vendor to take this role as they have the experience, knowledge and customer base for their solution to have the integrity and credibility to offer real actionable intelligence.  

Where’re standards?

Right now, there is no industry standard that we can hold as the benchmark by which all scorecarding should be graded against. Every vendor is vying to make their scoring mechanism the future of the security scorecard business. As vendors add new data sources or encounter new third-party services and configurations that could influence a score, they’re effectively making things up as they go along. This isn’t necessarily a bad thing and ideally the scoring will stabilize over time at a per vendor level, but we’re still a long way away from having an international standard agreed to. Bear in mind, despite two decades of organizations such as OWASP, ISSA, SANS, etc., the industry doesn’t yet have an agreed mechanism of scoring the overall security of a single web application, let alone the combined Internet presence of a global online business. 

The Cloud Becomes a Battleground

As companies and institutions have moved to the public cloud and enabled the bulk of the default security features that are freely available to them and are using the automated security alerting and management tools provided, are already fairly secure—much more so that their previous on-premise DIY efforts. As more organizations move to the public cloud, they all begin to have the same security features, so why would a third-party scorecard be necessary? We’re rapidly approaching a stage where just having an IP address in a major public cloud puts your organization ahead of the pack from a security perspective. Moreover, that the default security of public cloud providers will continue to advance in ways that are not easily externally discernable (e.g. impossible travel protection against credential misuse)—and these kinds of ML/AI-led protection technologies may be more successful than the traditional network-based defense-in-depth strategies the industry has pursued for the last twenty-five years.  

And that cloud doesn’t represent the reality of the on-prem network that needs to be scorecarded and graded; for that the vendor solutions—especially when they’re working within the eco system of the security offering—offer a valid evaluation of the real security situation. Cloud tools are a good off the shelf solution for some orgs, but certainly don’t reflect a full scope of metrics to be scorecarded (consider endpoints, authentication and other aspects to remind of this challenge).  

How to Score the Score?

Not only is there no standard for scoring an organization’s security, it’s not clear what you’re supposed to do with the scores that are provided. This isn’t a problem unique to the scorecard industry—we’ve observed the phenomenon for CVSS scoring for 10+ years. A critical aspect to have for any scorecarding to be credible is be to advise what should be done to mitigate the threat and improve the score. 

At what threshold should I be worried? Is a 7.3 acceptable, while a 7.6 means I must patch immediately? An organization with a score of 55 represents how much more of a risk to my business versus a vendor that scores 61?  Again, defining on the parameters of what the threat represents is key and should be a visible aspect of a scorecard. And It should it be as easy as an A-F score, one with color coding to alert the viewer of the severity of the situation.

Historical References and Breaches

The question of what happened in the past and how it relates to present and future threats that needs to be considered. If a business got hacked three years ago and the responsibly disclosed and managed their response—complete with reevaluating and improving their security, does another organization with the same current security configuration have a better score for not having disclosed a past breach?

Organizations get hacked all the time—it’s why modern security now works on the premise of “assume breach”. The remotely visible and attestable security of an organization provides no real insights in to whether they are currently hacked or have been recently breached.

In Defense Of Scoring

Defending the integrity and righteousness of an independent scoring mechanism is difficult and expensive. Practically all the scorecard providers out there explain their efficacy of operation as if it were a credit bureau’s Credit Score—as if that explains the ambiguities of how they score. Credit bureaus use in their credit rating systems and that’s valid in that industry, but you can be pretty sure they’re not port scanning websites, scraping IP blacklists, and enumerating service banners—and that the people being scored have as much control to modify the data that the scoring system relies upon. 

The key point here though lies with the repercussions of getting the score wrong or providing a score that adversely affects an organization to conduct business online—regardless of the score’s validity. The affected business will question and request the score provider to “fix their mistake” and to seek compensation for the damage incurred. In many ways it doesn’t matter whether the scorecard provider is right or wrong—costs are incurred defending each case (in energy expended, financial resources, lost time, and lost reputation). For cases that eventually make it to court, the “look at the financial credit bureau’s” defense will fall a little flat. 

So Is Cybersecurity Scorecarding Right For You?

IT managers strongly want a scoring mechanism to help distinguish good from bad, and to help prioritize security responses at all levels. And importantly, they want to be able to see that in an easy to overview dashboard and one that can shared to their managers higher up for actions to take and budgets to sign off to be truly secure in the knowledge that they are safe.  

Clavister’s CyberSecurity Scorecard—built by a European cybersecurity vendor with a security first mindset—takes all these elements into play and creates a holistic scoring for its parameters. They include six categories: Protection, Health, Behavior, Users, Connection and Devices. The intelligence is collected from the security infrastructure in dimensions including configuration data, actively polling status, external reference data, aggregated statistics and single events. Data is analyzed and scored daily to clearly show what level protection the current situation is giving. Using a color coded and A-F approach, they also give recommendations for how to improve your score. Compared to the the warnings stated in the first part of this blog, they mitigate many of the problems that the typical problems that plague scorecarding vendors. 

Providing a score isn’t a problem in the security world, the problem lies in knowing how to respond to the score you’ve been presented with!

Find out more about Clavister’s CyberSecurity ScoreCard and how it can give you an overview and action plan for your security: