Governance & Risk Management
,
IT Risk Management
,
Operational Technology (OT)
Individual Vulnerability Severity Not Always a Good Measure of Risk Exposure
A mainstay of IT security programs across the world, the Common Vulnerability Scoring System, may have terminal flaws when applied to the mirror universe of operational technology – a place where ordinary assumptions about risk don’t apply.
See Also: Frost Radar™ on Healthcare IoT Security in the United States
OT have long argued that CVSS is an inadequate measure for their purposes. In November 2023, the Forum of Incident Response and Security Teams, which maintains CVSS, sought to address those complaints with a new version, CVSS 4.0.
But a growing number of OT security experts believe it’s becoming clear that CVSS can’t be “fixed” – even putting aside issues such as the administrative burden required to implement CVSS 4.0 in the OT world.
What’s needed, a growing number of security experts argue, isn’t a better measure of the severity of specific vulnerabilities, but a different approach altogether. An approach that foregrounds the risk of cascading consequences, takes into account cross-sector dependencies and incorporates consequence management.
CVSS 4.0: an Imperfect Fix
Comparing the vulnerability score to a measurement of temperature, Patrick Miller, president and CEO of OT security consultancy Ampyx Cyber, told Information Security Media Group that previous versions of CVSS were like reading the temperature directly from a thermometer. “But we know that the temperature we experience, the ‘feels like’ temperature, is different based on a lot of factors like humidity and so on.”
CVSS 4.0 provided a way to incorporate more context into that initial “thermometer reading,” vulnerability score, he said, “so that measurement becomes much more valuable.”
Overall, the updated methodology “is a great step in the right direction and fills in a number of the gaps in previous versions,” said Kyle Miller, vice president for infrastructure cybersecurity at technology consultants Booz Allen Hamilton. He highlighted the addition of safety impacts as an element of the base score, and of additional environmental metrics like “subsequent system impacts” that can be added on to a base score.
But inputting that context to derive the “Environmental Score” add-on to CVSS 4.0 is proving resource intensive for OT owners and operators, said former U.S. Cybersecurity and Infrastructure Security Agency Senior Advisor Allan Friedman, now technologist-in-residence at the TPO Group, a defensive cyber consultancy.
“Very smart people thought it through,” he told ISMG of CVSS 4.0, “But by definition, if you’re trying to use my local context to help me make a decision [about what to do with a vulnerability], I’m the only one who can provide that data. That’s not something that can be folded in and sent out broadcast.
The problem, Friedman said, is that any measure that could capture enough different elements of environmental data to work for any kind of organization, “is going to be a little messy or maybe very messy.”
Organizations have – or should have – “a lot of information about their network,” he explained. But that data usually isn’t machine-readable or easily repurposed to provide contextual data for CVSS scoring. Collating and inputting that data “can be time intensive,” and may not be an affordable option for smaller or poorer organizations, Friedman concluded.
And it gets messier still, points out Booz Allen’s Miller. Many of the environmental or contextual factors introduced in version 4.0 really need to be measured separately for each asset with that vulnerability, he said. An enterprise might own dozens or even hundreds of assets impacted by a new vulnerability. “Each one may be located in a different part of the plant network with more or less compensating controls, and they may all control different downstream processes,” meaning their “subsequent system impact” would be different. And many of these various factors will only be known to – let alone properly understood by – individual plant managers, not the security teams responsible for inputting the additional data.
“This adds a lot of complexity for a large organization trying to manage thousands of vulnerabilities,” Miller said, which had slowed adoption of the updated standard. “While the context added in 4.0 is what makes it more powerful for OT, this is a manual process in most organizations which adds to its adoption challenges.”
FIRST, the cyber-responder non-profit that authored and maintains the CVSS standard, doesn’t track its adoption, said Neal Leali, volunteer co-chair of the CVSS Special Interest Group at FIRST and an incident manager with Cisco’s Product Security Incident Response Team.
“Given the relative newness of CVSS 4.0, the group hasn’t gotten much feedback specifically from OT systems owners,” he said, adding that the changes like incorporation of a safety metric and the environmental add-on had been “well-received” overall.
CVSS “was just one input to a vulnerability management system, and our guidance encourages use of other inputs as well,” he said. FIRST recently published guidance for users on how to implement the add-ons in CVSS 4.0. But while recommending augmentations and additions, the SIG still believes CVSS 4.0 is essential. “It’s the SIG’s hope that there are enough options in CVSS Environmental to allow OT asset owners to tailor vendor-provided assessments with their own asset information so that there is no need for alternative systems,” he said.
Correlations and Cross-Checks: Additional Dimensions of Vulnerability and Threats
Historically, CVSS vulnerability scores have been too noisy for OT defenders, said Sean Tufts, field CTO for OT security vendor Claroty. Even with the add-ons in version 4.0, Tufts said, it’s not uncommon for CVSS-based analysis of OT systems to produce lists of tens of thousands of vulnerabilities rated “high” or “critical.”
That’s in part because, in general, OT systems and their associated IT elements are never patched. Tufts, who used to work at “a large OEM supplier into the power industry,” explained how major industrial machinery such as power plant turbines would be sold with an anticipated three decade lifespan. “And along with that we would send them a computer, some random Dell or HP box with the latest version of Windows installed,” to use as the human machine interface.
Unlike the machinery they were controlling, the Windows boxes were never intended to last more than a few years. And they were designed to be protected by security programs and regularly updated to patch security vulnerabilities. But OEMs like the one Tufts worked for “put warranty information around those boxes that said [the asset owner/operator] cannot install antivirus or security patches because we didn’t know what that would do.”
The danger is not merely theoretical. In February 2017, NASA’s Office of the Inspector General published an audit of OT cybersecurity findings that a security patch applied to a desktop computer that monitored an industrial oven caused the monitoring and alarm systems to shut down, meaning that a fire in the oven went unnoticed until it damaged spacecraft parts.
“There’s a lot of legacy equipment out there that we just don’t know what would happen if we patched it. We don’t know if it could create a downstream cascade of outages. We don’t know what that full consequence tree would look like,” said Tufts.
Owners and operators can prioritize the endless lists of high and critical CVSS scores, explained Tufts, by cross-referencing vulnerabilities with:
- intelligence about what threat actors are currently targeting;
- databases like Exploit DB, which record whether a proof of concept (POC) exploit for the vulnerability has been published, or the CISA-curated Known Exploited Vulnerabilities catalog of vulnerabilities being currently exploited in the wild; and
- scores from the Exploit Prediction Scoring System – a machine-learning model that estimates the probability that a published vulnerability will be actively exploited in the wild within the next 30 days.
But even with that correlation work, and the hours of labor implied – unless it can be automated -vulnerability scores like CVSS generally can’t answer the question that asset owners and operators have, which is: What should I do, said Brian Proctor, the CEO and founder of Frenos, an OT security company.
At the end of the day, Ampyx Cyber’s Miller said, most operators were looking for a simple three-way categorization about exactly when they should take action with respect to a given vulnerability. “You can call it: Now, next, whenever,” he said.
A recent analysis by the ICS Advisory Project indicated that 98% of CVEs with a critical CVSS score “are never used in ICS incidents,” said Proctor. New data has also cast doubt on the effectiveness of EPSS, he added, since many vulnerabilities with high EPSS scores are likewise never exploited.
The reason? They’re not actually reachable by hackers. A lot of attack scenarios “while scary, are implausible in a specific environment,” Proctor said. But understanding “reachability” is tough, he adds, requiring knowledge of both the network pathways that need to be available to exploit a vulnerability as well as the pathways exposed in a particular organization. CVSS scoring models “do not have the data context of the exploit pathways or target environment to achieve these results, and organizations do not have the time or expertise to handle this manually,” Proctor concluded.
The Future for Vulnerability Scores: Automation or Alternatives?
Ampyx Cyber’s Miller said that automating these processes is essential, “without things like automated tools and machine exchangeable information, it’s much more difficult to do this on a system by system basis, across the entire fleet” of devices.
Some larger enterprises use artificial intelligence tools to scrape and reformat corporate data, to help automate and speed the process of incorporating context. “But that approach doesn’t work for everyone.”
One tool Miller highlighted is the Common Security Advisory Framework – a standardized, machine-readable format for security and vulnerability advisories. He called it “a very good attempt to standardize how we communicate these things so that they can be exchanged more accurately and more quickly at scale.”
In the years since its release in 2022, “There’s been some good traction,” Miller said, adding “the hard part is, we have to get the vendors on board.”
In addition to providing vulnerability disclosures in a standard and machine-readable format, vendors should prioritize offering alternatives to patching as a mitigation strategy, according to Friedman. “They need to be saying, ‘if you have this type of firewall, you’re going to need to use this rule, but that rule won’t work for this other type of firewall,'” he said, “and do all of that in a way that’s easy to digest,” like a machine readable standard.
Another tool which OT cyber defenders often find useful, Friedman said, is the Stakeholder-Specific Vulnerability Categorization – a decision-making framework designed to prioritize vulnerability remediation based on an specific contextual information about the organization and the threats its faces, rather than just technical severity.
“Essentially, it’s a series of questions that guide you through a decision tree that will ultimately get you to a place where you know, roughly: Do I need to do something immediately? Do I need to watch and monitor? Or can I ignore it?”
Originally developed at the Software Engineering Institute at Carnegie Mellon University, SSVC “has been strongly promoted by CISA as a general alternative or add-on to CVSS, and it’s what CISA uses to prioritize their own government announcements,” explained Friedman.
But increasingly, OT security specialists are coming around to the idea that supplementing or augmenting CVSS isn’t enough; a different approach is needed.
The hard fact is, said Proctor, the severity of a vulnerability, even if understood in the context of a particular organization or a specific deployment, is a poor way to measure what’s important for OT security. Vulnerabilities are just not as significant in OT as they are in IT, except at the edge. That’s because an attacker who understands the topography and functionality of OT in an organization can do terrible mischief just abusing the native capabilities of the system.
Vulnerabilities “are not necessary to achieve catastrophic results. If the industrial process does what you want just because you know how to ask it nicely, [vulnerabilities are] irrelevant” except as a way of gaining access, he said.
One alternative approach, Munish Walther-Puri told ISMG, is the cross-sector prioritization methodology developed by The Atlantic Council, a Washington, D.C.-based security think tank.
Essentially, the methodology downplays specific software vulnerabilities in favor of a “more holistic approach … prioritizing infrastructure criticality and policy ownership gaps in OT ecosystems,” said Walther-Puri, TPO Group’s head of critical digital infrastructure.
The aim is to move away from abstract measurements to consider real-world outcomes, based on which kinds of attacks were most likely to have the worst impacts, factoring in cross-sector dependencies and cascading consequences, he said. Power systems need precise timing from GPS, water systems need power, health systems need water – meaning a successful attack on GPS would impact not just the position navigation and timing services GPS directly provided, but all the services that were dependent on it.
Critical infrastructure organizations serve a particular locality, pointed out Danielle Jablanski, the methodology’s leading author. The exact same attack, with the exact same outcome on OT systems, can have wildly differing effects depending on whether it’s aimed at an organization serving 6,000 people or 6 million.
“We always say every emergency happens locally and so my emphasis was around local impact, which is why I chose to weight the severity of impacts on the ability of a scenario or incident to cause public panic and/or overwhelm local response capacity,” she said.
The idea was to provide a methodology that could be used “to rank scenarios and focus on those which you may have not prepared for cascading local impacts.”