Business Continuity Management / Disaster Recovery
,
Governance & Risk Management
,
Healthcare
Some EHRs Affected; Hospitals Cancel Patient Care in Latest Vendor Mega Incident
As if the healthcare sector doesn’t have enough technology problems, the global IT outage caused by a faulty CrowdStrike software update has forced some hospitals and other medical facilities worldwide to cancel patients procedures and resort to manual downtime procedures.
See Also: Healthcare in The Cloud: Detecting and Overcoming Threats to Ensure Continuity & Compliance
The Health Information Sharing and Analysis Center said it is hearing about system outages and disruptions affecting operations across a wide range of activities, said the organization’s chief security officer, Errol Weiss (see: Banks and Airlines Disrupted as Mass Outage Hits Windows PCs).
The outage is affecting patient services, lab collections, secure file transfers, transcription services, shipments, manufacturing, phone systems, electronic medical records, pharmacy orders, Medicaid and insurance billing, 911 communications and more. “Medical devices that rely on Windows are impacted too. Medical procedures are being rescheduled. Hospitals are operating under emergency downtime procedures. Plus, there are reports of third-party and partner disruptions that are impacting operations too,” he said.
Among those affected are reportedly about 40 hospitals in Massachusetts, including some of Boston’s largest medical centers.
“Many hospitals are cancelling elective procedures today. Patients should direct any questions to their providers because this is a practice-by-practice, hospital-by-hospital decision,” said the Massachusetts Department of Public Health in a statement.
The DPH is working with hospitals to move to downtime procedures and continue to provide care. The DPH also stood up its Department Operations Center in Marlborough, Massachusetts, to take any calls from healthcare providers across the state, the department said.
Mass General Hospital – Boston’s largest medical center – is among those canceling procedures. “A major worldwide software outage has affected many of our systems today. This means we are not able to access our clinical systems, including patient health records and scheduling,” said the hospital on its website Friday.
“As a result, all non-urgent visits at all Mass General Brigham hospitals and clinics are cancelled for today … If you have a non-urgent visit scheduled, a care team member will reach out to you to reschedule. Please do not call to reschedule today.”
Mass General said it is still open for urgent appointments and procedures and that its emergency rooms remain open. “We are working to resolve this issue. We apologize for the inconvenience.”
Electronic health records software giant Epic was among healthcare sector supply chain companies affected by the outage, and the effects vary among its customers.
“Nebula, Epic’s platform for cloud-based applications and services, was impacted last night by the Microsoft Azure Central region outage,” an Epic spokesperson told Information Security Media Group.
Some features, such as Epic Video Client for telehealth visits, were not available during the outage, but Epic has restored access to these features and is monitoring Nebula, the spokesperson said.
“The CrowdStrike update has not affected our software or services directly but has caused technical issues that prevent healthcare organizations from using their systems,” Epic said.
“Some groups have reported that the laptop and desktop workstations their staff use to access Epic are down. Others report that issues with data center software are preventing them from using multiple systems including Epic,” the company spokesperson said.
“Organizations that have been affected are following pre-established downtime protocols to continue delivering patient care. Epic staff are working with customer IT teams to restore access as fixes or mitigation approaches are available from CrowdStrike.”
The CrowdStrike/Microsoft outage and its disruptions to hospitals and other medical care providers comes just as the healthcare ecosystem in the U.S. is trying to snap back into shape from the devastating February ransomware attack on UnitedHealth Group’s Change Healthcare IT services unit, which disrupted thousands of entities for many weeks. On top of that, many healthcare organizations of all sizes have had to struggle with their own disruptive attacks – such as the recent ransomware incident at hospital chain Ascension, which affected facilities across 19 states.
“Change Healthcare demonstrated that healthcare delivery organizations needed to have redundant pathways to ensure a secure and resilient healthcare delivery environment,” said Brad Marsh, a registered nurse and executive vice president of government health security and technology at security and privacy consulting firm First Health Advisory.
“Today’s incident involving Microsoft and CrowdStrike demonstrates the prolific use of a single solution that, if compromised in any way, jeopardizes the ecosystem,” he said.
“Third-party organizations deliver capabilities to multiple HDOs as a service. Those services and capabilities rely on their own tech stack. If our redundant systems rely on the same vulnerable/compromised systems, we are no better than with a single point of failure,” he said.
“While there is an infinite number of permutations in this multiverse, it’s something that increases risk overall. Just like with SolarWinds, Change Health and countless others: these incidents all come down to the shared, single point of failure outside of the organization itself.”
Disruption Levels Vary
The American Hospital Association is in close communication with the hospital field and the federal government about the nonmalicious global technology outage, said John Riggi, AHA national cybersecurity adviser.
“While we continue to monitor the situation closely, we are hearing from hospitals and health systems that the impact varies widely. Some have experienced little to no impact while others are dealing directly with some disruptions to medical technology, communications and third-party service providers,” he said.
The disruptions are resulting in some clinical procedure delays, diversions or cancellations. Impact is also being felt indirectly as a result of local emergency call centers being down, Riggi said. “Impacted hospitals are working hard to implement manual restoration of systems and the CrowdStrike patch. Affected hospitals have also implemented downtime procedures to ensure that disruptions to patient care are minimized or avoided to the extent possible.”
The AHA issued an advisory to its members with tips for dealing with the issues.
“If you have instances of CrowdStrike in your networks, determine the impact and review your business and clinical continuity procedures,” the AHA said.
Use this opportunity to identify impact and downtime procedures for all internal and third-party, life-critical and mission-critical technology, services and supply chain, the AHA recommended.
In advance of such situations, healthcare organizations should test cyber incident response and emergency preparedness plans and communication channels, the AHA said.
“Plan for technology disruptions and cyber incidents on a regional basis. Be alert to increased phishing emails that may appear related to this disruption.”
“We’re going to see organizations taking cyber resilience more seriously now. It’s not just the need for security, but also making sure critical business operations can still function when systems are down,” said Health ISAC’s Weiss. “Cyber resilience is about quickly identifying, responding and recovering from IT incidents to minimize business impact. Unfortunately, the issue today is an example of how badly things can go when patches and updates go unchecked.”