Artificial Intelligence & Machine Learning
,
Governance & Risk Management
,
Next-Generation Technologies & Secure Development
The Computing Giant Faced a Wave of Criticism Over ‘Photographic Memory’ Feature
Microsoft is retreating somewhat from Recall, a planned feature it touts as “photographic memory” for personal computers. The company announced on Friday that it’s shifting the default setting for Recall to “off,” and express user consent will be required before Recall can be activated.
The operating system giant’s unveiling of Recall in May quickly went sour for Microsoft as security and privacy specialists raised alarms and regulators announced inquiries into the technology.
See Also: OnDemand | Practical Strategies for Accelerating AI Adoption in Cybersecurity
Microsoft’s documentation warned that a feature designed to capture screenshots every five seconds and index them with artificial intelligence could sweep in “information such as passwords or financial account numbers.”
“It’s semantic search over all your history,” Microsoft CEO Satya Nadella said. “We can recreate moments from the past, essentially.”
Redmond isn’t giving up on Recall, but on Friday it said it will require express user consent to activate Recall and give corporate system administrators the ability to disable the feature. /p>
It also said it would tighten security by making enrollment into the Windows Hello secure logon system a prerequisite and demand “proof of presence” to decrypt and access Recall snapshots. It will also encrypt the Recall search index database.
One strident critic of Recall, cybersecurity research Kevin Beaumont, said on May 31 that Recall has security “gaps you can drive a plane through.” On Friday, Beaumont said he’s still skeptical. “I’ll believe it is encrypted when I see it,” he said on social media. “There are obviously serious governance and security failures at Microsoft around how this played out that need to be investigated, and suggests they are not serious about AI safety.”
Alexander Hagenah, a former technology executive with commercial surveillance firm FinFisher, released a demo tool he dubbed “TotalRecall,” built to extract the Recall database. He told Wired earlier this month that he improved the tool after reading Recall-inspired research from Google’s James Forshaw on how to bypass Windows access control list restrictions.
Hagenah on Friday told Information Security Media Group that Microsoft’s announced improvements to Recall look “great on paper.” If implemented flawlessly, he said, they would appear to prevent TotalRecall from working.
“My main concern,” he said, “is that they decided just in the past three to four days to implement everything and still aim for a June 18 release. That leaves them with about 11 days to thoroughly threat model, develop and test. It’s hard to imagine that this will result in a secure, high-quality outcome. I wouldn’t be too surprised if there are still some issues, and all security researchers should definitely take another look at it soon.” Hagenah currently heads up an offensive cybersecurity team at Swiss financial market infrastructure firm Six.
Microsoft is battling perceptions that it overlooks security and privacy concerns, a reputation that has waxed and waned over the decades but is again ascendant after high-profile breaches by Russian and Chinese state hackers. The company has vowed to overhaul its security practices (see: Microsoft Overhauls Security Practices After Major Breaches).
Nadella in an all-staff memo in early May told employees that when “faced with the trade-off between security and another priority, your answer is clear: Do security.”
Even if Microsoft cures Recall’s vulnerabilities, the feature reveals a disconnect between the company and its customers, said Beaumont.
“A lot of Windows users just want their PCs so they can play games, watch porn, and live their lives as human beings who make mistakes,” he said. “They don’t always want to remember, and the idea other people with access to the device could see a photographic memory is very scary to a great many people on a deeply personal level. Windows is a personal experience. This shatters that belief.”