Artificial Intelligence Looms Large at Black Hat Europe

What’s artificial intelligence good for today, and what might be its next big cybersecurity use cases?

See Also: On Demand | The Dark Side of AI: Unmasking its Threats and Navigating the Shadows of Cybersecurity in the Digital Age

The topic of AI reality versus hype dominated the closing panel discussion Thursday at the annual Black Hat Europe conference in London. The panel featured conference founder Jeff Moss and members of the conference’s review board who helped evaluate the hundreds of submissions received by the conference this year, of which they approved about 50.

Review board members said that this year, AI-themed submissions of varying quality dominated. Some passed muster. Others included “AI talks written by AI, so we had to reject them,” said Vandana Verma, also member of the OWASP board of directors. The poor quality of AI-generated submissions made them easy to spot.

The accepted papers underpinned briefings tracks devoted to everything from vulnerability discovery and network security to privacy and incident response. The conference also included an “AI, ML and data science” track featuring eight sessions. These delved into a range of topics, including turning a generative AI agent against the application it was meant to serve; the difficulty of getting information excised from a training set; privacy concerns; tools for using LLM code assistants to aid developers without exposing sensitive information; and a major financial institution detailing its use of data analytics to streamline some business processes (see: Previewing Black Hat Europe 2024 in London: 20 Hot Sessions).

Cybersecurity: Powered by Buzzwords

Given interest in the topic, might it be time to launch a “Black Hat: AI” conference? Moss said he recently heard that question, and responded by noting that previous topics of overwhelming interest and focus, such as mobile and later the cloud, seemed to dominate for a little while before becoming “just kind of baked in.”

He predicts the same will happen with AI. “Everybody down the show floor has AI in the product, but they’ll sound weird in four years saying ‘now with AI,’ right?” he said.

Vendana said hot, stand-alone topics in recent years, including Zero Trust and supply-chain vulnerabilities, now feature in relatively few proposed briefings. For every new buzzword, vendors will claimed their “powered by it,” although what that means isn’t necessarily clear.

From a research standpoint, “the wide majority of the talks we selected were not about applying an LLM to something, because they will become a part of the tools we have, but they won’t replace all of the tools we have,” said Stefano Zanero, a cybersecurity professor at Politecnico di Milano.

Thematically speaking, “a lot of the talks that are about AI are more general talks, in the sense that if you’re attacking an LLM to do prompt injection, you’re just exploiting a product,” said panelist James Forshaw, a security researcher in Google’s Project Zero.

Killer Use Cases: What Next?

Since ChatGPT debuted for public use in November 2022, the use of large language models appears to have captured the public’s imagination. So far, the use case appears to focus on it as a prediction engine that functions like a “super autocomplete,” akin to Microsoft Clippy version 2.0, Moss said.

From a business standpoint, advances in AI are going to “make those predictions faster and faster, cheaper and cheaper,” he said. Accordingly, “if I was in the business of security, I would try to make all of my problems prediction problems,” so they could get solved by using predictions engines.

What exactly these prediction problems might be remains an open question, although Zanero other good use cases include analyzing code, and extracting information from unstructured text – for example, analyzing logs for cyber threat intelligence purposes.

“So it accelerates your investigation, but you still have to verify it,” Moss said.

“The verify part escapes most students,” Zanero said. “I say that from experience.”

One verification challenge is AI often functions like a very complex, black box API, and people have to adapt their prompt to get the proper output, he said. The problem: that approach only works well when you know what the right answer should be, and can thus validate what the machine learning model is doing.

“The real problematic areas in all machine learning – not just using large language models – is what happens if you do not know the answer, and you try to get the model to give you knowledge that you didn’t have before,” Zanero said. “That’s a deep area of research work.”

Another AI challenge can be the age of the training data. Moss offered a Python code-writing example, citing multiple cases of people using gen AI to quickly write working Python code, only to find that what’s been generated might be six or more years outdated compared to current practices, because it’s been trained on older data. So while the generated code might work well as a proof of concept, “it’s not modern,” and putting it anywhere that’s exposed to the internet might have security repercussions, he said.

For Now: Augmentation Tool

One audience member posed this question to the panel: Will AI replace cybersecurity jobs, such as the frontline analysts in security operations centers?

Zanero referenced an apocryphal meme attributed to Louis C.K., which says that “if you think that an immigrant that has no knowledge of the language, no connections, no degree, will steal your job, then maybe it’s you that suck.”

Expect to see AI being put to use isn’t in replacing jobs but augmenting them. “It’s the same thing we’ve seen with driving: self-driving is not working, it’s not going to work for the foreseeable future, but assistance for human driving up to the point where driving is safer and easier, that’s already there,” Zanero said. “The same thing is going to happen, at least for the foreseeable future, with AI.”