The 39C3 had a few interesting talks on (agentic) AI which I would like to highlight from security perspective.
A more political and a bit less technical view was provided in the talk “AI Agent, AI Spy”. The talks provides a great overview on the (security) risk that comes with the usage of AI - and especially when AI is introduced in the building blocks of our infrastructure like the operating system. While in my opinion agentic AI will not go away and will provide benefits, as an industry we need to be less reckless in deploying those agents. In case you provide a building block or platform this is also extremely important, because your success is very likely not only based on the feature set you deliver but is equally based on the trust the customers gives you. This must not be gambled with.
The talk “Agentic ProbLLMs: Exploiting AI Computer-Use and Coding Agents” provides insights to several CVEs and issues around Gen AI and LLMs. The sad key take away (in addition to the concrete examples and details provided): Prompt Injection will not go away. It is not fixable by some discipline and good frameworks like the similarity to SQL Injection implies. Prompt Injection is more like Social Engineering, which is also not fixable in a way that it does go away.
You can find all recorded talks (which are mostly awesome in my opinion) of the congress here: https://media.ccc.de/c/39c3
