What happened here is not related to agentic behavior or instructions in .md files. It's a binary a user runs, it scoops up their files and sends them to a third-party.
And the user even paid $99/month or more for having their data leaked.
When I read about MCP the first time and saw that it requires a "tools/list" API reminded me of COM/DCOM/ActiveX from Microsoft, it had things like QueryInterface and IDispatch. And I'm sure that wasn't the first time someone came up with dynamic runtime discovery of APIs a server offers.
Interestingly, ActiveX was quite the security nightmare for very similar reasons actually, and we had to deal with infamous "DLL Hell". So, history repeats itself.
Might be better to say that most companies think they have that kind of isolation, but pentesting, red teaming and incidents then later proof they don't. I have even seen companies routing prod traffic to test systems, it's not uncommon.
This makes me wonder why does OpenAI not build in a mitigation by default that requires a confirmation that they control? Why leave it up to the tool developers to mitigate, many of whom never heard of confused deputy attacks?
Seems like a missed opportunity to make things a little more secure.
Are these all vulnerable to Indirect Prompt Injections or is there a solution to this rising security challenge? Anything plugin developers should do to limit impact?
There aren't any specific limited amount of tokens to inject or mitigate against, there is an "infinite" amount of trickery the AI might misinterpret or be persuaded to do.
Annual security training will be needed for AI, to learn about the latest phishing attacks, much like for humans. Only have joking.
I was a bit underwhelmed by the depth of the conversation - it entirely lacked an opposing or at least an alternate view to try to understand the other side. Going in I thought Andrew would moderate it like that, but it was more of a bubble discussion.
Very interesting thought on how to mitigate this, because I think a solution like with parameterized queries isnt possible - at least with my current understanding (the attack is more of a "social engineering" attack on the AI).
Regarding the supervisor AI, in theory it would be vulnerable to the same attack but probably more difficult to perform. One could even have multiple supervisors (with different sensitivity levels or focus areas) to get a vote on the content I guess.
Not necessarily stolen, people just don't know better. The article explains that companies/schools operate in bulk, meaning they just drop thousands of in bulk - so they dont care if the are locked or not.
Does Lastpass have a requirement on the length of the master password?
Documentation says they recommend 12 characters as minimum, which seems short. Wondering if its enforced or if it's possible to have "password123" as master password?
Also, sounds like URL leaked in clear text - so targeted attacks will be very likely. This is quite bad with long running implications.
Most will tell you that its about luck or perseverance, but reality it's about the network and people you know majority of the times.
If you ever worked at FAANG or graduated from Ivy League, or have connected parents it's actually difficult to not get money from VCs - especially the last 2-3 years. Assuming that you have a reasonable sounding idea. Network and connections is highly undervalued. Once you have money, long term success becomes more likely also.
And the user even paid $99/month or more for having their data leaked.