In December I reported a data exfil in OpenAI Agent Builder and it was also closed as Not Applicable, so it's probably still there.
It's also unclear if anyone from OpenAI even ever saw the report. I don't know.
Maybe the incentives are off on some bug bounty platforms or programs, and triagers are evaluated on how fast they respond, and how quickly a ticket is closed rather then what kind of quality tickets they help produce.
It's the only explanation I have for this kind of decisions.
Claude (generally, even non Cowork mode) is vulnerable to exfil via their APIs, and Anthropic's response was that you should click the stop button if exfiltration occurs.
This is a good example of the Normalization of Deviance in AI by the way.
See my Claude Pirate research from last October for details:
Good point. Few thoughts I would add from my perspective:
- The model is untrusted. Even if prompt injection is solved, we probably still would not be able to trust the model, because of possible backdoors or hallucinations. Anthropic recently showed that it takes only a few hundred documents to have trigger words trained into a model.
- Data Integrity. We also need to talk about data integrity and availability (full CIA triad, not not just confidentiality), e.g. private data being modified during inference. Which leads us to the third....
- Prompt injection which is aimed to have the AI produce output that makes humans take certain actions (not tool invocations)
Generally, I call the deviation from don't trust the model, the "Normalization of Deviance in AI" where seem to start trusting the model more and more over time - and I'm not sure if that is the right thing in the long term.
Lots of interesting new prompt injection exploits, from data exfil via DNS to remote code execution by having agents rewrite their own configuration settings.
I'm currently doing a Month of AI bugs series and there are already many lethal trifecta findings, and there will be more in the coming days - but also some full remote code execution ones in AI-powered IDEs.
https://embracethered.com
Twitter: @wunderwuzzi23