Interactively Audit System Security with AI(naisys.org)
naisys.org
Interactively Audit System Security with AI
https://naisys.org/articles/interactively-audit-system-security-with-ai
2 comments
Thanks for the demo video; very cool. How do you prevent prompt injection attacks?
E.g., if you create a user account for the LLM models to run in, how to do prevent an attack where the LLMs can be leveraged to execute privilege escalation to get as close to admin privs as possible? Or, how to ensure PII/PHI standards in a functionality such as this?
E.g., if you create a user account for the LLM models to run in, how to do prevent an attack where the LLMs can be leveraged to execute privilege escalation to get as close to admin privs as possible? Or, how to ensure PII/PHI standards in a functionality such as this?
The LLM only communicates with the shell so other users shouldn’t be able to leverage it. Maybe if the LLM reads a file that a lower privilege user has access to that’d be a route for injection into the context at least. There’s also a ‘command protection’ system we’re trying out that uses a separate secondary LLM to approve the commands of the primary.
It will run the agents in a wrapper around your actual shell that is context friendly. It even has a custom mail client and browser wrapper that is agent/context friendly.
I have a demo video where a Claude3 and GPT4 agent build a website from scratch together on the command line. https://www.youtube.com/watch?v=Ttya3ixjumo