It’s technically possible to prompt inject like this. I actually reported this to OpenAI back in April 2023 but it was auto-closed. (I mean, I guess it’s not a true vulnerability but kinda funny it was closed within 5 mins)
Takes a random image from Unsplash and attempts to caption it, posts to social media if it's under 85% confident of the result - which leads to some interesting outcomes!
Really easy to work around, I used it to build a simple CLI to generate device screenshots of a webpage by modifying the user-agent and resolution to match each device.
It’s technically possible to prompt inject like this. I actually reported this to OpenAI back in April 2023 but it was auto-closed. (I mean, I guess it’s not a true vulnerability but kinda funny it was closed within 5 mins)