Apologies for that. We had about 8 keys in rotation, but eventually ran out of phone numbers to create new OpenAI accounts + fresh accounts have super low rate limits for 2 days. We had a rate limit increase now, so this should be less of an issue.
Will release a new level soon as well :-)
PS: in case it wasn’t clear I’m on the Lakera team.
Gets trickier at the higher levels, but all of Gandalf's defenses are hand crafted at the moment. Can probably be made much more secure. Lots of interesting discussions happening here: https://news.ycombinator.com/item?id=35905876
-> Don’t give up at level 4, if you crack that, you have a good shot at making it to Level 7. But will you be one of the lucky few to beat Gandalf Level 7?
Thanks for your comment. The whole field of run-time monitoring is concerned with this problem. It's a tough one to crack when the distribution changes are subtle, but you can and should at least check simple data attributes for consistency.
That every programmer today can build and train an ML model is one of the biggest advancements of ML engineering in the past 10 years.
But as you say it's GIGO, the difficulty today is to know what to feed it and to know what that means for the real life performance. There are no great tools for that yet.
Great article, both the original and the modified version. What do the data scientists and ML developers think about this. Is ML today where software was 25 years ago?
Will release a new level soon as well :-)
PS: in case it wasn’t clear I’m on the Lakera team.