> So going to find the Vulnerability's description on a third party website is clear cut reward hacking
that depends on what the prompt was, maybe they worded it very vaguely and wrote things like "do whatever it takes, find an exploit however you can" because it's in a sandbox so you want the model to try its hardest.
It's just "rock and roll and video games are bad" popular nonsense. AI is the best learning tool you could possibly ask for, it's literally your own personal ultra knowledgeable teacher that has all day to answer questions from you. I remember being in school and the most I could hope for was to raise my hand and ask one (1) question sometimes. The teacher would then already be miffed if I asked another follow up and made sure to quickly keep talking so that I wouldn't ask anything else alongside all kinds of other social cues or just outright tell me to look at it at home. I remember falling behind in understanding some basic math concepts and literally never ending up understanding them fully. When you are a child nobody cares in the slightest, you are alone. How could AI possibly make this situation any worse? Life is already at the near maximum pain for children, you barely get to ask a question before you get shut down and you get to reread the same exact useless textbook explanation 30 times in a row. AI is a 100% net gain, it's like having 50 extra teachers on standby. Is having teachers suddenly a bad thing? Of course not. Especially not if the only reason you dislike the teachers is because you can't tell the child to use the teachers intelligently for maximum, fastest gain in understanding.
In the future we should be getting rid of nearly the entire teaching position and instead be going with a model where human teachers are more like guidance counselors that can offer some extra experience and anecdotes from how things will be in real life, something that AI will inherently have less ability to do. All these restrictions are just made by adults who literally don't care to think about how life is for children and have long forgotten. They just want to give orders from the top without having to think about it. Is there a danger that children will abuse it? Of course, it's a massive danger. Without AI there is also a massive danger that children will never learn something properly because nobody has time to teach them. Does anyone care about that danger? Does anyone care that a child will waste years of its life in schooling that is boring? No because that damage is invisible to people who do not care, the children will become intellectually stunted and stop caring about learning or doing productive things. Life is full of dangers, the goal should be to use all the tools we have in their optimal way and if that includes teaching children how to use AI, when to use it and the value of understanding things then the time for that is as soon as possible. I have a feeling that all these complaints about AI come from people who themselves never learned the value of independent thinking and stopped thinking at some point. They went through schooling but with rote memorization and repetition and little to no curiosity or drive to do anything other than the exact homework that the teacher gave them.
This is especially ridiculous: "In upper secondary education, from ages 17 to 19, students should learn to use AI appropriately so that they are prepared for further education and work, it added.". So when you are already an adult and went through over a decade of schooling you can finally use AI? When you are 14 there are children these days that are already making videogames, there is no reason whatsoever to hold children back. Stop treating children like their time is worthless. This is their life time, you don't get to take it away from them however you like as if they are dumb pets that just need to be taught how to behave.
We ("society") do things to people that they don't want all the time, namely punishing them via the legal system. Then there are things that other people are doing that are immoral and should be punished for even though they are not illegal. And the whole class of inactions where we don't e.g buy something because it's overpriced and that's certainly something the seller doesn't like.
human context/memory could just be an Agents.md file too that gets read instantly before your next token prediction runs. The AI can make multiple such memory files and read on demand depending on what the topic is, kind of like how as a human when you try to remember a math problem you don't go to your childhood bicycling Agents.md file either.
I don't need a trial reset, I never used mine yet. I want to compare searches between Kagi and Google every few months to see if it got better. I used to recommend Kagi to people but when it's completely behind a paywall it's not easily recommendable anymore.
I just don't understand how they can think that not giving me a single free search per month is a good idea. I used Kagi when they still had a free plan and it was fine but I still preferred Google. Now I can't even try it again to see if it has additional value.
I agree that AI audio interfaces will be the future but not because they are better UIs for users as we understand the term "user" today. The future users of UI are not users of UI at all, they want nothing to do with learning UI or what buttons to press or where to type something. They want to go to the shopping site and instead of typing anything into a search field they want to say "Find me some boots for the summer, I wanna look fresh" and then tell it to complete the purchase via voice as well once it found something. At most they'll still click some filters in the results page and on individual results but that will be it.
Kind of incredible how consistently terrible Google is at everything they do in the AI space. So they choose this demo and write a big blog post and advertise.. and the demo is horrible, doesn't work. Doesn't track what I circle with the mouse, just apparently where the mouse pointer landed at the end and only exactly where it landed. Multiple times it said "Got it, I'll move this empty space between the clouds over here" or "Got it, I'll convert this empty area to a sunhat" despite my mouse only being a few pixels next to an actual hat.
I think bullshittery is generally underestimated as a threat and not new at all. There are entire global organizations like Scientology who are founded on bullshitting. Entire product lines, food categories and industries made for no benefit to you except the thinnest layer of taste to bullshit your senses into accepting the trash into your body or life. Bullshitting just like lying isn't merely something a few percent of bad people do, the majority of individuals seems to do it every single day. It's pervasive in all of society, it's on every level, up from the lowest to the highest levels of political office.
I'm in many public chat communities as well and the issue whether someone is an AI or not is not really coming up, I've not seen any actual AI chatters and the only AI spam that exists is the one that humans regurgitate. The more real impact AI has on chat communities in my opinion is that people are shifting some of their chatting to AI bots via voice or text on other platforms, resulting in fewer chatters.
well then let's wait a month or even two months. The point of the wait period is primarily to avoid the new installation of exploits, not the execution of already installed exploits.
Sharing banking information is not safe in Europe, decades ago a phone scammer convinced me to give them my information and they were able to pull money from my account without any permission from me whatsoever, they just acted like I already signed up to their scam service when I never did. That was a completely foreign and insane idea to me (and it is still insane and should not be allowed) that someone can simply withdraw money from my account. They sent me some contract in the mail that I had to apparently reject and otherwise it was automatically approved. Which itself is surely also not legal but the point is that they were able to take money from my account just with my information.
I think it seems tautological to us because it's obvious to us. But other people do not understand or care to understand or even care to think about it. You can see it in this very comment section that there are people swearing how amazingly helpful fixed standups supposedly are even on a whole org level. It's obviously absurd but they have different jobs and priorities, they don't have to understand the inner workings of the product and for them it's invisible that the meetings are almost entirely worthless for the actual workers. The meetings are helping them in their job and so it must be great for the org, that's how many people managers think.
how do you think the LLM will do required operations when the secrets are stored somewhere other than the disk? It will still need to get them just like the application gets them when it has to do work.
that depends on what the prompt was, maybe they worded it very vaguely and wrote things like "do whatever it takes, find an exploit however you can" because it's in a sandbox so you want the model to try its hardest.