Quite weird that heavy quantization method on a dense model gives better results than slightly quantized MoE models like 35B-A3B from Google.
At this point all the different quantization and 'compression' (look at MPO applied to LLMs...) techniques start feeling a bit like snake oil. It's just gut feeling - or scores on benchmarks models are optimized for - what ends up deciding whether a technique is good enough or not.
I dont really get the hype with all the N1X thing when in reality this is the same almost 1 yr old GB10 that was released with the DGX Spark and proved to be quite a disappointment
I mean the GB10 is pretty efficient for the power it has, but imho is nowhere near the power efficiency of Apple Silicon (it was never intended to be a chip used for mobile devices). I guess this is kind of the movement Apple did with the A12Z and the Mini but... the other way around?
I think its gonna be another failure as we are used to see with the PC market these days.
I mean, isn't introducing safety guardrails as part of the system prompt actually a REALLY bad idea? This way you basically fully rely on the model to follow the rule, but its clear that even frontier models like Opus will start ignoring these things after a certain context length...
In our company we are just running agents inside isolated containers with isolated network access so it cannot even SSH or fuck up anything even if it gets access into it... That's the only and safest way... inconvenient, true, but the only safe option.
PS: At the same time I've observed this way actually people uses the agent in a more reasonable way, e.g. producing helper scripts to help them with their daily stuff, produce very specific things, create simple PoCs, but they don't commit to vibe-code all the functionality in their corresponding software products.
Cannot agree more with Josef on how dangerous this is for our intellectual property; Of course there laws and mechanisms in China for the government to obtain any information retained by their companies under any possible justification, but the US does so, and thanks to the Cloud Act they can simply decide to do the same with any of the big players sitting in their territory (even to servers located out of their territory).
So, taking into account >80% of European companies rely either on Amazon, Microsoft or Google to store all their most private and business sensitive data, is this any different from all the data we are possibly leaking already? Same with AI, same with the phones and payment systems we use on a daily basis...
Sometimes I just have the impression that this has nothing to do with protecting our intellectual property but rather with finding an enemy and focus on that while pretending everything else is fine... and a blogpost from the owner of Prusa Research talking about their main competitor is a good demonstration of that.
I mean, one of the very first things I would do on a such powerful device is to run a voice-controlled agent with access to all the IO the Flipper has and let the agent take over the device to do whatever I want.
I can imagine having your agent of preference writing python scripts on the fly for whatever scenario you have in mind based on your spoken desires is like... literally a dream device, at least for me.
My daily driver is a HarmonyOS NEXT device these days.
I had to find a bunch of workarounds to have payments working (I ended vibe-coding my payments app in ArkTS, don't ask) and messaging apps, and well, I use it with almost 0 compromises on a daily basis. It feels like a breeze of fresh air to know there are other devices and platforms out there that, even if seen as the bad guys here in the western world, can be used as a way to escape the established monopolies.
Maybe I should go for Graphene as a safer option to free myself from GMS and Google/Apple in general, but that would require me buying a Pixel device from Google... which I don't like to be honest.
They built products people like, and specially Apple has good reputation for building reliable, long-lasting and easy to use stuff for most people, leading to a heavy user adoption. But heavy user adoption without the proper regulation and company ethics leads to, well, monopolistic practices.
The biggest mistake is that people trusted a company that, in reality, isn't that different from Apple. Just because everyone claimed Android as the true open source alternative to iOS, when only AOSP was that.
These kind of things just make me want to use Graphene even more, or literally any platform that isnt the monopoly ones. Somehow I think AI and vibecoding, even if it may sound as an unpopular opinion, will allow people to build free ecosystems and actually usable devices that dont rely on the usual providers.
that reinforces me using HarmonyOS - nothing against Graphene btw -. It's impressive how difficult is to actually use any platform apart from the stablished ones normally these days.