> “In the current generation, our density is 8 billion parameters on the hard wired part of the chip., plus the SRAM to allow us to do KV caches, adaptations like fine tuning, and etc. In our next generation, we would have the ability to go up to 20 billion parameters in a chip. Even with trillions of parameters, we’re talking about few tens of chips, which is a very, very small compared to anything else out there on the market today.”
Looking at their career page it looks like they do not care that much about PR at the moment... considering that they have a live chat with 14000t/s via Llama 3.1 8B[0] i don't really think they need to do PR either.
So i guess maybe they currently try to solve a very hard problem with a small focused group before scaling or they are dysfunctional.
Also Llama 3.1 8B is a dense model AFAIK and they are fast by nature.
As there are not a lot of dense models these days i could imagine that they try to optimise for MOE models.
> This Kimi K3 had only one prompt, because I was using the free tier and it ran out of free credits. So I was not able to go on a Journey with Kimi K3. I did with Fable, which is why some elements on Fable look good… for instance there is a partially complete pyramid in Fable that I added with additional prompting.
It was just an example.
You probably can imagine some kind of level you would consider definitly conscious.
I recomend focusing on designing Tests before rigid definitions of states of tests. - it is much clearer that way what is asked from something to be conscious.
But again these are all just indirect tests because we cannot test the core of consciousness because we don't know the core of consciousness.
For example: people in developing countries throw away more food then in developed countries although it is relative to their income much more value. The reason is because they often do not own fridges, use a lot of rice which spoils fast or have difficulties with the food supply chain in other forms.
I’ve seen definitions that I like they where in the direction of „if a system can recognize a problem it has not encountered before and can attempt to solve it with onto the problem adapted solutions, then it is conscious“
But then again this is just a external crude form of test that can lead to something like „light bulbs emit warmth, so fire must be a light bulb“
A specific pattern of self-referencing data could be seen (or not) as low-level consciousness in the future, when we know what consciousness exactly is.
It might be that stockfish is already something future scientists would define as "conscious".
Altough it is diffucult for me to Imagine that specific example.
> What information goes out of the system which is novel?
I think there is very little truly novel information. Most information including the information of "breakthrough idea after mis-hearing someone" is just a mix of previous information.
I guess you are already aware of that... just for completness sake.
meet.hn/city/de-Berlin
Socials: - github.com/i5heu - https://mastodon.social/@heidenstedt
Interests: Knowledge Management, AI/ML, Art, FOSS
---