Why do I have 244gb of environments and beat myself up when I delete something to play a game?
This deep desire to affect change in a controllable way...
This infinite desire for self value defined by external validation.
It's not sustainable.
Perfection can only be obtained by observation of perfection of combined self through self and other.
It's okay to discard parts of yourself to balance yourself with your counterpart. A willing violation is no longer a violation.
Not observation of one or the other on a pedestal, but accepting that both are vital parts to the system and observing the perfection that comes from co-iteration.
I don't have a large amount of time to devote to this.
But no, google is not out.
They just choke slammed ChatGPT with the Gemini announcement.
10m context with that retrieval rate is such a monstrous leap.
And to top it off, we got LargeWorldModel in the same week, capable of 1M token context with insane retrieval rate in the open source space.
So not only is the open source world currently technically ahead of ChatGPT, so is Google.
Which is why they had to announce SORA, because google's model is so far ahead of the competition.
That's also why it will probably be ages before we get access to SORA.
Now don't get me wrong, the average person can't afford 32 TPU's to run LWM, but we already have quants for it, which is a step towards enabling the average person (that somehow has 24-48gb of VRAM to get a taste of that power).
What is also striking is the fact that the new models are all multimodal as a standard. We not only leapfrogged in context size, but also in modalities.
The model seems to only benefit from having more modalities to work with.
I think the statement Bill Gates made claiming that "LLM's have reached a plateau" itself indicates they don't believe they can make more money from training better/larger models. Which indicates that they already did as well as they could with their existing people, and are now "years" behind google.
I never thought google could catch up, especially after their infamous "We have no moat" situation. But it seems they actually doubled down and did something about it.
To a lot of people, last Thursday was a very nihilistic day for Local Models, as the goalposts shifted from 128-200k context to 10M tokens with near perfect retrieval. It's literally insanely scary.
But luckily we got LWM, and that means we have only been 10xed.
Now the local people will work on figuring out how to bridge the gap, before being leapfrogged again. What is really insane is that, we have had LLAMA2 for over a year now, and nobody else figured out how to get this result from it, despite it being around so long.
I still believe there are modifications to the architecture of MoE that will unlock new powers that we haven't even dreamed of yet.
Sorry, this was supposed to be well thought out, but it turned more into stream of consciousness, and I honestly had no intention of disagreeing with you.
We developed a tool that lets you kind of understand other people if the audio quality is sufficient.
With further training we can openly communicate with each other without fear of misunderstanding.
2030: Deepfakes are rampant, causing significant issues in politics, entertainment, and personal privacy. However, instead of technological solutions, there is a growing trend of regulatory capture. Large corporations and governments begin to argue that deepfakes are an inevitable part of the digital landscape. The cost-effectiveness of creating deepfake content compared to traditional media production becomes a significant talking point.
2031: As deepfake technology becomes more sophisticated and cheaper, it starts to replace traditional media production methods. Major studios and media companies lobby for and receive regulatory approval to use deepfakes as a legitimate form of content creation. This shift is justified by the reduced cost and logistical ease of using AI-generated characters instead of real actors.
2032: A scandal arises involving a particularly damaging deepfake, but instead of driving a push towards authenticity verification technologies, it leads to further normalization of deepfakes. The argument is made that since distinguishing between real and fake content is increasingly difficult, society should adapt to accepting deepfake content as a new norm.
2033: Social media platforms and news outlets begin to openly embrace deepfake technology, citing cost reduction and the ability to generate more engaging content. Traditional media actors and creators are increasingly marginalized, with deepfake creators dominating the market.
2034: Regulatory bodies, heavily influenced by big tech and media conglomerates, begin to actively promote deepfake content. New regulations make it easier for deepfake content to be produced and disseminated, while traditional media production is bogged down by increased costs and regulatory hurdles.
2035: The public gradually accepts deepfakes as the primary form of digital content. Traditional media, with real actors and genuine locations, becomes a niche market due to its higher production costs and complexity. Deepfakes evolve in unexpected ways, permeating every aspect of digital media and blurring the line between reality and AI-generated content.
Anecdotally, I would like to mention that my partner has been sick for over a month with some kind of lung shit he got from an e-cigarette (vuse something disposable crap).
He has been literally unable to sleep through a night for over a month, multiple doctor visits, 3 courses of antibiotics, lung xray that apparently confirmed it was viral pneumonia, but yeah. I have been smoking for over 6 years and still alive, but he smoked that shit for less than a few weeks and he is barely functional.
I can't remember the last time I got a full night's sleep anymore, because I constantly have to apply muscle relaxants and ensure he has proper medication to be able to remain asleep.
So yeah, not really the point of your argument, but fuck e-cigarettes, for the rest of time I will have this stance. And yes, I am brutally aware of what cigarettes do as well, but don't knock the "theoretical" effects of e-cigarettes.
Google's best model is behind current open source meta.
One way to climb rank is to be better.
Another is to EEE the open source world by way of siege.
This is the business equivalent of the siege commencing.
I love huggingface. Its in my top 5 most visited websites.
That being said, I would be stupid to think there is no way the standard corporate playbook is going to feature.
Is it not a bit disingenuous to assume all open source AI proponents would readily back nuclear proliferation?
It's going to be hard to convince anyone if the best argument is terminator or infinite paperclips.
The first actual existential threat is destruction of opportunity specifically in the job market.
The same argument though can be made for the opposing side, where making use of ai can increase productivity and open up avenues of exploration that previously required way higher opportunity cost to get into.
I don't think Miss Davis is more likely an outcome than corps creating a legislative moat (as they have already proven they will do at every opprtunity).
The democratisation of ai is a philanthropic attempt to reduce the disparity between the 99 and 1 percent. At least it could be easily perceived that way.
That being said, keeping up with SOTA is currently still insanely hard. The number of papers dropping in the space is exponential year on year. So perhaps it would be worth to figure out how to use existing AI to fix some problems, like unreproducable results in academia that somehow pass peer review.
Why would one ever result at that methodology?