Yeah, this letter is just proof of (the by now long standing fact) of corporate power/corporate capture of American governance. Tons of popular protests against big AI? Makes no difference. Joint corporate letter about big AI? Possibly the government will do something.
Probably because claude suggested some kind of wack react based setup early on (because react dominates the training data) and it's be blasphemy worthy of termination for the Anthropic employees to question the sacred pronouncements of the llm.
> to be more specific, i'd argue LLMs “understand” awfully similarly to a brilliant (polymath) with early dementia or Alzheimer'
At which point i'd argue that you're kind of refuting your own claim as most humans are not brilliant polymaths with early onset alzheimer's.
Even admitting these kinds of comparisons to edge case human mental experience, there are more differences than similarities, and the similarities are misleading and superficial. There are still a lot of differences, even when it comes to the physical structure of the brains neurons as compared to digital neural nets, and i think it's far more beneficial to try to understand these machine in their uniqueness and for what they are than to draw hasty and shallow comparisons. You could be forgiven for that, though, because computing is rife with people who love to draw hasty unjustified analogies for some reason (example: people were already likening the brain to a computer when we didn't even have working implementations of neural nets yet and a computer was literally just a small number of logic gates lol)
I'm really starting to tire of people making broad, general claims about how LLMs work or how to use them with N = 1 or 2.
An LLM is a statistics machine for goodness sake. Basically any general claim about them needs to exploit the law of large numbers to be even remotely sensible. You cannot extrapolate from one-off behavioral successes. LLMs are not understanding anything in the way humans do. If they did, yeah, maybe you could extrapolate hard from small samples, but they don't work or understand things like we do. You need to show that the behavior you are documenting is an average behavior the LLM converges toward in the long run.
I don't know enough about the quantum level to really understand whether or not that would change my view of things.
From Shannon's theory, it's abundantly clear that "information" is just a mathematical description of difference and expectation. I can frame anything as information. If I know who my parents are, a statement about who they are is not informative to me because it constitutes no difference. The entropy and expectation/surprise is low. If someone approaches me with a compelling argument that no, actually, my parents are not my parents I was adopted and my real parents are so and so, this is informative. The entropy is high.
Crucially, this all depends on me my personal history, and my status as an observer. It is in this sense that I find information to be a communications concept (which is how it started!) or semiotic concept more than a useful abstraction for organizing our knowledge about other parts of reality. Yes, the abstraction might still be useful if e.g. quanta end up behaving in some way according to shannon's laws, that is interesting, but to me uh at is not an argument that turns information into some kind of "universal substrate" of the real—it is just a surprising applicability of laws in one domain to another. Is it surprising that empirical data fit the model? Maybe. Idk, lots of empirical data can be made to fit all kinds of models (there are ways in which hI can reasonably take a Newtonian view of various aspects of my life and the data generally won't contravene the laws).
What a stupid list. Heavily biased. I'd just like to point out that a large bulk of these papers depend on the existence of the computer and yet Alan Turing is not even on the list.
You are missing the point. Let me use "fabulashinaism" which is a theory I just made up that posits that the universe is made up of little gnomes called atoms.
There is a sense in which all of my observations hence will be "theory laden". When I seek "proof" i'm likely to do so within the constraints of fabulashinaism.
Science proceeds in the same way. Only crises, which are unpredictable and may have much more than simple "new data collected" behind them actually drive theory change. Beyond the theoretical assumptions, many of which are not proven but are constraining hypotheses, there are the social value assumptions I mentioned as well. As someone else said in the thread, science cannot prove science. You can be of the opinion that it's been more effective than alternatives, and I would agree, but there is no way to prove this because it is ultimately a question of theoretical assumptions and of values.
Let me try again. We use miles to measure distance. Hopefully you agree with me that, miles, being a system of measurement to express relationships of distance, have no ontological existence themselves. I would find it completely bizarre, for example, for someone to argue passionately that, "no miles are the real deal and something that exists in the universe and meters are a fabrication". Distance surely is a real phenomenon, but "miles" are a system of expression.
Bits and information are the same thing. They are a system of measurement to express relationships of difference. Difference and change as physical phenomena exist, but bits do not.
So, yes, pointing out that the limits of computation give us some useful facts about the way things change in reality is a useful perspective. Going further and saying something like "computation is the basis of reality" or similar extremes is too reductive and too abstract to really be a useful piece of knowledge in my view. Finally, to claim that information is somehow the important fundament of existence is patently absurd to me, since it would've similar to arguing that miles have some very special epistemic and ontological status.
At a glance, this seems heavily recency-biased and not adjusted for inflation. I would expect a lot of other names that predate the 21st century to be on the list. The methodology page doesn't even contain the word 'inflation'.
Yeah, but my point is that "information" is not really a substantive concept. It is an abstraction. It is a useful technical tool. For the record, I am a nominalist when it comes to mathematics. Thinking that information is some "real thing" with meaning independent of theoretical context and interpreters is a definitely Platonisist perspective (the idea that our mathematical representations are not merely useful technical models but that they correspond to some real substantive elements of the real).
Claiming that "information" is the underlying basis of the real is the same as effectively saying that "stuff" is the underlying basis of existence. It's essentially Platonism. It is not informative. What is informative is the operational use of the mathematical concept of information as defined by Shannon to understand real things or to structure theories. But this is a very different thing than identifying a substance that exists, independently of us in the universe. This "making real" of what is ultimately a mathematical abstraction is precisely reification.
Sure, others would also claim the "financial view" of the universe has also provided insight. That doesn't mean I go around claiming the money is some fundamental universal substance of the real.
Computation as realized by computers is a formalism invented by Turing, Church, Kleene and others to make precise the intuitive notion of "algorithm".
An algorithm, at its root, is a procedure rooted in human understanding that human beings can follow.
When Turing and others first introduced this formal notion, not all mathematicians were even fully convinced it was an adequate representation of the informal intuitive notion. For example, some argued it was too broad because traditionally knowing that an algorithm eventually terminates was one of the requirements (to some) for something to be a legitimate algorithm. Why? Because the idea is rooted in human practical concerns and human understanding. Depending on what one cares about, one could actually reject the Turing formalization of algorithm on grounds of the class containing non-convergent (partial) functions.
All this is to say that "computation" is very much a human invention and little more than a formal model of human behaviors (we want to manipulate things algebraically). Elevating it to some objective substance of the universe is just doing 17th/18th century philosophical Idealism wrapped in new clothes.
There is no single "scientific method" to speak of. Your epistemic position is ultimately just as faith based as you claim faith in God etc. is, the sciences (plural) just have a set of operations which, as you point out, seem to have practical utility and seem to be better than certain alternatives for building up shared epistemic structure.
The attitude of "we can't do better" is precisely the kind of attitude that would have hampered the creation of various sciences and their methods in the first place, and the production of scientific thought is often not as pure or logically consistent as all the methodological purists or Popperians seem to believe it is. Feyerabend, Quine, Von Foerster, Dataon and Gallison and several others have made this pretty clear and make the argument pretty forcefully in my opinion.
All sciences carry an ontology and value system along with them. Religion is also a rich and highly effective epistemic structure when your underlying ontological assumptions, values, and goals differ from those established and implicit in the scientific (predominantly utilitarian) Western enlightenment value system. Science is not able to found itself on some irrefutable bedrock "truth" correspondence any more than religion is. There is no basis by which we can actually solidify empiricism as somehow more privileged and more capital T true if we admit the possibility of human error.
This. I'll never understand people who reify "information" as though it is some object with independent existence from man.
No information without an interpretation. The "amount" of information is completely dependent on the observer.
You'd think people who work constantly with abstraction wouldn't fall prey to reifying abstractions but they actually seem more susceptible to it than anyone else.
Mamdani has been doing great in NYC so far. The hope is that he will inspire similar behavior in others. And drive similar approaches to politics in other constituencies.
"Hammers are now a very popular tool, and one can move quickly building exclusively with hammers, so we have decided to construct buildings strictly using nails, no more screws, bolts, or any other kind of fastener shall be used going forward."
People seem to have adopted very liberal definitions of "personal attack".
When someone criticizes you for things you could improve on that are within your control, it may be difficult to hear, but it is not a personal attack. Someone calling you a stink manager, and listing factors as to why (unreasonable expectations etc.) is not a personal attack.
Someone talking about the inexperience you used to have (being a beginner developer) is not a personal attack.
Someone listing your business decisions (taking VC money, etc) is not a personal attack.
Maybe the only comment in this entire piece that is remotely close to an "attack" is the bit about slop code, but even that, being about the person's work and not the person and aspects of their existence they cannot control or their personal lives, is hardly personal.
I find it really baffling that HN seems to be so sensitive to what's, in my opinion, a relatively tame post with maybe some prickly flavor (which imo is completely understandable given the complete nonsense that had ensured with Bun post acquisition and their very public, and I think unfair, negative portrayal of Zig, intentional or not).
Maybe the only quoted text that I would read as a personal attack is
"already writing slop well before he had access to LLMs" and maybe "low empathy".
Just because the other bits are negative and not so nice pieces of feedback one wouldn't want to hear doesn't necessarily make them attacks.
An attack is often far more personal (making fun of you for aspects of your existence you can't control, touching personal subjects like families, etc). idk, maybe I'm just not soaked enough in the goo goo baby corporate saccharine LLM speak or something but this post felt tame to me, if a bit cheeky. Nothing but a thumb jab at worst.
The history of humanity is full of biting polemics and attacks on the character of others, just read Voltaire. Sometimes, attacks are deserved.
Yes, a world in which everyone is professional and "plays nice" sounds great but it also quickly becomes a mechanism for suppression of the truth. Not the situation here, but "decorum" expectations are sometimes precisely one of the functions that help bad people maintain power (consider how difficult it can be for minorities or marginalized people to speak truthfully about what's happened to them). Sometimes one person being blunt, honest, and a little rude is a necessary step in clearing ground for others to be more upfront about the bad behavior of individuals.