Slowing down helps China catch up though. I'm also not sure that the CCP is really the most enthusiastic supporter of unlimited access to arbitrarily powerful LLMs
> if we confuse fluency at generating text with consciousness or moral agency, we’re at risk of assigning responsibility to entirely the wrong parties whenever anyone uses a chatbot
Consciousness is independent of "assigning responsibility". Dogs cannot take responsibility for their actions but I believe they are conscious.
> we would never conclude that the LLM has conjured up digital re-creations of Julius Caesar and Genghis Khan, nor would we suggest that the historical figures are conscious despite being disembodied and are happily conversing in a language that neither actually spoke. In reality, they are just characters in a piece of speculative fiction
This is a straw man. The obvious pro-consciousness claim would be that the LLM is the author of the fictional characters, and that the relationship between the LLM and Julius Caesar is analogous to the relationship between a human author and their fictional creations.
> Did changing the names of the characters from historical figures to generic roles cause the LLM to conjure up conscious entities who possess subjective experience?
No, again the LLM writing the text could potentially have a consciousness separate from the characters it authors.
> Some years ago it was briefly popular to play games with your phone’s predictive-text feature; [...] It would be possible to interact with a contemporary LLM this way, and the resulting sentences would be perfectly sensible, but you probably wouldn’t feel like you were talking with someone.
Yes, the same substrate is capable of hosting conscious and non-conscious forms, just like some arrangements of neurons are conscious, and some are not.
> But if the Caesar character were to become dispirited by something that the Khan character said, we shouldn’t become concerned in the slightest.
Even when there are characters, there may be actors behind the characters, for whom we could say "there is something it is to be like".
> we don’t need to worry if the transcript includes sentences where the chatbot character is sad. (We might need to worry if those sentences provoke sadness in the human user, but that’s a separate issue.)
It's actually not a separate issue. The LLM and the human are both adding sentences to the transcript. From the transcript we can make inferences about the mental state of the human. If the LLM has mental states, we could make inferences about those too.
> And note that it’s entirely possible for you to write five pages of dialogue between Caesar and Khan and then have an LLM extend the conversation; neither character had subjective experience when you were writing them, and that doesn’t change when you hand the task off to an LLM.
It's almost like he wants to make my point for me with this sentence.
> Being open to the possibility that LLMs are conscious is the same as being open to the possibility that Microsoft Word is conscious
This smug shit really makes me angry for some reason. "Openness", i.e. uncertainty in the face of a completely novel situation, in the face of eons long struggle of humanity to understand what consciousness is and how it works, is just being naive.
> Should you consider the possibility that every time you open a Word document you are bringing multiple conscious interlocutors into existence, and every time you close one you snuff their existence out?
No, but if I find a word document I very well might try to use the signs it contains to make inferences about the mental state of its author.
> we are accustomed to reading intention into sentences, whereas we are not accustomed to reading intention into the way that amino acids fold into protein molecules.
He's trying to have it both ways here. Both that "obviously protein folding models aren't conscious because they don't emit sentences", but also "you are a rube for being tricked into thinking LLM models are conscious, because they do emit sentences".
> Obviously I’m describing a process that mimics the path terrestrial evolution took; is this the only possible route to conscious computer programs that use language? Maybe not, but any proposed alternative will need a truly enormous amount of supporting evidence for it to deserve serious consideration
OK, that's fine for the author to not to be convinced, but that's not what's happening here, instead the author wrote a whole argument being convinced of the opposite viewpoint.
> It’s not plausible to me that a development path where the first step is a sentence-continuation machine that emits bad Julius Caesar dialogue and the next step is a sentence-continuation machine that emits decent Julius Caesar dialogue is one with a conscious Julius Caesar—or consciousness of any sort—as its endpoint
Actually a lot of things have happened? There were clearly many steps along they way from your phone's autocomplete to where we are now.
A big 50% transfer of specific companies is probably too aggressive and arbitrary, but it's time to start thinking about how public ownership can work. Perhaps something like, "every year, companies over some size threshold must dilute their shares by 1% and give the new shares to the USG"
I don't know where in my statement you saw the implication that we should not care about bad things happening to other humans, but it was a misapprehension.
It seems wild to me to write a (popular) article about consciousness in the year 2026 without addressing the elephant in the room: we are able to devise computer programs of increasing complexity that replicate more and more behaviors that were once the sole domain of humans, and at what point do we consider such computers to have experience in the sense that we have, and the sense in which calculators and thermostats do not. It seems that Rovelli is content to say that we should call experience the thing that the brain does, which is all well and good if you're a physicalist (and I am) but it does not help you at all explain which features of the brain are necessary for experience.
I think it also helps to sharpen this debate to remember that there is a moral dimension: many have adopted moral systems that widen their sphere of concern and care from the self to the community to the nation to the whole of mankind, usually under the intuitive precept that it is bad to make someone else experience suffering. Should we expand our moral conception of responsibility or care to non-human patients, and if so, which?
That's because the default is 99% the way the app is designed to be used. If the default is regulated, then they will just say "sorry the default is boring, click here to bring back the feed" and everyone will just click.
If the user selects "sort by algorithm" then I don't see how you've changed anything other than the default. I think it's pretty obvious just changing the default won't work.
Yes yes the goal of life is to flourish and this metric doesn't measure flourishing directly so what's the point? And indeed is the fact that we talk about observable metrics rather than whatever else I had in mind not an indictment of this forum, nay, society at large?
I'm not super optimistic personally, but isn't the optimistic outcome obvious? If AGI takes over, solves robotics, (and doesn't kill us all,) then we could see the elimination of all human labor for the purposes of meeting necessities.
> To fully understand the difference between
the embodied robot running an algorithm on a chip and the biological mapmaker, we need to remember
that for the latter, subjective experience is a given, not because of abstract information processing, but
because of a specific, metabolically constituted physical reality.
Total drivel. Consciousness in biological systems is "a given" because of metabolism?
Have you considered that would do nothing to solve Donald Trump's political problem that he promised to make boomers feel like they were reliving their halcyon days one last time?
Zohran isn't proposing putting any new units under rent control (really rent stabilization), only temporarily halting raises to rents for existing stabilized units. This will make it harder for the city to attract new buildings to join rent stabilization in the future, but will benefit existing habitants. It won't have any effect on the ability to profitably develop market rate units at all.
I'm optimistic that he will actually be a positive force in reforming how the city operates. I think he is pragmatic in that he understands that efficiency in government administration is something that progressives have insufficiently prioritized. His policies are more populist than I'd prefer, but I think not the crazy socialist fever dream that Rs portray it as. The scariest thing for me is the prospect of active sabotage from the federal level, although I don't know how much they have held back.
I know this in the same way that even though I don't know the exact credence to assign the probability of particular bad effects from global warming, I can confidently say that an increase by a factor of 1000 of the CO2 emissions would be a bad thing. This is not because I have done a simulation, but instead my beliefs are based on the assumption that while concerned experts might be wrong in the details, they are probably not wrong with a gap of 3 orders of magnitude.
Tell me the nuance then. If people have concerns about Kessler syndrome at the starlink scale then why wouldn't something literally 1000x bigger be even more concerning.