The first bit is why it is called Stochastic gradient decent. You follow the gradient of a randomly chosen minibatch at each step. It basically makes you "vibrate" down along the gradient.
Having a Scandinavian perspective on this, where birthrates also have fallen a lot recently, I don't think financial stability is the only factor. In my experience ppl here don't wait with having kids so long due to lack of money. With the social safety net we have here, free healthcare and education, paid parental leave etc, you are fine as long as you have a decent job. The reason that ppl wait until they are 35+ to get kids is rather that they want to do other shit first and just stay "young" longer and don't want the responsibility. Having kids is just not that important for ppl so they wait until they are kind of bored of other things and then they are often suddenly too old, or fail to find a good partner, or just don't want the responsibility.
Wonder if this perspective is compatible with Wolframs physics model based on hypergraphs?
Gravity, in this framework, is an emergent property arising from the statistical behavior of the hypergraph's evolution, suggesting that gravity is an "entropic force" arising from the tendency of the system to minimize its computational complexity
If you and I had the same exact input and outputs then why would we have different experiences? Seems very unlikely to me.
A better way of thinking about qualia is like embeddings in a neural network. Every time you run the training from a random initialization you will get different resulting embedding, but given the same training data all embeddings will all essentially be equivalent under rotation.
I.e., your internal representation of blue might be very different from mine in an absolute sense but our relation between the representation of different colors will be roughly the same.
The difference between traditional software and LLMs is the generality of their intelligence, and there are formal definitions of general intelligence such as https://en.m.wikipedia.org/wiki/AIXI
AIXI is a definition of the optimal agent and is hence uncomputable but LLMs are approximations which are approaching AIXI.
I recommend Fridman's interview with Hutter.
Funnily Bezos has probably made more for Harris by stopping the endorsement then if he let it go through. No one would have cared about the endorsement, now this story is everywhere..
The reason that ppl give up now is not that it is worse now than before, but rather the opposite. Now you can do almost nothing productive and still survive. Before, you would simply starve to death.
Compatibilists often argue that determinism and free will (and moral responsibility) are compatible since their view is that what constitutes freedom is not affected by whether or not determinism holds. Hence, in this case, the debate is not about the consequences of determinism, but rather what freedom is.
In my mind, that the known laws of nature do not permit "free will" in the way incompatibilists define it is trivial. (regardless of whether the universe is truly deterministic or also has some randomness sprinkled on top)
Yes, the Stanford Encyclopedia of Philosophy is great! (and it is referenced in the paper)
A quote from it: "Other compatibilists show less concern in rebutting the conclusion that the freedom to do otherwise is incompatible with determinism. Compatibilists of this stripe reject the idea that such freedom is necessary for meaningful forms of free will (e.g., Frankfurt 1969, 1971; Watson 1975, Dennett 1984)—the “varieties of free will worth wanting,” (Dennett 1984). And even more notably, some compatibilists simply deny that freedom of this sort is in any way connected to morally responsible agency (e.g., Fischer 1994, Fischer & Ravizza 1998, Scanlon 1998, Wallace 1994, Sartorio 2016)."
This is the position of the paper above essentially. It references Dennett. The kind of freedom that matters, and that ppl are talking about in everyday life, is not the type of freedom incompatibilists reject. It still makes sense to talk about freedom, as in "freedom of opinion" etc, even if agents could not have chosen differently.
The paper explains this position from the perspective of reinforcement learning, and also gives a theory for why it is beneficial for intelligent agents to model themselves as being able to have chosen differently even if they actually could not.
Well there are many versions of compatibilism I guess, but just reading the Wikipedia article on compatibilism I don't think most compatibilists think freedom relies on whether or not causal determinism holds. Please tell me if i'm wrong.
Defining free will:
Compatibilists often define an instance of "free will" as one in which the agent had the freedom to act according to their own motivation. That is, the agent was not coerced or restrained. Arthur Schopenhauer famously said: "Man can do what he wills but he cannot will what he wills."[14] In other words, although an agent may often be free to act according to a motive, the nature of that motive is determined. This definition of free will does not rely on the truth or falsity of causal determinism.[2] This view also makes free will close to autonomy, the ability to live according to one's own rules, as opposed to being submitted to external domination.
https://en.wikipedia.org/wiki/Compatibilism
Indeed. The problem is that the word "free" has different meanings when talking about free will and freedom in other contexts. What you talk about is value freedom, while ppl denying the existence of free will refer to "physcial" freedom. Few ppl notice the distinction though which makes the debate somewhat strange.
Yes, the strange thing with the debate is that ppl seem to think that there is a conflict between compatibilism and incompatiblism, which there isn't. They are just taking about different concepts.
"By clearly defining what freedom is, regardless of whether or not determinism holds, our conceptualization of value freedom allows for a compatibilist view on free will. We are not disagreeing with the classic incompatibilist
argument (McKenna & Coates, 2021) that determinism does not allow for free will; rather, what we propose is that the kind of freedom lay people actually refer to in everyday language is not the same kind of freedom incompatibilists reject. This distinction between what we here call physical freedom vs. value freedom allows for several potentially interesting perspectives."
> I could imagine that in reality all of that isn't needed. Whether the algorithm really feels something or not - it doesn't matter as long as the expression is realistic enough for humans to believe that it feels something. Then you get the consciousness "injected" into your algorithm from the outside. Can you convince someone with a realistic sophisticated simulation that something is conscious even though it isn't? Probably. Still, it won't get you anywhere.
The thing is that I can't prove that you or anyone else is conscious either, including myself. With conscious acting AIs it will be the same, and these AIs will believe they are conscious in the same way as we do. So yes, we will have to treat them as if they are conscious.
> Unless you really claim that modeling "pain" as some kind of variable in an algorithm can be equivalent to a biological being feeling pain?
Yes, I do. I guess it all boils down to whether or not you think the hard problem of consciousness is actually a problem or not. I doubt it is, but it is a totally respectable position if you do :)
> Where exactly in the cited paper is the claim that emotions are a belief? Can't find it.
As I wrote in my previous response, you can call it predictions too. Predictions are usually beliefs about the future. In the predictive brain literature, it is also used as predictions about the present. I use belief as the output of some inference algorithm that needs to deal with uncertainty.
Here is a quote from the paper:
"The brain continually constructs concepts and creates categories to identify what the sensory inputs are, infers a causal explanation for what caused them, and drives action plans for what to do about them. When the internal model creates an emotion concept, the eventual categorization results in an instance of emotion."
But you are right, the paper talks more about how emotional categories are created, and dodges the question of how the "subjective experience of having an emotion" emerges. In my mind, the step is not far though, and boils down to, as said above, how you view the hard problem or consciousness. That we "feel" stuff is a result of an algorithm/model that describes ourselves as having experiences, which is a good model, since how else would we describe ourselves? My belief is that the progress in AI and neuroscience will prove me right :)
> In other words: "Pain isn't real"? Are you serious?
Of course it is real. When I feel pain from an injury, the injury is very real, my pain sensor's signals are real and my brain's interpretation of those signals is real, and my behavior of trying to avoid further injury is real.
> Again, is there any reliable source for that other than a whole book that probably doesn't even talk about what you claim? If it is in the book, can you give a detailed reference where to find it? And if so, does the book provide references to research underpinning those claims?
Lisa Barrett is a very famous professor of neuroscience, so sure there is a lot of research in that book :)
Here is a short video where she explains it https://www.youtube.com/watch?v=0QfCvIJRtE0
I also recommend her interviews with Lex Fridman.
She talks more about emotions as predictions, but we mean the same thing.
> Or did you just come up with that on your own? It sounds absolutely ridiculous to me. Any attempt to "model" the experience is prone to fail by lack of evidence because pain is, by definition, subjective.
Your pain is very subjective yes, but that just means that it is created by your internal model of the world and yourself, which has a unique subjective perspective of the world.
> If you just search for the definitions of belief, emotion and pain, you will quickly find that their meanings are distinct. Claiming that pain is a belief is absurd.
These words are normally not used in the way I use them, but the point is that emotions are outputs of our model of ourselves. These outputs you can call beliefs, predictions or interpretations if you like.