By "understand" I mean the knowledge that is missing to the agent in order to control the environment toward achieving its goal. Reinforcement learning is concerned with this sort of interaction, between an agent (a decision maker) and an (unknown) environment. The (only) goal of the agent is to maximize its cumulative sum of reward (cf the Reward hypothesis(.
> And why are you calling "knowledge" that which is predictive?
No, I do not think I am saying that, but if it comes across like that, let me be more precise.
I mean that most (but not all) of the world knowledge is predictive. An example of not-predictive knowledge is factual knowledge, like mathematics. Knowledge being predictive is important for an autonomous decision maker because the knowledge can be verified solely by the agent (not a teacher, as it is in the framework of supervised learning). One crucial thing to understand about the framework of reinforcement learning is that it makes the agent solely responsible for its way of behaving. As it should be. Then, an effective way to do that in a scalable way (to not rely on some oracle teacher or anything else) is to be able to verify any knowledge that the agent wants to acquire, in order to achieve its goals.
"and investigate how much of the usual machinery of reinforcement learning algorithms can be replaced with the tools..."
These sounds as if the authors have confidently figured out that the current reinforcement learning formulation is not good enough.
On the other hand, I think the recent large language models have showed us that much of the world knowledge is indeed predictive. That, if you can predict accurately (next words), you can understand higher more abstract things. The hypothesis that much of world knowledge is predictive, is very important in the framework of reinforcement learning because that means that with enough General Value functions learned off-policy, one can predict almost anything about the world that is useful to the agent in achieving its goals. (cf the Horde paper).
IntelliJ certainly doesn't do this. IntelliJ and Kite have functionalities in common, which are fine. The other parts of Kite that IntelliJ hasn't, are problematic.
This kind of tool destroys ones ability to program long sustainable production code. For a novice programmer this has tremendous negative effect on the learning curve. For an experienced programmer this tool is useless, because an experienced programmer will NEVER rely on "popularity" of some code-snippet out there in the wild. Programming is a very intense and deep practice and it is certainly not crafted using this kind of tools. This tool helps people write poor quality code for customers. Makes me wonder, what Knuth would say on this?
Is it even good practice to have a dozen of tabs open?
Maybe it depends on the type of websites open? Travel related with 10 SO questions and 3 Youtube instances? That's not focused work right?
The point is I am very confused why people have dozens of tabs open at once. Please share with me the reason behind this habbit
I don't drink coffee at all. I think it has the placebo effect and we actually don't work harder. In my case, when I drink coffee, I get nervous and can't think straight anymore because of the energy rush
By "understand" I mean the knowledge that is missing to the agent in order to control the environment toward achieving its goal. Reinforcement learning is concerned with this sort of interaction, between an agent (a decision maker) and an (unknown) environment. The (only) goal of the agent is to maximize its cumulative sum of reward (cf the Reward hypothesis(.
> And why are you calling "knowledge" that which is predictive?
No, I do not think I am saying that, but if it comes across like that, let me be more precise.
I mean that most (but not all) of the world knowledge is predictive. An example of not-predictive knowledge is factual knowledge, like mathematics. Knowledge being predictive is important for an autonomous decision maker because the knowledge can be verified solely by the agent (not a teacher, as it is in the framework of supervised learning). One crucial thing to understand about the framework of reinforcement learning is that it makes the agent solely responsible for its way of behaving. As it should be. Then, an effective way to do that in a scalable way (to not rely on some oracle teacher or anything else) is to be able to verify any knowledge that the agent wants to acquire, in order to achieve its goals.