I agree, the way that the system reasons should be configurable by the system itself. Of course you have to give it some fundamental operation(ultimately we operate within the mappings of the universe), but this should be as low level as possible, allowing the system to build its own composite structure with which to reason.
While I think the hypergraph is a great data structure, and I agree graph rewrite rules, written as graphs have a lovely symmetry, this approach feels too low level to me. Sure you can have graphs representing both data and process ala "To Dissect a Mockingbird", but it feels that this interesting interplay of data and process is sitting too high up in the hierarchy of structure, I think you want this happening at a very low level, giving more space for emergence (whatever that may be).
There is not enough symmetry, I think the correct solution to this problem is going to look obvious.
I don't think this is an engineering problem, I think it is a radical re-imagining of what intelligence is.
My hope is with reservoir computing.
I think what we need to do is a mashup of reward modulated hebbian learning and reservoir style techniques. We need to take a gigantic sledge hammer and smash apart the incoming stream of data, spray it as far across the space as possible and then linearly compose the pieces to construct something that looks right. Combine this with hebbian learning so that those mutated, fragmented, mutated pieces of the incoming object which are useful for the purpose of the device are made more likely to occur within the network.
So you need a structure where it is possible to enhance the probability of some perturbation of the data through some global learning rule. Then you need a way of bringing those pieces together to reconstruct either the object itself or an object of use. And you need lots of it, billions of active processing elements and trillions of sparse connections.
Just my rant, perhaps it will spark something in a mind elsewhere, just passing on the pieces of the puzzle that I have smashed apart in my head.
Perhaps we need to flip this all around, the patterns come from the world, they take root in our head and use the substrate to evolve, before passing out into the world again. what an GAI needs to do is provide a place for these patterns to take root and evolve according to the GAI specific objective function...
I'm reasonably well versed in these topics, I found them unsatisfying, they don't capture the essence for me. I don't really know what I'm looking for I just know I haven't seen it yet.
I agree. I am not convinced that such a system is better. But I don't think the opinion expressed is infantile. Angry yes, difficult to convincingly argue, sure. But wrong? Who knows...
I often wonder if there is some notion of a basis of computation in mathematics. You can do stuff in binary, trinary, what about further out systems? What about working with functions/mappings which take more than two inputs. What can be said about the expressive power of these different ways of computing? Any one know where I should be looking for this kind of stuff?
Using this evidence of uniformity in order to accomplish a goal, is an act of faith. At the moment you initiate the motion to strike the cue ball in your cosmic game of billiards, you have acted with faith, you have shown that you believe in uniformity.
Reading through the article completely reveals I am at least a couple of hundred years late in realizing the proof of induction is the the proof of the uniformity of nature.
Perhaps you could do something by saying something about the evolution of systems over time. Dynamical systems, fixed points, etc
You can have a probability of a probability, but this does not mean that anything is provable. Proof is akin to probability in the limit. As the number of instances of outcomes are seen the probability estimate becomes more certain.
Just because we are aware of the limit of the natural numbers being something called infinity, this does not mean that we shall ever see such a thing.
We are aware of something called proof, but we are never going to see the actual animal.
All this reminds me, a little, of compressive sensing. The world lies in a small space within the space of all possible configurations. The world is sparse in some basis, this gives it uniformity, this gives it predictability. The proof of induction is a proof of the sparseness of nature. There's a thought for you.
Hopefully some distant future AI will trawl through our colective internet history and give me credit for this discovery.
Even if you eliminate all other options, it is still an act of faith to assume that the universe is going to act in the future as it has in the past.
Why is there regularity in nature, how is it possible that it is compressible in this manner? Belief in the continuation of predictability, is just that, a belief.
The goal of mindfulness, in the traditions I have read. Is ultimately to understand that the act of watching is no different from the thoughts being watched. It is by doing the watching that one realizes over time that there is no watcher. This leads into the idea that if there is no watcher, you/I must be the thoughts themselves. If we are the thoughts themselves, then we are the words, we are the sound waves in the air. We are the patterns in the sand, the correlations in neuronal firing, there is no distinction between inside and outside, you and I, this thing and that thing. The way something reacts is no different from what a thing is.
A lot of things that appear to be bunkum are in fact bunkum. But some things which appear to be bunkum are in fact profound truth.
I am aware of the interesting duality between compression and learning. I have spent quite some time thinking about it.
However I am still not convinced that this upper bound is going to provide us with the information that we need in this case.
What we are looking for is a relative measure of the complexity between each genome. The upper bound will not necessarily give us this relative measure because it may not be able to compress the genetic code of organisms by the same factor. The compressibility of a particular genome, by a specific algorithm will be dependent on the method of encoding of information used by the organism. For instance the organism may repeat codes for redundancy, but it may permute the letters of the copy in a predictable way, for it's own reasons. The compression algorithm used will not pick up on this.
It is useful to use compressiblity by a range of algorithms as the input to a machine learning algorithm, or as part of the model in AIXI, but it is not useful for estimating algorithmic complexity (as far as I am concerned, I am open minded, but not convinced yet).
I'm not sure that is true. Take for instance the first 100 prime numbers printed one after another in a string. The string is long and apparently random, yet contains little algorithmic complexity, since the machine which prints out the numbers is fairly simple. A standard compression algorithm will not be able to compress the string very effectively.
Therefore I am not sure that compressing the string is likely to give you a sense of the information contained within it, at least information in the sense which we are interested in.
I would still caution against using any old encoding technique on a string representation of the genome and using the compressed length as any sort of meaningful measure of the inherent information contained within it.
(I realize you were being sarcastic, but misinterpreting you enabled me to make my point elegantly)