Sequoia: 20 Petaflops, 1.6 million cores, 1.6 Petabytes RAM(hpcinfo.wordpress.com)
hpcinfo.wordpress.com
Sequoia: 20 Petaflops, 1.6 million cores, 1.6 Petabytes RAM
http://hpcinfo.wordpress.com/2009/02/05/sequoia-20-petaflops-16-million-cores-16-petabytes-ram-6-megawatts/
2 comments
> My own personal interest is protein folding.
> But computationally we're stuck with exponential time.
It's not that bad and it "recently" got a lot better.
Search for "shaw protein folding neutral territory".
> But computationally we're stuck with exponential time.
It's not that bad and it "recently" got a lot better.
Search for "shaw protein folding neutral territory".
It's not even close. Shaw's work is good, but we are still not close to folding proteins from a linear sequence. The physics is just not there
Protein folding is NP-complete (Berger et al 1998). This implies that we can get fast heuristics for certain subclasses of proteins, but there is no general linear time protein folding solution.
I don't know of anything in nature which solves the traveling sales person problem, but protein folding does progress in linear time in nature.
That may mean that proteins quickly collapse the probability space or have some other "trick" by which they are not NP-complete, or whatever, it is not magic.
There is a way and I'm hopeful we can find it.
That may mean that proteins quickly collapse the probability space or have some other "trick" by which they are not NP-complete, or whatever, it is not magic.
There is a way and I'm hopeful we can find it.
For what it's worth I attended a seminar by this guy
http://www.princeton.edu/~rajchak/
the other day and he was talking about using optimal control theory to optimize protein sequences for specific enzyme activity. He had a very sophisticated model for how the amino acid sequence determines the shape of the protein's active area. It was way over my head but he treated it as a solved problem. You may find his papers interesting.
The combinatorial search results on protein folding are theoretically interesting, but I don't think that they are show stoppers for finding, or optimising the folded configuration of a protein. As you mentioned, most of the degrees of freedom which lead to the exponential size of the search space are irrelevant. A detailed understanding of the physics of how amino acids interact in water with ions can allow one to make approximations that vastly reduce the size of the search space.
Plus how much time does it take a protein to find it's native conformation? Let's say the folding time is 10 ms. I think ultrafast pump probe spectroscopy has shown that alpha helices and other secondary structural motifs form on the time scale of 100 ps. Then the protein explores about 10^8 configurations before it finds the native one.
http://www.princeton.edu/~rajchak/
the other day and he was talking about using optimal control theory to optimize protein sequences for specific enzyme activity. He had a very sophisticated model for how the amino acid sequence determines the shape of the protein's active area. It was way over my head but he treated it as a solved problem. You may find his papers interesting.
The combinatorial search results on protein folding are theoretically interesting, but I don't think that they are show stoppers for finding, or optimising the folded configuration of a protein. As you mentioned, most of the degrees of freedom which lead to the exponential size of the search space are irrelevant. A detailed understanding of the physics of how amino acids interact in water with ions can allow one to make approximations that vastly reduce the size of the search space.
Plus how much time does it take a protein to find it's native conformation? Let's say the folding time is 10 ms. I think ultrafast pump probe spectroscopy has shown that alpha helices and other secondary structural motifs form on the time scale of 100 ps. Then the protein explores about 10^8 configurations before it finds the native one.
Hey this is from the abstract of:
Sequence optimization and designability of enzyme active sites - PNAS v. 102 (2005)
Full text is here:
http://www.pnas.org/content/102/34/12035.full
""" Since it was first observed that all known natural proteins adopt one of only ≈1,000 distinct protein folds (1) [emphasis mine], researchers working in the field of computational protein design have endeavored to understand the basis of this biophysical phenomenon and apply the resultant knowledge to the construction of unnatural proteins. Motivated by the observation that certain folds are associated with more natural sequences than others, the principle of fold designability (the number of unique sequences compatible with the backbone structure of the fold) was introduced (1). To place the problem on a mathematical foundation, an analogy has been drawn between sequence space and phase space in statistical physics, where the backbone structure represents a macroscopic state of the system and distinct sequences are microscopic states whose occupation of the macroscopic state is associated with an energy, the folding free energy of the protein. In this language, a fold's designability is correlated with its sequence entropy (2). """
Sequence optimization and designability of enzyme active sites - PNAS v. 102 (2005)
Full text is here:
http://www.pnas.org/content/102/34/12035.full
""" Since it was first observed that all known natural proteins adopt one of only ≈1,000 distinct protein folds (1) [emphasis mine], researchers working in the field of computational protein design have endeavored to understand the basis of this biophysical phenomenon and apply the resultant knowledge to the construction of unnatural proteins. Motivated by the observation that certain folds are associated with more natural sequences than others, the principle of fold designability (the number of unique sequences compatible with the backbone structure of the fold) was introduced (1). To place the problem on a mathematical foundation, an analogy has been drawn between sequence space and phase space in statistical physics, where the backbone structure represents a macroscopic state of the system and distinct sequences are microscopic states whose occupation of the macroscopic state is associated with an energy, the folding free energy of the protein. In this language, a fold's designability is correlated with its sequence entropy (2). """
I believe that the reason why proteins fold in polynomial time (not necessarily linear) is that nature's proteins are "easy" to fabricate.
To use the TSP as an analogy, think of a protein as a configuration of cities. The proteins in our body are the solutions of the easy configurations, i.e. a group of cities arranged in a circle. Nature hasn't been able to find the really hard solutions.
We would like to explore the full configuration space, to look for new proteins, and that's still hard. Or maybe I misunderstand.
To use the TSP as an analogy, think of a protein as a configuration of cities. The proteins in our body are the solutions of the easy configurations, i.e. a group of cities arranged in a circle. Nature hasn't been able to find the really hard solutions.
We would like to explore the full configuration space, to look for new proteins, and that's still hard. Or maybe I misunderstand.
I do think that really understanding molecular machinery (known proteins and their dynamic interactions in atomistic glory) and true molecular recognition problems are what we should be doing.
That said, we need better physics to represent these interactions. The fields been relatively static when it comes to forcefields that really work.
That said, we need better physics to represent these interactions. The fields been relatively static when it comes to forcefields that really work.
...and is it finally coming close to .00001 % of a human brain?
If someone is excited about that, what about the human brain itself? Just a simple information:
The human brain is the most complex thing in the whole universe, or: all of physical universe is less complex than a single human brain.
If someone is excited about that, what about the human brain itself? Just a simple information:
The human brain is the most complex thing in the whole universe, or: all of physical universe is less complex than a single human brain.
So the human brain has about 10^11 neurons and about 10^4 synapses per neuron. Naively representing the state of each synapse with one byte requires 10^15 bytes that's about 500,000 PCs.
But neurons aren't necessarily the optimal substrate for "neural" computation. They have a cycle time of about 100 Hz. Electronics are potentially a factor of 10^8 faster. Neurons also have a lot of noise which surely reduces their information transmission capacity. Also there are major constraints on the neural tissue involving power consumption and stability against siezures. Also there are vast sections of the brain e.g. the cerebellum which are apparently devoted to problems like stabilizing posture and making smooth movements which can be solved much easier with electronic controls.
It's very plausible that we could achieve a factor of 100 or even 1000 over the brain. The bottleneck is human imagination; we don't have a clue how the thing works.
But neurons aren't necessarily the optimal substrate for "neural" computation. They have a cycle time of about 100 Hz. Electronics are potentially a factor of 10^8 faster. Neurons also have a lot of noise which surely reduces their information transmission capacity. Also there are major constraints on the neural tissue involving power consumption and stability against siezures. Also there are vast sections of the brain e.g. the cerebellum which are apparently devoted to problems like stabilizing posture and making smooth movements which can be solved much easier with electronic controls.
It's very plausible that we could achieve a factor of 100 or even 1000 over the brain. The bottleneck is human imagination; we don't have a clue how the thing works.
Let me explicitly repeat your statement:
You say that the human brain would (in theory) be able to create something more intelligent than itself?
(Are you aware of the irony?)
You say that the human brain would (in theory) be able to create something more intelligent than itself?
(Are you aware of the irony?)
Hmm, I can use my physical strength to build an engine stronger than myself...
Are you aware of the empty rhetoric?
Are you aware of the empty rhetoric?
That's not the point.
The human brain is quite "infinitely" interconnected, and this is something which not only would be really, really hard to solve with a physical machine: it's simply impossible.
I understand the fascination of a real super computer, but it's simply nothing compared to a single human brain...
The human brain is quite "infinitely" interconnected, and this is something which not only would be really, really hard to solve with a physical machine: it's simply impossible.
I understand the fascination of a real super computer, but it's simply nothing compared to a single human brain...
> The human brain is quite "infinitely" interconnected, and this is something which not only would be really, really hard to solve with a physical machine: it's simply impossible.
Hmm. How does this "infinitely interconnected" thing work if not by physical means? (In physical, I include electromagnetic waves without a dedicated conductor, not just "wires".)
Hmm. How does this "infinitely interconnected" thing work if not by physical means? (In physical, I include electromagnetic waves without a dedicated conductor, not just "wires".)
Yes, you are always free to misunderstand.
The interesting thing is: some do, some do not.
(We have physics, chemistry, and others, exactly to distinguish different levels...)
The interesting thing is: some do, some do not.
(We have physics, chemistry, and others, exactly to distinguish different levels...)
I understood you to say that parts in the brain communicate by something other than physical means. Is that where I went wrong?
If some of the communications are not by physical means, how does the "not physical means" communication work?
If all of the communications are by physical means, how does the brain have "infinite" interconnect? Am I wrong in believing that one can't have infinite connectivity with physical means?
If some of the communications are not by physical means, how does the "not physical means" communication work?
If all of the communications are by physical means, how does the brain have "infinite" interconnect? Am I wrong in believing that one can't have infinite connectivity with physical means?
Preface: I'm not particularly well schooled in either neuroscience or in the mechanics of microprocessors.
However, I'm very close to certain that your post is basically a lot of hand-waving that is ultimately not only poorly reasoned but also actually incorrect.
- - -
The question we're trying to answer is basically, "What's the computational capacity of the brain?"
But that's not really a good question to work with, because "computational capacity" isn't well-defined. [Actually, I'm not sure about that. I know that either it's not well-defined, or its definition is something that's clearly not what we want in this situation.]
So let's ask something that we can actually reason about, like, "Assuming infinite memory, roughly how many floating point operations per second would a computer need to be able to execute in order to direct a human body in a way that's indistinguishable from what a real person does -- in real time?"
And, okay, we really only want the brain's conscious thought and maybe some of its unconscious thought, but it's hard to make a well-defined question involving just those things, and I doubt that the answer to our question will be more than a few orders of magnitude off of what we're really after.
- - -
Anyway, your post does a really, really bad job of answering that question (or anything like it). Your worst offense: > "So the human brain has about 10^11 neurons and about 10^4 synapses per neuron. Naively representing the state of each synapse with one byte requires 10^15 bytes that's about 500,000 PCs."
Come on. Did the fact that you had to use the word "byte" not tip you off? YOU'RE DISCUSSING MEMORY. Nobody cares about memory, because the amount of memory needed to store the brain's state is vastly less than the number of operations needed to simulate its function in real time.
I seriously am not able to articulate how frustrated I am that at least 12 people felt justified in upvoting you without taking the time to process the first two sentences of your post.
Before I go, another treat: > "Electronics are potentially a factor of 10^8 faster."
No. Electronics are a factor of 10^8 "faster", when "faster" is taken to mean "more rapid in executing a cycle." Which is an absolutely pointless irrelevent piece of shit statistic.
What the fuck.
- - -
Anyway, let me wrap up this post by actually answering the above question as best we can, given the current state of scientific understanding (or, more precisely, my understanding of that state of understanding):
There are on the order of 10^10 neurons and on the order of 10^11 glial cells in the adult human brain. These cells are directed by untold numbers of chemoattractants. We have no good theoretical model for the behavior of these cells in huge numbers of situations.
Therefore, WE HAVE ABSOLUTELY NO FUCKING IDEA how much processing power it would take to simulate the brain's macro function. We can put an upper bound on it: we know that if we could simulate every atom in the brain to a reasonable degree of accuracy, we'd get human behavior. Obviously the number of FLOPS it would take to do such a thing is unreachable. And beyond that, I don't think it's fair to say we know anything at all.
However, I'm very close to certain that your post is basically a lot of hand-waving that is ultimately not only poorly reasoned but also actually incorrect.
- - -
The question we're trying to answer is basically, "What's the computational capacity of the brain?"
But that's not really a good question to work with, because "computational capacity" isn't well-defined. [Actually, I'm not sure about that. I know that either it's not well-defined, or its definition is something that's clearly not what we want in this situation.]
So let's ask something that we can actually reason about, like, "Assuming infinite memory, roughly how many floating point operations per second would a computer need to be able to execute in order to direct a human body in a way that's indistinguishable from what a real person does -- in real time?"
And, okay, we really only want the brain's conscious thought and maybe some of its unconscious thought, but it's hard to make a well-defined question involving just those things, and I doubt that the answer to our question will be more than a few orders of magnitude off of what we're really after.
- - -
Anyway, your post does a really, really bad job of answering that question (or anything like it). Your worst offense: > "So the human brain has about 10^11 neurons and about 10^4 synapses per neuron. Naively representing the state of each synapse with one byte requires 10^15 bytes that's about 500,000 PCs."
Come on. Did the fact that you had to use the word "byte" not tip you off? YOU'RE DISCUSSING MEMORY. Nobody cares about memory, because the amount of memory needed to store the brain's state is vastly less than the number of operations needed to simulate its function in real time.
I seriously am not able to articulate how frustrated I am that at least 12 people felt justified in upvoting you without taking the time to process the first two sentences of your post.
Before I go, another treat: > "Electronics are potentially a factor of 10^8 faster."
No. Electronics are a factor of 10^8 "faster", when "faster" is taken to mean "more rapid in executing a cycle." Which is an absolutely pointless irrelevent piece of shit statistic.
What the fuck.
- - -
Anyway, let me wrap up this post by actually answering the above question as best we can, given the current state of scientific understanding (or, more precisely, my understanding of that state of understanding):
There are on the order of 10^10 neurons and on the order of 10^11 glial cells in the adult human brain. These cells are directed by untold numbers of chemoattractants. We have no good theoretical model for the behavior of these cells in huge numbers of situations.
Therefore, WE HAVE ABSOLUTELY NO FUCKING IDEA how much processing power it would take to simulate the brain's macro function. We can put an upper bound on it: we know that if we could simulate every atom in the brain to a reasonable degree of accuracy, we'd get human behavior. Obviously the number of FLOPS it would take to do such a thing is unreachable. And beyond that, I don't think it's fair to say we know anything at all.
Oy man, I'm not used to people getting this fired up over a back of the envelope calculation! I'm sorry if my post sounded arrogant - I'm just trying to have some fun here.
I don't think I can really comprehend what any of your objections are, but I can certainly think of some major flaws in the toy argument I gave, and maybe these are what you were getting at even if you couldn't really articulate them.
first, just saying the state of a brain is the state of each synapse assumes a fixed topology of the brain. This is obviously false - there is a lot of evidence that the topology of the brain is changed by experience e.g. the occular dominance columns. Okay, so how much extra space would we need to deal with this - not that much actually: how much space do we need to store an arbitrary weighted directed graph with 10^11 nodes and 10^15 edges? Well for each edge we need to know the two vertices and the weight. Give each neuron a unique id and this requires about 10-12 bytes per synapse instead of 1. This is only one order of magnitude difference between a fixed and arbitrary topology! And this is probably a gross overestimate of the storage space required because the topology of the nervous system is far from arbitrary - after all the thing is generated by a genetic code of very modest size. See my posts on why the large search space doesn't kill attempts to solve protein folding.
Obviously I ignore the functioning of the glia - I know for a fact that these are very important for the growth and development of the brain but I'm not sure they are the critical part of its computational function. If you know otherwise please point me to some references, I would love to know!
The reason that I mentioned the speed difference of electronics and neurons is that this is what makes it possible to update the state of those 10^15 synapses in real time. Of course it would be wasteful to use the standard PC architechture for this process, but still.
Hope this helps clear things up.
I don't think I can really comprehend what any of your objections are, but I can certainly think of some major flaws in the toy argument I gave, and maybe these are what you were getting at even if you couldn't really articulate them.
first, just saying the state of a brain is the state of each synapse assumes a fixed topology of the brain. This is obviously false - there is a lot of evidence that the topology of the brain is changed by experience e.g. the occular dominance columns. Okay, so how much extra space would we need to deal with this - not that much actually: how much space do we need to store an arbitrary weighted directed graph with 10^11 nodes and 10^15 edges? Well for each edge we need to know the two vertices and the weight. Give each neuron a unique id and this requires about 10-12 bytes per synapse instead of 1. This is only one order of magnitude difference between a fixed and arbitrary topology! And this is probably a gross overestimate of the storage space required because the topology of the nervous system is far from arbitrary - after all the thing is generated by a genetic code of very modest size. See my posts on why the large search space doesn't kill attempts to solve protein folding.
Obviously I ignore the functioning of the glia - I know for a fact that these are very important for the growth and development of the brain but I'm not sure they are the critical part of its computational function. If you know otherwise please point me to some references, I would love to know!
The reason that I mentioned the speed difference of electronics and neurons is that this is what makes it possible to update the state of those 10^15 synapses in real time. Of course it would be wasteful to use the standard PC architechture for this process, but still.
Hope this helps clear things up.
May I ask what is the basis of your estimate on the computational capacity of the human brain?
Especially pointed question since the statement implies someone has crossed the chasm of the physical universe in order to measure this.
(since "all of physical universe is less complex than a single human brain" and the physical universe contains around 6.7 billion brains (that we know of))
(since "all of physical universe is less complex than a single human brain" and the physical universe contains around 6.7 billion brains (that we know of))
I said "physical" to exclude the misunderstanding.
My point was that if you are restricting the comparison to the physical universe then logically you must be implying the brain extends beyond the physical universe somehow -- since there is more than one brain in the physical universe. Your subsequent statements in this thread confirmed you have this opinion.
If true, then it has an undefined and immeasurable complexity. You can't make all these statements about it, you can't measure it, and you have no context to even begin to talk about it. You can't even make assertions about what the measurement of complexity itself would be ("infinitely more" is just as nonsensical as a finite number more because we do not know what the units of a complexity measurement would be -- if "units" even makes sense in the "beyond").
If true, then it has an undefined and immeasurable complexity. You can't make all these statements about it, you can't measure it, and you have no context to even begin to talk about it. You can't even make assertions about what the measurement of complexity itself would be ("infinitely more" is just as nonsensical as a finite number more because we do not know what the units of a complexity measurement would be -- if "units" even makes sense in the "beyond").
Pretty sure you failed.
Intelligence can't be measured in huge numbers, and every single human brain is infinitely more intelligent than any possible computer.
It's not me saying this, but science.
It's not me saying this, but science.
I think you may have a fundamental misunderstanding of life, the universe, and everything. Suppose we can simulate interactions between atoms nearly perfectly on a computer... then it's just a matter of making an atom-for-atom digital replication of the brain (understanding of course that this is all easier said than done, but all possible). Once that is achieved, we optimize and make it even better by thinking faster and more accurately than we ever could. What are you missing from this? Or is hacker news really starting to be trolled... because I might just cry.
[deleted]
My own personal interest is protein folding.
Proteins are folding right now, as you read this, in every cell of your body, and they're doing it in linear time.
But computationally we're stuck with exponential time.
The day we find an algorithm for linear time protein folding, is the day the world changes at least as much as when penicillin was discovered.