IBM unveils microchip based on the human brain(newscientist.com)
newscientist.com
IBM unveils microchip based on the human brain
http://www.newscientist.com/article/dn20810-ibm-unveils-microchip-based-on-the-human-brain.html
12 comments
Very interesting, but my question is, what are the speed of these chips compared to conventional ones. The main problem with the human brain is that while it is fantastic at parallel processing, the speed at what it processes information is only a few MHz i think. The reverse is true for computers, great speed, terrible parallel. If these chips solve the problem of parallel processing but keep speeds the same it would be a great step for computers.
I always imagined the point of replicating the brain with microchips is to see if any emergent behavior (read: learning) is observed, not to improve the performance of any existing process.
> the speed at what it processes information is only a few MHz i think
More like Hz, actually. Neurons are slow - I am not sure how frequently can they fire, but I am sure it is not in the multi-thousand per second range.
More like Hz, actually. Neurons are slow - I am not sure how frequently can they fire, but I am sure it is not in the multi-thousand per second range.
Maximum firing rate differs between neuron types (i.e., cortical pyramidal neurons versus cerebellar Purkinje neurons) dependent upon cellular membrane properties (dendrite diameter and ion channel distributions, etc.), but a good rule-of-thumb to keep in mind is 1 kHz. That maximum "firing rate" is about as fast as a patch of membrane can generate an action potential (AP, or "spike"), reset, and fire another.
All that said, a relevant question to ask is, "What is the information content of a single action potential?" There is not an agreed-upon answer among neuroscientists. The metaphor linking brain and a personal computer is extremely strained: to start, it is not quite true that neurons can carry out logical functions. Neurons are firmly rooted in the analog domain and it is much more straightforward to infer polynomial-type computations from their physiology, the order and coefficients being dictated by the particulars of the scenario. Also, the current thinking is that information is encoded in bursts or repetitive APs, called "spike trains". But if we must, I'd throw out a range of 0.5 to 5 bits per spike [Rieke, et al.]. (If you want to go down a rabbit hole and explore one of the pivotal topics among people studying neural computation, look up "rate coding" versus "temporal coding". Perhaps even phrasing the question that way is misleading.)
Here's a side note that may be of interest: during an AP, the trans-membrane voltage swings from approximately -100 mV to +100 mV (very rough figures). If you consider that this potential difference is applied across the 3 nano-meter thick cell membrane, the resulting electric field is very close to the dielectric breakdown voltage for phospholipid membranes. Our neurons are constantly operating within a factor or two of literally destroying themselves! (The preceding paragraph contains very back-of-the-envelope reasoning. To be more rigorous, I'd have to dig out my notes and reference materials, but the general point nonetheless holds.)
All that said, a relevant question to ask is, "What is the information content of a single action potential?" There is not an agreed-upon answer among neuroscientists. The metaphor linking brain and a personal computer is extremely strained: to start, it is not quite true that neurons can carry out logical functions. Neurons are firmly rooted in the analog domain and it is much more straightforward to infer polynomial-type computations from their physiology, the order and coefficients being dictated by the particulars of the scenario. Also, the current thinking is that information is encoded in bursts or repetitive APs, called "spike trains". But if we must, I'd throw out a range of 0.5 to 5 bits per spike [Rieke, et al.]. (If you want to go down a rabbit hole and explore one of the pivotal topics among people studying neural computation, look up "rate coding" versus "temporal coding". Perhaps even phrasing the question that way is misleading.)
Here's a side note that may be of interest: during an AP, the trans-membrane voltage swings from approximately -100 mV to +100 mV (very rough figures). If you consider that this potential difference is applied across the 3 nano-meter thick cell membrane, the resulting electric field is very close to the dielectric breakdown voltage for phospholipid membranes. Our neurons are constantly operating within a factor or two of literally destroying themselves! (The preceding paragraph contains very back-of-the-envelope reasoning. To be more rigorous, I'd have to dig out my notes and reference materials, but the general point nonetheless holds.)
That was an impressive explanation. Thanks.
I doubt the complexity of the brain can be reduced to cycles and the speed at which it performs each cycle. Also, I can't imagine that every operation performed by the brain would be performed at the same speed, so getting sound from the vibrations in your ear to wherever it begins to be interpreted might be much faster than the processes that would trigger memories from such sounds (or any other type of input, be it touch, smell, taste, visual…
cpu frequency scaling. CPUfreq has been in linux since kernel 2.6.
A couple of other discussions on similar articles in the last few days:
http://news.ycombinator.com/item?id=2898229
http://news.ycombinator.com/item?id=2899299
http://news.ycombinator.com/item?id=2898229
http://news.ycombinator.com/item?id=2899299
Can we please stop sucking IBM's cock?
-Their chip only has a superficial connection to biological neurons. Any characterization of this chip in terms of "brains" is frankly bullshit. This is like measuring Google's computing clusters in terms of human brainpower.
-Everything their new chip can do has been done in software / FPGAs. While moving ANN's to ASICs can improve training speed (and is important), it does not help with the unsolved algorithmic challenges they present. Furthermore, the hardware structure severely constrains the ways in which these ANNs can be applied.
-Last time this IBM lab made a misleading announcement to the press, it was appropriately ripped to shreds by a very respected individual in the neuroscience community (http://spectrum.ieee.org/tech-talk/semiconductors/devices/bl...)
-Lastly, please refrain from drawing conclusions or extrapolating on science-related articles in the popular press. They are characterized by hyperbole and misinformation, and in many cases are flat-out wrong.
-Their chip only has a superficial connection to biological neurons. Any characterization of this chip in terms of "brains" is frankly bullshit. This is like measuring Google's computing clusters in terms of human brainpower.
-Everything their new chip can do has been done in software / FPGAs. While moving ANN's to ASICs can improve training speed (and is important), it does not help with the unsolved algorithmic challenges they present. Furthermore, the hardware structure severely constrains the ways in which these ANNs can be applied.
-Last time this IBM lab made a misleading announcement to the press, it was appropriately ripped to shreds by a very respected individual in the neuroscience community (http://spectrum.ieee.org/tech-talk/semiconductors/devices/bl...)
-Lastly, please refrain from drawing conclusions or extrapolating on science-related articles in the popular press. They are characterized by hyperbole and misinformation, and in many cases are flat-out wrong.
I'm sure you're quite right and the IEEE link makes oodles of sense, but for people examining notions of creativity in computational intelligence (for example), the idea of these chips is quite attractive.
In fact, I remember programming ANNs in horribly non-distributed C paradigms, and even PureData objects trying to come up with non-garbage computer music in the 90's and thinking we needed precisely the kind of chip they're trying to engineer.
This, and the advent of HTMs and other non-ANN ways of going about it, mean that chips that handle distributed processing for applications that model human creativity (which is necessarily about concurrent time-based activities), are a -good- thing no matter what the degree of success, IMHO.
In fact, I remember programming ANNs in horribly non-distributed C paradigms, and even PureData objects trying to come up with non-garbage computer music in the 90's and thinking we needed precisely the kind of chip they're trying to engineer.
This, and the advent of HTMs and other non-ANN ways of going about it, mean that chips that handle distributed processing for applications that model human creativity (which is necessarily about concurrent time-based activities), are a -good- thing no matter what the degree of success, IMHO.
If you had just posted this without the first sentence, you would't have needed to make a throwaway.
and for comparison:
"the ultra-efficient human brain is estimated to have 100 billion neurons and at least 100 trillion synapses"
This is amazing. IBM is actually reaching towards the old dream of creating a computer that is as powerful as the human brain. I'm aware that hardware alone doesn't do it, but still damn impressive.