Moore’s Law Is About to Get Weird (2015)(nautil.us)
nautil.us
Moore’s Law Is About to Get Weird (2015)
http://nautil.us/issue/21/information/moores-law-is-about-to-get-weird
16 comments
Yes, almost certainly the next phase for performance-intensive computing is more specialized chips, systems-on-a-chip, and whole specialized rigs. The TensorFlow Processing Unit is a good recent example of this, as are ASICs in bitcoin mining, specialized hardware in molecular biology simulation, and more examples. This is just the beginning.
I think the interesting question is what happens after that. Not too long from now, specialized systems will run up against the same physical limits that general purpose processors have begun to hit the past few years as well, as specialized fabs increase in capability and sophistication. We see this already with GPUs.
I think the interesting question is what happens after that. Not too long from now, specialized systems will run up against the same physical limits that general purpose processors have begun to hit the past few years as well, as specialized fabs increase in capability and sophistication. We see this already with GPUs.
Specialization is essentially trading off additional hardware design work--and additional software complexity--for performance. When Moore's Law was going strong, it usually didn't make sense; just wait a couple years and the next x86 generation will probably more or less catch up to whatever performance gains your specialization was going to buy you.
I think it's pretty clear that we're going to see the introduction of all sorts of specialized hardware that we already more or less know how to do but it just hasn't been worth the trouble before. But you're also right that, so long as we're still talking CMOS, this is probably a more or less one-time boost that's maybe worth a few CMOS generations depending on the specific case.
I think it's pretty clear that we're going to see the introduction of all sorts of specialized hardware that we already more or less know how to do but it just hasn't been worth the trouble before. But you're also right that, so long as we're still talking CMOS, this is probably a more or less one-time boost that's maybe worth a few CMOS generations depending on the specific case.
"Nautilus uses cookies to manage your digital subscription and show you your reading progress. It's just not the same without them"
Well, no thanks. No, it is not the same.
Well, no thanks. No, it is not the same.
As usual, memristor based computing architecture is left out.
Intel's 3DXPoint has already brought memristor memory units to market. The beauty of memristor architectures is that they can be used for logic and storage, so you can create some radical new architectures with them.
I'm too lazy to link the talk by Stan Williams of HP, but if you google it it's the ~45min one.
This is the next architecture because it's already here. All that other stuff is vaporware.
Intel's 3DXPoint has already brought memristor memory units to market. The beauty of memristor architectures is that they can be used for logic and storage, so you can create some radical new architectures with them.
I'm too lazy to link the talk by Stan Williams of HP, but if you google it it's the ~45min one.
This is the next architecture because it's already here. All that other stuff is vaporware.
From the article: "However, ternary computing may be spurred on by a new device called the memristor. [...] Memristors might become one of the industry’s hopes for extending Moore’s Law beyond the age of the silicon transistor."
Where can I buy one?
How about the breakthrough in silicene production? [1]
Could we use this to start stacking transistors in the vertical direction?
[1] http://spectrum.ieee.org/nanoclast/semiconductors/materials/...
[1] http://spectrum.ieee.org/nanoclast/semiconductors/materials/...
That will only work for passive components like caches otherwise you will see higher power consumption and heat dissipation.
That's true, but there are other options. For instance, clockless computing is one avenue worth pursuing.
Well, none of those listed technologies is practical.
Ternary logic is really cool, but it only gets you about a 10% efficiency improvement. I think it would be awesome to go in this direction, but retooling our entire tech stack for that isn't going to happen. I mean, look at x86.
The only thing that will really make progress is molecular nanotechnology. I had always believed that reaching the end of silicon process technology would necessitate the research into MNT.
What's funny is that we have arguably already reached the end on the silicon process technology, if Intel's desktop product offerings are used as a guide. There are gamers who are building new rigs using chips that are 2 or 3 generations old, because there isn't really anything better available. What's really hilarious are the guys building systems with used Xeon CPUs from retired servers!
So I don't know what's going on any more. You would think that the profits and stock prices of all the silicon companies would collapse if they weren't able to continue producing improvements on their existing products. Intel is spending massively on R&D, but the gains are nowhere close to what we saw 15 years ago.
Note that keep referencing Intel, because they've been at the leading edge. AMD doesn't have 14nm in production, for example.
Ternary logic is really cool, but it only gets you about a 10% efficiency improvement. I think it would be awesome to go in this direction, but retooling our entire tech stack for that isn't going to happen. I mean, look at x86.
The only thing that will really make progress is molecular nanotechnology. I had always believed that reaching the end of silicon process technology would necessitate the research into MNT.
What's funny is that we have arguably already reached the end on the silicon process technology, if Intel's desktop product offerings are used as a guide. There are gamers who are building new rigs using chips that are 2 or 3 generations old, because there isn't really anything better available. What's really hilarious are the guys building systems with used Xeon CPUs from retired servers!
So I don't know what's going on any more. You would think that the profits and stock prices of all the silicon companies would collapse if they weren't able to continue producing improvements on their existing products. Intel is spending massively on R&D, but the gains are nowhere close to what we saw 15 years ago.
Note that keep referencing Intel, because they've been at the leading edge. AMD doesn't have 14nm in production, for example.
Ternary logic actually happened - a bunch of Soviet computers used it [1], and it was somewhat faster/more cost efficient, but not decisively so.
In the end, the party decided that code compatibility with the west was more important than independent innovation, and the soviet computer industry slowly lost it's charm and died in the late 80s.
Never heard of a ternary computer since then.
[1] http://www.computer-museum.ru/english/setun.htm
In the end, the party decided that code compatibility with the west was more important than independent innovation, and the soviet computer industry slowly lost it's charm and died in the late 80s.
Never heard of a ternary computer since then.
[1] http://www.computer-museum.ru/english/setun.htm
An acquaintance wrote his thesis about Setun and later (2007) published a book about it: http://www.irmielin.org/setun-an-inquiry-into-the-soviet-ter...
Note that keep referencing Intel, because they've been at the leading edge. AMD doesn't have 14nm in production, for example.
AMD actually just released their first 14 nm products this summer in the form of graphics cards. They don't have any CPUs out in such an advanced process but I think that reflects the architectural cul de sac they've been stuck in. They're next architecture, Zen, is supposed to be 14nm and it doesn't look like it makes business sense to port a Bulldozer derivative to 14nm before Zen is ready.
AMD actually just released their first 14 nm products this summer in the form of graphics cards. They don't have any CPUs out in such an advanced process but I think that reflects the architectural cul de sac they've been stuck in. They're next architecture, Zen, is supposed to be 14nm and it doesn't look like it makes business sense to port a Bulldozer derivative to 14nm before Zen is ready.
"retooling our entire tech stack for that isn't going to happen. I mean, look at x86."
I wonder if today it's easier than people imagine, but the lessons of Itanium are used as a signal to not try.
Sure there are plenty of apps which make low-level use of x86 features, and those probably aren't going to be updated. However a lot of modern software is written in languages that go through interrupters. Java, C#, Python.. if you can update the runtime, in theory the apps can be supported anywhere.
I wonder if today it's easier than people imagine, but the lessons of Itanium are used as a signal to not try.
Sure there are plenty of apps which make low-level use of x86 features, and those probably aren't going to be updated. However a lot of modern software is written in languages that go through interrupters. Java, C#, Python.. if you can update the runtime, in theory the apps can be supported anywhere.
Switching to ternary means retooling not just the programming stack but the IC design stack. IC design is actually quite a conservative business, mainly because failures are so expensive.
You'd need to spend several years writing a ternary RTL synthesis, place, route and verification suite before starting.
You'd need to spend several years writing a ternary RTL synthesis, place, route and verification suite before starting.
I would expect that once silicon stops shrinking and both chips and fabs can be allowed to pay for themselves over longer time horizons we're going to see a lot more architectural experimentation.
I hope you're right. The costs of 7nm are so high that even the "big guys" are being very careful about what they do. It's become so expensive they just can't afford a miss in the market. I worry the high costs will stifle innovation and that we'll see something that's more like a tweak here and there for a while.
Oh, for sure that will happen.
I like the Mill CPU design, and they're saying it will have a 10x power/performance advantage over traditional architectures.
But we're just about done with the times where we had easy gains in price/performance/power by just moving to a smaller process node.
I like the Mill CPU design, and they're saying it will have a 10x power/performance advantage over traditional architectures.
But we're just about done with the times where we had easy gains in price/performance/power by just moving to a smaller process node.
Also, how reliable is ternary compared to binary? For instance, would a Schmitt-trigger device be as reliable? What would the consequence be for clock speed?
if Intel has reasonable guesses as to what their competitor's level of R&D is at, they can stall the current gen tech for as long as possible, in order to milk the max profits off the capital investments.
Chip design and manufacturing isn't something a garage tinkerer can do (at the level of cutting edge research to compete with existing silicon makers). Therefore, companies like Intel can afford to lax until their customers demand differently (in which case, they will probably put the gas on R&D).
Chip design and manufacturing isn't something a garage tinkerer can do (at the level of cutting edge research to compete with existing silicon makers). Therefore, companies like Intel can afford to lax until their customers demand differently (in which case, they will probably put the gas on R&D).
Samsung is mass producing the A9 in 14nm. TSMC is mass producing in 16nm.
https://en.wikipedia.org/wiki/Apple_A9
Intel, Samsung and TSMC have all demonstrated their 10nm processes with TSMC expected to be the first to mass production. https://en.wikipedia.org/wiki/10_nanometer
I believe nVidia and AMD use Samsung and TSMC as their advanced fabs which is how they stay alive. They get to keep the expensive factories off their books and the factories can use their capacity to make more than just AMD and nVidia parts keeping their lines full. At least that's the theory...
Intel, Samsung and TSMC have all demonstrated their 10nm processes with TSMC expected to be the first to mass production. https://en.wikipedia.org/wiki/10_nanometer
I believe nVidia and AMD use Samsung and TSMC as their advanced fabs which is how they stay alive. They get to keep the expensive factories off their books and the factories can use their capacity to make more than just AMD and nVidia parts keeping their lines full. At least that's the theory...
I tried to fix that by submitting this:
https://news.ycombinator.com/item?id=12227555
What do you think about that advancement? Think the quantom dot circuits can get practical enough to go anywhere? Or even good potentially for low-volume, high-price applications we see with some S-ASIC's or ASIC's?
https://news.ycombinator.com/item?id=12227555
What do you think about that advancement? Think the quantom dot circuits can get practical enough to go anywhere? Or even good potentially for low-volume, high-price applications we see with some S-ASIC's or ASIC's?
I think it will be difficult to bring some of these technologies into mass production with subtractive manufacturing like lithography. We really need additive manufacturing techniques instead.
This is an absolutely terrible article, it seems the author just quickly threw together a list of alternative computing technologies and added a clickbaity title. Fluidics as a post Moore's law computing technology is laughable. The maximum possible processing speeds are somewhere in the 10s of KHz. In addition, fluidics does not scale down well. Slime molds are purely a gimmick.
However, fluidics was/still is a practical technology for niche applications. Fluidic non-moving part logic devices have been used for for factory process control and integrated pneumatic controllers for aerospace for decades now. Fluidics can be very robust and reliable. In the time period of the 1960s-1970s fluidics was more reliable than similar electrical and electromechanical systems. One thing that still makes fluidics attractive today is its robustness in environments where transistors don't work. Fluidics can with stand high temperatures, high shocks, high radiation, high magnetic fields, etc. As long as the shape of the channel remains intact, fluidics still works. Fluidics has been demonstrated to work inside jet engines, nuclear reactors, and molten steel.
Because of fluidics inherent EMP resistance, fluidics has been considered for use in anti-ballistic missile guidance systems[0].
One interesting current day application of fluidics is kosher sound amplification[1]. Orthodox Judaism prohibits electronic sound amplification on the sabbath because it constitutes work. Using a speaker to produce sound constitutes labor because it creates a sound that did not exist before from electricity. Fluidics is able to get around this prohibition and can amplify sound in the range of human speech.
As far as molecular nanotechnology goes, some of the first molecular computers we may see might resemble the BZ media mentioned in the article. Right now we can make a whole range of interesting molecular devices including logic gates and motors among other things, but it is difficult to put them together into larger devices. It's hard to hook up a molecular motor to a molecular wheel to a molecular suspension to make a molecular car[2], much less wire up a bunch of molecular logic gates in a defined pattern to make a molecular computer.
However, it is significantly easier to make periodic structures. Metal Organic Frameworks(MOFs) offer a way to organize molecular machines into periodic structures. They are pretty great for this because they self-assemble and have a bunch of open space for molecular machines to move around, but still close enough that they can interact with each other.
Some researchers have proposed that we could use MOFs with molecular machines in them to make cellular automata with mole quantities of cells.[3]
[0]http://www.dtic.mil/cgi-bin/GetTRDoc?Location=U2&doc=GetTRDo... [1]http://acoustics.org/pressroom/httpdocs/132nd/2aaa8.html [2]http://www.cemes.fr/Molecule-car-Race?lang=en [3]http://pubs.rsc.org/en/Content/ArticleLanding/2012/CS/c1cs15...
However, fluidics was/still is a practical technology for niche applications. Fluidic non-moving part logic devices have been used for for factory process control and integrated pneumatic controllers for aerospace for decades now. Fluidics can be very robust and reliable. In the time period of the 1960s-1970s fluidics was more reliable than similar electrical and electromechanical systems. One thing that still makes fluidics attractive today is its robustness in environments where transistors don't work. Fluidics can with stand high temperatures, high shocks, high radiation, high magnetic fields, etc. As long as the shape of the channel remains intact, fluidics still works. Fluidics has been demonstrated to work inside jet engines, nuclear reactors, and molten steel.
Because of fluidics inherent EMP resistance, fluidics has been considered for use in anti-ballistic missile guidance systems[0].
One interesting current day application of fluidics is kosher sound amplification[1]. Orthodox Judaism prohibits electronic sound amplification on the sabbath because it constitutes work. Using a speaker to produce sound constitutes labor because it creates a sound that did not exist before from electricity. Fluidics is able to get around this prohibition and can amplify sound in the range of human speech.
As far as molecular nanotechnology goes, some of the first molecular computers we may see might resemble the BZ media mentioned in the article. Right now we can make a whole range of interesting molecular devices including logic gates and motors among other things, but it is difficult to put them together into larger devices. It's hard to hook up a molecular motor to a molecular wheel to a molecular suspension to make a molecular car[2], much less wire up a bunch of molecular logic gates in a defined pattern to make a molecular computer.
However, it is significantly easier to make periodic structures. Metal Organic Frameworks(MOFs) offer a way to organize molecular machines into periodic structures. They are pretty great for this because they self-assemble and have a bunch of open space for molecular machines to move around, but still close enough that they can interact with each other.
Some researchers have proposed that we could use MOFs with molecular machines in them to make cellular automata with mole quantities of cells.[3]
[0]http://www.dtic.mil/cgi-bin/GetTRDoc?Location=U2&doc=GetTRDo... [1]http://acoustics.org/pressroom/httpdocs/132nd/2aaa8.html [2]http://www.cemes.fr/Molecule-car-Race?lang=en [3]http://pubs.rsc.org/en/Content/ArticleLanding/2012/CS/c1cs15...
If anything this necessitates advances in parallel computing. We can keep up with Moores law for a bit longer if we add more cores too.
Moore's law is about transistor density.
Traditionally the term has been mostly about speed rather than density. Well, a lot of people say it's about transistors per chip since Moore wrote a paper on that doubling and that was the first time people noticed periodic doubling in the semiconductor industry. But it took almost 10 years for "Moore's Law" to come into currency and in the mean time Moore had made lots of speeches about lots of different doubling. And the term "Moore's Law" has never really referred to one aspect of semiconductor technology in particular since during the era of Dennard scaling they all improved together.
You don't have to increase density to increase the number of transistors.
Yes, but:
1. That's not Moore's Law, which is specifically about density.
2. When you aren't getting density increases, your processing power / memory size / whatever increases linearly with die area. We've been used to living in an age with regular process node shrinks, meaning that more and more powerful computers don't end up being more expensive. Sometimes the cost goes down while the performance goes up. We're seeing the end of that right now.
1. That's not Moore's Law, which is specifically about density.
2. When you aren't getting density increases, your processing power / memory size / whatever increases linearly with die area. We've been used to living in an age with regular process node shrinks, meaning that more and more powerful computers don't end up being more expensive. Sometimes the cost goes down while the performance goes up. We're seeing the end of that right now.
Probably live things had to develop massively-parallel async processing and then things akin to "deep learning" and other thick neural nets because their wetware was (and is) ridiculously slow.
What about Quantum computing?
Quantum computing has nothing to do with what what we would consider "regular" computing. Even if it became practical, at most it would be a specialized accelerator for certain types of computation, like factoring numbers.
A decent explanation can be found here: https://uwaterloo.ca/institute-for-quantum-computing/quantum...
A decent explanation can be found here: https://uwaterloo.ca/institute-for-quantum-computing/quantum...
Quantum computing is still in the "researchers are playing with a handful of Qbits" stage, approximately where transistors where a hundred years ago. Scaling quantum computers is really hard. Even though Qbits can be encoded in single atoms, the machinery that keeps the quantum states stable and manipulates them is most definitely not easy to shrink.
It's also completely unclear whether quantum computers would make your Javascript engine run any faster.
It's also completely unclear whether quantum computers would make your Javascript engine run any faster.
My understanding is Quantum computing isn't the next generation of computing, it's more of a new type of computing. It's an improvement on a certain class of problems (such as optimization algorithms).
I believe the point of this article was to discuss the possibilities outside of, and up until, that realm.
One thought that isn't on here:
Once the process is "mature" and isn't changing every few years, the error rate could be squeezed way down. That would make it practical and economical to make larger chips. There are cooling and power issues too but I'm sure those are solvable.
Make way for the 64-bit 128-core 1024gb RAM 12cm^2 SoC?
Call it More's Law? Or a BFC? (for Big F'ing Chip?)
Once the process is "mature" and isn't changing every few years, the error rate could be squeezed way down. That would make it practical and economical to make larger chips. There are cooling and power issues too but I'm sure those are solvable.
Make way for the 64-bit 128-core 1024gb RAM 12cm^2 SoC?
Call it More's Law? Or a BFC? (for Big F'ing Chip?)
>Make way for the 64-bit 128-core 1024gb RAM 8cm^2 SoC
This is already happening?
Intel Xeon Knights Landing already has an enormous 6.83cm^2 die size. It has 72 "cores," each of which has a 512-bit vector processor. In some sense, you could say that this chip has 9,216 "single precision cores." It also supports up to 384GB DDR4 RAM, which is not far from your 1024GB.
The term "core" is becoming vague, as NVIDIA refers to thousands of "CUDA cores" on its GPUs which are really vector processor elements.
China's Sunway SW26010 is also kind of a "SoC," 260 "cores" communicating via a Network-on-Chip interconnect. I think one could say that the Cell Broadband Engine processor had a similar design.
> make it practical and economical to make larger chips
I do wonder if die sizes can increase.
I'm not a hardware engineer, but my understanding is that latency is the main barrier to larger chips. The time it takes for a signal to propagate across a chip limits the clock frequency. One possibility is to replace the global synchronous clock design with an asynchronous chip design. However, from what I understand, this would make things worse because some form of synchronization is always needed - you could send back an "acknolwedge" message, but that would double the signal traffic.
On the other hand, supercomputers are regularly built with multi-socket systems and interconnects like QPI and PCIe between processors and nodes. It would seem logical to put multiple processors their interconnects on a single "SoC" and I don't know why that doesn't happen. Maybe it is because of error rates, as you say.
This is already happening?
Intel Xeon Knights Landing already has an enormous 6.83cm^2 die size. It has 72 "cores," each of which has a 512-bit vector processor. In some sense, you could say that this chip has 9,216 "single precision cores." It also supports up to 384GB DDR4 RAM, which is not far from your 1024GB.
The term "core" is becoming vague, as NVIDIA refers to thousands of "CUDA cores" on its GPUs which are really vector processor elements.
China's Sunway SW26010 is also kind of a "SoC," 260 "cores" communicating via a Network-on-Chip interconnect. I think one could say that the Cell Broadband Engine processor had a similar design.
> make it practical and economical to make larger chips
I do wonder if die sizes can increase.
I'm not a hardware engineer, but my understanding is that latency is the main barrier to larger chips. The time it takes for a signal to propagate across a chip limits the clock frequency. One possibility is to replace the global synchronous clock design with an asynchronous chip design. However, from what I understand, this would make things worse because some form of synchronization is always needed - you could send back an "acknolwedge" message, but that would double the signal traffic.
On the other hand, supercomputers are regularly built with multi-socket systems and interconnects like QPI and PCIe between processors and nodes. It would seem logical to put multiple processors their interconnects on a single "SoC" and I don't know why that doesn't happen. Maybe it is because of error rates, as you say.
Asynchronous does not help you on latency at all - it's not the latency of the clock that matters. It's the data latency. This is already kind of crippling, as it takes hundreds of cycles to get anything from RAM if it's not in your cache.
Only way to improve that is to make programming languages push either data locality or dataflow like designs, which may mean giving up or reimagining OO.
Error rates (yield) are a serious problem, but if your design is uniform enough you can very well get away with just disabling bits. There's also a less obvious problem with on-chip variation (OCV) - some physical properties have a gradient across the chip. At runtime, there's a temperature gradient as well.
Only way to improve that is to make programming languages push either data locality or dataflow like designs, which may mean giving up or reimagining OO.
Error rates (yield) are a serious problem, but if your design is uniform enough you can very well get away with just disabling bits. There's also a less obvious problem with on-chip variation (OCV) - some physical properties have a gradient across the chip. At runtime, there's a temperature gradient as well.
Or... continuing with the BFC concept... make chips with monster caches or even with all SRAM running at core speed. That's not been done in a while.
Caches effectively are SRAM. It's inherently much less dense than DRAM; I had a quick look on Intel and the largest I could see was 37Mb of cache.
It's not SRAM, but Kights Landing has 16GB of "MCDRAM" which is high bandwidth 3D-stacked DRAM. 400+GB/s throughput but latency is similar to conventional DRAM.
Latency is not a problem. The recent huge chips from Intel serve as a perfectly good example. Haswell puts the cores and IO controllers on 1-2 ring busses. Knights Landing uses a mesh network. In each cycle, data only has to travel between neighboring nodes. A memory controller that's 4 hops away is barely noticeable.
[deleted]
It's a lot more practical to put several dies insides one package, which is already happening (for example, the largest FPGAs from Xilinx have 4 dies in a single package).
Intra-package communication is not quite as good as intra-die communication (from a performance and energy point of view), but it's a lot better than inter-package communication, and several small dies have a higher yield than one large die.
From the article:
> "...Moore’s Law, which states that the number of transistors that can be squeezed onto a semiconductor chip of a given size doubles roughly every two years, has held true since the mid 1960s..."
I thought it was the case that Moore's "Law" hasn't held up since ~2012.
I believe Intel and others have stated that as of 2012 or so, with 22nm processes, transistor density now doubles every ~2.5 years and is likely to slow further within 5 years without more fundamental breakthroughs.
The related trend that processor clock speeds doubled every 1.5-2 years-- which is commonly mistaken as Moore's Law in lay press but is actually more related to Dennard Scaling, Koomey's Law and historical statements by Intel-- had ceased to hold as of the late Aughts as well, I believe.
Memsistors have been a promise for some time. I wonder how we will make the leap from current dense-transistor IC tech and processes to anything new, given the huge infrastructure investments in fabs, quality control processes, etc. It seems it will be difficult to scale out a new technology, whatever it is, and get it price-competitive with existing transistor-dense IC technology. Perhaps I misunderstand here though, and would be interested in thoughts from those with greater expertise in the field.
> "...Moore’s Law, which states that the number of transistors that can be squeezed onto a semiconductor chip of a given size doubles roughly every two years, has held true since the mid 1960s..."
I thought it was the case that Moore's "Law" hasn't held up since ~2012.
I believe Intel and others have stated that as of 2012 or so, with 22nm processes, transistor density now doubles every ~2.5 years and is likely to slow further within 5 years without more fundamental breakthroughs.
The related trend that processor clock speeds doubled every 1.5-2 years-- which is commonly mistaken as Moore's Law in lay press but is actually more related to Dennard Scaling, Koomey's Law and historical statements by Intel-- had ceased to hold as of the late Aughts as well, I believe.
Memsistors have been a promise for some time. I wonder how we will make the leap from current dense-transistor IC tech and processes to anything new, given the huge infrastructure investments in fabs, quality control processes, etc. It seems it will be difficult to scale out a new technology, whatever it is, and get it price-competitive with existing transistor-dense IC technology. Perhaps I misunderstand here though, and would be interested in thoughts from those with greater expertise in the field.
>I thought it was the case that Moore's "Law" hasn't held up since ~2012.
I thought it was before that, like 1999. I pretty sure I had an Intel Pentium III 2 GHz processor in 99. The computer I am using to type this comment is 2.5 GHz. More's law is long dead, we aren't going to see 12 GHz machines in the next 6 years.
I thought it was before that, like 1999. I pretty sure I had an Intel Pentium III 2 GHz processor in 99. The computer I am using to type this comment is 2.5 GHz. More's law is long dead, we aren't going to see 12 GHz machines in the next 6 years.
And the 2.5GHz chip you are using now n years later has (very) approximately 2^n more transistors - think about the increase in the number of cores, the large number of different execution units, the increased width of the data busses, that vast amount of on-die cache and so on. These all increase the transistor count, but feature size is also getting smaller, allowing the total transistor density to increase in line with Moore's law.
Moore's Law has nothing to do with clock speed.
Aside from the obvious problems in your comparison. Moore's Law says nothing about frequency. The "rule" that changed around 2006 was Dennard scaling.
https://en.wikipedia.org/wiki/Dennard_scaling#Breakdown_of_D...
https://en.wikipedia.org/wiki/Dennard_scaling#Breakdown_of_D...
One interesting avenue of research is embedded reconfigurable computing technologies like eMIPS [0]. Unlike other reconfigurable computing approaches that simply place a FPGA side-by-side with a CPU in the same package to act as a co-processor [1], eMIPS integrates reconfigurable logic directly into the processing pipeline, allowing the creation of custom processor instructions on the fly on a per app basis to accelerate apps. Pretty cool stuff.
[0] https://www.microsoft.com/en-us/research/project/emips/
[1] http://www.theregister.co.uk/2016/03/14/intel_xeon_fpga/
[0] https://www.microsoft.com/en-us/research/project/emips/
[1] http://www.theregister.co.uk/2016/03/14/intel_xeon_fpga/
Adamatzky seems to be churning out cookie cutter articles on slime molds and highways
https://scholar.google.com/scholar?q=Adamatzky+highways&btnG...
https://scholar.google.com/scholar?q=Adamatzky+highways&btnG...
For all the interesting things happening today, this is an incredibly dull article. I'm convinced I read this same article in the 90's.
If we're going to see any developments in alternative architectures and hardware, it's going to happen in the cloud computing space. From my perspective, this is the only sector that is in the unique position to sell specialized computing services to customers on a mass scale and make great margins in the process.
There are many interesting and useful things you can do outside of Von Neumann architectures and Turing Complete systems that were never practical on PCs. Simple services like Reddis could probably be implemented with alternative memory architectures like content-addressable memory. Many useful algorithms in the data analytic and machine learning space can be compiled down the specialized hardware using using a combination of transistors and memristors.
For companies like Amazon, Microsoft + Google, there's actually a financial incentive to harmonize hardware and software much in the same way Seymour Cray harmonized the software and hardware of Cray's supercomputers. These cloud companies don't want to be constrained to selling commodity virtual machines. They want to sell you the next generation PaaS solution like DynamoDB, SQL Azure or Firebase so they can lock you into their cloud platform.
If there's a future where developers are writing code in special DSLs that's compiled to gates on an FPGA, the cloud computing guys will be exploring it because they have a huge incentive to get their customer's into the walled garden that yields better margins. If you can create one platform service that developers love and runs on cheaper specialized hardware, you'll be able to destroy your competitors and signal an alternative hardware arms race.
If we're going to see any developments in alternative architectures and hardware, it's going to happen in the cloud computing space. From my perspective, this is the only sector that is in the unique position to sell specialized computing services to customers on a mass scale and make great margins in the process.
There are many interesting and useful things you can do outside of Von Neumann architectures and Turing Complete systems that were never practical on PCs. Simple services like Reddis could probably be implemented with alternative memory architectures like content-addressable memory. Many useful algorithms in the data analytic and machine learning space can be compiled down the specialized hardware using using a combination of transistors and memristors.
For companies like Amazon, Microsoft + Google, there's actually a financial incentive to harmonize hardware and software much in the same way Seymour Cray harmonized the software and hardware of Cray's supercomputers. These cloud companies don't want to be constrained to selling commodity virtual machines. They want to sell you the next generation PaaS solution like DynamoDB, SQL Azure or Firebase so they can lock you into their cloud platform.
If there's a future where developers are writing code in special DSLs that's compiled to gates on an FPGA, the cloud computing guys will be exploring it because they have a huge incentive to get their customer's into the walled garden that yields better margins. If you can create one platform service that developers love and runs on cheaper specialized hardware, you'll be able to destroy your competitors and signal an alternative hardware arms race.
I think you are right on, on every point. (Including it being a dull article, at least for the audience of folks here.) This is great insight and it's something I've thought for the past few years as well.
Big cloud computing players will almost certainly move this way. Actually surprised we haven't seen more of this already. But TensorFlow and the Tensor Processing Unit were smart moves by Google and I think this is just a preview of many, many things to come.
I think IBM probably belongs on the list of top contenders to make a move. Between SoftLayer, IBM's hardware and chip design and fab partnerships, their cloud/saas businesses, and their Enterprise penetration, they're well-positioned to lead here.
I think that you're right as well that all these folks will push to create developer-friendly services and APIs backed by hardware. However, I think the real wins will come from capturing market share in the Enterprise sooner than later. If I were them and I had to prioritize between the two, I'd target Enterprise customers first and foremost. You can sell services to help them migrate and that helps you get out to market more quickly than creating a fully robust self-serve platform. Moreover, if your custom hw-backed platform can offer either better price or better performance, Enterprises will pay big bucks for it. Harder to get it right with self-serve first, but you certainly need to open up to self-serve as quickly as possible over time.
Big cloud computing players will almost certainly move this way. Actually surprised we haven't seen more of this already. But TensorFlow and the Tensor Processing Unit were smart moves by Google and I think this is just a preview of many, many things to come.
I think IBM probably belongs on the list of top contenders to make a move. Between SoftLayer, IBM's hardware and chip design and fab partnerships, their cloud/saas businesses, and their Enterprise penetration, they're well-positioned to lead here.
I think that you're right as well that all these folks will push to create developer-friendly services and APIs backed by hardware. However, I think the real wins will come from capturing market share in the Enterprise sooner than later. If I were them and I had to prioritize between the two, I'd target Enterprise customers first and foremost. You can sell services to help them migrate and that helps you get out to market more quickly than creating a fully robust self-serve platform. Moreover, if your custom hw-backed platform can offer either better price or better performance, Enterprises will pay big bucks for it. Harder to get it right with self-serve first, but you certainly need to open up to self-serve as quickly as possible over time.
Maybe the cloud computer guys will go there, but the high performance computing people are already well settled. Anton [1], a machine running custom ASICs for molecular dynamics, has already produced some very interesting biomolecular simulations. MDGRAPE3 was even earlier (completed in 2006).
There was also a (sadly failed) initiative about ten years back in Germany to build a custom ASIC for multigrid Poisson equation solvers. I'm sure there are more examples outside of my interests.
[1] https://en.m.wikipedia.org/wiki/Anton_(computer)
There was also a (sadly failed) initiative about ten years back in Germany to build a custom ASIC for multigrid Poisson equation solvers. I'm sure there are more examples outside of my interests.
[1] https://en.m.wikipedia.org/wiki/Anton_(computer)
> If you can create one platform service that developers love and runs on cheaper specialized hardware, you'll be able to destroy your competitors and signal an alternative hardware arms race.
I think this was what Google is trying with their TPU for running TensorFlow for machine learning applications.
https://cloudplatform.googleblog.com/2016/05/Google-supercha...
I think this was what Google is trying with their TPU for running TensorFlow for machine learning applications.
https://cloudplatform.googleblog.com/2016/05/Google-supercha...
I feel bad for calling this a dull article. This is actually a decent and eye opening article that introduces people to unconventional computing. For middle-aged people who have a cursory interest in the direction of alternative computing, it reads like a hyped Popular Science article from the 90's.
This is one of the most insightful things about Moore's Law I've read: https://www.quora.com/How-long-will-Moores-Law-continue-to-a...
Moore's Law is a limit that ultimately arises from economics. The costs to keep Moore's Law going are exponential (just like the gain), and fewer and fewer companies are willing to pay the increasingly high price.
I was always primed to believe the exponential gains were essentially "free" and came with a fixed amount of investment into the industry. Understanding that this isn't true explains why Moore's Law will end unless demand is exponentially increasing.
Moore's Law is a limit that ultimately arises from economics. The costs to keep Moore's Law going are exponential (just like the gain), and fewer and fewer companies are willing to pay the increasingly high price.
I was always primed to believe the exponential gains were essentially "free" and came with a fixed amount of investment into the industry. Understanding that this isn't true explains why Moore's Law will end unless demand is exponentially increasing.
Moore's Law seems to reach its limit only for CPU clock. But it still moving very fast for GPU, FPGA, DDR, SSD, etc, right?
Or even in term of Network connection speed - 100mbps, 1G, 10G, 25G, 40G (at lease in data center where it matters a lot.)
The simple usb connection speed also continue to increase at a fast rate.
Or even in term of Network connection speed - 100mbps, 1G, 10G, 25G, 40G (at lease in data center where it matters a lot.)
The simple usb connection speed also continue to increase at a fast rate.
In the short term I really think we are entering the era of accelerators. Accelerators like Micron's Automata Processor (1) for graph analysis, accelerators for compute intensive applications like Convolutional Neural Nets, continued innovation in DSPs, GPUs, etc.
Accelerators that can either outperform traditional processors or use significantly less power I think represent one of the major next steps after Moore's law. What comes next may well be a slime mold computer but I think at this point is pure speculation.
(1) - http://www.micronautomata.com - actually I heard about this on HN!