Systems Software Research is Irrelevant by Rob Pike (2000)(doc.cat-v.org)
doc.cat-v.org
Systems Software Research is Irrelevant by Rob Pike (2000)
http://doc.cat-v.org/bell_labs/utah2000/
5 comments
There is a reason Plan 9 dumped PoSix, there is APE ( http://doc.cat-v.org/plan_9/4th_edition/papers/ape ) but to properly maintain it would take way more effort than the rest of the OS.
And it is becoming even worse, 'all the world is a vax' has become 'all cpus are x86(-64), all the kernels are linux, all the libcs are glibc, and all the compilers are gcc', portability of most software is awful, and auto*hell only makes things much worse.
And it is becoming even worse, 'all the world is a vax' has become 'all cpus are x86(-64), all the kernels are linux, all the libcs are glibc, and all the compilers are gcc', portability of most software is awful, and auto*hell only makes things much worse.
On the other hand, in that world at least you only have the broken insanity of one operating system/library stack to worry about instead of three or four.
``Not Linux'' is not the same as 'perfect', no matter what the *BSD people say.
``Not Linux'' is not the same as 'perfect', no matter what the *BSD people say.
Written August 5th, 2000
"Irrelevant: Does not influence industry"
Two years later came Xen, a systems research project that has definitely influenced industry.
"Irrelevant: Does not influence industry"
Two years later came Xen, a systems research project that has definitely influenced industry.
And two years earlier had come VMware.
The explosion of interest in virtualization was started by VMware, a commercial entity, which rather makes Pike's point. Xen was started in response to the lack of an academia-friendly platform for virtualization research; "paravirtualization" was not initially a design goal, but rather an expedient choice to avoid the difficulties of correctly virtualizing the x86.
And while I'm biased, VMware's solution to this problem (dynamic binary translation from supervisor x86 to user-level x86) is massively more inventive, interesting and useful than Xen's solution (hack up the kernel). I emphasize "useful", because a lot of VMware's customers were most interested in running Windows, which was not paravirtulization-ready until 2008, a full ten years after VMware's products were available.
The explosion of interest in virtualization was started by VMware, a commercial entity, which rather makes Pike's point. Xen was started in response to the lack of an academia-friendly platform for virtualization research; "paravirtualization" was not initially a design goal, but rather an expedient choice to avoid the difficulties of correctly virtualizing the x86.
And while I'm biased, VMware's solution to this problem (dynamic binary translation from supervisor x86 to user-level x86) is massively more inventive, interesting and useful than Xen's solution (hack up the kernel). I emphasize "useful", because a lot of VMware's customers were most interested in running Windows, which was not paravirtulization-ready until 2008, a full ten years after VMware's products were available.
VMware's awesome... but its roots are also in academia, the company was started by a Stanford professor and his PhD students.
It seems like a circular argument if the criteria of an academic project "successfully influencing industry" is widespread adoption but that adoption cannot be facilitated by a commercial entity.
[As a side note -- the impetus for Xen, and also somewhat for VMware, was in a vision of global appliance-based computing, not in "virtualization research." I think that is really interesting given what just happened in this area in the last 3 or 4 years.]
It seems like a circular argument if the criteria of an academic project "successfully influencing industry" is widespread adoption but that adoption cannot be facilitated by a commercial entity.
[As a side note -- the impetus for Xen, and also somewhat for VMware, was in a vision of global appliance-based computing, not in "virtualization research." I think that is really interesting given what just happened in this area in the last 3 or 4 years.]
[deleted]
http://news.ycombinator.com/item?id=245227
EDIT: Here's Pike's update: http://interviews.slashdot.org/article.pl?sid=04/10/18/11532...
EDIT: Here's Pike's update: http://interviews.slashdot.org/article.pl?sid=04/10/18/11532...
The following passage from that second piece is fascinating (I had never heard of the "Unix room"... it sounds like an illustration of Conway's Law, and a lot of fun too).
One odd detail that I think was vital to how the group functioned was a result of the first Unix being run on a clunky minicomputer with terminals in the machine room. People working on the system congregated in the room - to use the computer, you pretty much had to be there. (This idea didn't seem odd back then; it was a natural evolution of the old hour-at-a-time way of booking machines like the IBM 7090.) The folks liked working that way, so when the machine was moved to a different room from the terminals, even when it was possible to connect from your private office, there was still a `Unix room' with a bunch of terminals where people would congregate, code, design, and just hang out. (The coffee machine was there too.) The Unix room still exists, and it may be the greatest cultural reason for the success of Unix as a technology. More groups could profit from its lesson, but it's really hard to add a Unix-room-like space to an existing organization. You need the culture to encourage people not to hide in their offices, you need a way of using systems that makes a public machine a viable place to work - typically by storing the data somewhere other than the 'desktop' - and you need people like Ken and Dennis (and Brian Kernighan and Doug McIlroy and Mike Lesk and Stu Feldman and Greg Chesson and ...) hanging out in the room, but if you can make it work, it's magical.
One odd detail that I think was vital to how the group functioned was a result of the first Unix being run on a clunky minicomputer with terminals in the machine room. People working on the system congregated in the room - to use the computer, you pretty much had to be there. (This idea didn't seem odd back then; it was a natural evolution of the old hour-at-a-time way of booking machines like the IBM 7090.) The folks liked working that way, so when the machine was moved to a different room from the terminals, even when it was possible to connect from your private office, there was still a `Unix room' with a bunch of terminals where people would congregate, code, design, and just hang out. (The coffee machine was there too.) The Unix room still exists, and it may be the greatest cultural reason for the success of Unix as a technology. More groups could profit from its lesson, but it's really hard to add a Unix-room-like space to an existing organization. You need the culture to encourage people not to hide in their offices, you need a way of using systems that makes a public machine a viable place to work - typically by storing the data somewhere other than the 'desktop' - and you need people like Ken and Dennis (and Brian Kernighan and Doug McIlroy and Mike Lesk and Stu Feldman and Greg Chesson and ...) hanging out in the room, but if you can make it work, it's magical.
Thank you for posting this.
OK, to play devil's advocate, Xen is nothing new in the same way that Pike claims Linux is nothing new.
Hypervisors and virutalization have been around since the late '60s.
Hypervisors and virutalization have been around since the late '60s.
Sure, I agree with you, there was no "huge paradigm shift" (that is so extremely rare in CS).
But there was a virtualization efficiency advance that got an impressive number of companies to actively jump into the project (with time and money) which (along with it being free) made it explode, mature, and eventually make things like commodity computing rental via EC2 possible to implement just years later.
I agree with him that there is so much established infrastructure/standardization out there that doing something completely game changing is hard if not impossible. Which is why I think weighing things against such a behemoth industry, Xen's story is impressive.
It did what he wanted: "Make the industry want your work." But it's because it is useful, not just because it is cool. As for grant money's role in all this, it's the same issue. Like arts and humanities vs. energy/science, the lion's share of research funding is always going to go towards what will most likely advance the country's economy/technology.
But there was a virtualization efficiency advance that got an impressive number of companies to actively jump into the project (with time and money) which (along with it being free) made it explode, mature, and eventually make things like commodity computing rental via EC2 possible to implement just years later.
I agree with him that there is so much established infrastructure/standardization out there that doing something completely game changing is hard if not impossible. Which is why I think weighing things against such a behemoth industry, Xen's story is impressive.
It did what he wanted: "Make the industry want your work." But it's because it is useful, not just because it is cool. As for grant money's role in all this, it's the same issue. Like arts and humanities vs. energy/science, the lion's share of research funding is always going to go towards what will most likely advance the country's economy/technology.
> because it is useful, not just because it is cool
Seems like Pike's own opinions migrated towards an acceptance of this, from jackchristopher's slashdot link [1]:
"Applications - web browsers, MP3 players, games, all that jazz - and networks are where the action is today, and aside from irritating little incompatibilities, the kernel has become a commodity. Almost all the programs I care about can run above Windows, Unix, Plan 9, and on PCs, Macs, palmtops and more. And that, of course, is why these all have a POSIX interface: so they can support those applications."
[1] - http://interviews.slashdot.org/article.pl?sid=04/10/18/11532...
Seems like Pike's own opinions migrated towards an acceptance of this, from jackchristopher's slashdot link [1]:
"Applications - web browsers, MP3 players, games, all that jazz - and networks are where the action is today, and aside from irritating little incompatibilities, the kernel has become a commodity. Almost all the programs I care about can run above Windows, Unix, Plan 9, and on PCs, Macs, palmtops and more. And that, of course, is why these all have a POSIX interface: so they can support those applications."
[1] - http://interviews.slashdot.org/article.pl?sid=04/10/18/11532...
> But there was a virtualization efficiency advance
You are just reinforcing Rob's point. Xen doesn't do anything new, it is simply an optimization, that doesn't mean it is not useful (as he also points out), but confirms his point that all 'research' this days is just optimizations and 'phenomenology', and nothing that is and feels truly different and new.
In the end Xen is a textbook illustration of the problems he is pointing at.
You are just reinforcing Rob's point. Xen doesn't do anything new, it is simply an optimization, that doesn't mean it is not useful (as he also points out), but confirms his point that all 'research' this days is just optimizations and 'phenomenology', and nothing that is and feels truly different and new.
In the end Xen is a textbook illustration of the problems he is pointing at.
People across the globe are doing something that is and feels truly different and new: they're thinking and arranging their entire stacks in terms of remote virtual machines.
We could argue about the definition of novelty all day and in the end I think it's simply a judgement call what one deems as exciting or worth talking about. If you're looking for something so profoundly different along the lines of Einstein and Heisenberg in physics, I agree with him through and through, nothing in systems will probably happen like that given the current funding paradigms (I can't actually think of anything like that in the history of computer science -- you can for example be as reductionist as you want if you want to take Pike's side of the argument: "oh, it's all just a collection of logic gates"...).
We could argue about the definition of novelty all day and in the end I think it's simply a judgement call what one deems as exciting or worth talking about. If you're looking for something so profoundly different along the lines of Einstein and Heisenberg in physics, I agree with him through and through, nothing in systems will probably happen like that given the current funding paradigms (I can't actually think of anything like that in the history of computer science -- you can for example be as reductionist as you want if you want to take Pike's side of the argument: "oh, it's all just a collection of logic gates"...).
> People across the globe are doing something that is and feels truly different and new: they're thinking and arranging their entire stacks in terms of remote virtual machines.
Except that there had been people doing precisely that for ages. Seems that computer science is condemned to eternally reinvent wheels... sigh
Except that there had been people doing precisely that for ages. Seems that computer science is condemned to eternally reinvent wheels... sigh
Well I guess your definition of "precisely" must be much different than mine.
Well, in the seventies IBM mainframes had the Control Program (CP) and what is now the z/VM hypervisor, so yes precisely the same paradigm was dominant in architecting systems back then.
Sure, squint hard enough and it's the "precisely the same." Being in a virtualization/utility-computing job for the last five years, I've heard this kind of quip over and over and I agree with it on the surface but it's true only in the most general ways.
I guess I was not specific enough above with simply saying "remote virtual machines." With a credit card almost anyone can rent thousands of machines programatically (with redundancy across multiple datacenters if you want) and be using them in minutes. And make them go away just as quickly. And construct the VMs themselves programatically or by hand on a laptop. And the people renting out the hardware don't need to even trust you. All of this is a new development that changes the way business is done. Like I said above, you can say everything is "just" logic gates plus electricity and claim nothing new ever happens (and I'll agree with you, too, I just won't think the conversation is going anywhere interesting).
I guess I was not specific enough above with simply saying "remote virtual machines." With a credit card almost anyone can rent thousands of machines programatically (with redundancy across multiple datacenters if you want) and be using them in minutes. And make them go away just as quickly. And construct the VMs themselves programatically or by hand on a laptop. And the people renting out the hardware don't need to even trust you. All of this is a new development that changes the way business is done. Like I said above, you can say everything is "just" logic gates plus electricity and claim nothing new ever happens (and I'll agree with you, too, I just won't think the conversation is going anywhere interesting).
[deleted]
Agreed. My research: http://v3vee.org/. I don't even know what to say about this article - it's just so poorly thought out and researched that I can't believe it.
> I don't even know what to say about this article - it's just so poorly thought out and researched that I can't believe it.
You are clearly missing the point by a few miles...
You are clearly missing the point by a few miles...
Yeah. I see the date now. Ignore that.
In that case, you're just wrong. Look at what has come out of systems research: Xen and its ilk created a huge jump in virtualization research. Microsoft and VMWare now have large paravirtualization as parts of their virtual machines, which we are seeing more and more uses for every day.
There are a number of other things that come to mind, mostly in high performance computing. Parallel FORTRAN is one of the most common HPC languages and I believe the primary development was based on work at Rice university.
The problem is that people too often get caught up on implementation: most academic departments do not develop "products," they work to develop ideas. Many of these ideas have been built on recently to produce successful products.
There are a number of other things that come to mind, mostly in high performance computing. Parallel FORTRAN is one of the most common HPC languages and I believe the primary development was based on work at Rice university.
The problem is that people too often get caught up on implementation: most academic departments do not develop "products," they work to develop ideas. Many of these ideas have been built on recently to produce successful products.
The date has nothing to do with it, the kind and nature of the research is the issue.
[deleted]
If Xen is the best you can come up as a result of research impacting industry in the last ten years... IBM had been doing basically the same for decades, so I'm not sure one could consider Xen particularly revolutionary.
Edit: Also note that Xen in no way fulfills the definition of Systems Research, it is simply a new minor twist on a very old idea.
Edit: Also note that Xen in no way fulfills the definition of Systems Research, it is simply a new minor twist on a very old idea.
I'm wondering if it's time for a revival of the capability-based OS's of the 70's, such as Hydra/C.mmp, CMU's flagship OS of the time.
Seems like the raw performance is there, and the CPU parallelism is there. Certainly the time is ripe for a more secure basis for operating systems, and capabilities still seem like one of the best theoretical foundations around.
Seems like the raw performance is there, and the CPU parallelism is there. Certainly the time is ripe for a more secure basis for operating systems, and capabilities still seem like one of the best theoretical foundations around.
Maybe someone should inform SOSP. By the way guys virtualization is hardly the only thing going on in systems research. As far as the Web replacing the OS... Honestly if HTTP/HTML is the sole future of innovation in computing then that is a bit sad. As for Cloud Computing, well, let's say the reports of the demise of systems research have been greatly, greatly exagerrated. Actually, looking at my dept this just seems to be a pitifully naive view.there are plenty of projects and not the time nor enough grad students to do them all.
New OS stagnation could be showing the trend away from the OS. A return to the terminal? Cloud computing could be the first new step of many. In many ways we may no longer need the local OS.
This, of course, has to be done. Otherwise there will be a whole bunch of software future Haiku users will miss out on, and a whole bunch of trivial software the developers would have to rewrite from scratch. Oh, and then it would take way too much effort to port Perl, Python, Ruby, Vim ...
These days, "portable software" pretty much means "targets POSIX".