Ask HN: Is a distcc style plugin for professional workgroups an idea whose time has come?
4 comments
Mac OS X has this baked into Xcode, as well as general grid computing capabilities. This implemented with distcc and Bonjour (zeroconf), and Xgrid, respectively.
Linux, too, can implement this.
Operating systems as far back was Windows' arguable predecessor OpenVMS have distributed batch capabilities with clustering and with distributed batch queues, and you can toss build-related jobs into queues. Nowhere near distcc, but it works.
If nobody's already gotten traction with this distributed environment within the Visual tools over on Microsoft Windows, well, have at. Microsoft and its related business decisions would centrally affect your future here. Ignore, buy, trump, etc.
Linux, too, can implement this.
Operating systems as far back was Windows' arguable predecessor OpenVMS have distributed batch capabilities with clustering and with distributed batch queues, and you can toss build-related jobs into queues. Nowhere near distcc, but it works.
If nobody's already gotten traction with this distributed environment within the Visual tools over on Microsoft Windows, well, have at. Microsoft and its related business decisions would centrally affect your future here. Ignore, buy, trump, etc.
Two things to read about:
1. Electric Cloud (which is a company I started) that does parallel compilation (amongst other things) for mostly very large teams.
2. Xoreax which does something like your idea. Typically for smaller teams.
1. Electric Cloud (which is a company I started) that does parallel compilation (amongst other things) for mostly very large teams.
2. Xoreax which does something like your idea. Typically for smaller teams.
The plugin should be smart about avoiding the user noticing that other users are compiling software on their machine. Locking itself to a single processor is a good start, and refusing jobs if the user on the local machine is compiling code or doing some other processor intensive activity.
The plugin should do this? How about letting the OS do it, that's the whole point of an OS after all :) Just nice down the distcc thing and let the OS do the rest. (Hint: the OS' algorithm is probably better than the one you proposed, so why reimplement it?)
The plugin should do this? How about letting the OS do it, that's the whole point of an OS after all :) Just nice down the distcc thing and let the OS do the rest. (Hint: the OS' algorithm is probably better than the one you proposed, so why reimplement it?)
I've never used it, but I hear that IncrediBuild does exactly that:
http://www.xoreax.com/main.htm
http://www.xoreax.com/main.htm
It occurs to me that in recent months, dual and quad core CPUs are now becoming more or less the default choice for new workstations. Typically, they'll also come with 2 or more GB of RAM. While I don't have load averages to back this up, it's my guess that the average developers CPU sits almost idle about 95% of the time. When they're writing code, browsing documentation, using the source control system or bug tracker or just at lunch or chatting idly.
This means that in a typical team consisting of some 10 or 12 developers, and supporting staff such as managers, architects, integration coordinators, etc. etc. You have thousands of processor cycles a second going to waste. Typically, all these workstations will be networked together on a single wired LAN segment that's considerably underutilised.
I think it'd be great if someone makes a decent distcc style plugin for some common dev environments like VC++, or Java/Eclipse. The plugin ought to advertise itself via zeroconf networking, and automatically utilise as much spare capacity on the network as can be found. The plugin should be smart about avoiding the user noticing that other users are compiling software on their machine. Locking itself to a single processor is a good start, and refusing jobs if the user on the local machine is compiling code or doing some other processor intensive activity.
Does this sound like a good idea to anyone else? In some of my past jobs, something like this would have saved me whole hours a week.