IBM's infamous "Black Team"(t3.org)
t3.org
IBM's infamous "Black Team"
http://www.t3.org/tangledwebs/07/tw0706.html#
15 comments
Does anyone have a link to the version where the Black Team member found a bug in rigourously (mathematically) proven code? edit Ah, here it is: http://www.penzba.co.uk/GreybeardStories/TheBlackTeam.html
That's almost exactly one of the parables between Achilles and the Tortoise that Hofstadter gives in Godel, Escher, Bach, where the Tortoise keeps breaking Achilles' record player by playing records of the machine's resonant frequency. Hofstadter gives it as an example of the 2nd Incompleteness Theorem, that in any mathematical system there are always statements that can neither be proven true nor false using the system's axioms (although I'm not fully convinced this resonant frequency example is actually an example of Godel's theorem).
In any case, that's funny, I bet Hofstadter based the example on this story of the Black Team.
In any case, that's funny, I bet Hofstadter based the example on this story of the Black Team.
While I don't know the veracity of that particular story (and the "in front of the press" bit seems really unlikely), the old washing-machine-sized drives could indeed walk across a room due to seeking, at least according to the Jargon File: http://www.catb.org/jargon/html/W/walking-drives.html .
It's a law of nature that you can build up a lot of energy in something by adding just a tiny bit of energy each pass. This was a bitter team that was beaten and decided to use physics to add this tiny bit of energy each time the tape rewound--mind you, at a steady rate chosen to add in the energy perfectly-- to embarrass somebody who had beaten them.
Or is my understanding of a tape drive incorrect, and they would wind and unwind at a set rate? Because the original article said that it would often stop, rewind, move forward to retrieve data.
Or is my understanding of a tape drive incorrect, and they would wind and unwind at a set rate? Because the original article said that it would often stop, rewind, move forward to retrieve data.
I don't think it really wound all the way to the ends of the tape, like my link said. I think it just wound forward for a set amount of time, and then back, to match the resonant frequency of the device.
Also, I think the point is that it's the Black Team's job to minimize bugs shipped, and they were really impossibly good at it. It was the programmers' job to write bug-free code, and this guy was inspired by the Black Team to write perfect code. It seems like both sides were doing exactly what they were supposed to do.
Also, I think the point is that it's the Black Team's job to minimize bugs shipped, and they were really impossibly good at it. It was the programmers' job to write bug-free code, and this guy was inspired by the Black Team to write perfect code. It seems like both sides were doing exactly what they were supposed to do.
On the other hand any physical system in motion is also loosing energy constantly to friction, air resistance, and other factors. Because of that and because the winding/unwinding description seemed implausible I strongly suspect the story is apocryphal.
I think you're underestimating how mechanical old hardware really was.
There's the classic story of "Walking Drives" from esr/jargonfile:
http://dictionary.die.net/walking%20drives
Or search for "washing" here for the story about huge DEC drives being ripped apart: http://www.rinkworks.com/stupid/cs_drives.shtml
Old hardware employed a lot of force, so "bad things" (TM) really could happen.
There's the classic story of "Walking Drives" from esr/jargonfile:
http://dictionary.die.net/walking%20drives
Or search for "washing" here for the story about huge DEC drives being ripped apart: http://www.rinkworks.com/stupid/cs_drives.shtml
Old hardware employed a lot of force, so "bad things" (TM) really could happen.
Normally you'd expect the Q factor of a standing tape drive to be small enough that you'd be unable to come anywhere close to knocking it over with the tiny kicks from the reels.
Wouldn't you have said the same thing of the Tacoma Narrows Bridge?
I probably would, yeah, unless I actually took the time to examine the design and calculate Q. I'd've been wrong in that case.
Actually, I probably wouldn't, because if it had occurred to anyone that they ought to ensure that the bridge design had no high-Q vibrational modes, the problem would have been found and averted before the first caisson was sunk.
Actually, I probably wouldn't, because if it had occurred to anyone that they ought to ensure that the bridge design had no high-Q vibrational modes, the problem would have been found and averted before the first caisson was sunk.
I have the sudden urge to build a weighted cabinet and see if I can knock it over with fast-forwards/rewinds. I've got a couple engineer friends who could run the numbers, but nothing would quite compare to the satisfaction I would feel if I could actually do it.
Can't we get Mythbusters to do it?
I think that's actually a great idea. Anyone know how to throw suggestions their way?
Here's the message board that takes submissions for Historical "Fact" myths. http://community.discovery.com/eve/forums/a/frm/f/9551919888
Exactly. How much of IBM's money had gone into developing the new product? How many sales did they lose as a result of this public humiliation?
That's the problem with "elite" teams in organizations. You don't get the actual elite. You just get the best connected, and they think they're untouchable.
That's the problem with "elite" teams in organizations. You don't get the actual elite. You just get the best connected, and they think they're untouchable.
"Beware of bugs in the above code; I have only proved it correct, not tried it." -- Donald Knuth
Searching for "the black team ibm -mustaches -infamous" on google returns (nearly) nothing. I find it astonishing that there is no record of such a team that doesn't mention mustache twirling.
I suspect that some hacker wanted a version of the Black Watch to look up to, so he invented one. I don't object to the invention of legends, but we should include some traditional legend-flagging phrases: "long ago", "never heard from again", &c.
I suspect that some hacker wanted a version of the Black Watch to look up to, so he invented one. I don't object to the invention of legends, but we should include some traditional legend-flagging phrases: "long ago", "never heard from again", &c.
The original article is a somewhat munged copy of an account in Peopleware 2nd ed (I wonder about the plagiarism possibility there).
And said book does have the mustache twirling.
And said book does have the mustache twirling.
If these events didn't actually transpire the way that the author of this story suggests, does that make the story any less valid?
[deleted]
Having worked on both the hardware and the software for most of IBM's tape drives, from the '60s era 2400s through the '70s 3480s, which cover the time of the Black Team, I find this story difficult to believe. On all these drives adjusting the start/stop mechanism required the engineer to be inside the enclosure with hands on the the very read head area whilst running many patterns of start/stop/rewind/fast forward. If I had felt any significant enclosure movement it would have indicated to me a major problem.
This reminds me of the article about the developers and testers who worked on software for the shuttle. In all the years, they only had 6 bugs ever reported on the shuttle, all of which were fairly minor as I recall.
It's so true that bugs are now simply part of life, and it has to do with the speed at which development must happen. I wonder what the Black team of old would think of today's web development wild west sort of approach.
Here it is: http://www.fastcompany.com/node/28121/print - They Write the Right Stuff (2007)
It's so true that bugs are now simply part of life, and it has to do with the speed at which development must happen. I wonder what the Black team of old would think of today's web development wild west sort of approach.
Here it is: http://www.fastcompany.com/node/28121/print - They Write the Right Stuff (2007)
Reading this story makes me sad.
There's also another story (google fails me) about a legendary IBM programmer around whom IBM built an entire team of testers, documenters, etc, all to keep this one guy's way above average productivity going. That story also makes me sad.
These stories make me sad because I know how huge a difference the environment makes to everyone's job.
The key points about the black team:
1. A few individuals that happen to be a bit above average at finding defects.
2. Bring them together, create a team.
3. Support them, but mostly just get out of their way and don't distract them with management B.S.
Very little change and support results in a huge jump in their productivity!
Same thing with the single legendary programmers, simply relive him of non-programming tedious tasks, give him enough support staff to keep up with his output and again HUGE productivity boost.
What's so sad about this is that is so rarely happens. I think most people are capable of having this productivity jump, if only they'd get the same support. OK, let me back of a bit from most and be more precise and say, you should be at least a bit above average.
But why does this so rarely happen? Sadly I think for most sizable companies minor process changes are a huge obstacle.
The bright side of this? Startups. Startups are like these kinds of teams within a behemoth like IBM, except without the behemoth. Or actually a startup up ought to be like that, because that is one of the key advantages a small business should have over the big ones.
There's also another story (google fails me) about a legendary IBM programmer around whom IBM built an entire team of testers, documenters, etc, all to keep this one guy's way above average productivity going. That story also makes me sad.
These stories make me sad because I know how huge a difference the environment makes to everyone's job.
The key points about the black team:
1. A few individuals that happen to be a bit above average at finding defects.
2. Bring them together, create a team.
3. Support them, but mostly just get out of their way and don't distract them with management B.S.
Very little change and support results in a huge jump in their productivity!
Same thing with the single legendary programmers, simply relive him of non-programming tedious tasks, give him enough support staff to keep up with his output and again HUGE productivity boost.
What's so sad about this is that is so rarely happens. I think most people are capable of having this productivity jump, if only they'd get the same support. OK, let me back of a bit from most and be more precise and say, you should be at least a bit above average.
But why does this so rarely happen? Sadly I think for most sizable companies minor process changes are a huge obstacle.
The bright side of this? Startups. Startups are like these kinds of teams within a behemoth like IBM, except without the behemoth. Or actually a startup up ought to be like that, because that is one of the key advantages a small business should have over the big ones.
I am thinking that this might just be humbug. Possibly motivational humbug, however.
But I did witness first hand some shenanigans done by the Field Engineers on XDS tape drives in the 1970s. They did use a kind of resonant thing to test the limits of how well a particular tape drive was working. It would do a lot of rewinding, stopping, reversing and the like. These drives had long vacuum (work with me here) chambers, one on each side where a loop of tape would be suspended. Thus, a fast back-and-forth operation could be performed on a short section of the tape without moving the reel. The goal was to try to get the tape moving in such a way that it would pop out of the vacuum chamber and fault the tape drive.
Somewhat like the Black Team's efforts are alleged to do, the net result was that all the tape drives, after adjustment, were able to pass this tough diagnostic.
But I did witness first hand some shenanigans done by the Field Engineers on XDS tape drives in the 1970s. They did use a kind of resonant thing to test the limits of how well a particular tape drive was working. It would do a lot of rewinding, stopping, reversing and the like. These drives had long vacuum (work with me here) chambers, one on each side where a loop of tape would be suspended. Thus, a fast back-and-forth operation could be performed on a short section of the tape without moving the reel. The goal was to try to get the tape moving in such a way that it would pop out of the vacuum chamber and fault the tape drive.
Somewhat like the Black Team's efforts are alleged to do, the net result was that all the tape drives, after adjustment, were able to pass this tough diagnostic.
Does anyone know what became of the Black Team? Presumably it's a defunct group, but when did that happen, and under what context?
Lots more comments on the previous discussion: http://news.ycombinator.com/item?id=985965 as well as http://news.ycombinator.com/item?id=994358
As a hardware tester at IBM, this makes me happy. I never see any adoration. People think of it as a 9-5; not as a chance to "best" somebody.
I remember hearing about the Black team from a training manual given to new hires who worked at Sperry/Univac in the late 70's - early 80's. There was a passage where the Black Team considered it a "failure" when they couldn't identify a defect during a testing round. And conversely they considered it a "success" when they identified a problem. Its almost as if they were doing TDD before anyone knew what to call it.
Ruthless and thorough QA is a good thing, but has nothing to do with TDD. TDD should in theory help prevent the developer from ever hearing from the Black Team to begin with. If the Black Team found the bug, the dev's TDD fu wasn't strong enough.
As much as I appreciate your praise of TDD here, there are large categories of defects that TDD does nothing to prevent. TDD only helps programmers write the code they intended to write. It doesn't protect against programmer misunderstandings.
As a result, well-TDD'd code may still have defects that involve:
- the programmers misunderstood what needed to be built ("requirements" defect) - the programmers interfaced with an external system that behaves differently than they thought - the programmers used a third-party library or framework incorrectly - there is a systemic error in the programmers' approach to the problem (e.g., not knowing about SQL injection attacks)
As I say in my "Let's Play TDD" series (http://jamesshore.com/Blog/Lets-Play/), TDD does a great job of helping a programmer write the code she intended to write. But it can't check the programmer's fundamental assumptions, so it's still important to check those assumptions using other techniques.
As a result, well-TDD'd code may still have defects that involve:
- the programmers misunderstood what needed to be built ("requirements" defect) - the programmers interfaced with an external system that behaves differently than they thought - the programmers used a third-party library or framework incorrectly - there is a systemic error in the programmers' approach to the problem (e.g., not knowing about SQL injection attacks)
As I say in my "Let's Play TDD" series (http://jamesshore.com/Blog/Lets-Play/), TDD does a great job of helping a programmer write the code she intended to write. But it can't check the programmer's fundamental assumptions, so it's still important to check those assumptions using other techniques.
Agree with all of this, and I've made the same arguments myself. That's one reason code coverage metrics are of limited use -- they can't measure the quality of the tests.
I wasn't intending to diminish the role of QA (essential) or assert that TDD cures all (doesn't), just that The Black Team weren't doing TDD.
I wasn't intending to diminish the role of QA (essential) or assert that TDD cures all (doesn't), just that The Black Team weren't doing TDD.
I guess the part I should have emphasized better in my original comment was "in theory"...
OR in an agile/integrated testing environment, the Black Team would be working with the dev to help develop unit tests and/or find the bug before it was delivered.
Nice article. This "Black Team" is sighted as an example in the book 'Peopleware: Productive Projects and Teams'.
FYI, "sighted" -> "cited".
Indeed, this story is in Peopleware, and it surprises me that I can find no account of the Black Team which has any details other than those I can find in Peopleware. It makes me wonder whether the story is apocryphal.
Indeed, this story is in Peopleware, and it surprises me that I can find no account of the Black Team which has any details other than those I can find in Peopleware. It makes me wonder whether the story is apocryphal.
It would be nice if there was info about the methods they developed for testing, rather than just how effective it was.
Pity the websites text takes up only 20% the width of my monitor... text looks cramped and awkward to read while 80% of my monitor is blank white space...
Longer line lengths are harder on the eyes. There's a reason why newspapers and magazines use columns.
I'm a big believer in columns. See the recent redesign of http://hackerstream.com (clicking on stories or users creates new streams/columns), and my previous project, http://readwarp.com
[deleted]
Luckily, the wonderful Readability can be used for these situations:
https://www.readability.com/
Which then proceeds to cut off the first paragraph.
Instapaper's Text view works better here.
http://www.instapaper.com/extras
http://www.instapaper.com/text?u=http%3A%2F%2Fwww.t3.org%2Ft...
Instapaper's Text view works better here.
http://www.instapaper.com/extras
http://www.instapaper.com/text?u=http%3A%2F%2Fwww.t3.org%2Ft...
Now it takes up 33%. Could be better still, but this is good enough - it is readable now! Thanks.
Can instapaper be customized? Can I make it wider still?
Can instapaper be customized? Can I make it wider still?
Yes, click the slightly non-obvious "Aa" item added at the top and it will open options to customize.
There are two rectangle icons that allow you to increase and reduce the margin.
There are two rectangle icons that allow you to increase and reduce the margin.
Awesome, thank you.
One of the few unique features of Safari is the "Reader" button (similar to readability), which I find very useful in reading such content (it's also great when applied to Anandtech, Arstechnica or other sites where article page count goes to double digits).
I used readability on this link. It actually dropped the 'Epilogue' on this one which left me scratching my head for a couple seconds before I backed out. But readability is incredibly handy on almost every ugly/noisy content page, it's one of the few tools I use every day.
I didn't know Safari had a "Reader" button (I usually stick to FF/Chrome), out of curiosity do you know who implemented the idea first?
I didn't know Safari had a "Reader" button (I usually stick to FF/Chrome), out of curiosity do you know who implemented the idea first?
Safari's "Reader" button is implemented with code from Readability. (Note the presence of Arc90 license verbiage in Help > Acknowledgements.)
I get extremely peeved by all of those blog themes that have 400px content columns. Grr.
All throughout reading about the Black Team, i couldn't help but recall the Stanford Prison experiment(http://en.wikipedia.org/wiki/Stanford_prison_experiment) and how the article says a lot about how people behave in groups ... Hmm....odd,given the goal of the article..
If anyone doubts whether they should read this article, let me quote:
"Team members began to affect loud maniacal laughter whenever they discovered software defects. Some individuals even grew long mustaches which they would twirl with melodramatic flair as they savaged a programmer's code."
"Team members began to affect loud maniacal laughter whenever they discovered software defects. Some individuals even grew long mustaches which they would twirl with melodramatic flair as they savaged a programmer's code."