The Illusion of Certainty(app.spectator.co.uk)
app.spectator.co.uk
The Illusion of Certainty
https://app.spectator.co.uk/2020/04/22/the-illusion-of-certainty/content.html
11 comments
Thanks for the overview. For those like me who mostly visit HN for the comments, a ~concise summary like this is helpful. Upvoted to counteract the (imho aggressive) down.
Thanks. I found the essay persuasive, even if (as others here have mentioned) the anecdote about Formula 1 teams is not entirely convincing.
The essay is about much more than that one anecdote.
The essay is about much more than that one anecdote.
>This essay asks the question, why are larger organizations so bad at performing feats of brilliance most of the time?
I'm not sure I buy this premise: Menlo Park, Bell Labs, Xerox PARC, IBM Research, major universities, the US military/government. Giant organizations like these provided enormous innovation and discovery in the last century. Maybe they don't anymore, but the question of why isn't the same as proclaiming "larger organizations [are] so bad at performing feats of brilliance most of the time".
I'm not sure I buy this premise: Menlo Park, Bell Labs, Xerox PARC, IBM Research, major universities, the US military/government. Giant organizations like these provided enormous innovation and discovery in the last century. Maybe they don't anymore, but the question of why isn't the same as proclaiming "larger organizations [are] so bad at performing feats of brilliance most of the time".
"giant" != "brilliant", and the question why something is the case does imply the statement that it is the case. You say you don't buy the premise, but then say that maybe it's true.
> illusion of certainty.
Never seen that idea summarized before, but it resonates with a lot of thing of how the world works. The system wants some subset of reality and elects it as the reality because the system is stabler this way for a period.
I also think that past 2 decades went overboard with that.
Never seen that idea summarized before, but it resonates with a lot of thing of how the world works. The system wants some subset of reality and elects it as the reality because the system is stabler this way for a period.
I also think that past 2 decades went overboard with that.
I disagree with his opening statement:
"Formula 1 teams have been able to design, prototype and manufacture essential health equipment incredibly quickly"
As far as I can see they have done none of these things and are only helping in providing manufacturing capability [1]. The ventilator they are helping to manufacture is an iteration on a previous design made by a medical specialist Penlon [2].
Not that there is anything bad with that strategy: experts in a given field are the experts so best to use the medical expertise of one company and the rapid batch production capabilities of companies like Formula 1 teams. Racing teams often have to turn components around for an update at the next grand prix e.g. 1 - 2 weeks. This means having processes and suppliers that can operate at that fast pace.
[1] https://www.inceptivemind.com/penlon-prima-eso2-newly-adapte... [2] https://www.penlon.com/
As far as I can see they have done none of these things and are only helping in providing manufacturing capability [1]. The ventilator they are helping to manufacture is an iteration on a previous design made by a medical specialist Penlon [2].
Not that there is anything bad with that strategy: experts in a given field are the experts so best to use the medical expertise of one company and the rapid batch production capabilities of companies like Formula 1 teams. Racing teams often have to turn components around for an update at the next grand prix e.g. 1 - 2 weeks. This means having processes and suppliers that can operate at that fast pace.
[1] https://www.inceptivemind.com/penlon-prima-eso2-newly-adapte... [2] https://www.penlon.com/
Thanks for this context. Here's the statement on McLaren's website: https://www.mclaren.com/group/news/articles/ventilatorchalle....
It seems consistent with what you're saying. There's some broad consortium of companies, but there are already designs approved by regulators and they're more or less providing manufacturing capacity to make more of them as fast as possible.
It seems consistent with what you're saying. There's some broad consortium of companies, but there are already designs approved by regulators and they're more or less providing manufacturing capacity to make more of them as fast as possible.
This story supports that opening statement:
"The [University of Southampton & NIHR] researchers worked alongside F1 team McLaren and industry experts including INDO and Kemp Sails to come up with the design, which is now available as open-source to be manufactured around the world.
https://www.itv.com/news/2020-04-23/hospital-first-to-widely...
"The [University of Southampton & NIHR] researchers worked alongside F1 team McLaren and industry experts including INDO and Kemp Sails to come up with the design, which is now available as open-source to be manufactured around the world.
https://www.itv.com/news/2020-04-23/hospital-first-to-widely...
In this case, a prototype (which is probably worse than existing PAPRs [powered air-purifying respirators]) is demonstrated. It certainly hasn't been manufactured and the Formula 1 team involvement looks to be just of the design of a few components for 3D printing [1].
[1] https://engrxiv.org/rvcs3/
[1] https://engrxiv.org/rvcs3/
I'm completely ashamed by the level of b.s. and failure exhibited by Western companies during the corona crisis.
We'd be better off if they just admitted the failure so we can move on to the next step, rather than be distracted by utter b.s.
The reality is even if some science-project ventilator was made, no hospital would approve its use. But we knew that in advance, before the b.s. even started.
And we've known since Jan. that ventilators kill their patiences 66% to 90% or more often.
We'd be better off if they just admitted the failure so we can move on to the next step, rather than be distracted by utter b.s.
The reality is even if some science-project ventilator was made, no hospital would approve its use. But we knew that in advance, before the b.s. even started.
And we've known since Jan. that ventilators kill their patiences 66% to 90% or more often.
F1 teams making ventilators is probably a pretty bad example. I bet these ventilators are very expensive, hard to maintain and have probably a lot of little problems. I work in medical devices and if there is one thing I have learned it’s that there is a big difference between having something that works somehow and something that’s ready to be produced and maintained in the long run. I agree that big companies tend to stifle innovation but they are also very optimized and there is a lot of invisible work that has to go into achieving that state.
I see the same in software. We sometimes have interns whip up some system that on the surface looks good and innovative and makes the old timers look bad and unimaginative. But once you try to deploy them they usually fall apart because they ignored some inconvenient real world details.
I see the same in software. We sometimes have interns whip up some system that on the surface looks good and innovative and makes the old timers look bad and unimaginative. But once you try to deploy them they usually fall apart because they ignored some inconvenient real world details.
Your comment and the article's referance to honey bee behavior reminds me of a section that has always troubled me from an old Charlie Munger talk on human misjudgement [1]
<QUOTE> ... 22. And then I’ve got development and organizational confusion from say-something syndrome.
And here my favorite thing is the bee, a honeybee. And a honeybee goes out and finds the nectar and he comes back, he does a dance that communicates to the other bees where the nectar is, and they go out and get it. Well some scientist who is clever, like B.F. Skinner, decided to do an experiment. He put the nectar straight up. Way up. Well, in a natural setting, there is no nectar where they’re all straight up, and the poor honeybee doesn’t have a genetic program that is adequate to handle what he now has to communicate. And you’d think the honeybee would come back to the hive and slink into a corner, but he doesn’t. He comes into the hive and does this incoherent dance, and all my life I’ve been dealing with the human equivalent of that honeybee. [Laughter] And it’s a very important part of human organization so the noise and the reciprocation and so forth of all these people who have what I call say-something syndrome don’t really affect the decisions. <END QUOTE>
So this "Say-Something-Syndrome" is a negative input that introduces development and organizational confusion in to an otherwise well tested method. It doesn't scale. It is to be ignored.
This troubles me as an absolute position. Yes there is danger funneling good resources down a rabbit hole to nowhere and yur core strategies should not flap around like a faulty nic card chasing shiny new things. But, you cannot close a system off completely or it will get stuck. Contuious improvement, iteration, hot wash lessons learned, red teams, squad checks, all these are dependent on effective positive & negative feedback processing. Not writing off all dissenting opinion as noise.
It's a balance. It's form of art. Dynamic systems are messy in real time both for managers and engineers and not much about that will change. Best we can do is learn what we can from some old bee and monkey [2] stories.
[1] https://buffettmungerwisdom.wordpress.com/2013/01/14/full-te...
[2] https://www.wisdompills.com/monkey-ladder/
<QUOTE> ... 22. And then I’ve got development and organizational confusion from say-something syndrome.
And here my favorite thing is the bee, a honeybee. And a honeybee goes out and finds the nectar and he comes back, he does a dance that communicates to the other bees where the nectar is, and they go out and get it. Well some scientist who is clever, like B.F. Skinner, decided to do an experiment. He put the nectar straight up. Way up. Well, in a natural setting, there is no nectar where they’re all straight up, and the poor honeybee doesn’t have a genetic program that is adequate to handle what he now has to communicate. And you’d think the honeybee would come back to the hive and slink into a corner, but he doesn’t. He comes into the hive and does this incoherent dance, and all my life I’ve been dealing with the human equivalent of that honeybee. [Laughter] And it’s a very important part of human organization so the noise and the reciprocation and so forth of all these people who have what I call say-something syndrome don’t really affect the decisions. <END QUOTE>
So this "Say-Something-Syndrome" is a negative input that introduces development and organizational confusion in to an otherwise well tested method. It doesn't scale. It is to be ignored.
This troubles me as an absolute position. Yes there is danger funneling good resources down a rabbit hole to nowhere and yur core strategies should not flap around like a faulty nic card chasing shiny new things. But, you cannot close a system off completely or it will get stuck. Contuious improvement, iteration, hot wash lessons learned, red teams, squad checks, all these are dependent on effective positive & negative feedback processing. Not writing off all dissenting opinion as noise.
It's a balance. It's form of art. Dynamic systems are messy in real time both for managers and engineers and not much about that will change. Best we can do is learn what we can from some old bee and monkey [2] stories.
[1] https://buffettmungerwisdom.wordpress.com/2013/01/14/full-te...
[2] https://www.wisdompills.com/monkey-ladder/
> I see the same in software. We sometimes have interns whip up some system that on the surface looks good and innovative and makes the old timers look bad and unimaginative. But once you try to deploy them they usually fall apart because they ignored some inconvenient real world details.
You’ve criticized the article but ended up reaching essentially the same conclusion. Having interns spin up a new feature which can later be tidied up and put into production is a great use of time. Interns are cheap, less busy, see things that more jaded people may not, and are liable to be less risk averse too. If you take what the interns build and put it directly into production, that’s on you, but the interns may be able to get you 80% of the way there in half the time.
You’ve criticized the article but ended up reaching essentially the same conclusion. Having interns spin up a new feature which can later be tidied up and put into production is a great use of time. Interns are cheap, less busy, see things that more jaded people may not, and are liable to be less risk averse too. If you take what the interns build and put it directly into production, that’s on you, but the interns may be able to get you 80% of the way there in half the time.
"get you 80% of the way there in half the time."
From my experience they don't. Either it gets thrown away or the cleanup requires as much or more even more work than if experienced people had done it from scratch.
Edit: I am not saying there is no value in work from newcomers but I think it's often overrated. They usually do the easy part but finishing it takes much more work than people think.
From my experience they don't. Either it gets thrown away or the cleanup requires as much or more even more work than if experienced people had done it from scratch.
Edit: I am not saying there is no value in work from newcomers but I think it's often overrated. They usually do the easy part but finishing it takes much more work than people think.
Agreed, interns are extremely expensive and anyone who thinks that they are cheap labor is not aware of the day to day details. Only reason to ever have interns is if you want to give back to the community and grow your senior talent. You might end up with some loyal developers that know your stack, but I wouldn't bank on it. They certainly have no place in an early stage startup.
Why one team can create a project during a 2-day hackathon, while it takes a year to finish professionally?
(Game, chat app, anything.)
Well, large organizations have overhead for communication, management, etc (see v). But also there is a statistical effect - if we get many, many shots - sometimes we get lucky (in most cases - not; think of a typical hackathon project - usually it never gets properly started). So, in short, we see a huge survival bias.
...and it takes time to polish rough edges. For example, game Superhot started as a hackathon demo (with a single scene). It took Kickstarted crowdfunding, and a year of work to turn in into a fully-fledged commercially-viable game. (Personally, I think it is one of the most creative game ideas in the last years, especially for VR. I am happy that I crowdfunded it. :))
(Game, chat app, anything.)
Well, large organizations have overhead for communication, management, etc (see v). But also there is a statistical effect - if we get many, many shots - sometimes we get lucky (in most cases - not; think of a typical hackathon project - usually it never gets properly started). So, in short, we see a huge survival bias.
...and it takes time to polish rough edges. For example, game Superhot started as a hackathon demo (with a single scene). It took Kickstarted crowdfunding, and a year of work to turn in into a fully-fledged commercially-viable game. (Personally, I think it is one of the most creative game ideas in the last years, especially for VR. I am happy that I crowdfunded it. :))
well 99/1 rule... 99% of the product (visible bits) comes from 1% of the effort... but 99% of the effort and cost is required to deliver the last 1% of difficult soul crushing detail to get it done. The reason is that we optimize, align and facilitate organizationally the 99% because it's common.
This is also why the evolution of multicellular creatures has (meta-)evolved to proceed by discrete generations (birth & senescence) and Darwinian rather than Lamarckian inheritance.
Otherwise you over-adapt (overfitting) and lose the reservoir of variability required to cope with extraordinary circumstances.
Gregory Bateson describes it in one of his essays, I haven't had enough coffee yet but if you're interested bug me and I'll go look up which one. (It's in "Steps to an Ecology of Mind" or "Mind and Nature: A Necessary Unity" IIRC.)
Otherwise you over-adapt (overfitting) and lose the reservoir of variability required to cope with extraordinary circumstances.
Gregory Bateson describes it in one of his essays, I haven't had enough coffee yet but if you're interested bug me and I'll go look up which one. (It's in "Steps to an Ecology of Mind" or "Mind and Nature: A Necessary Unity" IIRC.)
I love this, having seen it in operation so many times. Accountants promoted to senior managers whose only way of increasing profits is to trim the costs, reducing all flexibility and removing any resilience from the organisation. Good ideas for new stuff getting veto'd because they're "risky" (risky to the management bonus if they fail).
The last 3 paragraphs talk about explore/exploit ratio in bee colonies - most bees "exploit" what others found, small number of bees "explore" new grounds.
We do exactly the same. We exploit in the times of plenty and explore when the going get tough. The same evolutionary forces act on us as a society as on the bees, so it is unfair to say that bees know better than us how to get that ratio right.
https://en.wikipedia.org/wiki/Multi-armed_bandit#Optimal_sol...
We do exactly the same. We exploit in the times of plenty and explore when the going get tough. The same evolutionary forces act on us as a society as on the bees, so it is unfair to say that bees know better than us how to get that ratio right.
https://en.wikipedia.org/wiki/Multi-armed_bandit#Optimal_sol...
I think the point he’s making is some groups (currently too many?) outweigh certain costs compared to uncertain benefits— limiting innovation. It’s along the lines of “nobody ever got fired for recommending IBM.” I don’t think it generalizes beyond the single organization level well considering all the money going into venture capital as of late. And I don’t see it at Apple/Alphabet/Microsoft/Amazon where some of the world’s largest companies continue to innovate and pivot. But, you do see it in terms of Berkshire Hathaway’s investment strategy (which interestingly includes investments in Apple), which is exceedingly conservative by some measures. But part of that is because Buffet is risk-averse (which is different than uncertainty-averse).
I think the article stops short of saying how we would know enough resources are being put into innovation. And for that, I have no idea either.
I think the article stops short of saying how we would know enough resources are being put into innovation. And for that, I have no idea either.
Apple/Alphabet/Microsoft/Amazon are innovating?
Mostly they seem to be acquiring smaller companies that have come up with interesting ideas.
Apple, especially, appears to be churning out annual updates on a product line that is not changing at all (I'm not blaming them, they're making more money than god, but still).
And pivoting? I guess Alphabet's "it didn't go viral in the first two years so we're dropping it" strategy could be thought of as pivoting, but not really.
Mostly they seem to be acquiring smaller companies that have come up with interesting ideas.
Apple, especially, appears to be churning out annual updates on a product line that is not changing at all (I'm not blaming them, they're making more money than god, but still).
And pivoting? I guess Alphabet's "it didn't go viral in the first two years so we're dropping it" strategy could be thought of as pivoting, but not really.
Apple created the iPhone 13 years ago. Then Macbook Air a year later, the iPad line and so on. The A13 processor is the fastest mobile processor in widespread use, as was the A12 when it was launched. Do you really expect a company to innovate every single year? Google/Alphabet created K8s seven years ago, Glang, Android, and consistently refines its cloud platform. Its search and advertising platform is very effective and gives enormous value to its customers, even if it's not the "strictly do no evil" innovation of yore. Amazon has grown to enormous scale, innovating along its supply chain and AWS. Microsoft created Azure in almost no time, supports Linux on its own operating system and has embraced JavaScript and has some open source projects. I think, if you remember the Microsoft of 10 years ago (software-only, bitterly proprietary), that's quite a pivot.
> Do you really expect a company to innovate every single year?
I'd say a company as large as Apple would need to innovate hundreds or thousands of times per year, to be comparable to a small startup, in innovativeness per person.
I'd say a company as large as Apple would need to innovate hundreds or thousands of times per year, to be comparable to a small startup, in innovativeness per person.
Agreed. Considering their size the innovation output of these companies is really low. Google still hasn’t found anything besides selling ads and total surveillance and Apple mainly makes things thinner.
>One problem with this pretence of certainty is that cost-savings are more easily quantified than potential gains — so business and government are increasingly geared towards providing people with more, poorer things at an ever-lower price. Yet much evidence suggests that people like fewer, better things at a slightly higher price.
I would dispute that paragraph. The popularity of "Primani" is proof that there are plenty of people like to be able to buy a larger range of cheap stuff. The number of times I've heard female friends, on being complimented on a garment, say "and it was only a fiver", or "10 quid at Primark".
I would say there's plenty of both types in this world. Those who prefer quality over quantity, and those who don't.
I would dispute that paragraph. The popularity of "Primani" is proof that there are plenty of people like to be able to buy a larger range of cheap stuff. The number of times I've heard female friends, on being complimented on a garment, say "and it was only a fiver", or "10 quid at Primark".
I would say there's plenty of both types in this world. Those who prefer quality over quantity, and those who don't.
Fashion isn't the best example for this, because often people are paying for prestige more than quality. A $3,000 purse isn't 100X as good as a $30 purse; the exclusivity of a brand that expensive is the attraction. People who opt-out of this competition however, may instead brag about what savvy shoppers they are.
There is a relevant theory in economics called "the market for lemons" that speaks to what you are saying though. The idea is this: if someone is trying to buy a washing machine as cheaply as possible, they would rationally pick an $1000 machine that lasts 15 years over a $500 machine that lasts 5. The problem, however, is that the consumer often can't tell if the expensive machine is really better quality, or just going for a higher markup. With that uncertainty, it is safer to buy the cheap machine. Manufacturers know this, so they compete on price and features more than longevity. Without a time machine, consumers can't see if the extra money they spend is worth it, so cheap crap is the equilibrium for the market.
There is a relevant theory in economics called "the market for lemons" that speaks to what you are saying though. The idea is this: if someone is trying to buy a washing machine as cheaply as possible, they would rationally pick an $1000 machine that lasts 15 years over a $500 machine that lasts 5. The problem, however, is that the consumer often can't tell if the expensive machine is really better quality, or just going for a higher markup. With that uncertainty, it is safer to buy the cheap machine. Manufacturers know this, so they compete on price and features more than longevity. Without a time machine, consumers can't see if the extra money they spend is worth it, so cheap crap is the equilibrium for the market.
To quote: No one ever got fired for buying IBM.
If you're working for a large, bureaucratic organization, producing a radically different solution can be a career-limiting move, even if it's a success and the problem is quite important. (Or perhaps especially if it's quite important.)
If you're working for a large, bureaucratic organization, producing a radically different solution can be a career-limiting move, even if it's a success and the problem is quite important. (Or perhaps especially if it's quite important.)
> a radically different solution can be a career-limiting move, even if it's a success
Why / how can that happen?
Why / how can that happen?
Typically politics, turf wars, etc. If you succeed quickly at something someone with pull has been working on unsuccessfully for a while, you can easily end up with a powerful enemy or just plain fired.
In the real world, getting things done is not your job. Your job is to make your boss look good to his boss. In reasonably functional organizations, the former is aligned to the latter. But they are hardly the rule.
In the real world, getting things done is not your job. Your job is to make your boss look good to his boss. In reasonably functional organizations, the former is aligned to the latter. But they are hardly the rule.
Thanks for explaining and the examples
:fire:
Two things:
Two things:
1. From my own personal experience, I think the observations here are _accurate_
2. I really appreciate that is article offers an implicit good-side to tough time. That is, success in tough times is made by ditching safe behavior
The author sees two obstacles. One is what he calls "the boredom threshold:" The creative individuals who could perform those feats quickly get bored with the mind-numbing minutia of bureaucratic processes at larger organizations and leave.
The second obstacle is "the illusion of certainty:" Conventional operational improvements, cost-cutting efforts, legal reorganizations, etc. can easily be 'proven' to work in advance, but more interesting lines of inquiry come with career-threatening unknowability. Many bureaucrats and businesspeople, the OP argues, are attracted to the illusion of certainty.
Quoting from the OP: "If you engage engineers, you don’t know what you are going to get. You may be unlucky and get nothing. Or their solution may be so outlandish that it is hard to compare with other competing solutions. On average, though, what you get will be more valuable than the gains produced by some tedious restructuring enshrined in a fat PowerPoint deck. But in business, let alone in government, it is only in crises that people find a budget for probabilistic interventions of this kind."
The essay concludes that "one reason why the world is in a mess is because, for a long time, the ratio between 'explore' and 'exploit' has been badly out of whack. Entities like procurement have been allowed to claim full credit for money-grabbing cost-savings without commensurate responsibility for delayed or hidden costs."