Want more policies based on evidence?(review.chicagobooth.edu)
review.chicagobooth.edu
Want more policies based on evidence?
http://review.chicagobooth.edu/accounting/2018/article/want-more-policies-based-evidence
12 comments
The underlying problem is politization. You can assume that if some political party wants some something and another political party wants the opposite something, you could find a set of impartial experts that would provide hard data and solve the question. In the real world, there are two sets of experts, holding opposite views and providing contradictory data. Everybody will make a big noise and eventually nobody knows what happened, just that the question got muddied and you aren't so sure about anything anymore.
The problem underlying politicisation is confidence. Science isn't binary, it's a set of circles of decreasing confidence that spreads out from a core of propositions that we're very confident about - more or less what you'll learn on an undergrad physics course - to a set of increasingly tentative hypotheses.
A lot of arguments about science are really arguments about confidence. E.g. most climate change scientists are fairly sure about their models, but the lack of absolutely certainty makes it possible for deniers to cherry pick a tiny collection of outlier scientists who will argue in public that it's all nonsense.
Policy makers and the media are some combination of corrupt and clueless, so they're happy to go with the false equivalence this creates.
One way to depoliticise science would be to have an international science foundation, which was funded independently of any individual government.
Of course there would be squeals of disapproval from vested interests, but that would simply highlight the problem - the vested interests don't want independent criticism or oversight. Their entire MO is based on regulatory capture which gives them the freedom (for themselves only) to operate as they want with no personal or financial consequences.
Scientific accountability would set them on the path to democratic accountability, which is the last thing they'll accept.
A lot of arguments about science are really arguments about confidence. E.g. most climate change scientists are fairly sure about their models, but the lack of absolutely certainty makes it possible for deniers to cherry pick a tiny collection of outlier scientists who will argue in public that it's all nonsense.
Policy makers and the media are some combination of corrupt and clueless, so they're happy to go with the false equivalence this creates.
One way to depoliticise science would be to have an international science foundation, which was funded independently of any individual government.
Of course there would be squeals of disapproval from vested interests, but that would simply highlight the problem - the vested interests don't want independent criticism or oversight. Their entire MO is based on regulatory capture which gives them the freedom (for themselves only) to operate as they want with no personal or financial consequences.
Scientific accountability would set them on the path to democratic accountability, which is the last thing they'll accept.
> A lot of arguments about science are really arguments about confidence. E.g. most climate change scientists are fairly sure about their models, but the lack of absolutely certainty makes it possible for deniers to cherry pick a tiny collection of outlier scientists who will argue in public that it's all nonsense.
I think scale/proportion is also a problem. Humans seem to place a lot of value in narratives/stories but we aren't so good with quantities (e.g. https://en.wikipedia.org/wiki/Conjunction_fallacy ). Pretty much everything (economics, climate, etc.) has factors pushing it in different directions, so we can always find a counterargument to any position (e.g. we can rebuff climate change by pointing to solar cycles, CO2 causing extra plant growth, etc.); that's fine, but some factors are overwhelmingly more important than others, whilst we seem to cling on to these stories/narratives and give them more equal weighting than we should.
As a concrete example, a family member used to leave their lights on overnight, claiming that "they use more energy than normal when they're first switched on". Whilst true, the saving is cancelled out after seconds ( e.g. https://www.energy.gov/energysaver/when-turn-your-lights )
I think scale/proportion is also a problem. Humans seem to place a lot of value in narratives/stories but we aren't so good with quantities (e.g. https://en.wikipedia.org/wiki/Conjunction_fallacy ). Pretty much everything (economics, climate, etc.) has factors pushing it in different directions, so we can always find a counterargument to any position (e.g. we can rebuff climate change by pointing to solar cycles, CO2 causing extra plant growth, etc.); that's fine, but some factors are overwhelmingly more important than others, whilst we seem to cling on to these stories/narratives and give them more equal weighting than we should.
As a concrete example, a family member used to leave their lights on overnight, claiming that "they use more energy than normal when they're first switched on". Whilst true, the saving is cancelled out after seconds ( e.g. https://www.energy.gov/energysaver/when-turn-your-lights )
There is also an issue of getting what you measure for since humans game systems to their benefit. Look at standardized tests - they guided education from an early age as opposed to actual educational outcomes. I remember vividly being in elementary school and they multiple workbooks with pages of analogies with occasional ambiguous answers. There wasn't any real learning just a bunch of drilling that depended on existing knowledge. Then the SAT dropped it for a writing section and analogies practically disappeared off the face of the earth. They showed up four times a year at most - literally. Usually because the quarterly state tests had one question with them.
There's a middle ground.
Practically no systemic analysis is done within government. At least in Chicago. Government systems are compartmentalized in ways that makes interfacing with them impossible for any worthwhile analysis. Example: the only analysis that Chicago's finance department had done on its parking tickets is a single very high level spreadsheet.
Some analysis is better than no analysis.
Practically no systemic analysis is done within government. At least in Chicago. Government systems are compartmentalized in ways that makes interfacing with them impossible for any worthwhile analysis. Example: the only analysis that Chicago's finance department had done on its parking tickets is a single very high level spreadsheet.
Some analysis is better than no analysis.
> Some analysis is better than no analysis.
I don't agree with that. Doing some analysis on a limited and possibly skewed data set can lull you into a false sense of understanding. It makes your ideas seems objective when they can in fact be completely baseless.
I don't agree with that. Doing some analysis on a limited and possibly skewed data set can lull you into a false sense of understanding. It makes your ideas seems objective when they can in fact be completely baseless.
As always, you need to take the limitations of the available information into account. That ought to be part of the analysis, though of course it often, or perhaps usually, isn't.
Not at all. Analysis of limited data gives you wide credible intervals, the exact thing that guards against unwarranted confidence.
How is no analysis just as good?
It's hard to believe that the percentage of successes with no analysis is GTE the percentage of successes with some analysis.
It's hard to believe that the percentage of successes with no analysis is GTE the percentage of successes with some analysis.
I presume because faulty analysis or bad data can actually get you more off track than when its just a "back of the napkin" based guess or hunch.
And additionally, you now have the pride/confidence thing in your even worse results because you did "analysis"...
And additionally, you now have the pride/confidence thing in your even worse results because you did "analysis"...
Often the pride is constant.
Are you saying that zero middle ground exists?
Obviously a middle ground exists, but it's possible that in part of the scale you get worse results as you head toward the middle.
> Practically no systemic analysis is done within government
This is so far from being true, it undermines your point and your post.
The Federal Reserve does no systematic analysis? The U.S. Treasury does no systematic analysis? The Bureau of Labor Statistics does no systematic analysis? The Congressional Budget Office does no systematic analysis? The Centers for Disease Control do no systematic analysis?
This is so far from being true, it undermines your point and your post.
The Federal Reserve does no systematic analysis? The U.S. Treasury does no systematic analysis? The Bureau of Labor Statistics does no systematic analysis? The Congressional Budget Office does no systematic analysis? The Centers for Disease Control do no systematic analysis?
Systemic, not systematic.
I don't even understand what you mean. I have read quite a few government papers (mainly federal) and they usually were well researched. I don't know how things are on a local level but the politicians in Congress have a lot of well researched data available if they want to listen (which they often don't).
Sorry - should have been more clear. I'm mostly talking about local policy, which is where most of my experience with government comes from, through many FOIA requests. Federal is much more calculated (read: slow) in comparison to local government. I've found local government to be very "we've checked the box, let's move on", which doesn't leave room for analysis, let alone the acknowledgement that analysis is even possible.
Makes sense. The way local governments deal with things like pensions is truly horrifying. Even the simplest analysis would quickly show that they are setting themselves up for disaster.
"We could add more money to the pension fund. Or we could assume a 10% market rate of return forever, and spend that money on new office chairs and computers instead. We can't get a tax increase just for those, but we could for the prospect of homeless old people eating cat food, and the next guy will get blamed for it."
They're actually setting other people up for disaster, hoping that they will already be gone when it hits.
They're actually setting other people up for disaster, hoping that they will already be gone when it hits.
If it is that easy to predict then it is probably career suicide to be the person who produces the analysis that proves a disaster is going to happen.
Once you resolve questions of values and fairness, you should be able to leverage data and metrics to achieve outcomes consistent with those values. Say you’re designing a jet engine. You decide what output parameters you care about optimising (maximum thrust versus fuel effiency, for example). You know that you can manipulate certain input parameters to influence those outputs. And you can use data to verify and iterate on your design. But all you can do is change inputs to a very complex system. The unbending rules of the system itself decide how those affect the outputs.
The problem is that most in government simply are not systems thinkers. They are focused on values and fairness, and believe that once you’ve identified those values, you can directly legislate those into outcomes. This thinking leads to spectacular failures (the war on drugs, the war on poverty, tough on crime, etc).
The problem is that most in government simply are not systems thinkers. They are focused on values and fairness, and believe that once you’ve identified those values, you can directly legislate those into outcomes. This thinking leads to spectacular failures (the war on drugs, the war on poverty, tough on crime, etc).
> The problem is that most in government simply are not systems thinkers.
I don't think the main problem is that "people in the government are stupid". Real life and societies are much more complex than a jet engine. There are thousands of value goals, too many to be able to put numerical targets on each one.
What is the optimal ratio of potholes to unsolved murders?
And when you forget to include any of the value goals in your model, then you get a paperclip AI scenario with terrible effects somewhere else.
I don't think the main problem is that "people in the government are stupid". Real life and societies are much more complex than a jet engine. There are thousands of value goals, too many to be able to put numerical targets on each one.
What is the optimal ratio of potholes to unsolved murders?
And when you forget to include any of the value goals in your model, then you get a paperclip AI scenario with terrible effects somewhere else.
I didn’t say they were stupid. I said they are not systems thinkers. There are lots of very smart people who don’t view the world in terms of cause and effect, cost and benefit, action and reaction. Irrational and harmful, I might say that...
> In terms of public policy you have to decide what you're optimizing for and that decision can't be made with data alone because it does not help resolve questions of value and fairness.
Sure, but once you decide, then you should use data to optimize it. Seems pretty straightforward, and I don't think anything in this article would disagree with that.
Sure, but once you decide, then you should use data to optimize it. Seems pretty straightforward, and I don't think anything in this article would disagree with that.
Even after I know what I'm optimizing for I cannot use data to optimize for it. I know from experience that unintended results will happen. Some of them will be very base which will force me to come up with a whole new set of things to optimize for.
You are also assuming we can agree on what to optimize for. In fact we do not, and will not.
You are also assuming we can agree on what to optimize for. In fact we do not, and will not.
> Even after I know what I'm optimizing for I cannot use data to optimize for it. I know from experience that unintended results will happen. Some of them will be very base which will force me to come up with a whole new set of things to optimize for.
You can't use data to optimize anything? Google and Facebook must be wasting their time collecting all that user data, then.
> You are also assuming we can agree on what to optimize for. In fact we do not, and will not.
We agree on lots of things to optimize for. There are cases of disagreement, but very very broad agreement as well.
You can't use data to optimize anything? Google and Facebook must be wasting their time collecting all that user data, then.
> You are also assuming we can agree on what to optimize for. In fact we do not, and will not.
We agree on lots of things to optimize for. There are cases of disagreement, but very very broad agreement as well.
>You can't use data to optimize anything?
No, I can use data, the problem is I don't know what all the effects of that optimization will be. So I constantly have to change what I'm optimizing for.
> We agree on lots of things to optimize for.
Broadly, but there are limited resources and each thing effects the other. So even though we agree that A and B are worth optimizing for, we will disagree on which is more important. Worse in many cases we will agree on A and B, but the data shows you cannot optimize for one without pessimism the other.
That is the broad agreement isn't really enough to do anything with, we need the details and there we disagree.
No, I can use data, the problem is I don't know what all the effects of that optimization will be. So I constantly have to change what I'm optimizing for.
> We agree on lots of things to optimize for.
Broadly, but there are limited resources and each thing effects the other. So even though we agree that A and B are worth optimizing for, we will disagree on which is more important. Worse in many cases we will agree on A and B, but the data shows you cannot optimize for one without pessimism the other.
That is the broad agreement isn't really enough to do anything with, we need the details and there we disagree.
> No, I can use data, the problem is I don't know what all the effects of that optimization will be. So I constantly have to change what I'm optimizing for.
What are you trying to say here? Optimizing things with data is hard?
> Broadly, but there are limited resources and each thing effects the other. So even though we agree that A and B are worth optimizing for, we will disagree on which is more important. Worse in many cases we will agree on A and B, but the data shows you cannot optimize for one without pessimism the other.
I think we even agree broadly enough on the relative weights of many things to optimize for them. And I think we at the very least agree enough on things to partially optimize them, or pareto optimize them. In many cases there are low-hanging fruit to be picked that can optimize a metric that we all agree is good without sacrificing another.
What are you trying to say here? Optimizing things with data is hard?
> Broadly, but there are limited resources and each thing effects the other. So even though we agree that A and B are worth optimizing for, we will disagree on which is more important. Worse in many cases we will agree on A and B, but the data shows you cannot optimize for one without pessimism the other.
I think we even agree broadly enough on the relative weights of many things to optimize for them. And I think we at the very least agree enough on things to partially optimize them, or pareto optimize them. In many cases there are low-hanging fruit to be picked that can optimize a metric that we all agree is good without sacrificing another.
> In terms of public policy you have to decide what you're optimizing for and that decision can't be made with data alone because it does not help resolve questions of value and fairness.
Not to mention that matters of value and fairness also change the availability of, and interest in, research itself. For example, if some people don't like the results of your research (despite presenting no challenge to the facts or methodology), they can now apparently replace it without notice after publishing it in a journal (and do so in a manner which is intended to make it difficult to publish it in other journals).
The academy is the wrong place to direct politics, because politics already direct the academy.
From my perspective, a more likely overall improvement in the ongoing quality of policy would be a requirement that all policies sunset reasonably soon by default, even if they don't seem divisive at the time they're passed. As it regards controlled substances, this would lower the bar for repeal to "nobody particularly cares to renew it" from "nobody particularly cares to do the work to repeal it".
The academy is the wrong place to direct politics, because politics already direct the academy.
From my perspective, a more likely overall improvement in the ongoing quality of policy would be a requirement that all policies sunset reasonably soon by default, even if they don't seem divisive at the time they're passed. As it regards controlled substances, this would lower the bar for repeal to "nobody particularly cares to renew it" from "nobody particularly cares to do the work to repeal it".
This is a great point. I wish more people with an interest in policy/politics also had an interest of in intellectual history.
A lot of the most important debates of the last 100+ years has been on exactly this debate.
Rationalists like Ayn Rand (sort of), Rosa Luxemburg (red Rosa) and others on one side, and the like of Karl Popper & Hayek on the other.
When people talk about "testability" of a theory to decide if it's a scientific theory... they are borrowing from Popper's criticism of Freudianism, Marxism & the concept of metric based "government science."
A lot of the most important debates of the last 100+ years has been on exactly this debate.
Rationalists like Ayn Rand (sort of), Rosa Luxemburg (red Rosa) and others on one side, and the like of Karl Popper & Hayek on the other.
When people talk about "testability" of a theory to decide if it's a scientific theory... they are borrowing from Popper's criticism of Freudianism, Marxism & the concept of metric based "government science."
I didn't understand the article's point at first because to me, medicine is not a great example of evidence-driven policies. The American health insurance system does not appear to be optimized by any sort of metric, much less to be driven by evidence. Many key decisions, like how many doctors should be given medical degrees each year, are not subject to any evidence at all. And just at a high level in terms of performance for cost something is fundamentally "not working" in medicine over the past couple decades.
It's true that for the micro decisions, like does drug X treat condition Y, medicine appears to be much more evidence-based than, say, sociology. But the overall dysfunction of medical policymaking makes me dubious of using medicine as a model for other fields of decisionmaking.
It's true that for the micro decisions, like does drug X treat condition Y, medicine appears to be much more evidence-based than, say, sociology. But the overall dysfunction of medical policymaking makes me dubious of using medicine as a model for other fields of decisionmaking.
It's optimised for efficient transfer of tax payer money to the industry with as little distraction as possible, such as wasting money on actual care.
As with everything the USA sets out to do, it excels at it.
As with everything the USA sets out to do, it excels at it.
Even at the micro level: many childbearing related dictates (don’t drink, do breastfeed) are based on extremly weak evidence. Emotion trumps data in many cases even in medicine.
What do you mean by "don't drink" being based on extremely weak evidence? Do you mean after the pregnancy, or while trying to conceive?
Because the harms of drinking during pregnancy are well known. Fetal Alcohol Syndrome is a real thing, as are other Fetal Alcohol Spectrum disorders.
Because the harms of drinking during pregnancy are well known. Fetal Alcohol Syndrome is a real thing, as are other Fetal Alcohol Spectrum disorders.
I don't know if this is what the OP was referring to but to quote the wikipedia article:
> Evidence of harm from less than two drinks per day or 10 drinks per week is not clear
It seems there is strong evidence that "heavy" drinking is bad, the evidence that you shouldn't drink at all is much less clear.
> Evidence of harm from less than two drinks per day or 10 drinks per week is not clear
It seems there is strong evidence that "heavy" drinking is bad, the evidence that you shouldn't drink at all is much less clear.
I was aware of Fetal Alcohol Syndrome but not Fetal Alcohol Spectrum Disorders [0]. Thank you for sharing. From what I ahve read the dont drink and do breast feed have fairly strong research behind them so I am a bit confused by the weak correlation comment as well.
Anecdotally, while in Europe I do see it extremely commonplace for women to have a few wines while pregnant.
[0] https://en.wikipedia.org/wiki/Fetal_alcohol_spectrum_disorde...
Anecdotally, while in Europe I do see it extremely commonplace for women to have a few wines while pregnant.
[0] https://en.wikipedia.org/wiki/Fetal_alcohol_spectrum_disorde...
I’m concerned the medical field is what is being aspired too.
The number of examples of where standard of Care deviates from empirical research is staggering.
We still lose hundreds of thousands of lives a year in the US to preventable medical errors.
Many hospitals still refuse to use things like simple checklists that are proven to save lives.
The number of examples of where standard of Care deviates from empirical research is staggering.
We still lose hundreds of thousands of lives a year in the US to preventable medical errors.
Many hospitals still refuse to use things like simple checklists that are proven to save lives.
So this is an area of my interest. I actually hope to one day be able to improve diagnosis of certain diseases through apps.
I have widespread tendinitis caused by a reaction to a medication. I have seen many physicians and ALL OF THEM are just terrible with dealing with stuff they don't have expertise in.
Also, my cousin recently got diagnosed with EDS she suffered for over 20 years but they just made the diagnosis now.
I cannot comprehend why doctors aren't just open when they have not diagnosed a disease with some symptoms and ask to schedule another appointment. They go type some terms into google and at least come up with a checklist or a plan to test for diseases. The worst thing is when you have done some ACTUAL RESEARCH and can even provide certain studies but they straight up don't have the will to look at it. I think many doctors are just getting burned out by the current state of the medical system in the US, which is definitely not helping.
As a SE I often say, I have never done that before but I'm sure with a little time I can get it done. Medicine is supposed to be based on science but some doctors do not trust drug companies and studies anymore, as that is another shit show. There is pressure on researchers to obtain positive results.
I have widespread tendinitis caused by a reaction to a medication. I have seen many physicians and ALL OF THEM are just terrible with dealing with stuff they don't have expertise in.
Also, my cousin recently got diagnosed with EDS she suffered for over 20 years but they just made the diagnosis now.
I cannot comprehend why doctors aren't just open when they have not diagnosed a disease with some symptoms and ask to schedule another appointment. They go type some terms into google and at least come up with a checklist or a plan to test for diseases. The worst thing is when you have done some ACTUAL RESEARCH and can even provide certain studies but they straight up don't have the will to look at it. I think many doctors are just getting burned out by the current state of the medical system in the US, which is definitely not helping.
As a SE I often say, I have never done that before but I'm sure with a little time I can get it done. Medicine is supposed to be based on science but some doctors do not trust drug companies and studies anymore, as that is another shit show. There is pressure on researchers to obtain positive results.
Too many people know that evidence will indicate that a policy that they don't want changed should be changed. This makes evidence gathering much, much harder.
> Who wouldn’t want science and empirical evidence to guide policy decisions.
One look at crime stats and this will not be a favorable idea.
One look at crime stats and this will not be a favorable idea.
Care to elaborate?
I assume this is a reference to the crime per race per capita numbers.
[deleted]
I think what OP alluded too is not understanding preexisting bias.
"Where should police patrol" ->where crime is highest.
Where is crime highest ->existing reports point to X neighborhood having the most arrests.
Without asking "why" it may ignore That neighborhood is black, so historically racist decisions lead to extra patrolling (but no evidence of increased crime rates) which led to more arrests.
Without asking "why" it may ignore That neighborhood is black, so historically racist decisions lead to extra patrolling (but no evidence of increased crime rates) which led to more arrests.
Your logic could be misleading if not parsed carefully. It's not that police shouldn't patrol where crime is highest--of course they should patrol there, if they know where that is. Rather, the point is that "most arrests" is not necessarily an accurate way to detect where crime is highest.
Precisely. Thank you for clarifying!
This is easy. What are we optimizing for? Crime fighting effectiveness of police officers. Can we measure that? Sure, it's number of convictions per officer-hour of patrol.
You could have pointed out that number of arrests per area is the wrong measure and why. That would have been rational. Instead, you chose to drag identity politics into it. Brilliant.
You could have pointed out that number of arrests per area is the wrong measure and why. That would have been rational. Instead, you chose to drag identity politics into it. Brilliant.
I'm not saying "crime arrests per area is wrong".
@xamuel explained it better than I could:
>It's not that police shouldn't patrol where crime is highest--of course they should patrol there, if they know where that is. Rather, the point is that "most arrests" is not necessarily an accurate way to detect where crime is highest.
> I'm not saying "crime arrests per area is wrong".
But that's what you should have said if you were making a rational argument. Instead you made it a political statement by playing the racism card.
But that's what you should have said if you were making a rational argument. Instead you made it a political statement by playing the racism card.
fein(1)
Yes, except this is trivial to normalize for - just track arrests per cop. I feel like I constantly see criticisms like this, that use extremely naive metrics as straw men to argue against the idea of measuring things. It's extremely misguided (I know you're just explaining what he meant and not necessarily promulgating that idea yourself).
Sure, but then a conservative outcry is guaranteed: "Everybody can see that blacks are more criminal, the statistics show it clearly. But now the government is trying to hide it by gaming the data!"
As the public doesn't understand statistics it would be easy to make that claim a truth by repeating it a few times.
As the public doesn't understand statistics it would be easy to make that claim a truth by repeating it a few times.
Everyone manipulates statistics in their favor, be that to support a claim ("the numbers say that if we do X then Y", guns/less gun deaths, social healthcare/improve quality of life) or to cause outcry ("you can't say X is/are/has Y, the numbers are Z", gender A/higher standard deviation for Q/sexist, monopolies/damaging/biased) regardless of the side of politics you are on in the US.
Statistics are hard and the implications of some result are sometimes not intuitive. People don't care about what a standard deviation is, they just want more standard deviation than the other people over there because that makes them better.
Statistics are hard and the implications of some result are sometimes not intuitive. People don't care about what a standard deviation is, they just want more standard deviation than the other people over there because that makes them better.
I'm not sure what your point is here: "Sometimes people lie or obfuscate...therefore let's not bother with the truth" ?
More like: "In a political context gathering data is not at all a guaranteed way to get further to the truth."
I don't want to make a case for dismissing an empirical approach per se!
An current example from Germany right now: There's a part of the country that's in uproar because of new immigrants that came in 2015. Some people felt threatened and demanded more police in public spaces were immigrant youth gathers. So more police is present at those public spaces. Naturally this resulted in them being aware of more violent events (mostly groups of immigrant kids scuffling), which made the immigrant share of violent crimes double, and people feel "rightfully" afraid, as the stats now proof their suspicions! But the reality of course is that German kids have fights in public, too; police just didn't take note and so those fights don't end up in the data gathered.
I don't want to make a case for dismissing an empirical approach per se!
An current example from Germany right now: There's a part of the country that's in uproar because of new immigrants that came in 2015. Some people felt threatened and demanded more police in public spaces were immigrant youth gathers. So more police is present at those public spaces. Naturally this resulted in them being aware of more violent events (mostly groups of immigrant kids scuffling), which made the immigrant share of violent crimes double, and people feel "rightfully" afraid, as the stats now proof their suspicions! But the reality of course is that German kids have fights in public, too; police just didn't take note and so those fights don't end up in the data gathered.
> More like: "In a political context gathering data is not at all a guaranteed way to get further to the truth."
It's not guaranteed, no. But it's the only way we have any hope of it.
> An current example from Germany right now: There's a part of the country that's in uproar because of new immigrants that came in 2015. Some people felt threatened and demanded more police in public spaces were immigrant youth gathers. So more police is present at those public spaces. Naturally this resulted in them being aware of more violent events (mostly groups of immigrant kids scuffling), which made the immigrant share of violent crimes double, and people feel "rightfully" afraid, as the stats now proof their suspicions! But the reality of course is that German kids have fights in public, too; police just didn't take note and so those fights don't end up in the data gathered.
Is that actually true, though? I'm not familiar with the figures, but it's not unreasonable to imagine there may genuinely be more violent incidents among migrants from a war torn region like that. There were certainly serious cultural assimilation problems:
https://en.wikipedia.org/wiki/New_Year%27s_Eve_sexual_assaul...
It's not guaranteed, no. But it's the only way we have any hope of it.
> An current example from Germany right now: There's a part of the country that's in uproar because of new immigrants that came in 2015. Some people felt threatened and demanded more police in public spaces were immigrant youth gathers. So more police is present at those public spaces. Naturally this resulted in them being aware of more violent events (mostly groups of immigrant kids scuffling), which made the immigrant share of violent crimes double, and people feel "rightfully" afraid, as the stats now proof their suspicions! But the reality of course is that German kids have fights in public, too; police just didn't take note and so those fights don't end up in the data gathered.
Is that actually true, though? I'm not familiar with the figures, but it's not unreasonable to imagine there may genuinely be more violent incidents among migrants from a war torn region like that. There were certainly serious cultural assimilation problems:
https://en.wikipedia.org/wiki/New_Year%27s_Eve_sexual_assaul...
> But the reality of course is that German kids have fights in public, too
Not deadly knife fights, though. We don't want to erase the difference between brutal murder (25 stab wounds) and "a scuffle", do we?
Not deadly knife fights, though. We don't want to erase the difference between brutal murder (25 stab wounds) and "a scuffle", do we?
Well, this isn't about murder. Murder rates didn't raise, but statistics for scuffles did. In both cases this is about crimes among immigrants, though, not about crimes of immigrants against Germans.
> In both cases this is about crimes among immigrants, though
Does that make it less of a problem?
Does that make it less of a problem?
Yes, as the entire point of this thread is problematic application of data in a political context. The data is violent crime stats. The false interpretation is fear of the native polulation. The native population is not affected by the violence, ergo the fear is misplaced.
That you brought up the murder in the first place is exactly the misuse of statistics I spoke of. About 400 people get murdered every year in Germany, and you bring up exactly the one recent case were the suspects are immigrants.
That you brought up the murder in the first place is exactly the misuse of statistics I spoke of. About 400 people get murdered every year in Germany, and you bring up exactly the one recent case were the suspects are immigrants.
Here are the murder offender data from the FBI [1]. Your job is to bring down murder rates. You have limited resources. How would you now allocate them using "evidence based policies"?
[1] - https://ucr.fbi.gov/crime-in-the-u.s/2015/crime-in-the-u.s.-...
[1] - https://ucr.fbi.gov/crime-in-the-u.s/2015/crime-in-the-u.s.-...
I think the comment above yours may be misleading. Perhaps lawmakers are following a different set of metrics (contributions from private prison industry) when considering their rulemaking.
"Evidence-based" seems to end up meaning practices like NHST where there is evidence involved, but the analysis is so messed up you can conclude anything you want from it.
And its really funny they use "evidence-based medicine" as a positive example. Basically this is what we should expect to happen:
>'“Evidence-based medicine has drifted in recent years from investigating and managing established disease to detecting and intervening in non-diseases,” Greenhalgh and colleagues wrote in BMJ in June.
Furthermore, the “evidence” favoring various treatments typically comes from trials in which companies decide which drugs to test, at what doses, on how many people. Often the experimental statistical “evidence” in such studies establishes a benefit for a drug that is of little practical value; the supposed benefit, while perhaps real in a mathematical sense, is so slight as to be meaningless for real patients. In other cases, perfectly sound evidence regarding a particular disease is rendered irrelevant in patients afflicted with more than one disorder (patients who are commonly seen in medical practice, but typically excluded from the trials that produced the evidence).
On top of all that, Greenhalgh and colleagues point out, the sheer volume of medical evidence makes assessing it all intelligently essentially impossible. Even the guidelines summarizing the evidence are too voluminous to be useful to doctors.
“The number of [evidence based] clinical guidelines is now both unmanageable and unfathomable,” Greenhalgh and coauthors note. In one 24-hour period in 2005, for instance, a hospital in the United Kingdom admitted 18 patients with 44 diagnoses. The relevant U.K. national guidelines for those patients totaled 3,679 pages. Estimated reading time: 122 hours.'" https://www.sciencenews.org/blog/context/evidence-based-medi...
And its really funny they use "evidence-based medicine" as a positive example. Basically this is what we should expect to happen:
>'“Evidence-based medicine has drifted in recent years from investigating and managing established disease to detecting and intervening in non-diseases,” Greenhalgh and colleagues wrote in BMJ in June.
Furthermore, the “evidence” favoring various treatments typically comes from trials in which companies decide which drugs to test, at what doses, on how many people. Often the experimental statistical “evidence” in such studies establishes a benefit for a drug that is of little practical value; the supposed benefit, while perhaps real in a mathematical sense, is so slight as to be meaningless for real patients. In other cases, perfectly sound evidence regarding a particular disease is rendered irrelevant in patients afflicted with more than one disorder (patients who are commonly seen in medical practice, but typically excluded from the trials that produced the evidence).
On top of all that, Greenhalgh and colleagues point out, the sheer volume of medical evidence makes assessing it all intelligently essentially impossible. Even the guidelines summarizing the evidence are too voluminous to be useful to doctors.
“The number of [evidence based] clinical guidelines is now both unmanageable and unfathomable,” Greenhalgh and coauthors note. In one 24-hour period in 2005, for instance, a hospital in the United Kingdom admitted 18 patients with 44 diagnoses. The relevant U.K. national guidelines for those patients totaled 3,679 pages. Estimated reading time: 122 hours.'" https://www.sciencenews.org/blog/context/evidence-based-medi...
Medicine and life sciences adopted bad statistics. Every single paper claims something along the lines of "what we observed is so incredibly unlikely (<5%) assuming complete randomness, it must mean our theory is true, our new drug works, the bad pollutant causes cancer, etc." Papers that should claim "what we observed is completely ordinary and there is nothing to see here" never get published.
Even if this was done correctly (it isn't, as John Ioannidis keeps pointing out), you'd get lots of false positive results (Ioannides again). The moment you look at a collection of statistics, the only available conclusion is "They all contradict each other!!1"
Until they ask physicists for help, medicine, life sciences, and many other fields will not be evidence based. We can call them "argument based", but that's not the same...
Even if this was done correctly (it isn't, as John Ioannidis keeps pointing out), you'd get lots of false positive results (Ioannides again). The moment you look at a collection of statistics, the only available conclusion is "They all contradict each other!!1"
Until they ask physicists for help, medicine, life sciences, and many other fields will not be evidence based. We can call them "argument based", but that's not the same...
Politics is mostly about power and ideology. Not truth/evidence/practicality. Even if people agree on the evidence they will still disagree on what to do about it.
Government tends to conduct studies prior to engaging in an effort but it's questionable how much evidence from the studies actually gets used in decision making.
Coming from the Chicago School of Economics, specialized in voodoo theories that don't pan out in the real world this is pretty funny.
Compared to what? Other economic schools of theory that worked out exactly as predicted?
Pretty much. The Chicago school was the one pushing for financial deregulation, IIRC.
I think the problem here is that we elect governments to solve collective problems, not personal problems. Collective problems arise from the interactions of large groups of people and they are generally intractable. Even if an economist tries to accurately model human behavior, it's really hard to do as you start adding more rational agents, and then you need a theory of production and a theory of expectations formation and all of a sudden what you are doing is just a toy to be tractable.
So the zone where there is clear unambiguous science is very small, and it's generally not the zone of contention in politics. Stuff like "don't swallow arsenic" -- it doesn't really help us to govern or solve any of our political questions.
So if we can't use science to show us how to organize ourselves, the debate turns to meta-approaches which are basically philosophies of how we approach problems.
For example, a liberal might start with a Rousseau-type blank slate view of human nature and a various declaration of rights, both of which are basically statements of faith, and then try to think of some way that government can secure those rights through a period of debate, and then adopt whatever seems right to the people doing the debating, even if it means overturning long established customs.
A conservative might go -- woah, there is hidden information in customs and traditions, which we may not appreciate (it's hidden) but, as an article of faith, the fact that these customs have lasted so long means that they are probably useful for _something_, and I doubt your ability to think of every unintended consequence of your policy, so I'm going to oppose it, and instead suggest you do something much smaller, as a pilot, over here for a generation, and then see what happens.
And a libertarian might be led to say, "Your proposal infringes on my personal liberties, and thus, as an article of faith, must lead to bad outcomes."
Etc.
So the real debates we have are actually methodological debates given that we are working under imperfect information and limited knowledge about how complex interacting social systems work. Saying "we should use more science" is vacuous B.S. What we need is a better functioning public policy discourse that supports more course corrections, allows more people to air their views, and eliminates certain known distortions, like the influence of big money groups on campaigns. That's a hell of a lot more useful than pretending that scientists can be politicians or that politicians should be scientists.
I think the problem here is that we elect governments to solve collective problems, not personal problems. Collective problems arise from the interactions of large groups of people and they are generally intractable. Even if an economist tries to accurately model human behavior, it's really hard to do as you start adding more rational agents, and then you need a theory of production and a theory of expectations formation and all of a sudden what you are doing is just a toy to be tractable.
So the zone where there is clear unambiguous science is very small, and it's generally not the zone of contention in politics. Stuff like "don't swallow arsenic" -- it doesn't really help us to govern or solve any of our political questions.
So if we can't use science to show us how to organize ourselves, the debate turns to meta-approaches which are basically philosophies of how we approach problems.
For example, a liberal might start with a Rousseau-type blank slate view of human nature and a various declaration of rights, both of which are basically statements of faith, and then try to think of some way that government can secure those rights through a period of debate, and then adopt whatever seems right to the people doing the debating, even if it means overturning long established customs.
A conservative might go -- woah, there is hidden information in customs and traditions, which we may not appreciate (it's hidden) but, as an article of faith, the fact that these customs have lasted so long means that they are probably useful for _something_, and I doubt your ability to think of every unintended consequence of your policy, so I'm going to oppose it, and instead suggest you do something much smaller, as a pilot, over here for a generation, and then see what happens.
And a libertarian might be led to say, "Your proposal infringes on my personal liberties, and thus, as an article of faith, must lead to bad outcomes."
Etc.
So the real debates we have are actually methodological debates given that we are working under imperfect information and limited knowledge about how complex interacting social systems work. Saying "we should use more science" is vacuous B.S. What we need is a better functioning public policy discourse that supports more course corrections, allows more people to air their views, and eliminates certain known distortions, like the influence of big money groups on campaigns. That's a hell of a lot more useful than pretending that scientists can be politicians or that politicians should be scientists.
We need more people like yourself sowing discourse on HN.
Good read, and great take on the current stream of consciousness.
Good read, and great take on the current stream of consciousness.
I agree that a lot of policy decision are value judgements and can't be fully explained by scientific approaches.
But I definitely would like to see more hard numbers in political discourse. For example, what was the effect of previous tax cuts that increased deficits? Just put out the numbers: government revenue, business investment and so on. At least people (maybe) could have a rational foundation for their opinions. But I guess I am asking too much...
But I definitely would like to see more hard numbers in political discourse. For example, what was the effect of previous tax cuts that increased deficits? Just put out the numbers: government revenue, business investment and so on. At least people (maybe) could have a rational foundation for their opinions. But I guess I am asking too much...
I was horrified at the business tax cuts that we recently passed, and I was sure they wouldn't result in any real benefits for investment.
But, to me, that's more of an example of money in politics and money having a big loudspeaker rather than "more science needed".
Moreover, I don't think that people who voted for Trump, voted for him because they favored corporate tax cuts, but I think it was more of a protest vote.
Of course, I could be wrong, but the problem is the outsized influence of lobbyists and the wealthy in our political system -- that was a Koch brothers job. I don't think conservatives love the Koch brothers, to be honest.
Let's remember "science" was how the economists pushed through all the banking de-regulation and liberalization of international capital flows which I am so opposed to, as well as other disastrous reforms. For example, the sadism against Greece in the EU was led by PhDs, and many PhDs pushed for more austerity.
So, for me, "more science" isn't the solution, but something like public financing of campaigns might be a better solution. I think the skepticism of technocrats is quite healthy.
But, to me, that's more of an example of money in politics and money having a big loudspeaker rather than "more science needed".
Moreover, I don't think that people who voted for Trump, voted for him because they favored corporate tax cuts, but I think it was more of a protest vote.
Of course, I could be wrong, but the problem is the outsized influence of lobbyists and the wealthy in our political system -- that was a Koch brothers job. I don't think conservatives love the Koch brothers, to be honest.
Let's remember "science" was how the economists pushed through all the banking de-regulation and liberalization of international capital flows which I am so opposed to, as well as other disastrous reforms. For example, the sadism against Greece in the EU was led by PhDs, and many PhDs pushed for more austerity.
So, for me, "more science" isn't the solution, but something like public financing of campaigns might be a better solution. I think the skepticism of technocrats is quite healthy.
I guess I would like to see more data and less opinion. Especially in economics.
Your comment shows a severe lack of knowledge what economics research is all about, please correct me if I'm wrong.
I have two graduate degrees (econometrics) and while I understand and can reason with most of these models I'd argue exactly the opposite. Economics is not a "real science" or rather, it is the science of humans, and we know for sure that humans are fickle and irrational.
Economics is not a hard science yet sometimes is treated as such, and this research is then taken true as gospel (which I argue is wrong). There is a lot of things wrong with economics (I'm a fan of behavioral economics in particular) but "I want to see more data" is a non-argument. You're acting like economists are some armchair scientists waving around wet fingers in the air.
I have two graduate degrees (econometrics) and while I understand and can reason with most of these models I'd argue exactly the opposite. Economics is not a "real science" or rather, it is the science of humans, and we know for sure that humans are fickle and irrational.
Economics is not a hard science yet sometimes is treated as such, and this research is then taken true as gospel (which I argue is wrong). There is a lot of things wrong with economics (I'm a fan of behavioral economics in particular) but "I want to see more data" is a non-argument. You're acting like economists are some armchair scientists waving around wet fingers in the air.
This sort of radical skepticism is generally unhelpful. It is simply not the case that the economics literature, on the whole, conveys no information. It conveys quite a bit, and often correctly answers a myriad of serious, useful, practical political questions. There are certainly edge cases that get a lot of attention upon which the data and consensus are split (e.g. minimum wage disemployment effects), but using those edge cases to disregard the research of an entire field (in favor of what, anyway?) is profoundly misguided.
Economic decisions have to be made by politicians. That is an unavoidable fact. They can either attempt to use the best data available to optimize the values that they care about...or they can just do whatever they feel is best, informed by, what? Their gut? Their down home common sense?
Economic decisions have to be made by politicians. That is an unavoidable fact. They can either attempt to use the best data available to optimize the values that they care about...or they can just do whatever they feel is best, informed by, what? Their gut? Their down home common sense?
Me, too, but especially in an area like economics, the rich are going to fund studies and be able to dominate the profession, so "more data" is tricky. I guess I am just asking for better economics that had a better track record in out of sample predictions, but that's wishful thinking in any social science.
Well, outside of Oakeshott (Rationalism in Politics) I've not heard a conservative talk like that, but great comment.
A story from Chesterton's Heretics (1905):
Suppose that a great commotion arises in the street about something, let us say a lamp-post, which many influential persons desire to pull down. A grey-clad monk, who is the spirit of the Middle Ages, is approached upon the matter, and begins to say, in the arid manner of the Schoolmen, “Let us first of all consider, my brethren, the value of Light. If Light be in itself good–” At this point he is somewhat excusably knocked down. All the people make a rush for the lamp-post, the lamp-post is down in ten minutes, and they go about congratulating each other on their unmediaeval practicality. But as things go on they do not work out so easily. Some people have pulled the lamp-post down because they wanted the electric light; some because they wanted old iron; some because they wanted darkness, because their deeds were evil. Some thought it not enough of a lamp-post, some too much; some acted because they wanted to smash municipal machinery; some because they wanted to smash something. And there is war in the night, no man knowing whom he strikes. So, gradually and inevitably, to-day, to-morrow, or the next day, there comes back the conviction that the monk was right after all, and that all depends on what is the philosophy of Light. Only what we might have discussed under the gas-lamp, we now must discuss in the dark.
A story from Chesterton's Heretics (1905):
Suppose that a great commotion arises in the street about something, let us say a lamp-post, which many influential persons desire to pull down. A grey-clad monk, who is the spirit of the Middle Ages, is approached upon the matter, and begins to say, in the arid manner of the Schoolmen, “Let us first of all consider, my brethren, the value of Light. If Light be in itself good–” At this point he is somewhat excusably knocked down. All the people make a rush for the lamp-post, the lamp-post is down in ten minutes, and they go about congratulating each other on their unmediaeval practicality. But as things go on they do not work out so easily. Some people have pulled the lamp-post down because they wanted the electric light; some because they wanted old iron; some because they wanted darkness, because their deeds were evil. Some thought it not enough of a lamp-post, some too much; some acted because they wanted to smash municipal machinery; some because they wanted to smash something. And there is war in the night, no man knowing whom he strikes. So, gradually and inevitably, to-day, to-morrow, or the next day, there comes back the conviction that the monk was right after all, and that all depends on what is the philosophy of Light. Only what we might have discussed under the gas-lamp, we now must discuss in the dark.
I think most conservatives defer to and value tradition and customs because they feel this, even though they may not articulate it in the same way. Roger Scruton gives a nice summary, in his book on conservatism:
https://www.amazon.com/Conservatism-Invitation-Tradition-Rog...
Similarly liberals who say things like "all men are created equal and have the same rights and can institute whatever practices seem right to them" may not know anything about Rousseau and blank slates, but they are expressing these views.
Thanks for the Chesterton quote. Chesterton is awesome. A bit of a lunatic, but also insightful and fun to read.
Similarly liberals who say things like "all men are created equal and have the same rights and can institute whatever practices seem right to them" may not know anything about Rousseau and blank slates, but they are expressing these views.
Thanks for the Chesterton quote. Chesterton is awesome. A bit of a lunatic, but also insightful and fun to read.
I don't know whether you've also read these books or not, but for others who are really interested in this, I can't recommend Anti-Fragile and Skin In The Game by Taleb that explores this view and topic. I just finished them both and I feel the book can explain it in a reasonable way, but don't get too serious on his name-calling. It gets a bit funny when you know where he comes from.
typo: I meant I can't recommend them enough to have more understanding about this topic.
I know of Taleb, but haven't read him. I've read some of his essays, and he has good points to make. The name calling doesn't bother me (since he hasn't gored one of my sacred cows, not because I am especially enlightened)
> So the zone where there is clear unambiguous science is very small, and it's generally not the zone of contention in politics. Stuff like "don't swallow arsenic" -- it doesn't really help us to govern or solve any of our political questions.
So, what, we should just make things up? Do you have some other decision procedure aside from the scientific method you would like to employ?
> What we need is a better functioning public policy discourse that supports more course corrections, allows more people to air their views, and eliminates certain known distortions, like the influence of big money groups on campaigns. That's a hell of a lot more useful than pretending that scientists can be politicians or that politicians should be scientists.
And what is going to inform that public policy discourse? Are we just going to talk at each other about our prior beliefs and implicit biases until one side gets tired? Or should we use the scientific method to answer empirical questions about the impacts of policy, and then debate which values we should use the knowledge gleaned therefrom to optimize? I know which one i'd prefer.
So, what, we should just make things up? Do you have some other decision procedure aside from the scientific method you would like to employ?
> What we need is a better functioning public policy discourse that supports more course corrections, allows more people to air their views, and eliminates certain known distortions, like the influence of big money groups on campaigns. That's a hell of a lot more useful than pretending that scientists can be politicians or that politicians should be scientists.
And what is going to inform that public policy discourse? Are we just going to talk at each other about our prior beliefs and implicit biases until one side gets tired? Or should we use the scientific method to answer empirical questions about the impacts of policy, and then debate which values we should use the knowledge gleaned therefrom to optimize? I know which one i'd prefer.
> So the zone where there is clear unambiguous science is very small, and it's generally not the zone of contention in politics.
Unless it's climate science, reproductive health, evolutionary biology, the existence of "electromagnetic hypersensitivity", whether fracking causes earthquakes, or a list of other topics where denialism and, yes, outright lying have arisen to challenge science that the scientific community regards as unambiguously settled.
Unless it's climate science, reproductive health, evolutionary biology, the existence of "electromagnetic hypersensitivity", whether fracking causes earthquakes, or a list of other topics where denialism and, yes, outright lying have arisen to challenge science that the scientific community regards as unambiguously settled.
In terms of public policy you have to decide what you're optimizing for and that decision can't be made with data alone because it does not help resolve questions of value and fairness.