The Data/Human Goal Gap(ken.arneson.name)
ken.arneson.name
The Data/Human Goal Gap
http://ken.arneson.name/2016/06/the-data-human-goal-gap
6 comments
I think Goodhart's law is closest to what he is getting at. There is also the aphorism of Deming, that the most important figures for managing your business are unknown and unknowable. We want to measure successful learning. Instead, we measure test scores. Whatever figures actually measure successful learning are unknown and likely unknowable. "Management", in this case the parents, teachers and administration, are focused on test scores and other marks of achievement. College admissions departments have their own measures that may include all of these things. But, in the end, hitting high values for these targets may not be reaching the actual goal. It's a good way of looking at an age-old problem.
I imagine this is a corollary of the phrase, "you manage what you measure."
Usually the metrics we measure are an indirect substitute for what we actually want to measure, e.g., lines of code as a measure of programmer productivity. What inevitably happens is that behaviour is optimised for the metric, and not what the metric was intending to measure. It's really just gaming the system.
Usually the metrics we measure are an indirect substitute for what we actually want to measure, e.g., lines of code as a measure of programmer productivity. What inevitably happens is that behaviour is optimised for the metric, and not what the metric was intending to measure. It's really just gaming the system.
Tl;dr if you don't know what your goals are, you won't achieve them. Shocking!
Though in half his examples (education, politics, baseball), the complaint actually seems to be "other people don't share my goals".
What this has to do with measurement, I have no idea. Would this problem occur less if we non-quantitatively tried to achieve the wrong goal?
Though in half his examples (education, politics, baseball), the complaint actually seems to be "other people don't share my goals".
What this has to do with measurement, I have no idea. Would this problem occur less if we non-quantitatively tried to achieve the wrong goal?
the complaint actually seems to be "other people don't share my goals
I guess. But i think what he is driving at is, in data driven world, we can lose sight of bigger picture ie in quest for test scores we can unreasonably stress kids out, in quest for profits we can shit on employees and the environment, etc.
Just taking the last point, people generally like parks and nature. What person would say "oh yes id love to pollute my closest river?"
but then suppose you ran a factory on a river, and it would cost your company 90% of profits to dispose of your factory's waste in an environmentally sound way, and you are measured on how much profits you bring in. Or you can dump in the river for free.
Would you choose to dump your waste into the river? I think most people would.
I guess. But i think what he is driving at is, in data driven world, we can lose sight of bigger picture ie in quest for test scores we can unreasonably stress kids out, in quest for profits we can shit on employees and the environment, etc.
Just taking the last point, people generally like parks and nature. What person would say "oh yes id love to pollute my closest river?"
but then suppose you ran a factory on a river, and it would cost your company 90% of profits to dispose of your factory's waste in an environmentally sound way, and you are measured on how much profits you bring in. Or you can dump in the river for free.
Would you choose to dump your waste into the river? I think most people would.
I'd like to think that most people wouldn't. I'd rather run no business at all.
One solution to this problem is making "flipped classroom" a mandatory at all the schools.i.e. students go thru lectures/videos at home and do homework collaboratively in the Classroom. The Real world is more collaborative compared to our schools. Flipped Classroom is proven to increase scores as well. Question: What efforts are you willing to put in making this happen?
> The Real world is more collaborative compared to our schools
People keep saying this. I must be living in a different shard of the real world than those people. Collaboration is nice, but at the end of the day, the work has to actually be done, not just talked about.
People keep saying this. I must be living in a different shard of the real world than those people. Collaboration is nice, but at the end of the day, the work has to actually be done, not just talked about.
Distinguishing between work requiring a heads-down approach and work that can be sped up by collaboration (shared understanding, parallelisable tasks) needs to be learned and school's a good a place to get started on that as any.
I haven't seen any sources showing that flipped classrooms were proven to increase scores- what studies are you referring to?
Edit: grammar
Edit: grammar
I used to call them 'Prosthetic Goals' http://michaelfeathers.typepad.com/michael_feathers_blog/200...
https://en.wikipedia.org/wiki/Goodhart%27s_law https://en.wikipedia.org/wiki/Campbell%27s_law https://en.wikipedia.org/wiki/Reification_(fallacy)