> You didn't really address the examples which mschuster91 provided and that's important to understanding the problem:
On the contrary, I addressed it entirely. mschuster91 seems to be under the impression that the teacher evaluation schemes boil down to nothing but the simplest possible before-after comparison of grades of students, ignoring all issues of demographics, differing student quality, differing school circumstances, etc. Such a scheme is indeed absurd, as his counterexample proves, but it is not what has been proposed by pretty much everyone! The actual proposals are well aware of what he thinks is the fatal problem, and go to often elaborate lengths to model and adjust for these sorts of heterogeneities in order to quantify the value-added of a particular teacher. The problem is recognized, included, and mostly dealt with. Whether the solution works entirely or is worthwhile is unclear, but he's arguing against a strawman.
> One estimate has ~12% of NYC public school teachers being punished by the flawed VAM in use there:
So I've looked at http://mathbabe.org/2015/04/02/the-arbitrary-punishment-of-n... and I have zero idea what she is trying to show. She assumes independence and treats it as a coin flip. Ummm.... what? With that sort of logic, you could show no one could expect to score a 1600 on the SAT. When criticized she links to a real analysis†, which shows considerable non-independence which means her numbers are wrong and will overstate how many will be denied tenure based on the VAMs. By the way, why are you phrasing it as 'punished'? That sounds like you're assuming your conclusion. If VAM doesn't affect hiring decisions, there's no point to bothering with it in the first place is there, but if it does affect hiring decisions, that means teachers are being 'punished'...?
† not that I think too much of it either, since it relies mostly on an argument from incredulity and pointing angrily at some scatterplots, and tries to ignore the r=.35 correlation of ratings from two subjects; to put an r=.35 in perspective, the correlation between years of education and intelligence is only ~r=.55! Even the best IQ tests won't correlate with Gf more than r=.7 or so. r=.35 is pretty good for a single pair. I don't know why he thinks a .24 is 'minuscule' when that means you're predicting half of variance... (I wonder if this is a graphing problem? He doesn't seem to jitter the datapoints, which for a large amount of discrete data will hide a lot of the density; a plot of r=.35 of n=6k should look much more striking, like this: http://imgur.com/KcwmJJH ) For implications, look at the first graph and think about classification rates. Look at the datapoints at 100 along one axis, then look across to see how many correspond to <10 on the other; hardly any do, and the 100s are almost all mapped onto 80+ on the other axis. Or look at the 0s. In terms of identifying the bottom decile, it's doing a good job.
> I'm not saying there aren't any valid reasons for doing it.
> I'm just saying the there's a "meta-lesson" there that has to be corrected.
And I'm saying that your meta-lesson is not a good idea as it will tend to teach underconfidence. The real world does not always let you off with a "I don't know"; you may not know to some high degree of certainty whether a cancer treatment is a good idea, but nevertheless you must decide whether or not to do it.
> The fallacy with using tests of the kids as a marker for the quality of the teachers is that you just can't do that and get reliable results.
Which is not what is being proposed by people arguing for teacher evaluations drawing on standardized testing (https://en.wikipedia.org/wiki/Value-added_modeling), as the very name 'value-added' implies.
Test motivation differs from person to person, so if you don't encourage everyone to at least try every question and guess, you'll get differences in scores which don't reflect the child's difference in knowledge (which is what the test is trying to measure) so much as willingness to try or guess. This willingness can differ systematically so you might get drastically lower scores for poorer children than they should. (This is one of the reasons schools like psychologists to do IQ tests, because they can spot when a child isn't trying or is deliberately underperforming.) So that's one reason. Another reason is that it's rare to have no idea whatsoever; even if you feel entirely uncertain, you can still often guess at above chance rates, showing that you did know more than nothing. Forced-choice methodologies expose that knowledge and again make the tests more accurate, because more items means more effective at distinguishing between students.
(Imagine a test of 10 questions, each substantially harder; one student manages to answer correctly up to question 5 before starting to feel uncertain and refusing to answer any more, and a second gets up to question 6. How sure are you that #1 knows less than #2? Now imagine that they instead 'guessed' on the remaining 5 questions, and #1 got 3/5 right and #2 got 1/4. Now how sure are you? Haven't you learned something from this apparently 'useless' guessing?)
> But in the real world, there is no 25% credit for guessing.
You can no more refuse to guess in the real world than you can refuse to make choices, take actions, or let time pass.
Looks like one'd describe him as an insurance adjuster:
> Kafka was rapidly promoted and his duties included processing and investigating compensation claims, writing reports, and handling appeals from businessmen who thought their firms had been placed in too high a risk category, which cost them more in insurance premiums.[41] He would compile and compose the annual report on the insurance institute for the several years he worked there.
And of course, Wallace Stevens was an insurance lawyer.
> This was a favorite zinger that my evolutionary biology teach liked to spring on unsuspecting students that tried to argue that they could demonstrate that low iq among AAs was due to genetic differences.
And what happens when AAs migrate out of America, or when Africans migrate into America, hmm? The debate is not that simplistic and easily resolved, and your professor did you a disservice by pretending that it is and not discussing why his anecdote is not airtight (for example, immigrant samples are almost always contaminated by serious selection effects which are hard to measure and vary by group). By the way, how sure are you that your professor was even right in the first place (https://unsafeharbour.wordpress.com/2012/01/13/burakumin-and...)?
You can download your genotype SNP data as a backup. I don't know about the health reports (are they so voluminous that you cannot open up each in a tab and then save them all as a batch of HTML files?) but there's a partial replacement in the form of the Promethease service which will take a SNP export and try to summarize any interesting hits.
> The desire to believe in equality is weird sometimes.
It's not that weird. If you look back in the HN archives, you can find one or two submissions from transsexual authors who argue, essentially, that after changing genders, they were discriminated against more, and that this proves that society/Silicon Valley/etc must be extremely sexist & discriminatory because nothing else about them changed; many of the HN commenters agreed with the claims.
> The point remains the article specifically cites a situation where the adoption process fails
No, it doesn't. The article vaguely alludes to a third-hand description of the failure of a legal tactic which taken literally is nonsensical; and as far as I can tell, when those two people tried in the sensible standard manner used by everyone else (the manner in which the article is about), did succeed.
"adopting each other" is meaningless and the citation is vague. Some googling of The Advocate's website suggests this is probably a reference to Baker and McConnell, where the regular adoption did succeed: http://www.nytimes.com/2015/05/17/us/the-same-sex-couple-who...
> This is completely fine if you're tooling around in a research lab or industrial lab, but even a 1% loss is probably too much for a human brain to bear and remain the same as before.
You probably lose at least that many neurons over a lifetime; consider the shrinking volume of the brain with age. And losses definitely easily exceed 1% in early Alzheimer's or dementia, but while not fun, people with early Alzheimer's clearly have not died or ceased to exist!
(And anyway, the real question about the dead cells is whether they still have the information about their synaptic weights and other functional information. Being able to revive the cell is overkill; the focus on revival is as an _a fortiori_ argument, since any revival proves that, even in the absence of future scanning advances, it must be possible.)
It also sounds like a motivating example for machine-checked proofs: if one could feed Mochizuki's proof into Coq or something and be assured that it was correct, even if only in a purely formal sense, I suspect there would be much more interest in grappling with the concepts to understand what the proof is doing, whether it's acceptable, and why the proof is correct. As it stands, there's the risk of a wild goose chase.
That doesn't matter if they're doing paired comparisons, as they are. 2 comparisons of a few hundred images yields far more reliable results than a few hundred comparisons of 2 images, because there are measurements both within and between images.
An analogous situation: if someone runs a blood pressure clinical trial, whose results will you believe more - a trial which measures one person's blood pressure on and off a drug several hundred times over a year or two, or a trial which measures several hundred peoples' blood pressure at the beginning and end of the trial?
Obviously the latter, because we know that there are big differences between people which must be measured if we want to make reliable predictions about the effect of the drug in the rest of the population, while additional blood pressure measurements of a person only reduces variability a little bit and helps only a little (because most of the sampling error was removed by the first pair of measurements, and further measurements leave the bulk of variance unaffected).
That the Russians have been doing it for years and it's been so slow to catch on suggests the opposite, for the same reason that the increasing spread of autonomous cars is not a sign of the decay & fall of the West but its continued technological vitality.
> That's right, that terribly communist government suppressing its people and free speech, is actually responsible for lifting 500 million people out of poverty over the last 30 years.
Indeed, China's success has been a great testimony to the efficacy of centralized planning and communism...
Based on what criteria, exactly? 606 comparisons is more than enough to rule out large differences, especially considering that the testers were heavily primed to look for even the tiniest difference and making forced-choices about difference or no difference. Less than 1% difference suggests no real difference.
Then use reproducible builds. You inspect the source code, and if it does what it's supposed to, compile it reproducibly, and compare the binary with the official one which has a signature. If they match, then everything is kosher.
"I discovered that at least one person in Europe in the 1730s not only knew about Buddhism but had studied Buddhist philosophy for years. His name was Ippolito Desideri, and he had been a Jesuit missionary in Tibet. In 1728, just before Hume began the Treatise, Desideri finished his book, the most complete and accurate European account of Buddhist philosophy to be written until the 20th century. The catch was that it wasn’t published. No Catholic missionary could publish anything without the approval of the Vatican—and officials there had declared that Desideri’s book could not be printed. The manuscript disappeared into the Church’s archives...Desideri accepted the challenge. He spent the next five years in the Buddhist monasteries tucked away in the mountains around Lhasa. The monasteries were among the largest academic institutions in the world at the time. Desideri embarked on their 12-year-long curriculum in theology and philosophy. He composed a series of Christian tracts in Tibetan verse, which he presented to the king. They were beautifully written on the scrolls used by the great Tibetan libraries, with elegant lettering and carved wooden cases...He also translated the work of the great Buddhist philosopher Tsongkhapa into Italian. In his book, Desideri describes Tibetan Buddhism in great and accurate detail, especially in one volume titled “Of the False and Peculiar Religion Observed in Tibet.” He explains emptiness, karma, reincarnation, and meditation, and he talks about the Buddhist denial of the self...Desideri overcame Himalayan blizzards, mountain torrents, and war. But bureaucratic infighting got him in the end. Rival missionaries, the Capuchins, were struggling bitterly with the Jesuits over evangelical turf, and they claimed Tibet for themselves. Michelangelo Tamburini, the head of the Jesuits, ordered Desideri to return to Europe immediately, until the territory dispute was settled. The letter took two years to reach Tibet, but once it arrived, in 1721, Desideri had no choice. He had to leave. He spent the next 11 years writing and rewriting his book and appealing desperately to the Vatican to let him return to Tibet. It had clearly become the only place where he really felt that he was himself. In 1732 the authorities finally ruled—in favor of the Capuchins. His book would not be published and he could never return. He died four months later."
How truly awful. He mastered Tibetan, spent decades studying the profoundest and subtlest Buddhist philosophy with great success, wrote it all up for the Europeans back home whose understanding was de minimis, and from reading old books I could well credit that his work was superior to anything published until 1900 (or perhaps later) - implying the Catholic Church singlehandedly set back understanding by over 179 years, for reasons that strike one as either incompetent or psychopathicly malicious.
And this is just the summary by Gopnik, who is trying to paint the Catholic Church in as positive, tolerant, and globalizing a way as possible!
On the contrary, I addressed it entirely. mschuster91 seems to be under the impression that the teacher evaluation schemes boil down to nothing but the simplest possible before-after comparison of grades of students, ignoring all issues of demographics, differing student quality, differing school circumstances, etc. Such a scheme is indeed absurd, as his counterexample proves, but it is not what has been proposed by pretty much everyone! The actual proposals are well aware of what he thinks is the fatal problem, and go to often elaborate lengths to model and adjust for these sorts of heterogeneities in order to quantify the value-added of a particular teacher. The problem is recognized, included, and mostly dealt with. Whether the solution works entirely or is worthwhile is unclear, but he's arguing against a strawman.
> One estimate has ~12% of NYC public school teachers being punished by the flawed VAM in use there:
So I've looked at http://mathbabe.org/2015/04/02/the-arbitrary-punishment-of-n... and I have zero idea what she is trying to show. She assumes independence and treats it as a coin flip. Ummm.... what? With that sort of logic, you could show no one could expect to score a 1600 on the SAT. When criticized she links to a real analysis†, which shows considerable non-independence which means her numbers are wrong and will overstate how many will be denied tenure based on the VAMs. By the way, why are you phrasing it as 'punished'? That sounds like you're assuming your conclusion. If VAM doesn't affect hiring decisions, there's no point to bothering with it in the first place is there, but if it does affect hiring decisions, that means teachers are being 'punished'...?
† not that I think too much of it either, since it relies mostly on an argument from incredulity and pointing angrily at some scatterplots, and tries to ignore the r=.35 correlation of ratings from two subjects; to put an r=.35 in perspective, the correlation between years of education and intelligence is only ~r=.55! Even the best IQ tests won't correlate with Gf more than r=.7 or so. r=.35 is pretty good for a single pair. I don't know why he thinks a .24 is 'minuscule' when that means you're predicting half of variance... (I wonder if this is a graphing problem? He doesn't seem to jitter the datapoints, which for a large amount of discrete data will hide a lot of the density; a plot of r=.35 of n=6k should look much more striking, like this: http://imgur.com/KcwmJJH ) For implications, look at the first graph and think about classification rates. Look at the datapoints at 100 along one axis, then look across to see how many correspond to <10 on the other; hardly any do, and the 100s are almost all mapped onto 80+ on the other axis. Or look at the 0s. In terms of identifying the bottom decile, it's doing a good job.