Why Kids Should Grade Teachers(theatlantic.com)
theatlantic.com
Why Kids Should Grade Teachers
http://www.theatlantic.com/magazine/archive/2012/10/why-kids-should-grade-teachers/309088/?single_page=true
73 comments
For example the tough math teacher would get a bad evaluation, but the cream-puff teacher who never assigned homework and gave everyone A's would get a good evaluation.
I always hear this objection, and it's usually brought up to discourage even experimental data gathering. I don't buy it at all, and I think it says more about the objectors' mentality than about the students'.
I always hear this objection, and it's usually brought up to discourage even experimental data gathering. I don't buy it at all, and I think it says more about the objectors' mentality than about the students'.
It's not a crazy objection; in other environments (for instance, inside corporations), peer and customer review regimes do foster CYA cultures. Also: the best teachers tend not to be the friendliest or easiest, and the issue is not just "will those teachers get bad performance reviews", but rather "will the act of measuring disrupt the behavior we're trying to measure?"
In many other situations that foster CYA cultures, the people doing the evaluating have as much to lose (might lose my job, lose a bonus, etc).
Student/teacher situations are somewhat unique in that students don't choose to be there, and will usually be gone in 1 year, and definitely in a few years. Students have far less to lose in this situation. That may make some of the be more harsh, but it likely (based on the article) will just make more of them honest.
I can't be honest with my boss because he might make life a living hell or fire me. I can certainly be honest with a teacher in a school who I'll never see again in my life.
Student/teacher situations are somewhat unique in that students don't choose to be there, and will usually be gone in 1 year, and definitely in a few years. Students have far less to lose in this situation. That may make some of the be more harsh, but it likely (based on the article) will just make more of them honest.
I can't be honest with my boss because he might make life a living hell or fire me. I can certainly be honest with a teacher in a school who I'll never see again in my life.
The story addressed this at length. Actually, the student evaluations were an excellent predictor of test score gains.
The article discusses this issue extensively; I already quoted one part here: http://news.ycombinator.com/item?id=4560031 .
Here's just an example of what high school students do for fun when the "popular kids" band together http://www.detroitnews.com/article/20120924/METRO/209240341
Basing teachers' pay and job security on surveys from students seems like a good idea, especially given the numbers mentioned in the article. One problem is that it might give too much power to students.
I was a dick back in high school. The hacker I was back then would have figured out exactly how the testing and metrics were set up (public information) and organized a union of students to manipulate it. I can't do much with standarized test scores, they reflect on me. But a teacher quality survey? That's just a weapon.
Things like this make me wish that we had some kind of Hacker in Chief, to figure out how to circumvent new systems before they get implemented.
I was a dick back in high school. The hacker I was back then would have figured out exactly how the testing and metrics were set up (public information) and organized a union of students to manipulate it. I can't do much with standarized test scores, they reflect on me. But a teacher quality survey? That's just a weapon.
Things like this make me wish that we had some kind of Hacker in Chief, to figure out how to circumvent new systems before they get implemented.
>One problem is that it might give too much power to students.
This is discussed extensively, midway through the article, in approximately four paragraphs:
Students were better than trained adult observers at evaluating teachers. This wasn’t because they were smarter but because they had months to form an opinion, as opposed to 30 minutes. And there were dozens of them, as opposed to a single principal. Even if one kid had a grudge against a teacher or just blew off the survey, his response alone couldn’t sway the average.
“There are some students, knuckleheads who will just mess the survey up and not take it seriously,” Ferguson says, “but they are very rare.” Students who don’t read the questions might give the same response to every item. But when Ferguson recently examined 199,000 surveys, he found that less than one-half of 1 percent of students did so in the first 10 questions. Kids, he believes, find the questions interesting, so they tend to pay attention. And the “right” answer is not always apparent, so even kids who want to skew the results would not necessarily know how to do it.
Even young children can evaluate their teachers with relative accuracy, to Kane’s surprise. In fact, the only thing that the researchers found to better predict a teacher’s test-score gains was … past test-score gains. But in addition to being loathed by teachers, those data are fickle. A teacher could be ranked as highly effective one year according to students’ test gains and as ineffective the next, partly because of changes in class makeup that have little to do with her own performance—say, getting assigned the school’s two biggest hooligans or meanest mean girls.
Student surveys, on the other hand, are far less volatile. Kids’ answers for a given teacher remained similar, Ferguson found, from class to class and from fall to spring. And more important, the questions led to revelations that test scores did not: Above and beyond academic skills, what was it really like to spend a year in this classroom? Did you work harder in this classroom than you did anywhere else? The answers to these questions matter to a student for years to come, long after she forgets the quadratic equation.
This is discussed extensively, midway through the article, in approximately four paragraphs:
Students were better than trained adult observers at evaluating teachers. This wasn’t because they were smarter but because they had months to form an opinion, as opposed to 30 minutes. And there were dozens of them, as opposed to a single principal. Even if one kid had a grudge against a teacher or just blew off the survey, his response alone couldn’t sway the average.
“There are some students, knuckleheads who will just mess the survey up and not take it seriously,” Ferguson says, “but they are very rare.” Students who don’t read the questions might give the same response to every item. But when Ferguson recently examined 199,000 surveys, he found that less than one-half of 1 percent of students did so in the first 10 questions. Kids, he believes, find the questions interesting, so they tend to pay attention. And the “right” answer is not always apparent, so even kids who want to skew the results would not necessarily know how to do it.
Even young children can evaluate their teachers with relative accuracy, to Kane’s surprise. In fact, the only thing that the researchers found to better predict a teacher’s test-score gains was … past test-score gains. But in addition to being loathed by teachers, those data are fickle. A teacher could be ranked as highly effective one year according to students’ test gains and as ineffective the next, partly because of changes in class makeup that have little to do with her own performance—say, getting assigned the school’s two biggest hooligans or meanest mean girls.
Student surveys, on the other hand, are far less volatile. Kids’ answers for a given teacher remained similar, Ferguson found, from class to class and from fall to spring. And more important, the questions led to revelations that test scores did not: Above and beyond academic skills, what was it really like to spend a year in this classroom? Did you work harder in this classroom than you did anywhere else? The answers to these questions matter to a student for years to come, long after she forgets the quadratic equation.
In your rush to attack hooande for not reading the entire article, you neglected to read the entire comment and missed the part about organized group effort. If you consider the prospect of being a teacher who is gay, Muslim, an immigrant (or one from the wrong country) etc. it's quite easy to believe that the risk isn't limited to a single student in one class.
Fortunately the authors weren't so careless — that concern is obviously key to the repeated cautions about assigning too much weight to such tests. I'd like to believe anyone setting policy would be similarly slow to jump to conclusions but this is a field notoriously prone to chasing fads and quick fixes…
Fortunately the authors weren't so careless — that concern is obviously key to the repeated cautions about assigning too much weight to such tests. I'd like to believe anyone setting policy would be similarly slow to jump to conclusions but this is a field notoriously prone to chasing fads and quick fixes…
It would really depend across classes and groups. I've worked a lot with kids in academic (schools) and semi-academic settings (scientific summer camps), and I've had groups that had an awareness for "what was best for them".
For example, in one programming summer camp I led, I've had kids come to me at the end of the 10 days and tell me that I was their favorite educator and a great director because I was tough but fair and expected them to learn and progress, while some other educator was lame because even though he was super nice and laid back, they didn't learn anything with him.
Now of course, during the camp itself, I've overheard more than one kid telling another that I was super mean and terrible. But in the end, most of them were vocal about how they appreciated me. Kids in general tend to be very aware of and appreciate fairness, consistency, and holding them up to standards and pushing them to get better.
Cases like that are not the exception, but sadly not the majority either (the specific example I cited was with French "gifted" kids between 14-18 years old). So it would really depend.
For example, in one programming summer camp I led, I've had kids come to me at the end of the 10 days and tell me that I was their favorite educator and a great director because I was tough but fair and expected them to learn and progress, while some other educator was lame because even though he was super nice and laid back, they didn't learn anything with him.
Now of course, during the camp itself, I've overheard more than one kid telling another that I was super mean and terrible. But in the end, most of them were vocal about how they appreciated me. Kids in general tend to be very aware of and appreciate fairness, consistency, and holding them up to standards and pushing them to get better.
Cases like that are not the exception, but sadly not the majority either (the specific example I cited was with French "gifted" kids between 14-18 years old). So it would really depend.
A lot of kids mature enough to go to scientific/programming camps are usually the ones mature enough to understand that they learn under guidelines. Also, those kids aren't being graded.
As a recent high schooler, I've seen kids who absolutely hated their teachers because they gave them bad grades because they challenge their students. However, this specific teacher was probably the most influential for those who appreciated him.
I'm just afraid that the majority will figure out the system and use it as a weapon, as was mentioned.
As a recent high schooler, I've seen kids who absolutely hated their teachers because they gave them bad grades because they challenge their students. However, this specific teacher was probably the most influential for those who appreciated him.
I'm just afraid that the majority will figure out the system and use it as a weapon, as was mentioned.
"But a teacher quality survey? That's just a weapon."
Only if it's used as part of a (simplistic) algorithm.
And besides, why would students want to fire the good teachers? That doesn't really make sense. (As someone who was sent to the principal's office on a semi-regular basis.)
Only if it's used as part of a (simplistic) algorithm.
And besides, why would students want to fire the good teachers? That doesn't really make sense. (As someone who was sent to the principal's office on a semi-regular basis.)
I agree that other factors would prevent students having too much power, but "good" is a very subjective term. Even "effective" depends on how the students learn. Just because I think a teacher is good for challenging me, another student might hate them for being so tough.
Even if students are colluding to give poor ratings to teachers they dislike, it won't significantly alter the relative rankings of the teachers - the better the teacher, the fewer students that are willing to systematically exaggerate the teacher's weaknesses, and the best teachers (the ones who make learning fun) won't have their scores hurt at all. Students colluding will make it hard to set a threshold for "good enough" to not fire, but even if the students catch on, it should still be a good tool for identifying the best teachers.
I mean at the end of the day it's not really an opinion question; either the survey is empirically validated or its not. At least for now it appears to be, whether it will remain so when widely used is anyone's guess. (But not up for debate, since again it's something that can be tested empirically.)
My english teacher in high school was an older, fairly harsh and not too charming lady. She was mostly disliked by my classmates. At one point they even organized a petition to replace her with someone else. I didn't sign it. In my perception, she wasn't unfair, just demanding and quite strict when it comes to grammar.
Later, whenever I met a new teacher (at university, in the workplace) lessons always seemed easy to me. I had to work hard for a good grade at high school, and her tests demanded precision and attention to detail. I started to appreciate her more.
Students are likely to prefer teachers with a pleasant attitude and lower difficulty level. A teacher easy to distract with an off-topic conversation. But at the end of the day, a teacher's job is to teach. Sure, a great teacher would combine that with charisma, ability to inspire and natural charm, but nobody's perfect.
Later, whenever I met a new teacher (at university, in the workplace) lessons always seemed easy to me. I had to work hard for a good grade at high school, and her tests demanded precision and attention to detail. I started to appreciate her more.
Students are likely to prefer teachers with a pleasant attitude and lower difficulty level. A teacher easy to distract with an off-topic conversation. But at the end of the day, a teacher's job is to teach. Sure, a great teacher would combine that with charisma, ability to inspire and natural charm, but nobody's perfect.
Excellent -- then as part of a balanced review process her students' test scores would be excellent, and the principal evaluating her would have the same perspective as you.
Win!
Win!
I think you don't realize how politicized teaching has become. It doesn't matter if her students have good test scores: they all hate her. That one black mark is plenty for a determined administrator to get her fired. That a balanced review process would show see is a good teacher is simply immaterial. At least, that is how teachers see things.
Part of the reason you see so much backlash from teacher's unions against these initiatives (despite the fact that virtually all teachers acknowledge that some are better than others, and have a pretty fair idea who the worst ones are) is that they don't see this as a way to fairly evaluate teachers and improve the quality of education. Rather, they see it as a weapon to be wielded by school districts, school administrators, and angry parents ("Mrs. X told little Johnny (age: 17) where babies come from!" "Mr. Y is a member of $MARGINALIZED_SUBGROUP!" "Ms. Z believes something that is the current scientific consensus" "Mr. Q gave my son an A-") against unpopular teachers. It is an unfortunate fact of life, and even more unfortunate that so few are willing to admit it (especially here on HN, where the quality of discussion is usually above average).
We would probably have much greater buy-in from teaching unions if these evaluations were perceived as something other than politics as usual. Unfortunately, the proposals all seem to keep a some level of human involvement in the firing process (so only the very most outstanding teachers are immune to the usual political pressures). It's useful at this point to recall a bit of history: Teachers today are so hard to fire because they have tenure. Teaching unions successfully forced tenure the tenure system on school districts because school administrators of the past abused their authority to fire good teachers who happened to be politically unpopular.
Part of the reason you see so much backlash from teacher's unions against these initiatives (despite the fact that virtually all teachers acknowledge that some are better than others, and have a pretty fair idea who the worst ones are) is that they don't see this as a way to fairly evaluate teachers and improve the quality of education. Rather, they see it as a weapon to be wielded by school districts, school administrators, and angry parents ("Mrs. X told little Johnny (age: 17) where babies come from!" "Mr. Y is a member of $MARGINALIZED_SUBGROUP!" "Ms. Z believes something that is the current scientific consensus" "Mr. Q gave my son an A-") against unpopular teachers. It is an unfortunate fact of life, and even more unfortunate that so few are willing to admit it (especially here on HN, where the quality of discussion is usually above average).
We would probably have much greater buy-in from teaching unions if these evaluations were perceived as something other than politics as usual. Unfortunately, the proposals all seem to keep a some level of human involvement in the firing process (so only the very most outstanding teachers are immune to the usual political pressures). It's useful at this point to recall a bit of history: Teachers today are so hard to fire because they have tenure. Teaching unions successfully forced tenure the tenure system on school districts because school administrators of the past abused their authority to fire good teachers who happened to be politically unpopular.
Only in education is it controversial to give some semblance of power to the supposed customers.
They aren't the customers, though, not even officially. If anyone's the customer of K-12 schools, it's the adults of society who believe the schools should exist, and should educate kids in a particular way (and who also pay for the schools' upkeep). That's why kids don't get to choose their curriculum, for example; adults decide what should be taught in schools. Kids aren't allowed to opt-out, either (they can be arrested for truancy if they try).
Yes, I'm very aware of truancy laws. Perhaps I should have said supposed beneficiaries.
[deleted]
It seems very similar to up- and downvotes. You'd want to know how to filter out the noise, but you're going to get a lot of good feedback from them. For instance, as much as I loved bad teachers in HS who let me get away with not doing anything, I never would've told anyone those teachers were "good" -- it was just that my immediate desire to slack off in high school outweighed my desire to do work on my own in spite of having a bad teacher who didn't require it. But grading that teacher as bad wouldn't take any special effort, so I would've done it.
There is a problem here: The evaluations would be most helpful if they occurred partway through the year/semester, rather than solely at the end of it, because it would give teachers time to react to the feedback. Would you be willing to honestly critique a bad teacher if it meant you would have to deal with them actually doing their job for the next few months?
I'm also not so sure that everyone would be so willing to honestly critique their teachers at the end of the year, either. A lot of kids come to like their "bad teachers who lead fun classes" almost as much or more than the teachers who do their jobs well. Kind of like how many kids are unwilling to "snitch" on kids who have wronged them, even when guaranteed anonymity, I think a lot of them wouldn't want to get their teachers in trouble, even if they weren't doing their job properly. In my experience with such teachers, even poorly behaved students would act extremely respectfully when the principal came around to observe. Then again, maybe they were just afraid of getting on the teacher's bad side.
Even worse are instances where the administration knows fully well that a teacher leads a "joke class" and keeps them around to funnel the "bad kids" into. I grew up in a school system that straddled the border between a middle class college town and Section 8 housing packed with families ruined by the collapse of unskilled industrial labor in the area. I have no doubt that a large number of the elective courses existed solely to fill credits for disruptive students and keep them from causing problems in the "real classes" that the middle class and brighter/better-adjusted lower class students took. It didn't matter if the class actually learned anything, the school was just desperate to raise their graduation rates.
I do think these evaluations can help good teachers improve and possibly help to identify people who don't deserve to work as teachers, but they can't solve fundamental problems like "the early education and home environment of these students has failed them, and now the administration of their high school is ready to roll over to keep them from dropping out."
I'm also not so sure that everyone would be so willing to honestly critique their teachers at the end of the year, either. A lot of kids come to like their "bad teachers who lead fun classes" almost as much or more than the teachers who do their jobs well. Kind of like how many kids are unwilling to "snitch" on kids who have wronged them, even when guaranteed anonymity, I think a lot of them wouldn't want to get their teachers in trouble, even if they weren't doing their job properly. In my experience with such teachers, even poorly behaved students would act extremely respectfully when the principal came around to observe. Then again, maybe they were just afraid of getting on the teacher's bad side.
Even worse are instances where the administration knows fully well that a teacher leads a "joke class" and keeps them around to funnel the "bad kids" into. I grew up in a school system that straddled the border between a middle class college town and Section 8 housing packed with families ruined by the collapse of unskilled industrial labor in the area. I have no doubt that a large number of the elective courses existed solely to fill credits for disruptive students and keep them from causing problems in the "real classes" that the middle class and brighter/better-adjusted lower class students took. It didn't matter if the class actually learned anything, the school was just desperate to raise their graduation rates.
I do think these evaluations can help good teachers improve and possibly help to identify people who don't deserve to work as teachers, but they can't solve fundamental problems like "the early education and home environment of these students has failed them, and now the administration of their high school is ready to roll over to keep them from dropping out."
Right. The effectiveness of student reviews needs to be studied in a context where the kids know the reviews are important. Might be a different result.
Modify the weight of a vote based on how far the student's votes deviate from the norm over time. A historically positive or negative student would have their vote neutered.
This doesn't prevent one class from ganging up to downvote a teacher on a single occasion, but if that happens that semester's score for the teacher will appear as an anomaly compared to previous scores.
This is probably the root cause of grade inflation in American universities. Terrible, terrible idea.
Actually that would be the Vietnam war.
Are you being facetious? If not, would you mind expanding on this? I'm greatly interested in the recent trend towards grade inflation.
No that's the actual reason. During the Vietnam war you would get drafted unless you were in college, which means that anyone who failed out of college was basically put on the next plane to the jungle to get killed. Since most of the faculty were anti-war (or at least relatively liberal) most colleges either dropped the grading system entirely or else made grading significantly easier.
OK, but the Vietnam War has been over for almost 40 years now, but grade inflation keeps getting worse. Why?
- Grades don't mean anything, so there's no reason to give people bad grades. I know I quote this all the time on HN, but only because it's accurate: "A grade can be regarded only as an inadequate report of an inaccurate judgment by a biased and variable judge of the extent to which a student has attained an undefined level of mastery of an unknown proportion of an indefinite amount of material."
- Despite the fact that grades don't predict workplace performance at all, companies still use them to hire people. (Basically because it's a legal way to keep out black people.) Because of this, schools can boost their US News & World Report rank by giving people better grades, since starting salary factors into the rankings in many years.
- Schools are basically run as businesses these days, and that's what the customers want. Asking why schools keep inflating grades is kind of like asking why McDonald's keeps making their chicken nuggets taste so fucking delicious.
- Despite the fact that grades don't predict workplace performance at all, companies still use them to hire people. (Basically because it's a legal way to keep out black people.) Because of this, schools can boost their US News & World Report rank by giving people better grades, since starting salary factors into the rankings in many years.
- Schools are basically run as businesses these days, and that's what the customers want. Asking why schools keep inflating grades is kind of like asking why McDonald's keeps making their chicken nuggets taste so fucking delicious.
-I don't know about you, but if I had to hire between 50 A students and 50 D students, I'd hire the A students.
-Schools in Africa grade students, who are they trying to keep out?
-Some schools might be run as a business; I think most aren't (they're subpar and inefficient). If they are, they're run as a mom and pop. If San Francisco schools were businesses, they'd have gone bankrupt and displaced a while ago.
-Schools in Africa grade students, who are they trying to keep out?
-Some schools might be run as a business; I think most aren't (they're subpar and inefficient). If they are, they're run as a mom and pop. If San Francisco schools were businesses, they'd have gone bankrupt and displaced a while ago.
"I don't know about you, but if I had to hire between 50 A students and 50 D students, I'd hire the A students."
The science suggests that if you we're assigned both groups at random, you wouldn't be able to tell the difference.
The science suggests that if you we're assigned both groups at random, you wouldn't be able to tell the difference.
"Any citations for this?"
There are only a handful of studies because it's considered kind of a dumb question to ask. But Alfie Kohn references some of them in his books, I think in What Does It Mean To Be Well Educated? But the ones that exist consistently show the same thing.
This includes the huge internal study that Google HR did:
"Unfortunately, most of the academic research suggests that the factors Google has put the most weight on — grades and interviews — are not an especially reliable way of hiring good people."
"When all this was completed, Dr. Carlisle set about analyzing the two million data points the survey collected. Among the first results was confirmation that Google’s obsession with academic performance was not always correlated with success at the company."
http://www.nytimes.com/2007/01/03/technology/03google.html?e...
There are only a handful of studies because it's considered kind of a dumb question to ask. But Alfie Kohn references some of them in his books, I think in What Does It Mean To Be Well Educated? But the ones that exist consistently show the same thing.
This includes the huge internal study that Google HR did:
"Unfortunately, most of the academic research suggests that the factors Google has put the most weight on — grades and interviews — are not an especially reliable way of hiring good people."
"When all this was completed, Dr. Carlisle set about analyzing the two million data points the survey collected. Among the first results was confirmation that Google’s obsession with academic performance was not always correlated with success at the company."
http://www.nytimes.com/2007/01/03/technology/03google.html?e...
The article you link to and quote, while saying grades are less important than Google thought, goes on to say that progressive Google after doing this study and at least partially implementing its new system "Last week...hired 6 people who had a below 3.0 GPA." 6 out of an estimated 200 weekly hires. That's a far far cry from D student = A student and nobody can tell the difference. Also, all of the direct quotations from Google people in the article talk about years spent in academia, degrees like doctorates, and interviews being the poor predictors. Some of the best programmers I know dropped out of high school. Doesn't mean I couldn't tell apart a room of A students from D students or that I couldn't use the A students better for many jobs I might need to fill.
"That's a far far cry from D student = A student and nobody can tell the difference."
That's what the results found. Google just didn't change their hiring policy accordingly.
That's what the results found. Google just didn't change their hiring policy accordingly.
Any citations for this? I can buy that this is the case for some jobs, from some school districts, for many students, across a certain distribution of grades, some number of years after graduation. I have a hard time believing that a D student who barely passed college will do just as well in a new consulting job as an A student, simply because the A student probably has actually learned a hell of a lot more useful information.
[deleted]
(Basically because it's a legal way to keep out black people.)
Could you apply the principle of charity and think of any other possible reason why employers might care about grades?
Like, I dunno, employers believe grades are correlated with conscientiousness, agreeableness and intelligence, and employers find these traits useful?
Could you apply the principle of charity and think of any other possible reason why employers might care about grades?
Like, I dunno, employers believe grades are correlated with conscientiousness, agreeableness and intelligence, and employers find these traits useful?
"Could you apply the principle of charity and think of any other possible reason why employers might care about grades?"
If you knew twice as much about the issue, do you think you'd be more or less charitable than you are today?
If you knew twice as much about the issue, do you think you'd be more or less charitable than you are today?
I try to apply the principle of charity no matter how much I know about something.
https://en.wikipedia.org/wiki/Principle_of_charity
It's a far better way to learn from other people than simply assuming they are racist/communist/terrorist and dismissing them.
https://en.wikipedia.org/wiki/Principle_of_charity
It's a far better way to learn from other people than simply assuming they are racist/communist/terrorist and dismissing them.
I think that's a reasonable principle to apply, but at least as stated by the Wiki article it refers to avoiding "attributing irrationality, logical fallacies or falsehoods to the others' statements, when a coherent, rational interpretation of the statements is available."
Now if law schools have a rational argument for why they still use grades to determine who gets into law school even though it has been shown that GPA has zero correlation with ability as a lawyer, and that its main effect is to keep black people out of law school, then I'd love to hear it. But note that they would need a rational reason for doing this, not just any reason.
(Which Malcolm Gladwell talks about here: http://www.newyorker.com/online/video/conference/2008/gladwe...)
Now if law schools have a rational argument for why they still use grades to determine who gets into law school even though it has been shown that GPA has zero correlation with ability as a lawyer, and that its main effect is to keep black people out of law school, then I'd love to hear it. But note that they would need a rational reason for doing this, not just any reason.
(Which Malcolm Gladwell talks about here: http://www.newyorker.com/online/video/conference/2008/gladwe...)
...even though it has been shown that GPA has zero correlation with ability as a lawyer...
A charitable interpretation of the actions of law school admissions departments would simply be that they don't believe this "fact" (a citation for which is desperately needed).
Further, you are attributing anti-black racism to a group of people (law school admins/profs) who routinely engage in pro-black racism (see, e.g., Grutter v. Bollinger). It doesn't even make sense.
A charitable interpretation of the actions of law school admissions departments would simply be that they don't believe this "fact" (a citation for which is desperately needed).
Further, you are attributing anti-black racism to a group of people (law school admins/profs) who routinely engage in pro-black racism (see, e.g., Grutter v. Bollinger). It doesn't even make sense.
http://www.mindingthecampus.com/originals/2009/11/grade_infl...
"Finally, the "draft fear" explanation cannot explain why a very similar pattern of grade inflation has occurred over the decades in Canadian universities which were spared the tumult of the Vietnam protest era and student fears that they might be drafted and sent overseas to fight and die."
"Finally, the "draft fear" explanation cannot explain why a very similar pattern of grade inflation has occurred over the decades in Canadian universities which were spared the tumult of the Vietnam protest era and student fears that they might be drafted and sent overseas to fight and die."
Once it starts in one place it's a race to the bottom everywhere.
[deleted]
To help students get draft deferments, schools gave kids better grades. I don't quite understand it, since GWB dodged Vietnam with bad grades.
It has been written about, probably in the Atlantic or NYT, maybe 10 years ago.
It has been written about, probably in the Atlantic or NYT, maybe 10 years ago.
University grading is seldom second-guessed with wide-scale standardized testing, which is the perfect vehicle for counteracting the grade inflation incentive. In primary schools, if the data shows your students love you but they keep flunking, you won't get a bonus, you'll get demoted to gym teacher.
A completely false and unsupported statement.
This is one way this process has been validated, from the submitted article: "The responses did indeed help predict which classes would have the most test-score improvement at the end of the year. In math, for example, the teachers rated most highly by students delivered the equivalent of about six more months of learning than teachers with the lowest ratings. (By comparison, teachers who get a master’s degree—one of the few ways to earn a pay raise in most schools —delivered about one more month of learning per year than teachers without one.)
. . . .
"The survey did not ask Do you like your teacher? Is your teacher nice? This wasn’t a popularity contest. The survey mostly asked questions about what students saw, day in and day out.
"Of the 36 items included in the Gates Foundation study, the five that most correlated with student learning were very straightforward:
1. Students in this class treat the teacher with respect.
2. My classmates behave the way my teacher wants them to.
3. Our class stays busy and doesn’t waste time.
4. In this class, we learn a lot almost every day.
5. In this class, we learn to correct our mistakes."
Here is earlier reporting (10 December 2010) from the New York Times about the same issue:
http://www.nytimes.com/2010/12/11/education/11education.html Here is the website of Ronald Ferguson's research project at Harvard:
http://tripodproject.wpengine.com/about/our-team/
And here are some links about the project from the National Center for Teacher Effectiveness:
http://www.gse.harvard.edu/ncte/news/NCTE_Conference_Using_S...
LAST EDIT: I'm amazed at how many of the comments in this thread appear to be about issues thoroughly discussed in the submitted article, but unresponsive to what the submitted article said. On this kind of issue, it's an especially good practice to read the fine article before assuming what is being discussed. We all know about school, but specific proposals for school reform have specific details that make some worse than others, and can be empirically tested.
. . . .
"The survey did not ask Do you like your teacher? Is your teacher nice? This wasn’t a popularity contest. The survey mostly asked questions about what students saw, day in and day out.
"Of the 36 items included in the Gates Foundation study, the five that most correlated with student learning were very straightforward:
1. Students in this class treat the teacher with respect.
2. My classmates behave the way my teacher wants them to.
3. Our class stays busy and doesn’t waste time.
4. In this class, we learn a lot almost every day.
5. In this class, we learn to correct our mistakes."
Here is earlier reporting (10 December 2010) from the New York Times about the same issue:
http://www.nytimes.com/2010/12/11/education/11education.html Here is the website of Ronald Ferguson's research project at Harvard:
http://tripodproject.wpengine.com/about/our-team/
And here are some links about the project from the National Center for Teacher Effectiveness:
http://www.gse.harvard.edu/ncte/news/NCTE_Conference_Using_S...
LAST EDIT: I'm amazed at how many of the comments in this thread appear to be about issues thoroughly discussed in the submitted article, but unresponsive to what the submitted article said. On this kind of issue, it's an especially good practice to read the fine article before assuming what is being discussed. We all know about school, but specific proposals for school reform have specific details that make some worse than others, and can be empirically tested.
>I'm amazed at how many of the comments in this thread appear to be about issues thoroughly discussed in the submitted article, but unresponsive to what the submitted article said.
I'm not, unfortunately. It seems like most people read a headline and perhaps a paragraph or two, then active their pre-existing beliefs about whatever the subject happens to me, and move on from there. That's certainly been my experience with commenters on my blog, anyway, and it's been experience in observing both online communities and in reading student papers.
I'm not, unfortunately. It seems like most people read a headline and perhaps a paragraph or two, then active their pre-existing beliefs about whatever the subject happens to me, and move on from there. That's certainly been my experience with commenters on my blog, anyway, and it's been experience in observing both online communities and in reading student papers.
I am worried about adding another metric to the way that we measure performance in schools. It's already been shown that there are problems with the standardized tests. Not that they are terrible things, but students in the US tend to disagree with their proliferation. (Perhaps they would feel less this way if there were less tests)
Having students grade the teachers I think is also the same way. One of the problems is that students don't know what makes a good teacher. The tests can gear students in that direction (ie "I feel challenged but not overwhelmed in this classroom"), but if we look for too much insight from the students I think we will be misdirected. Just like we are misdirected when we pay too much attention to standardized tests.
My fear is that some schools would start to look at these performance measurements as golden bullets sort the way that we've started to look at standardized tests as golden bullets. Most of jr. high was geared towards getting perfect scores on the state exams. My Junior and Senior years of high school were almost 100% (a few teachers went outside of the scope, but it was a teacher decision and not an administrative decision) geared around AP tests and the ACT.
The ACT and the AP tests have their place. And I think that student evaluations of teachers have their place as well. Both are very useful when applied appropriately.
I just don't want to see the system (d)evolve in such a way that too much emphasis is placed on empirical data.
Having students grade the teachers I think is also the same way. One of the problems is that students don't know what makes a good teacher. The tests can gear students in that direction (ie "I feel challenged but not overwhelmed in this classroom"), but if we look for too much insight from the students I think we will be misdirected. Just like we are misdirected when we pay too much attention to standardized tests.
My fear is that some schools would start to look at these performance measurements as golden bullets sort the way that we've started to look at standardized tests as golden bullets. Most of jr. high was geared towards getting perfect scores on the state exams. My Junior and Senior years of high school were almost 100% (a few teachers went outside of the scope, but it was a teacher decision and not an administrative decision) geared around AP tests and the ACT.
The ACT and the AP tests have their place. And I think that student evaluations of teachers have their place as well. Both are very useful when applied appropriately.
I just don't want to see the system (d)evolve in such a way that too much emphasis is placed on empirical data.
"One of the problems is that students don't know what makes a good teacher."
Students don't need to know what makes a good teacher. They only need to be able to assess whether they learned, and whether they had fun in the process. Whether they learned enough is pretty much an orthogonal issue, and one that can and will be dealt with through standardized testing. Students also don't really need to give any thought to a teacher's specific methods in order to offer useful information.
Students don't need to know what makes a good teacher. They only need to be able to assess whether they learned, and whether they had fun in the process. Whether they learned enough is pretty much an orthogonal issue, and one that can and will be dealt with through standardized testing. Students also don't really need to give any thought to a teacher's specific methods in order to offer useful information.
If the student evaluations correlate strongly with the standardized tests' measures of student progress, then:
a) What's the purpose of having both?
b) If the evals are preferred over the tests, will "good" teachers continue to teach to a predictable, standardized curriculum?
c) Is the correlation additional evidence in favor of "differential compensation", that is, a compensation program based at least in part on exam scores?
d) Even if the information supplied is similar, doesn't this extra test/survey administration detract from instructional time? Is the information gleaned sufficient to compensate for the loss of instructional time?
e) Atlanta (Georgia, USA) is still reeling from a years-long cheating scandal. If such evaluations become "high stakes" (and there will likely be a push to do so, despite likely union opposition), won't these results be exploitable as well? (And perhaps even more so, through campaigning, social engineering, etc?)
a) What's the purpose of having both?
b) If the evals are preferred over the tests, will "good" teachers continue to teach to a predictable, standardized curriculum?
c) Is the correlation additional evidence in favor of "differential compensation", that is, a compensation program based at least in part on exam scores?
d) Even if the information supplied is similar, doesn't this extra test/survey administration detract from instructional time? Is the information gleaned sufficient to compensate for the loss of instructional time?
e) Atlanta (Georgia, USA) is still reeling from a years-long cheating scandal. If such evaluations become "high stakes" (and there will likely be a push to do so, despite likely union opposition), won't these results be exploitable as well? (And perhaps even more so, through campaigning, social engineering, etc?)
"What's the purpose of having both?"
What's the purpose of looking at multiple polls when you're trying to predict the outcome of the upcoming election? More evidence gives you higher confidence and lower margin of error. And as the article says, these student surveys provide clean, stable data that doesn't fluctuate very much from year to year and doesn't require much correction for race and family income.
These surveys take on the order of 10-15 minutes. That's nothing compared to a battery of standardized tests. They wouldn't have to be very informative at all in order to be worth the small sacrifice of instructional time, and if they're the second-best predictor of class achievement, then they're certainly worth the time (if the results are actually used).
I don't think there's going to be much movement to stop paying attention to standardized tests and curriculums, since these surveys don't measure the same thing - roughly speaking, the standardized tests seek to measure how much was learned, and these surveys add a dimension of why.
What's the purpose of looking at multiple polls when you're trying to predict the outcome of the upcoming election? More evidence gives you higher confidence and lower margin of error. And as the article says, these student surveys provide clean, stable data that doesn't fluctuate very much from year to year and doesn't require much correction for race and family income.
These surveys take on the order of 10-15 minutes. That's nothing compared to a battery of standardized tests. They wouldn't have to be very informative at all in order to be worth the small sacrifice of instructional time, and if they're the second-best predictor of class achievement, then they're certainly worth the time (if the results are actually used).
I don't think there's going to be much movement to stop paying attention to standardized tests and curriculums, since these surveys don't measure the same thing - roughly speaking, the standardized tests seek to measure how much was learned, and these surveys add a dimension of why.
The point being, if they're strongly correlated, the "why" can already be inferred (assuming adoption of this particular theoretical causal chain).
I agree the movement toward testing is likely to continue unabated. The point was the futility of additional questions; when a strong correlation is already known, the responses can be predicted with a high degree of accuracy, so the additional information is not really "additional". Yes, social scientist prefer multi-item measures and avoiding a single source bias (particularly when publishing theory), but in the end, if the correlation is maintained, not much new is really learned.
(And one of the reasons for multiple election polls is that the results change over time, and leading up to an election, that's relevant. Additionally, the entire electorate isn't polled each time, so the "cost" to the system is lower, relatively speaking.)
I agree the movement toward testing is likely to continue unabated. The point was the futility of additional questions; when a strong correlation is already known, the responses can be predicted with a high degree of accuracy, so the additional information is not really "additional". Yes, social scientist prefer multi-item measures and avoiding a single source bias (particularly when publishing theory), but in the end, if the correlation is maintained, not much new is really learned.
(And one of the reasons for multiple election polls is that the results change over time, and leading up to an election, that's relevant. Additionally, the entire electorate isn't polled each time, so the "cost" to the system is lower, relatively speaking.)
Let's make some simplifying assumptions: the survey identifies two causes of poor performance, a lack of academic rigor, and a poor classroom environment (such as a teacher that's mean and unresponsive to requests for help or clarification). Those two categories are weighted equally on the survey - a teacher who gets 100% on the survey is good in both categories, a teacher who gets 0% is bad in both categories, and a teacher has more than one way to score 50%.
So, if both of those factors affect student performance on standardized tests, then there will be a strong correlation between the overall survey scores and the test scores. But analyzing the details of the survey results can offer actionable guidance that the test scores can't - the survey does provide useful information for how mediocre teachers can improve, even when it matches the test scores in predictive power for future test scores.
So, if both of those factors affect student performance on standardized tests, then there will be a strong correlation between the overall survey scores and the test scores. But analyzing the details of the survey results can offer actionable guidance that the test scores can't - the survey does provide useful information for how mediocre teachers can improve, even when it matches the test scores in predictive power for future test scores.
As with any self-sustaining system (politics, economy, ecology) you need a feedback loop to form among the parties with conflicting interests. If the feedback loop is broken, the system deteriorates and ultimately falls apart. If it's inefficient, the system tends to be inefficient as well. (Examples: soviet-style planned economy, rabbits in Australia etc.)
This kind of thing (teachers grade students, students grade teachers) could improve the efficiency of the feedback loop and thus efficiency of the education system as a whole.
This kind of thing (teachers grade students, students grade teachers) could improve the efficiency of the feedback loop and thus efficiency of the education system as a whole.
next step , let kids grade there parents , and the administration calls social services when scores are low.
I know more about student evaluations at the university level than at the grade levels discussed in this article. Here is an excellent overview of some of that research: http://home.sprynet.com/~owl1/sef.htm
The message one takes away from that is that (i) yes, student evaluations are a good predictor of some objective properties of a class (and other measures don't even achieve that much), but those properties aren't what teachers should be optimizing. I'd grant that (ii) it does seem worthwhile for students to see that their interests make a difference to what happens in the classroom. I'd also grant that (iii) some classroom situations may be so bad that optimizing student satisfaction may, even if not educationally ideal, still be a big improvement. And for all I know, this may be widely true at the pre-university level; but on the other hand, for all I know, giving these evaluations a big institutional role at the pre-university level could also be counter-productive...the evidence cited in the article hardly enables us to say. Any deliberation about giving student evaluations an institutionalized role should take the evidence behind (i) seriously.
One promising message from the research reported in the Atlantic article is that (iv) the specific tests being discussed have been designed in ways that seem novel and especially revealing. But the article mixed that together with an indiscriminate enthusiasm for student evaluations quite generally. And I think many people will read this and say, "Duh, that's a no brainer." Yes, it is a brainer! These kinds of policy issues aren't settleable from the armchair. Even if we cleared all the political hurdles and made someone the educational policy dictator, he or she isn't going to be able to tell just from the armchair what the results of rolling out one policy rather than another is going to be. So I get frustrated with articles like this one, that report some interesting evidence but mix it together with the kind of insensitivity to the details exhibited in comments like "That research had shown something remarkable: if you asked kids the right questions, they could identify, with uncanny accuracy, their most—and least—effective teachers. The point was so obvious, it was almost embarrassing."
Neither does this inspire confidence: "Some studies...have shown that professors inflate grades to get good reviews. So far, grades don’t seem to significantly influence responses to Ferguson’s survey: students who receive A’s rate teachers only about 10 percent higher than D students do, on average." I hope that readers of this site don't need an explanation of why the clause after the colon is only barely relevant to whether grades get inflated because that leads to better evaluations. It's almost irrelevant. In the first place, teachers needn't be aware of the cited fact; they may experience grade-inflation pressures differently. Also, the cited fact is compatible with the majority of current A-getters scoring their teachers in ways that are largely insensitive to the grades they get, but a minority of current A-getters and a majority of current B-getters being extremely responsive to the grades they get. The cited fact is just not what we need to know.
The message one takes away from that is that (i) yes, student evaluations are a good predictor of some objective properties of a class (and other measures don't even achieve that much), but those properties aren't what teachers should be optimizing. I'd grant that (ii) it does seem worthwhile for students to see that their interests make a difference to what happens in the classroom. I'd also grant that (iii) some classroom situations may be so bad that optimizing student satisfaction may, even if not educationally ideal, still be a big improvement. And for all I know, this may be widely true at the pre-university level; but on the other hand, for all I know, giving these evaluations a big institutional role at the pre-university level could also be counter-productive...the evidence cited in the article hardly enables us to say. Any deliberation about giving student evaluations an institutionalized role should take the evidence behind (i) seriously.
One promising message from the research reported in the Atlantic article is that (iv) the specific tests being discussed have been designed in ways that seem novel and especially revealing. But the article mixed that together with an indiscriminate enthusiasm for student evaluations quite generally. And I think many people will read this and say, "Duh, that's a no brainer." Yes, it is a brainer! These kinds of policy issues aren't settleable from the armchair. Even if we cleared all the political hurdles and made someone the educational policy dictator, he or she isn't going to be able to tell just from the armchair what the results of rolling out one policy rather than another is going to be. So I get frustrated with articles like this one, that report some interesting evidence but mix it together with the kind of insensitivity to the details exhibited in comments like "That research had shown something remarkable: if you asked kids the right questions, they could identify, with uncanny accuracy, their most—and least—effective teachers. The point was so obvious, it was almost embarrassing."
Neither does this inspire confidence: "Some studies...have shown that professors inflate grades to get good reviews. So far, grades don’t seem to significantly influence responses to Ferguson’s survey: students who receive A’s rate teachers only about 10 percent higher than D students do, on average." I hope that readers of this site don't need an explanation of why the clause after the colon is only barely relevant to whether grades get inflated because that leads to better evaluations. It's almost irrelevant. In the first place, teachers needn't be aware of the cited fact; they may experience grade-inflation pressures differently. Also, the cited fact is compatible with the majority of current A-getters scoring their teachers in ways that are largely insensitive to the grades they get, but a minority of current A-getters and a majority of current B-getters being extremely responsive to the grades they get. The cited fact is just not what we need to know.
Expanding the final example: say I currently give 20 students As, 15 of them score me at 1.05, 5 of them are so happy to get As they score me at 1.65. I give my 80 other students Fs, and they score me at 1.0. Then on average my A students will score me at 1.2, and my F students will score me at 1.0, and my score for the whole class will be 1.04. But suppose I were to give 10 more F students As, and they would also be so happy that they'd also score me at 1.65. By doing so, I'd increase my average score from 1.04 to 1.105---for all we've been told, that may be several standard deviations across other teachers, and may translate to substantially higher salary and job security for me.
Forgive me for being glib, but this seems like a lot of words to simply say "You can't implement it because, well... you just can't!"
Thanks for your comment. I'm sorry that---despite using so many words---I didn't make myself understood. I didn't intend to argue that student evaluations can't or shouldn't be widely deployed. I was criticizing the sloppy, casual attitude towards the evidence displayed in the Atlantic article. And I thought I had linked to a literature review, and pointed out passages in the article, that supported that criticism.
This survey was part of the larger study funded by Bill Gates. The claim of "prediction" is not supported by the evidence.. the correlations with test scores were low.. and in any case, this is another example of reifying test scores and stupid concepts such as "a months worth of learning" as if all learning is the same, regardless of the subject, the grade level, the prior experience of the kids and so on. The very low correlations with test scores are not surprising and a by-product of the survey questions, designed to check whether the "good" teacher gets kids to comply with rules, defer to the teacher's authority, think of learning as not making mistakes (and for the sake of Gates, stay on task all the time). Perfect conditioning for students being taught that education is a matter of doing well on fill-in-the bubble tests that Gates and this researcher seem to value as the single best measure of great education. See this and the links within it. http://voices.washingtonpost.com/answer-sheet/guest-bloggers...
For example the tough math teacher would get a bad evaluation, but the cream-puff teacher who never assigned homework and gave everyone A's would get a good evaluation.