Udacity Unfair Grading(udacity-forums.com)
udacity-forums.com
Udacity Unfair Grading
http://www.udacity-forums.com/cs373/questions/9076/hw1-4-localization-program-graded-unfairly
2 comments
Suzam Pal: "How does it really matter if the course is non-profit or not? Why should that stop us from calling a mistake as a mistake, a bug as a bug, an ugly solution as an ugly solution? I've participated in a few small and medium-sized open source projects where we give away the product of our hard work for free, and users benefit from it without paying anything. If there is a bug in my work, or if a user suffers due to my work, I take responsibility for it and I fix it. If a student submitted an answer without violating Udacity's specifications, and it was graded incorrectly, then it is clearly either a bug or a mistake in the specification, and in a free society, people have the right to complain about it."
For someone who doesn't know anything about Udacity, what's going on here?
A student didn't follow directions and is unhappy at being penalized. Doesn't belong on HN at all, it's an issue between student and udacity.
AndyAtUdacity:
We have seen a lot of questions about homework 1.4. While we don't have time to check everyone's code, it seems like the majority of problems fall into one of a few cases.
1. Students deleted or modified the code above the comment section. This section was necessary for grading purposes. Admittedly this is not the most robust system and we will try our best to make future homework more clear about what you can modify.
2. Students redefined the show() function. While this wasn't explicitly forbidden, this had the same affect on the grading as modifying the show command. The reason you couldn't do this is because we actually redefine the show command in our grading script. This was an effort to prevent us from grading you wrong if your solution included print statements for debugging.
3. Students did not make their solution general enough. These solutions tended to only work for a world of 20 cells. We tested with a variety of worlds, so such a solution would not have been marked as correct.
We apologize for the confusion and the effect this had on some of your grades for the first hw. I hope everyone still enjoyed the homework, we put a lot of work into making it and we are as disappointed as you when it doesn't turn out perfectly.
Please mark this solution as correct so other students can see the "official" answer. Thanks!
---
k_l_s:
I guess I'm not sure what to make of this answer. While I understand that you have what seems to be thousands of users and so individual attention would be inplausible, you are presenting a rating system through which you are communicating to folks (admittedly, I am one of them) that their material mastery is poor when it likely is not. It would seem to me that the solution is to fix the grading system or to do away with grades, if it is technologically infeasible to correctly evaluate an individual's mastery of the material. Simply explaining that the grading technology is lacking but leaving the grades in place is demotivating and, quite honestly, perplexing. We students don't know the full implication of these grades, but almost certainly they will remain in your database and might even have significance, in the future, that would make some of us regret simply ignoring them. I presume that I am not the only one who would like to see the system work.
---
We have seen a lot of questions about homework 1.4. While we don't have time to check everyone's code, it seems like the majority of problems fall into one of a few cases.
1. Students deleted or modified the code above the comment section. This section was necessary for grading purposes. Admittedly this is not the most robust system and we will try our best to make future homework more clear about what you can modify.
2. Students redefined the show() function. While this wasn't explicitly forbidden, this had the same affect on the grading as modifying the show command. The reason you couldn't do this is because we actually redefine the show command in our grading script. This was an effort to prevent us from grading you wrong if your solution included print statements for debugging.
3. Students did not make their solution general enough. These solutions tended to only work for a world of 20 cells. We tested with a variety of worlds, so such a solution would not have been marked as correct.
We apologize for the confusion and the effect this had on some of your grades for the first hw. I hope everyone still enjoyed the homework, we put a lot of work into making it and we are as disappointed as you when it doesn't turn out perfectly.
Please mark this solution as correct so other students can see the "official" answer. Thanks!
---
k_l_s:
I guess I'm not sure what to make of this answer. While I understand that you have what seems to be thousands of users and so individual attention would be inplausible, you are presenting a rating system through which you are communicating to folks (admittedly, I am one of them) that their material mastery is poor when it likely is not. It would seem to me that the solution is to fix the grading system or to do away with grades, if it is technologically infeasible to correctly evaluate an individual's mastery of the material. Simply explaining that the grading technology is lacking but leaving the grades in place is demotivating and, quite honestly, perplexing. We students don't know the full implication of these grades, but almost certainly they will remain in your database and might even have significance, in the future, that would make some of us regret simply ignoring them. I presume that I am not the only one who would like to see the system work.
---
My experience with the AI class was weird in the sense that the profs would say things like "I don't blame you if you got this one wrong" but then score the answer wrong.
This gets to a greater question (which they also addressed) is that a deeper more involved question makes you think more, and hence they felt that people who got things wrong perhaps did more work and challenged themselves more, which as a whole a good thing, and I'd agree.
Some things just don't lend well to being binary right or wrong. In the ML class however, their code was a little more modular, and they had a lot more clear warnings about what you could and couldn't touch, and then the grading system was interactive and you could repeat your tests (and they did like what was described here and tried for more general problem spaces to see if your answer was written generally as well)...
So this is tough! The cynic in me says that this can make one go down the rabbit whole that "the man" is trying to indoctrinate you into an arbitrary system of their own choosing that isn't necessarily empirically correct, but then I remember my intro to Philosophy where they state that you really have to frame the debate or else all bets are off, and you just can't get any work done in chaos in like that.
This gets to a greater question (which they also addressed) is that a deeper more involved question makes you think more, and hence they felt that people who got things wrong perhaps did more work and challenged themselves more, which as a whole a good thing, and I'd agree.
Some things just don't lend well to being binary right or wrong. In the ML class however, their code was a little more modular, and they had a lot more clear warnings about what you could and couldn't touch, and then the grading system was interactive and you could repeat your tests (and they did like what was described here and tried for more general problem spaces to see if your answer was written generally as well)...
So this is tough! The cynic in me says that this can make one go down the rabbit whole that "the man" is trying to indoctrinate you into an arbitrary system of their own choosing that isn't necessarily empirically correct, but then I remember my intro to Philosophy where they state that you really have to frame the debate or else all bets are off, and you just can't get any work done in chaos in like that.