Humans, the Latest MOOC Feature(insidehighered.com)
insidehighered.com
Humans, the Latest MOOC Feature
https://www.insidehighered.com/news/2016/09/02/massachusetts-institute-technology-experiments-instructor-grading-massive-open
2 comments
The two scenarios are vastly different in my opinion. Grading multiple choice and short answer questions in parallel is a much simpler task than grading a written essay. The essay is complex human thought, not a formal system as in multiple choice or near formal system as in short answers.
The chemistry and engineering diagram grading scenario is closer to the first cases as well. The diagram is a formal visual representation of the underlying solution. There is less variance in the range of correct-looking diagrams than the written word. If the question was more complex, "Please draw a UML diagram of a complete user authentication system," then a human would have to grade it. There are an infinite amount of ways to design and diagram a solution to an open-ended problem.
The chemistry and engineering diagram grading scenario is closer to the first cases as well. The diagram is a formal visual representation of the underlying solution. There is less variance in the range of correct-looking diagrams than the written word. If the question was more complex, "Please draw a UML diagram of a complete user authentication system," then a human would have to grade it. There are an infinite amount of ways to design and diagram a solution to an open-ended problem.
I'm one of the founders of Gradescope (the company in the other post).
I see where you're coming from, but I actually think that both of these posts describe ways for students to get more open ended assessment, rather than automatically graded multiple choice questions (i.e. what's typical in MOOCs and Scantrons).
I see where you're coming from, but I actually think that both of these posts describe ways for students to get more open ended assessment, rather than automatically graded multiple choice questions (i.e. what's typical in MOOCs and Scantrons).
I agree with that, but would you say that the "trend", if there is one, is towards providing that through AI, or more humans?
If I recall, they had computers grade the GMAT essays for at least 10 years, but they had to have a human in the loop because that is the ultimate measure for whether a computer is "correct" in terms of grading an essay.
If I recall, they had computers grade the GMAT essays for at least 10 years, but they had to have a human in the loop because that is the ultimate measure for whether a computer is "correct" in terms of grading an essay.
The trend is towards neither, I'd say. The biggest trend has been towards online automatically graded multiple choice & short answer questions that aren't really open ended.
The automated essay grading stuff typically looks at writing style more than content, but it's true that it's a problem that tons of people have worked on and there's been some cool progress there as well. We're not really working on bringing AI to essay grading ourselves though.
The automated essay grading stuff typically looks at writing style more than content, but it's true that it's a problem that tons of people have worked on and there's been some cool progress there as well. We're not really working on bringing AI to essay grading ourselves though.
https://news.ycombinator.com/item?id=12414764
https://blogs.nvidia.com/blog/2016/09/02/gradescope-brings-a...