Machine Scoring of Essays Means Better Tests for Less
Existing assessments tend to emphasize multiple choice, or “bubble in”, type of questions because they are easier, more timely, and cheaper to score. Research has shown that these types of questions do not provide an accurate reflection of students’ critical thinking skills. To garner an accurate assessment of these skills, more in-depth questions, where the student provides a written response, are now being more widely adopted. This raises questions about the cost associated with grading constructed response items.
The ASAP Pricing Study, published today, addresses these cost questions. The paper concludes that, “In a higher quality assessment, human scoring of student essays can comprise over 60 percent of the total cost of the assessment. Automating this scoring process holds tremendous promise in making higher quality assessments affordable.” Machine scoring of essays may be 20 to 50 percent as much as human scoring with large volumes (see Exhibit 4 on p35 for details).
Concerns remain regarding the quality of automated scoring but a previous study demonstrated that machine scoring of essays is as accurate as scoring done by trained graders and short answer responses were nearly as accurate.
Learn more and download the full case study here. Join the conversation on Facebook at Facebook.com/gettingsmart and on Twitter at @Getting_Smart.
Dave Saba
I totally agree with you concerns of multiple choice vs essays. There is a gaping hole with short-form open-ended responses, like fill-in-the-blanks. I heard about a grading service that can reliably grade those types of tests here in Dallas called Scribesense - http://www.scribesense.com that is getting it done. They are just getting started and after a couple of meetings with them - they are the real deal. They are looking for pilot schools this spring, so if you are interested, give them a shout!! Great team - great work.