Pacific Metrics Acquires Prize-Winning Automated Scoring Software: Automated Scoring Engine to Incorporate New Software
Pacific Metrics, already an innovator in automated scoring (AS) and a pioneer in the use of AS in educational assessments, is innovating again. The company has entered into an agreement to acquire prize-winning automated scoring technology, to be integrated into the company’s high- performing automated scoring engine, CRASETM.
This past spring, Pacific Metrics was one of nine companies to compete in a Hewlett Foundation-funded Automated Student Assessment Prize (ASAP) to design innovative software to help teachers and school systems assess student writing. The results of the competition were announced at the National Council on Measurement in Education’s April meeting in Vancouver, and Pacific Metrics’ CRASE was a top performer, delivering results comparable to or exceeding the performance of human scorers. In addition to the nine corporate participants, the competition also included over 200 independent teams of researchers and data scientists. The winning independent team, consisting of Jason Tigg, Momchil Georgiev, and Stefan Henss, outperformed all other participants based on a weighted Kappa measure of agreement rates. Their collaborative effort brought together their diverse skill set in computer science, physics, and language and created the most innovative, effective, and applicable scoring model. Pacific Metrics’ acquisition agreement with this team gives Pacific Metrics two new pieces of powerful software: one developed for the essay scoring competition and one developed for the short answer competition. With this software purchase agreement, and its integration with CRASE, Pacific Metrics is continuing on its innovative path within the industry. CRASE is already one of the most advanced, research- based products on the market, and the integration of the new code reflects Pacific Metrics commitment to continual leadership in automated scoring technologies and applications.
In addition to purchasing the winning software, Pacific Metrics has engaged the winning team in a consulting capacity. Jason Tigg, an Oxford-educated physicist, Momchil Georgiev, a senior software engineer at the National Oceanic and Atmospheric Administration, and Stefan Henss, a graduate student studying computer science, will be welcome additions to the Pacific Metrics team. Sue Lottridge, Ph.D., Director of Machine Scoring for Pacific Metrics states, “This is a very exciting initiative for Pacific Metrics. We look forward to working with Momchil, Jason, and Stefan to incorporate their software and unique techniques into CRASE to achieve the most accurate and reliable scoring possible.”
The two multi-state consortia funded by the U.S. Department of Education to develop next-generation assessments, the Partnership for Assessment of Readiness for College and Careers (PARCC), and Smarter Balanced Assessment Consortium, are committed to the challenge of transitioning from paper-and-pencil to computer-based testing. As state agencies implement the Common Core State Standards, they are making decisions about the next generation assessments and how to accurately measure the new level of rigor. Innovative software that can faithfully replicate how trained educators evaluate a student’s written response offers a new approach for states to meet the challenge. Software scoring programs like CRASE do not independently assess the merits of an essay; instead they predict, very accurately, how a person would have scored the essay. The software is calibrated to replicate the scores assigned by trained educators and can deliver them in significantly less time. Its purpose is not to replace teachers, but to support the increased assignment of written tasks to students and provide fast and accurate feedback.
Currently utilized by several state departments of education and in a U.S. Department of Education-funded Enhanced Assessment Grant, CRASE is an innovative information technology that uses artificial intelligence to score a wide range of constructed-response items. Pacific Metrics’ technology has been used successfully in large-scale formative and summative assessment environments for the past six years. Along with time and cost savings, CRASE can interoperate with digital learning and assessment platforms, allowing easy utilization of its feature set with a variety of testing programs and products. For more information about CRASETM, visit www.automatedscoring.com, email [email protected], or call (831) 646-6400.