NAF is a network of more than 500 high school academies across the US serving more than 60,000 students in four themes: Finance, Hospitality & Tourism, Information Technology, and Engineering.
NAF announced a new certification system today. We think it signals a real commitment on NAF’s part to getting serious about outcomes. We are bullish on certification systems as market signaling strategy that will grow in importance over time. We like the way NAF approached the design of this system. They are one of small handful of organizations that has the reach to do this at scale. In other words, we think this is a pretty big deal.
— — —
The National Academy Foundation (NAF) has partnered with WestEd, a leading educational research, development, and service agency, to develop a student certification assessment system for its national network of high school, career-focused academies. The system is designed to measure students’ knowledge and application of key career-related technical content and foundational skills, in a nationally-recognized, industry-authenticated certificate.
NAF’s system uses multiple methods to assess a broad range of career-related content and skills and allows students to demonstrate their learning in a variety of ways. At the same time, it employs rigorous measurement practices to ensure a valid and reliable product. Most of the foundational skills and dispositions measured by the NAF Student Certification Assessment System are also recognized as being important for college readiness.
The state assessments currently being designed in support of the Common Core State Standards are widely viewed as stronger measures of college readiness than career readiness. They will also, to some extent, assess 21st Century skills that are equally important for both college and career readiness. The NAF assessments are a valuable complement to the state assessments: they are stronger measures of career readiness and offer additional evidence of the student proficiencies needed for both post-secondary study and the workplace.
NAF’s work in student certification assessment directly addresses current federal and state requirements that career and technical education programs assess technical knowledge and skill proficiencies “aligned with industry-recognized standards.” However, it is not meant merely to help high schools comply with federal and state accountability requirements.
Students earning the NAF Student Certificate can expect multiple forms of recognition by postsecondary institutions (through articulation agreements) and employers (through scholarship programs and recruitment for entry-level positions). NAF is actively pursuing such agreements.
Student Certification Assessment System Components. The components of the certification assessment system include:
- Four Culminating Project Assessments—students’ performance in curriculum-based, classroom-embedded projects is assessed for demonstrated proficiency in technical content and skills, both specific to a career theme and those that are broadly useful, using a standardized rubric and anchor sample assessments.
- Four End-of-Course Examinations—proctored, curriculum-based, online exams comprised of both selected response and constructed response item types assess the breadth and depth of career and technical knowledge and skills across industry-authenticated course units.
- Culminating Work-Based Learning Experience Assessments—supervisors’ assessments of students’ demonstrated proficiency with application of career and technical knowledge and 21st Century skills essential for both college and career readiness.
The satisfactory completion of all nine components leads to an end of program certification.
Student Certification Assessment System Development. Industry recognition and authentication of the NAF student certification assessment system was established by including industry professionals in every phase of curriculum and assessment development.
Working professionals from the finance, hospitality & tourism, and information technology career areas (NAF’s career themes for which we have four years of curriculum developed) participated as subject matter experts in development of every course, and continue to participate in annual updating of course curricula. In addition to the relevance and appropriateness of course learning objectives and content knowledge, industry representatives recommended and reviewed project- and problem-based learning activities and the culminating project in each course.
All the components for NAF’s student certification assessment system (exam items, scoring guides for constructed response exam items, project assessments, and the internship assessment) were created by WestEd’s curriculum and assessment development experts. Industry professionals participated in the assessment development at two critical junctures: selecting and vetting targeted learning outcomes; and reviewing exam items and subcomponents of project assessments (e.g., what student work samples were most appropriate as evidence of learning, and how work samples and student reflections would be evaluated) and the internship assessment (e.g., what learning outcomes would be assessed, and reviewing appropriateness of performance level descriptions, and examples of evidence incorporated into the scoring rubric).
Student Certification Assessment System Operations. The entire student certification assessment system is being delivered online using a platform developed by Internet Testing Systems. Internet Testing Systems (ITS) is an industry leading provider of customized solutions for online assessment programs, reliably delivering over four million assessments each year. ITS clients span the education, certification, healthcare, information technology and corporate sectors.
To complete project assessments, students upload work samples from planning and implementation phases of their projects and the final product (or a digital representation thereof), and write reflections explaining how the uploaded work demonstrates attainment of the targeted learning outcomes. At the end of each course in which a project is completed, teachers use a rubric, anchor examples, and scoring instructions created by WestEd to perform the summative assessment of students’ work. For each project, students are assessed on (1) evidence of content knowledge, (2) evidence of application of knowledge and skills, (3) awareness of own ability and performance in relation to learning objectives, and (4) evidence of teamwork and collaboration.
End of course exams, created by WestEd in collaboration with curriculum developers, teachers and administrators, and industry representatives, are taken online in a proctored setting using a secure platform provided by Internet Testing Systems. Multiple choice items are automatically scored; constructed response items are hand scored online using scoring guidelines created by WestEd.
Teachers are trained by WestEd and NAF to assess project work and score constructed response items from the end of course exams. Work-based learning supervisors receive orientation and training from academy personnel. Adherence to the scoring guidelines and inter-rater reliability will be constantly monitored through an audit process in which trained independent raters will re-assess/re-score a percentage of project work and constructed response items randomly selected from completed project assessments and exams.
Current Status. As of June 2012, all course-based assessment components have been completed and have been piloted in the online platform. The work-based learning assessment (Supervisor Assessment of College and Career Readiness) is currently in its third and last iteration of field-testing. Pilot data will be analyzed and final revisions and refinements will be made this summer. The complete assessment system will be available for academies to begin using in the fall of 2012. It is anticipated that the first cohort of students to complete all components and earn the NAF certificate will be the class graduating in the spring of 2014.
Acknowledgements.The National Academy Foundation thanks its corporate and foundation partners for their support of this important work: Capital One Foundation, Citi Foundation, The Edna McConnell Clark Foundation, The James Irvine Foundation, KPMG Foundation, The McGraw-Hill Companies, The Moody’s Foundation, The Travelers Companies, Inc.