Reflections on NAEP: Breaking the Reactionary Cycle

Earlier this month, the National Center for Education Statistics released the results from the 2017 administration of the National Assessment of Educational Progress (NAEP) in reading and mathematics. These tests measure 4th and 8th grade student proficiency at the state and selected large urban school district level. The results are further reported by a variety of demographic groupings. The NAEP test is praised for administering a common metric across states and over time, providing a consistent measure by which to evaluate student progress at the national and state level. These scores are posted as part of The Nation’s Report Card.

The Test Score Cycle

In responding to these results and trends, many civic leaders–governors, state education commissioners, editorial boards, and chambers of commerce–are celebrating, bemoaning, critiquing, and questioning themselves and their colleagues. Statutes, policies, and adjustments to systems of accountability will occur.

Assertions that systemic change will take time are currently being recited as people infer the meaning of low single digit variances from the previous round of testing. This year, a highlight in the official release documents includes the observation that “All reading and math scores are higher than the 1990s.” And the reading score difference is in the single digits–from a generation ago, during which two cohorts of students have matriculated through the K-12 system.

This familiar routine of releasing results, parsing the meaning, adjusting policies, amending accountability, and tweaking budgetary allocations is not limited to responding to NAEP. This occurs in each state on an annual basis in response to the once NCLB-, and now ESSA-, mandated assessments to receive federal funds. This process is undertaken with added intensity when the state issues school and/or district “grades.” And the ritual is done triannually at the international level with the release of the Programme for International Student Assessment (PISA) results.

This cycle of designing and administering assessments, and then responding to the results, is the dominant input to decision making related to improving educational outcomes. Decisions on maintaining or adjusting educational approaches, on setting resource allocation priorities, and on what feelings leaders convey about the public educational system emerge from reacting to the test scores. It is how we gauge whether or not specific school systems are doing right by their kids in preparing them for the future.

As a result of these score-generated perceptions of schools, people make decisions about how schools are structured and function. These decisions are made with the constraints of limited resources, with agendas of varying priorities, and interest groups ascribing differing causes to observations. And over the fifty-year history of NAEP and other assessments, reactions have been characterized by urgency, by the replacement of previous practice with a new silver bullet, and accountability systems with dashboards of output metrics.

Putting Test Scores in Context

These measures are important. But, during the generation of time which has seen the policy world engage in reading wars, fights over standards and assessments, and performance pay, there has been little growth in overall performance. We are not necessarily condemned to another fifty years of stagnation, though. If we look more broadly, there are areas of educational research and practice beyond those traditionally considered paramount in educational reform conversations that have significant positive effects on outcomes. We will not likely see significant movement on what NAEP seems to value until we embed what assessments are telling us into the larger context of educational progression and human development

To move beyond the knee-jerk, reactive response, educational policymakers and practitioners should consider including development-informed priorities like social-emotional learning, family engagement, and 21st century skills as a central part of their approach to academic growth.

Social Emotional Learning

Social Emotional Learning (SEL), which gives students the tools to manage emotions, connect with others, develop and maintain healthy relationships, and establish and work toward goals, is the most prominent of these approaches. Though many recognize SEL as important for student developmental success, doubt over whether SEL progress can be effectively measured, systemically embedded in academic curriculum, and supported under federal and state funding structures often means it is omitted from conversations around strategies to promote academic achievement. Those myths should be reconsidered.

Companies and organizations are providing evidence-based SEL solutions that raise student achievement by focusing on healthy, age-appropriate human development. Panorama Education, a provider of services to school districts that utilizes data gathering, analytics, and improving practices has a framework that gauges twenty-two attributes of social emotional learning. These attributes are grouped into three domains of Student Competencies, Student Support & Competencies, and Teacher Skills & Perspectives. Panorama asserts that “Research shows that investment in SEL has led to 13% gains in academics, improved classroom behavior, better stress management, and provided an 11:1 total ROI.”  Similar evidence showing the role of SEL in promoting academic achievement have been found by Center on Standards and Assessment Implementation, CASEL, and the OECD, the publisher of the PISA exam.

Interestingly, the Every Student Succeeds Act (ESSA) explicitly expanded the option for States to incorporate SEL as a central component of accountability frameworks. To facilitate this, the Rand Corporation provided an evidence review publication, free for States and districts to utilize in completing their ESSA applications, outlining evidence-based SEL interventions. Yet, after the submittal of each state’s initial ESSA application, not one State opted to measure SEL in its accountability plan. What could have been core was made tangential. Indeed, the state plans submitted to USDOE relegated expectations, initiatives, and funding opportunities related to SEL as non-core and supplementary. Policy makers had nowhere near a NAEP-like response when offered the opportunity to do so. And whether this is inertia, a lack of creativity, or simply fear at taking a different path, this response runs counter to the experience of educators and researchers who know that health–physical, social, emotional, and mental–precedes achievement. State Education Agencies and local school districts can still prioritize budgeting federal (Title I – V), state, and local funds to SEL practices. Hopefully, they will do so.

Family Engagement

One key area of research and practice that has been significantly linked to academic success, generally, and the positive development of literacy skills, specifically, is family engagement. The research on this is clear, well-established, and unambiguous: family engagement in a child’s learning has marked benefits for all students across a wide spectrum of outcome metrics. A 2007 meta-analysis by William Jeynes found positive impacts of family engagement across a range of measures of student achievement including grades, reading levels, standardized test scores, and teacher ratings. These gains hold true across all socioeconomic, racial, and ethnic groups. In short, all students benefit from family involvement in their education.

And we also know that certain types of family engagement are particularly effective. The Dual Capacity-Building Framework for Family-School Partnerships, developed by Harvard’s Dr. Karen Mapp for the U.S. Department of Education, articulates that family engagement that is directly linked to student learning has substantial benefit to overall academic success. Further, engagement that builds capacity in parents and caregivers, models effective practice, and increases confidence in a family member’s ability to be a partner in student learning, has lasting benefits in building literacy skills as well as creating an overall positive outlook toward learning.

This kind of engagement that has been shown to be most effective is not intuitive for many parents and caregivers.  Parents need scaffolded support to know how to engage their reader with texts of varying levels, topics, and degrees of reader interest. In fact, the Jeynes study did show that reading is one of the most valuable ways for parents to engage with their children. This experience of shared reading is much more meaningful when students are engaged with those around them in conversations, observations, and experiences about what they are reading. If school systems provide both structured and open-ended conversation starters and activities, they connect families to student learning, promote the development of literacy skills, and plant the seeds for a lifelong love of reading.

21st Century Skills & Deeper Learning

Employer groups, civic advocates, and higher education systems all report that 21st century skills, developed through deeper learner, are fundamental to educating college- and career-ready students. Educational systems that focus on 21st century skills promote the development of collaboration, critical thinking, communication, and creativity. Designing and implementing student experiences that promote these skills have been shown to strengthen life-readiness and result in higher traditional test scores. Deeper Learning has been shown to raise student achievement in math and reading; focusing on math and reading in isolation have yet to be shown to produce deeper learning. The tests do not, and never will, tell the whole story of student development.

And, a new NAEP instrument has begun to report some aspects of 21st century skills on the Nation’s Report Card. NAEP’s Technology and Engineering Literacy (TEL) Exam was first administered in 2014, and then again in 2018. The NAEP website explains the test assesses 8th graders on “how well students can apply their understanding of technology principles to real-life situations.” These results merit a similar engagement of press conferences, opinion pieces, and policy focus groups when released later this year. And, hopefully, the urgency to create academic systems and structures that promote achievement through the development of 21st century skills will follow.

Getting Off the Pendulum

While it is important to consider the results of assessments like NAEP, educational policy and practice should be designed based on more than a reaction to a narrow snapshot. Our nation’s schools have been hanging on the swinging pendulum of public expectations for decades. Educators continue to try to find a balance between the advocates for phonics and whole language approaches, between an emphasis on knowledge versus skills, between the hunger for data and the hope for authentic assessment in responding to the “assessment crisis” of the moment. Decision-makers have structured and re-structured our educational systems with the explicit goal of moving these very specific outcome measures. And, yet, the public has seen little progress in reading due to the lack of consistency from the vagaries of community reaction.

Educators, administrators, and policymakers need to take a step back and shift their responses to concerns about academic achievement to include more nuanced understandings of student development. Conversations around these holistic approaches to learning have become more prevalent. Initiatives that include mindfulness, problem-based inquiry, and growth mindsets are now seen as fairly mainstream and, often, desirable. However, in the wake of test score releases, especially those showing mediocre or negative results, political will for these more innovative programs often subsides. History has shown that attention turns toward activities that seemingly directly, and certainly reactively, prepare students for the next test administration.

These behaviors do not support what we have learned about how to have students learn deeply and authentically. And when there is a gap between what we know about how students learn and grow–about what truly builds foundational understanding rather than quick recall, about what kind of learners will be successful in an ever-evolving marketplace–and what we measure, good practice gets lost. We need to value long-term, systematic approaches to student learning that research tells us work. We also need to question what we are measuring and whether it matches those values. In doing this, we will ultimately increase test scores. But more importantly, we will create a generation of students who are able to meaningfully and productively take on the challenges of the future.

For more, see:

• Innovating Large-Scale Assessments: Design Considerations for Education Leaders
• Performance Assessment in Different Learning Environments
• An Assessment for Change: “Is It Best for Kids?”

Stay in-the-know with all things EdTech and innovations in learning by signing up to receive our weekly newsletter, Smart Update.