After nearly a decade of research, the National Assessment Governing Board (NAGB) released in May the first outcomes of its efforts to use the results of the 2013 12th grade National Assessment of Educational Progress (NAEP) to report on the academic preparedness of U.S. 12th graders for college. It found that only 38% of 12th graders meet its preparedness benchmark in reading, and 39% meet its preparedness benchmark in math. NAGB’s efforts to track college readiness in the United States is uniquely important as it has the only assessment program that reports on the academic performance of a representative national sample of high school students.
That said, the group that issues the Nation’s Report Card deserves a grade of “Incomplete” for its work. Reading and math are obviously necessary indicators of academic preparation for college and careers after high school, but higher education and employers say it’s not enough. When it comes to the ability to complete college level work (and to being career ready), writing skills are essential. Yet, despite the fact that NAGB also administers a 12th grade writing test, it inexplicably chose not to include writing as an indicator of readiness.
If NAEP wants to remain the “gold standard” for assessment, NAGB must remedy this situation quickly. Postsecondary institutions and systems throughout the nation assess writing in order to determine whether students have the academic skills to succeed in first year courses. According to ACT, approximately one third of ACT test takers do not meet its readiness standard for English Composition. Recent data from Florida indicates that 32% of first year students are placed into developmental writing courses. Using preparedness indicators that do not include writing will not only provide incomplete information to the public but will send the wrong signal about the importance of writing for high school graduates. And states that assess writing need an independent external benchmark they can rely on, which NAEP has always provided with their reading and math assessments.
Unfortunately, the current NAEP 12th grade writing assessment, starting with the Writing Framework that guides the development of test items, will need substantial revisions to be a valid indicator of academic preparedness. One of the most important advances made through the development of the Common Core State Standards (CCSS) English Language Arts/Literacy standards is the understanding that preparation for both postsecondary education and careers requires the ability to read texts of appropriate complexity and mobilize evidence from the text to make a clear and logical written argument. Achieve’s earlier research with states on college and career readiness for the American Diploma Project provides a strong foundation for expecting high school students to be able to write coherent arguments supported by evidence from credible sources. The CCSS are quite explicit on this issue, building the idea of "writing to sources" into the grade-by-grade progression of the writing standards. Focus groups of postsecondary faculty conducted by PARCC assessment consortium powerfully underscored the importance of these skills.
While NAGB does not need to align its assessments and their frameworks to the CCSS, it does need to pay careful attention to the evidence upon which they rest.
A review of the 2011 NAEP Framework and sample items makes clear that the assessment does not address the ability of students to draw on evidence to make persuasive arguments. In fact, the released 2011 12th grade items do not come close to assessing writing to sources.
One item asks students to write an essay describing how he/she uses technology. It includes a prompt that presents survey data on how students use computers, but doesn't require use of or reference to the data in order to respond to the prompt.
12th Grade NAEP Writing Prompt:
Write an essay for a college admissions committee about one kind of information or communications technology you use. Describe what it is and explain why the technology is important to you. Develop your essay with details so the admissions committee can understand the value of this technology. You may use information from the presentation in your essay.
NAGB’s web site shows several sample responses, including one that was rated Effective (the highest score), one rated Competent, and one rated Adequate. None of those highly ranked essays made any use of the survey data presented in the question. Those data were window dressing. In short, this item does not require students to read anything (except the question), nor to make an argument based on the evidence provided.
Another item asks 12th grade students to write a persuasive letter to the local council on whether or not to build a discount store in the area. It too is also of limited value for assessing preparedness in writing. It asks students to read a contrived "newspaper article" regarding plans to build a store in the community. First, the text is considerably less complex than what 12th graders should be able to handle and even less complex than what would be found in many newspaper stories.
12th Grade NAEP Writing Item:
The following article recently appeared in your local newspaper. Write a letter to the local council members arguing for or against the building of Big Discount in your area. Support your argument and defend it against the arguments the opposing side might make.
And while students are expected to marshal evidence to support their positions, the sample responses include assertions about evidence and facts, but with no sources cited, and no useful evidence provided in the article students were asked to read. Students could simply make up evidence for their response. That’s not the type of preparation for college we should encourage.
If NAGB wants to make a significant contribution to the national conversation about college readiness, it will have to quickly step up its game. Both multi-state assessment consortia, PARCC and SBAC, have developed assessments that incorporate “writing to sources” into their high school assessment programs, and many states will begin to administer them next year.
PARCC 11th Grade Sample Writing Task:
Today you will read a biography of Abigail Adams, and then you will read two examples of correspondence between Abigail and her husband, John Adams, who served as President of the United States from 1797 to 1801. As you read these texts, you will gather information and answer questions that will help you understand John and Abigail Adams’s relationship and opinions. When you are finished reading, you will write an analytical essay.Question: Both John and Abigail Adams believed strongly in freedom and independence. However, their letters suggest that each of them understood these terms differently based on their experiences.
Write an essay that explains their contrasting views on the concepts of freedom and independence. In your essay, make a claim about the idea of freedom and independence and how John and Abigail Adams add to that understanding and/or how each illustrates a misunderstanding of freedom and independence. Support your response with textual evidence and inferences drawn from all three sources.
Sending the right signal to the public and to state policymakers about the importance of assessing writing for college readiness is particularly important now as some states are contemplating buying off the shelf tests or creating their own.
In addition, if NAGB is serious about having a complete indicator of college readiness, they should revise the schedule for administering the 12th grade writing assessment. The last 12th grade writing assessment was given in 2011, and it is not scheduled to be administered again until 2017. Every six years simply isn’t enough.
In the decade it took NAGB to conduct its academic preparedness research, states moved rapidly to make college and career readiness the mission of their K-12 systems, and a national priority. Today, every state has adopted college- and career-ready standards in literacy and mathematics, either the CCSS or their own state standards. And, states are working to develop and administer tests that measure college ready skills – and are honored by postsecondary institutions – to high school students statewide. Twenty states have raised course-taking requirements for high school graduation, and many are working to incorporate indicators of college-readiness into their accountability and reporting systems.
In short, the states are way out in front on promoting and assessing college readiness. NAGB doesn’t have a moment to waste.
Michael Cohen is the president of Achieve.
This article originally appeared in Achieve's June 2014 Perspective newsletter.
