Editor's note: On Tuesday, February 2, Fordham hosted the ESSA Acountability Design Competition, a first-of-its-kind conference to generate ideas for state accountability frameworks under the newly enacted Every Student Succeeds Act (ESSA). Representatives of ten teams, each from a variety of backgrounds, took the stage to present their outlines before a panel of experts and a live audience. We're publishing a blog post for each team, comprising a video of their presentation and the text of the proposal. Below is one of those ten. Click here to see the others.
ESSA Accountability Design
A Proposal by the Prichard Committee Student Voice Team
The objectives of our state accountability system are:
- to create a holistic view of school quality through both academic and nonacademic indicators of success; and
- to provide students, parents, teachers, administrators, and other stakeholders with the information they need to be effective advocates, understand potential problems within their schools, and strive for all students to receive a beneficial educational experience.
School accountability measures are most effective when they identify what a school is contributing to students, not what students are contributing to a school.1 Our system, by focusing on these objectives, will encourage schools to use their resources in the most effective ways and facilitate improvement by including all invested parties, building on the idea that it takes a village to educate a child.
Schools will be evaluated on four indices that will be individually scored, weighted, and combined to indicate a school’s overall rating:
- Academic Proficiency: 31.25 percent
- Student Progress Over Time: 37.5 percent*
- Closing Performance Gaps: 25 percent*
- School Climate: 6.25 percent
*If the school has fewer than twenty students in a subgroup, that subgroup will not be included in Index Four, and 6.25 percent will be moved to Index Two.
Index 1: Academic Proficiency
Annual statewide testing would measure and test student competence for both English language arts and math with equally weighted scores, beginning in the third grade and continuing through fifth. The National Research Council has concluded that students who are not moderately skilled readers or adequately exposed to mathematics by the end of the third grade are unlikely to graduate from high school; results that reflect these trends prove powerful resources for states to advocate for more coordinated, effective learning systems.2 The critical analysis, reading-to-learn, text interpretation, language, and writing skills emphasized through the English language arts tests will create strong skills for future subjects.3 Similarly, emphasizing a strong foundation in math will prepare students for upper-level math and science classes.
Index 2: Student Progress Over Time
Student progress over time will be assessed using the Colorado Growth Model.4 Schools will be compared graphically by their median student growth percentiles (SGP) and the percentage of proficient/distinguished students in a school. The graphs will provide students, teachers, parents, administrators, and other stakeholders with information on the impact that an individual school has on its students when compared to other schools across the state. As a result, the public will be able to easily distinguish schools that are truly serving their students from those that are only benefiting from well-prepared students, informing stakeholders which schools are the best models. Using the Colorado Growth Model facilitates the general public’s understanding of how a low-performing school might be serving its students just as well as a high-performing school, while also revealing the low-performing schools that need the most assistance.
Index 3: Closing Performance Gaps
Students defined as being heavily affected by an “achievement gap,” as well as certain subgroups that regularly perform below or near average, should always be an indicator of academic achievement when assessing accountability. Students will be broken up into four traditional achievement gap subgroups for comparison, unless a student group does not meet the minimum size of twenty students. The four subgroups will be:
- Black/Hispanic/Native American students
- Economically disadvantaged
- Former/current English language learners
- Students with disabilities
These categories are based on the Tennessee and Massachusetts District Accountability Systems.6, 7
English Language Proficiency
To ensure that the needs of all English language learners are being met within their educational institutions, students who are deemed Limited English Proficient (LEP) will be tested to determine their level of English language proficiency. LEP students are defined as “any student who does not speak English or whose native language is not English, and is unable to perform ordinary tasks in English.” Students who are multilingual but fluent in English will not be classified as LEP. Preliminary indicators of LEP will come from a home survey to determine the students' native language and whether they are English Proficient. Additional input on students' progress can be taken from their respective teachers.
Once deemed LEP, students will be tested on their listening, reading, and writing abilities. While tests will vary according to grade level (e.g., writing words in kindergarten versus writing a short composition in the fifth grade), every student will be tested on English language arts each year until deemed English Proficient.
Once a student achieves a benchmark score, determined by individual schools, teachers will evaluate students, and a parent conference will be held to determine whether the student can be classified as English Proficient.
Before a student is classified as English Proficient, he will receive the following accommodations on state assessments:
- First year of enrollment in U.S. schools:
- Students are tested in all areas
- Students’ scores are not calculated into their school’s scores, but the aggregate scores must be reported by the school
- Second year of enrollment in U.S. schools:
- Students’ scores will only be measured as part of Subindex 3B
- Third year of enrollment in U.S. schools:
- Students' scores are calculated normally
Index 4: School Climate
It is appropriate that student ideas are at the forefront of identifying issues concerning the school climate, as students are the chief stakeholders in schools. Students would take surveys that would include questions about teacher and student engagement, constructive student voice, communication among students, communication between students and teachers/administrators, cohesiveness of the student body, effective use of resources, encouragement of creativity, and student support. Fairman and Clark found these factors be present in positive school climates.8
The survey will be created by the state and evaluated by the state. The student will evaluate each of the factors on a one-to-five scale. The responses will be averaged for each survey, and each survey will be averaged to find the school’s overall climate score. The responses to each question will also be averaged across all surveys and released so that the school and the public will know how the school performs on specific measures of school climate.
Students will also be asked a short-answer, open-response question about any issues or concerns facing his school. These responses will not be included in the school’s score. The state will report to the school all of the responses, but will only release to the public responses which appeared in 5 percent or more of the surveys. These responses would only have to relate to the same general issue (i.e., not identical to one another), and all identifying information will be removed. This will help schools address problems not directly addressed by the ranked questions.
The intent of these surveys is to create a knowledge base of what can be done to improve each school on a specific, student-based level. Positive change in schools is more effectively brought about through changes in climate rather than changes in structure.9 This measure of school climate effectively identifies problems within schools, which is one of our objectives, and therefore gives administrators the information they need to design beneficial changes that will enhance the educational experience of students.
***
School Rating System
*For every subgroup exemption a school receives, five points will be subtracted from the points possible for closing performance gaps and five points will be added to the points possible for student progress over tIme
Index Raw Scores
Index 1: Academic Proficiency
Index 2: Student Progress Over Time
For every student who demonstrates “typical growth” (i.e., exhibiting an SGP of 40 or greater), a school will be awarded .6 points. For every student who demonstrates “advanced growth” (i.e., exhibiting an SGP of 60 or above), the school will be awarded 1 point. The sum of these two values, divided by the total number of participating students, will be the school’s raw score for Index 2. (This calculation will include students’ scores in both English language arts and math)
This is based on Kentucky’s Unbridled Learning Accountability Model.10
Index 3: Closing Performance Gaps
The goal for all groups is to achieve or exceed an SGP of at least one point above the “typical growth” for the state. Groups with a median SGP of 41 or higher receive .6 points. Groups that achieve “advanced growth” will receive 1 point. The sum of these will be the school’s raw score for Index 3. (This calculation will include students’ scores in both English language arts and math.)
Index 4: School Climate
The score for school climate will be found by:
- Averaging the answers of each survey to find each student’s overall school climate assessment. Answers to the open-ended question will not be included.
- Rounding the average to the nearest whole number score. Digits 1-4 round down, digits 5-9 round up.
- Finding the student body mean overall assessment of school climate.
School Ratings
Schools will be put into four categories based on the table below. The score for each Index will also be reported. The performance of the subgroups listed in Index 3 with twenty or more students will also be reported. Participation rates overall, for each index, and for the subgroups listed in Index 3 with twenty or more students will be reported.
Participation Rate
All schools must meet a participation rate of 95 percent overall, for each index, and for all subgroups listed in Index 3 that have twenty or more students. Any school failing to meet the 95 percent participation benchmark for any component will:
? still have the scores for all indices reported, including indices with inadequate participation,
? be rated as “Below Expectations: Inadequate Participation,” regardless of the total score of other indices,
? and be subject to intervention.
Interventions
Schools ranking in the bottom 5 percent of the state will be required to propose goals or targets and plan steps to achieve them. This will occur through a planning committee that will be made up of students, parents, teachers, administrators, members of the state education department, and other stakeholders. Involving all affected stakeholders in the process is vital to the system's effectiveness because it will increase the probability that all ideas are heard and all resources used.
The planning committee must pinpoint where the school needs to be, identify barriers stopping the school from getting there, and recommend actions to transcend the barriers. The committee’s evaluation of what the school needs to accomplish must be reasonable, achievable in no less than ten years, and not allow the school to tout progress when little is taking place. The committee must then identify the barriers that hinder the school’s ability to reach the goals set forth. The barriers should be grouped together if they deal with a similar issue or area, and there should be no duplicates. The members of the committee then have to recommend the best course of action to break down the barriers. The committee should consider the views of all stakeholders included and comments from the public during the process with the underlying goals of inclusion and transparency.
The school must then begin the implementation of the committee’s recommendations. To ensure that the recommendations are being followed and implemented, all stakeholders should monitor the school’s implementation process.
This tier process is based off Florida's Guide to 8-Step Planning & Problem Solving for Continuous Improvement.11
_______________________
Notes
1. Hanushek, E. A., Machin, S., & Woessmann, L. (2010), Handbook of the economics of education, 3, 383-421 (DOI: 10.1016/S0169-7218(11)03008-5).
2. Achieve (2013), Creating a P-20 continuum of actionable academic indicators of student readiness. Retrieved from http://www.achieve.org/files/StudentReadinessIndicators.pdf.
3. Chingos, M. M., & West, M. R. (2015), "Why Annual Statewide Testing Is Critical to Judging School Quality," Brookings Institution. Retrieved from http://www.brookings.edu/research/papers/2015/01/20-chalkboard-annual-t….
4. Carey, K. & Manwaring, R, (2011), "Growth models and accountability: A recipe for remaking ESSA," Education Sector.
5. Screenshot of Colorado Department of Education’s SchoolView. Retrieved from https://www.cde.state.co.us/schoolview.
6. Tennessee Department of Education (2015), 2015 District Accountability Overview [PowerPoint slides]. Retrieved from https://www.tn.gov/assets/entities/education/attachments/acct_overview_….
7. Massachusetts Department of Elementary and Secondary Education (2015), School Leader’s Guide to the 2015 Accountability Determinations. Retrieved from http://www.mass.gov/edu/docs/ese/accountability/annual-reports/school-l….
8. Fairman, M. & Clark, E. (1982), Organizational problem solving: An organizational improvement strategy (Fayetteville, AK: Organizational Health Diagnostic and Development Corp.).
9. Kytle, A. W. & Bogotch, I. E. (2000), "Measuring reculturing in national reform models," Journal of School Principalship, 10, 131–157.
10. Kentucky Department of Education (2012), Unbridled learning accountability model (With focus on the next-generation learners component). Retrieved from http://education.ky.gov/comm/UL/Documents/WHITE%20PAPER%20062612%20fina….
11. Differentiated Accountability State System of School Improvement, Florida Administrative Code, Rule 6A-1.099811 (12 Dec. 2014).
