ASSESSING STUDENT LEARNING: THE CURSE OF STANDARDIZED TESTS

By Ronald Fox

 

It’s that time of year; public school academic test scores are in and are being reported in the news. The Sacramento Bee reported on August 30 that 70% of schools in the Sacramento region had their Academic Performance Index (API), a composite of student test scores, decline from 2012 to 2013. Curses! Already fingers of blame are being pointed. Teachers blame larger classrooms, reduced school funding, and the inappropriateness of standardized tests for measuring academic performance; principals cite disruptions associated with implementing new California state curriculum standards (called Common Core); district chiefs emphasize that scores have been rising over the last dozen years or so and perhaps the schools have “topped out,” and a slight slip was inevitable; many school “reformers” and politicians insist poor teaching is at fault; and, free-market fundamentalists would have us believe, it's those damned teachers' unions.  What's with all the ado over test scores? 

I don’t know who or what's at fault for the API decline, or even if the falling scores, in fact, really mean anything.  You see, standardized school tests, as administered in the U.S., are notoriously flawed instruments for measuring student knowledge and learning; they're even worse as an indicator of teaching effectiveness.  What I do know is that standardized test scores can have monumental consequences. Falling scores predict trouble for students, teachers, principals, schools, and even states.

Schools are fully aware of the significance of the yearly API scores.  When schools fall below prescribed API standards, low-scoring students can be held back, budgets cut, schools closed, privatized, or turned into charters, and teachers and principals dismissed. Rising scores can guarantee teachers, as well as principals, good performance evaluations, job security and increased pay.  Financial support from the federal government to states is linked to meeting mandated benchmarks established under the Obama Administration's Race to the Top Program, which are even stricter than the Bush Administration's draconian No Child Left Behind standards. 

With so much riding on test scores--the fates of students, teachers, principals, districts, and even states--it is no surprise motivation is high to produce high scores, whatever this requires. The centrality of standardized tests in federal and state education models reflects the ascendance of modern day school "reformers," who believe that by raising academic standards and applying pressure, schools can be improved. There is not a shred of evidence to back this claim, yet the reformers have stuck to this vision even when evidence counters their assertions. They also continue to back testing as a mechanism to assess effective teaching, despite mounting evidence to the contrary.

Standardized tests are poor indicators of student learning because the tests themselves are error-prone and because varying scores can reflect random variation owing to factors not related to a student's education.  Technical errors, such as when poorly worded questions allow more than one correct response, and scoring keys have incorrect answers, occur with some frequency. Across districts and states, tests can vary widely in degree of difficulty, thus yielding misleading results. Student performance can also vary because of non-education factors, such as poverty, personal problems, lack of sleep, hunger, or even the weather.  In addition, variation in test scores may simply reflect changes in the student population: an influx of a different number of high or low achievers, students with disabilities, or English as second language learners than in previous years. Home situations may have changed, resulting in less parental engagement. Idiosyncratic factors, such as a student’s personal motivation, may also be involved. Flaws in the testing instrument, along with non-educational factors associated with test performances, make it unreasonable to draw definitive conclusions about student learning and the effectiveness of individual teachers, or schools.  Nevertheless, this is precisely how standardized testing regimes are used in most states.

The extremely high stakes in the test-based, learning assessment system incentivizes all the players in the education business--students, teachers, principals, and administrators--to game the system to produce higher scores, or at least the appearance of higher scores. Rising scores make them all look good. Gaming devices include teaching students test-taking methods, kicking low-performing students out of school, dumbing-down tests by making them less challenging or by lowering the passing score necessary to meet government-imposed standards, expanding the pool of students needing extra test time or other assistance, tossing out scores of students not continuously enrolled in the district, and for schools that can choose their students, restricting admission to highly motivated students, or, conversely, refusing to admit students with poor academic records. Such practices are rarely reported, and even if they were, as long as they lift a school's API profile, few would complain.  Rising scores make parents happy, reward educators, provide politicians a credit-taking opportunity, and pad the pockets of the companies that provide the tests.  And all along, the media goes along for the ride.  Sure they report the yearly test results, good and bad, but rare is the media outlet that investigates whether test score gains are real and meaningful.

 

Worst of all, impressive improvement in test scores can result from outright cheating, such as oft-reported instances when student answers are erased and correct responses penciled in, test questions are leaked in advance, and wrong answers are pointed out by teachers while tests are being taken. Test cheating has grown in direct proportion to rising pressure for higher scores. Evolving information technologies are now opening new cheating opportunities for students. For example, the Sacramento Bee reported on August 9 that California education officials flagged 246 schools statewide, including 15 in the Sacramento area, for possible cheating because students shared test-related photos on social media. Similar cases are turning up in other states; most notoriously, perhaps, when a student at the super-elitist, Stuyvesant High School in New York City took a photo of a Spanish regent test and texted it to his fellow students.  Can we expect anything different? 

In most professional organizations, when performance appears to fall below a specified standard, the lion’s share of blame usually falls heaviest on employees well down the organizational chain, be they sales people, service personnel, entry level workers, or, in this case, teachers. Teachers invariably take the biggest hit. But they are not solely responsible for falling, or rising, test scores, even if such scores were accurate indicators of student learning. As I stated, other factors are also important, especially the influence of social class.

And what about the the rarely discussed role of students themselves? Many people seem to believe students are merely passive recipients of a teacher’s effectiveness: good teachers produce good results, and poor teachers, the opposite. But it isn’t so simple. What is the student’s role? Do they attend class, do reading assignments, complete homework, pay attention in class, take pride in their work, make a serious effort? Such things affect student performance at least as much as a teacher’s skills, and probably more.

Don’t get me wrong; I’m not one of those former teachers opposed to learning assessment. Accountability models that integrate standardized testing with other evidence of teacher effectiveness based on multiple measures of student learning are much more meaningful devices for ensuring improved student learning and teacher accountability. If used wisely, such models can inspire improved teaching through better subject knowledge, course organization, teaching techniques, mentoring, knowledge of students, and a better understanding of what works and what doesn’t. A positive learning outcome, however, will not likely occur when the primary incentive for educators is to produce higher test scores. Studies have shown that more effective teaching is also not likely to occur when financial incentives are dangled in front of teachers.

Broader-based accountability models would help not only teachers, but also principals, administrators, and parents wanting to know if effective teaching and learning is taking place. As long as standardized testing remains the exclusive, or near exclusive, criteria upon which vital educational judgments and decisions are made, we can expect to see more of what’s wrong with the current system: teachers taking up valuable class time to teach standardized test-taking skills, often at the expense of teaching critical thinking, complex reasoning and written communication; educator’s tampering with tests; schools manipulating the pool of test takers; tests being made less challenging; passing scores lowered; and, students and teachers being unfairly punished, or rewarded, for test scores for which they share only partial responsibility.  In the scandal-driven world of high stakes testing, everyone loses.

Given growing educator and parent frustration with high stakes, academic achievement tests, it is no surprise resistance to these tests is growing.  In 2012, California rejected tying teacher evaluations to test scores and bucked the federal government's Race to the Top Program priorities. In Seattle, student and teachers boycotted the Measures of Academic Progress Test (MAP), which resulted in Seattle high schools being allowed to opt out of MAP tests.  Similar battles are taking place throughout the nation.  Notable developments, to be sure, but here's still a long way to go if standardized tests are to be dethroned from their exalted perch.