NASHVILLE — Offering middle-school math teachers bonuses up to $15,000 did not produce gains in student test scores, Vanderbilt University researchers reported Tuesday in what they said was the first scientifically rigorous test of merit pay. The results (pdf) could amount to a cautionary flag about paying teachers for the performance of their students, a reform strategy the Obama administration and many states and school districts have favored despite lukewarm support or outright opposition from teachers’ unions. The U.S. Department of Education has put a great deal of effort into prodding school districts and states to try merit-pay systems as part of its Race to the Top competition, although teachers’ unions have often objected on the grounds that they don’t have fair and reliable ways to measure performance. In most school districts, teacher pay is based on years of experience and educational attainment levels. The report’s authors, of the National Center on Performance Incentives (NCPI) at Vanderbilt University’s Peabody College of Education, stress that theirs is just one approach. The Nashville teachers who hit the mark based on their students’ test scores received a bump in their paychecks but no additional mentoring or professional development. Neither their principals nor fellow teachers knew who participated in the experiment or who received bonuses. Matthew G. Springer, director of the federally funded NCPI, said pay-for-performance is not “the magic bullet that so often the policy world is looking for.” At least in this experiment, Springer said, “it doesn’t work.” The study was conducted from 2006 to 2009 in partnership with the nonprofit RAND Corporation . A local industrialist and Vanderbilt benefactor, Orrin Ingram, put up the nearly $1.3 million in bonuses. Some 296 middle-school math teachers — two-thirds of the district’s middle-school math teachers — volunteered to participate in the experiment. Half were placed randomly in a control group, while the rest were eligible for bonuses of $5,000, $10,000 or $15,000 if their pupils scored significantly higher than expected on the statewide exam known as the Tennessee Comprehensive Assessment Program. One third of the eligible teachers — 51 of 152, or 34% — got bonuses at least once. Eighteen teachers received bonuses all three years. Except for some temporary gains for fifth-graders, though, their students progressed no faster than those in classes taught by the 146 other teachers. The local teachers’ union in Nashville agreed to the experiment in collective bargaining, according to Erick Huth, president of the Metropolitan Nashville Education Association. He said the results were not at all surprising. “I’ve believed for a long time that what improves instruction is having an instructional leader who is able to get all players in a school to collaborate,” Huth said The bonuses amounted to as much as 30% of teachers’ yearly salaries here in the Music City, where the scale runs from $36,000 to $64,000, Huth said. The nation’s 3.2 million public school teachers earned $53,910 on average in 2008-09, according to the National Center for Education Statistics. The study was released at a two-day conference, “Evaluating and Rewarding Educator Effectiveness,” at Vanderbilt’s Peabody College that drew participants from Colorado, Georgia, Tennessee, Washington, D.C. and other places conducting their own experiments with performance pay. Some states have moved to tie teacher and principal evaluations to student test scores. The study did not shake the faith of U.S. Secretary of Education Arne Duncan in merit pay. “While this is a good study, it only looked at the narrow question of whether more pay motivates teachers to try harder,” said Sandra Abrevaya, a spokeswoman for Duncan. It did not address the Obama administration’s push to “change the culture of teaching by giving all educators the feedback they need to get better.” The Nashville experiment, known as POINT (Project on Incentives in Teaching), doled out the $15,000 bonuses to those teachers whose students performed “at a level that historically had been reached by only the top 5% of middle school math teachers.” Teacher performance was calculated by using a value-added model, which predicts how students will do in a given year based on how they performed in the previous year. The teachers had to hit the 80th and 90th percentiles to pocket the $5,000 and $10,000 bonuses, respectively. The study’s design, in which teachers were judged against percentile benchmarks rather than their colleagues’ performance, sought to preserve collaboration among teachers. In surveys about the program, most teachers said they were already effective without the incentive of additional pay. Eight in ten said they didn’t change the way they taught to improve their odds of earning a bonus. Many teachers came close to getting a bonus — so close that they would have qualified if their pupils answered two or three more questions correctly on the 55-question state exam. The Nashville math teachers, according to the study, “expressed moderately favorable views toward performance pay in general, though less so for POINT in particular.” The experiment ran smoothly, although the teachers became less enthusiastic over the three years. “They did not come away … thinking it had harmed their schools,” the study said. “But by and large, they did not endorse the notion that bonus recipients were better teachers.” The fact that many fifth-grade teachers teach multiple subjects to the same students may have been a reason for the positive impact of merit pay found in fifth grade, according to the study’s authors. But “the effect did not last. By the end of 6th grade it did not matter whether a student’s 5th grade math teacher had been in the treatment or control group,” the study said. The researchers said the Nashville experiment didn’t stir the negative reactions that have attended some other merit pay programs, but it “simply did not do much of anything.” Springer of NCPI said the study lays a foundation for further experiments on a topic that educators have been debating “for over a century.” Tennessee Commissioner of Education Timothy Webb said the Nashville study shows, “Money alone is not enough to encourage people to go into challenging schools and teach the most difficult students.” He stressed the importance of improving teachers’ working conditions, not just their pay. Frederick M. Hess, director of education policy studies at the American Enterprise Institute in Washington, D.C., said he does not believe the study says much of value and worries it will only confuse the issue. “The fact that that teachers don’t respond to cash bonuses like rats do to food-pellets does nothing to diminish my confidence that it’s good for schooling if teacher pay better reflects the contributions that teachers make,” Hess said. “Serious proponents of merit pay believe the point is not any kind of short-term test-score bump but making the profession more attractive to talented candidates.” William Slotnik, executive director of the Boston-based Institute for Compensation Reform and Student Learning at the Community Training and Assistance Center, has argued that performance-based compensation tied directly to the educational mission of a school district can be a lever to transform schools. But he said it will take more than financial incentives to improve student achievement and that merit pay “is hard to get right. … If all you are doing is focusing on money, there is no track record in that resulting in the kind of changes needed to do this work well.” Contributing: Christopher Connell is a freelance writer. Liz Willen contributed to this article. It was produced by The Hechinger Report, a nonpartisan education-news outlet affiliated with the Hechinger Institute on Education and the Media at Columbia University

On Friday afternoons between work and rugby practice, Brittany Wolfe would rush to the campus library hoping copies of her advanced algebra textbook had not all been checked out by like-minded classmates. It was part of the math major’s routine last quarter at the University of California , Los Angeles: Stand in line at the reserve desk in the library’s closing hours with the goal of borrowing a copy for the weekend. The alternative was to buy a $120 book and sell it back for far less. If she could sell it back at all. “It’s like this terrible game of catch your books when you can,” said Wolfe, a new graduate who estimates she saved $800 a year using books on reserve and who now shares textbook tips as a counselor to incoming UCLA students. “It’s frustrating when you’re already stressed about school. Being stressed about textbooks doesn’t seem right.” Maybe, just maybe, relief is on the way. A new federal law requires publishers to provide textbook price information to professors and calls on colleges to identify course textbooks during registration, giving students more time to shop around. Experts call it a step in the right direction, but not a game-changer. ‘OLD SCHOOL’: Arizona college cuts book costs the old-fashioned way PROFESSORS: Some stopped from cashing in on textbooks At the same time, a robust online marketplace of used books and recent inroads by textbook rental programs give students more options than ever. The prospect of digital books and slow-but-steady growth in free online “open” content loom as developments that could upend the textbook landscape and alleviate the perennial problem of rising prices. “Change is coming, but it’s not going to happen immediately,” said David Lewis , dean of the Indiana University-Purdue University Indianapolis University Library and assistant vice president for digital scholarly communications at Indiana University. “If you’re in junior high school, you can be sure it’ll be better. If you’re in high school, there’s a shot. If you’re starting college as a freshman, you might see it as a senior. It’s on more and more people’s agenda.” According to a 2005 study by the U.S. Government Accountability Office, college textbook prices increased at twice the rate of inflation over the previous two decades, though not as dramatically as tuition. More recent data from the National Association of College Stores show textbooks costs climbed 14% from the 2006-2007 academic year to 2008-2009. A 2010 survey by the group found students spent an average of $667 per year on required course materials including textbooks, although other studies have put the figure at about $900. In 2008, Congress responded by including textbook-affordability provisions in the Higher Education Opportunity Act. Along with the price-disclosure clause meant to push professors toward cheaper options, it requires publishers to offer textbooks separately from extra items like workbooks and CDs. The practice of “bundling” products leads to markups of 10 to 50% and makes books harder to sell, according to the Student Public Interest Research Groups, which pressed for the reforms. “We have more lower cost options than ever before, and professors are going to have more information than ever before,” said Nicole Allen, textbook advocate for the student PIRGs. Like the music and media businesses, the textbook industry has been revolutionized by the Internet. Although used books have long been an option for students, the Web opened up a world of bargain-hunting beyond the campus bookstore. These days, sites such as BIGWORDS and BestBookBuys let students search several online stores at once. The 13th edition of the seminal textbook “Marketing Management,” which lists for $190 new, can be had for as little as $19.99 used. More recently, textbook rental sites such as Chegg, BookRenter and CollegeBookRenter have arrived, offering rentals at roughly half the cost of buying. Their business model — Netflix goes to college — has prompted college bookstores and publishers to play catch up and offer rentals themselves. Textbook publisher Cengage Learning began renting directly to students last spring and has expanded its online rental inventory to 3,000 titles. Campus bookstore operator Follett will introduce rentals at more than 800 bookstores this fall, and Barnes & Noble will do the same on more than 300 campuses. Earlier this summer, BookRenter, which has contracts with Amazon.com and other online booksellers to fill orders, announced that more than 75 campus bookstores would use its platform to rent textbooks. Chegg keeps its own inventory of nearly 5 million books at a warehouse outside Louisville The start-up aspires to forge direct relationships with students, shipping products in their own packaging, offering a liberal return policy and promising to plant a tree for every order, said CEO Dan Rosensweig , a former Yahoo executive. Behind the scenes, publishers get a share of the rental revenue — something they can’t say about used book sales. Open access textbooks pose a bolder challenge to the status quo. The start-up Flat World Knowledge contracts with authors to write new textbooks and publishes them for free under an open content license, allowing professors to edit the raw material and add their own contributions while giving students access to a Web-based HTML book. Last fall, about 480 professors adopted one of the company’s initial 10 business and economics titles, said co-founder Eric Frank. About 1,200 professors are expected to use 22 titles to teach 95,000 students this fall. The company is betting students will pay a reasonable price for greater convenience. Flat World’s revenue comes from selling everything from $30 black-and-white copies of its books to $3 audio chapters, as well as study aids like digital flash cards. About 55% of students are buying something at this point, Frank said. So far, the main drawback to open access is the dearth of titles, said Albert Greco, a professor at Fordham University ‘s Graduate School of Business Administration and an authority on the textbook publishing industry. Greco and others forecast a major shift in the next five years to digital textbooks, which already cost about half as much as new print editions on CourseMart.com, a kind of textbook iTunes launched in 2007 by the major textbook publishers. That would doom the used book and print rental marketplace, Greco said. As for immediate relief from the new price disclosure law, Greco said it won’t do any good for students unlucky enough to have four courses with brand-new books. “Whether it will help students comes down to, ‘It depends,’” he said. Sophie Stanish, a junior at Fordham University in New York, fumes about paying $200 for a new math textbook she couldn’t sell back and a $10 short-story collection that fetched 75 cents at trade-in. She likes the concept of Fordham’s “E-RES” program — short for “electronic reserve” — in which professors scan sections of textbooks to the extent allowed by copyright law and then put the material online for free. But, she said, “I can’t read off a screen and retain the knowledge as well. It’s a personal thing. I like to highlight.” Other colleges seeking to provide relief have adopted textbook loan programs. At City College of San Francisco, Kathy Gill said she misses class to line up early for a popular loan program for students on financial aid. The limit is two loaned books, so the business major still shops online for used and rental options each semester. “You do get a little bit of a break,” Gill said. “Every little thing helps.” Copyright 2010 The Associated Press. All rights reserved. This material may not be published, broadcast, rewritten or redistributed.