Research turns thumbs down on using student test scores to evaluate teachers.

[proud2BlibKansan]

Value added teacher evaluation, teacher performance reviews based on student test scores is all the rage. Arne Duncan and President Obama rave about it. Arne even claims, as he did all over the talking head shows on Sunday, that teachers want it.

None that I know.

Even the national union leadership, pressed by the Department of Ed and the media, concede that the problem with it is that it needs to be part of a larger evaluation process.

But four former presidents of the American Educational Research Association; two former presidents of the National Council on Measurement in Education; the current and two former chairs of the Board of Testing and Assessment of the National Research Council of the National Academy of Sciences; the president-elect of the Association for Public Policy Analysis and Management; the former director of the Educational Testing Service’s Policy Information Center; a former associate director of the National Assessment of Educational Progress; a former assistant U.S. secretary of education; a member of the National Assessment Governing Board; and the vice president, a former president, and three other members of the National Academy of Education, have all put their names on a report that turns thumbs down on the concept.

http://preaprez.wordpress.com/2010/08/30/research-turns-thumbs-down-on-using-student-test-scores-to-evaluate-teachers/

Link to report: http://epi.3cdn.net/724cd9a1eb91c40ff0_hwm6iij90.pdf

[TexasObserver]

1. These tests have been a disaster since inception, and gotten worse.

These tests pretend that good scores are mainly the result of what teachers teach, instead of the teachers simply playing the hand they're dealt by the parents and the district. Teachers who teach at poorer schools have a lot harder time producing scores than those who teach at more financially sound schools.

[ibegurpard]

2. doesn't matter

corporations want to run our schools.
they've paid for it and they're going to get it.

[proud2BlibKansan]

3. I believe you are probably right

Unfortunately

[TexasObserver]

4. It started with Coke and Pepsi contracts.

Shove a little money their direction, and many districts will do damn near anything.

[hobbit709]

5. Don't let facts get in the way of their corporate agenda.

[proud2BlibKansan]

6. I'm sure they're looking for new facts to support their idiotic ideas.

[End Of The Road]

8. Question for you...

Pardon my ignorance, but can you give me the basics on WHY the corporations want to take over public education? I can't find any logic in it. Moreover, I don't see how it will be profitable for them. It seems obvious to me that corporations want to take over public ed and higher ed, and just as obvious that they will ruin it, but damn if I can figure out the WHY of it.

I love your posts on education, by the way.

[proud2BlibKansan]

9. Madfloridian and Hannah have both connected this to Wall Street profiteers

It's about money. Of course.

[End Of The Road]

10. I'm sure it's about money

But in the long term, I don't see how it can be profitable. Oh well.

[Gabi Hayes]

12. profitability, per se, is just one aspect of what corporate world has in mind for public education:

From 'Share the world's resources'

Part 3 - Corporations, Government and Public Opinion

http://www.stwr.org/multinational-corporations/multinational-corporations-mncs-beyond-the-profit-motive.html#education

''The education system provides arguably the most fertile ground on which to influence public opinion. In the US, corporations are making significant in-roads by sponsoring teaching materials and aggressively marketing and supplying junk foods through vending machines and lunch programs. Of greatest concern are corporate sponsored curriculum modules, public education propaganda videos, and grants and sponsorship programs that refocus education to pro-corporate aspects of law and economics. Competition, economic growth and profitability are emphasized- qualities that secure future corporate opportunity. There is a simultaneous shift away from learning the benefits of cooperation, community endeavor and goodwill. Together such tactics effectively skew public opinion from an early age and further enshrine the neo-liberal, corporate agenda. Unsurprisingly there is a trend in the US, the EU and developing countries for corporations to operate public schools for profit, capitalizing on yet another market opportunity. ''

[Starry Messenger]

19. I'm not sure either.

I don't know anything about money, investing or anything. But there are a lot of Wall Street guys who seem to think this is the golden ticket.

http://edreform.blogspot.com/2007_05_01_archive.html

A money manager recently sent an e-mail to some partners, congratulating them on an investment of $1 million that yielded an estimated $400 million. The reasoning was that $1 million spent on trying to lift a cap on the number of charter schools in New York State yielded a change in the law that will bring $400 million a year in funding to new charter schools.



The money managers who were among the main investors in this law — three Harvard MBAs and a Wharton graduate named Whitney Tilson, Ravenel Boykin Curry IV, Charles Ledley, and John Petry — are moving education-oriented volunteerism beyond championing a single school. They want to shift the political debate by getting the Democratic Party to back innovations such as merit pay for teachers, a longer school day, and charter schools.



The organization from which they hope to launch their revolution, Democrats for Education Reform, does some of its work at cocktail parties hosted in Mr. Curry's Trump Plaza penthouse. The group — actually two separate political action committees — has raised money for senators Obama, Clinton, and Lieberman; Governor Spitzer; Rep. George Miller; state senators Malcolm Smith and Antoine Thompson; assemblymen Sam Hoyt, Hakeem Jeffries, and Jonathan Bing, and City Council Member Vito Lopez. They count the charter cap lift, signed by Mr. Spitzer in April, as their first major victory.

<snip>

Teachers' unions may give a big boost to the Democratic Party, but so do those working in finance. If Democrats for Education Reform can convince them to press issues like length of the school day and merit-based teacher pay, it could force a dramatic swing in the party itself.

<snip>

As investors, the group's leaders spend their days searching for hidden diamonds in the rough: businesses the market has left for dead, but a savvy investor could turn for a profit. A big inner-city school system, Mr. Tilson explained, is kind of like that — the General Motors of the education world. "I see very, very similar dynamics: very large bureaucratic organizations that have become increasingly disconnected from their customers; that are producing an inferior product and losing customers; that are heavily unionized," he said. A successful charter school, on the other hand, is like "Toyota 20 years ago."

Many philanthropists are also getting in on this. Evidently they get hefty tax breaks for funneling their loot towards education in certain ways. I'll try to find a link, I think Hannah Bell has it in her journal.

[End Of The Road]

20. Thanks for this.

I think I need to spend some time reading HannahBell's journal.

That last paragraph really got to me -- I disagree with the "customer service" approach to education. IMHO this type of business model will not work.

[Swamp Rat]

7. k&r

[EFerrari]

11. Oh, there you go with your anti-Obama disinformation again.

How any rational person can look at this program and say with a straight face that it's credible is beyond me. First year econ students know better.

[Catshrink]

13. Did you forget the sarcasm smilie?

Because otherwise your post is unclear.

[EFerrari]

18. Yes, I did. n/t

[Catshrink]

21. I thought so... thanks for clearing that up!

[proud2BlibKansan]

14. I remember when we used to value research in education

It actually guided our work.

Imagine that. LOL

[EFerrari]

17. Another world, I guess. n/t

[LWolf]

15. I wish we could pin this to the top.

I'll bookmark it to link to when necessary.

Thanks for bringing it to us.

[Whisp]

16. my understanding is that these test scores are about individual students progress.

they compete with themselves and any improvement is considered a success rather than a roomful being groomed to get the highest marks possible to compete and leave so much that is necessary in an education behind because they are pushed so hard to 'pass' that they lose out on a lot of other things.

sounds like a most fair way to gauge success.

What are the tests you are talking about?

[Catshrink]

22. Here's one problem I've seen.

Every year I give my students two assessments at the beginning of the school year and again at the end to measure progress. This is my assessment, not the school's, but it's also part of a larger study on the method we use to teach science. (This data consistently shows that this "newer" method is far superior to traditional methods and we have 15 years of data to back up that claim. PM me if you want more information.)

These assessments measure prior knowledge, preconceptions, and reasoning ability. At the beginning of the year, most students seem to make an effort but about a third of them "is this for a grade?" I used to tell them no, that it was part of a study and very important, used to measure me, etc. but found that they wouldn't take it seriously and just made pretty patterns. So I tell them it's for placement to be sure they're in the correct level of course - and that does have validity. This year I had some that were far below and I talked to their parents about my concern. I also had some that were above and convinced one to move up to honors where he is thriving. (BTW, the one of the low scoring kids admitted to his dad that he bs'd it and his dad insisted that he take it again after school. Hooray for parents!)

At the end of the school year, I administer the same assessment and compare results using something called the Hake Gains Test that measures student improvement. The problem is again that many kids randomly fill in the bubbles which skews the results. I've had students fill in A's all the way down the sheet or make zig zag patterns. I've had students who do this to the bottom of the scantron then realize that they've just bubbled in 50 answers when the test only has 30 or 35 questions depending on the test. If it isn't part of their grade, they don't care.

I've had students who did very well on unit tests and final exams, students I know have done well and know content, reasoning, etc., totally bomb these assessment because they filled in the form randomly. In a nutshell, this is a huge reason why many teachers resist having their evaluations and salaries pegged to tests like this. If students took it seriously, it is a measure of how well I did because, by using the Hake Gains, I can see how much a student improved. The problem is that too many blow it off. What do I do about this? I haven't figured that out yet.

[WolverineDG]

23. Error: you can only recommend threads which were started in the past 24 hours

damn. sorry!! but here's a kick from me on behalf of my cousins who left teaching because of the stupid testing. and they were excellent teachers, too, btw.

dg

[xchrom]

24. Kick

[Catshrink]

25. again

[izzybeans]

26. Thanks for posting. Forwarding this report.

From my experiences working with a couple of schools on their data and in business with these types of models, I find many people tend to take them as gospel and not understand the provisional nature of mathematical modeling. They confuse them with science and mistakenly believe science establishes absolute truths. In business it is even worse, educators are less naive about these things. Many of our economic problems can be traced back to the over-realiance on similar "valued added" models related to balance sheets. They turn concrete reality into abstractions, hide how the world actually works, and make it easy for disconnected leaders to make counterproductive decisions, but just because it "clears the ledger" they rake in big bucks and can feign strength to their shareholders.

btw, I say educators are less naive about this, because they at least have the mental capacity to question this from the beginning prior to it being implemented. Statistical modeling has been a shiny distraction for business for some time. It's not that it doesn't have its place, its that it has its proper uses...and it is improper to automate your decisions based on them. You have to have experienced people in the trenches incorporating them as just one bit among many other bits of information necessary to make a decision. Investors are the worst with this. If you have no qualitative understanding of the reality you are measuring, your quant is crap.