Which do you prefer? FL formula or NY formula for grading teachers? Follow the formulas.

[madfloridian]

Students and teachers are being judged by formulas that few understand. Formulas that refuse to acknowledge that students are people, not products. Formulas that refuse to admit that a good teacher has characteristics that go way beyond a graph or chart.

SO...what you do think? Which looks best? One or both or neither?

The Florida formula.



Advanced calculus high school teacher, Orlando Sarduy, writes out the formula that will grade and help determine the pay of Florida teachers. Even for a college math major like him, the formula is too confusing to understand. He calls it a "mathematical experiment."

From State Impact Florida 2012

Inside the Mathematical Equation for Teacher Merit Pay

StateImpact Florida and the Miami Herald partnered up to deconstruct the equation and try to figure out what’s going on here. We asked statisticians and policymakers how the formula works. The answer we got: No lay person, teacher or reporter can understand it. So just trust us.

“I would really challenge any sort of decision maker to look at [the formula] and explain it,” Sarduy said. “I understand just the basics, but this is really the technical nitty-gritty of what’s going on, and to me it looks the same as it would to a lay person, like ‘what’s going on here?”

The infamous formula for New York teachers that often leaves the best teachers out in the cold. No personal factors considered, no consideration given to problems beyond teacher control....just that formula.



A statistical model the school system uses in calculating the effectiveness of teachers.
By MICHAEL WINERIP Published: March 6, 2011

Highly respected NYC teacher placed in bottom 7% by Value Added method of scoring. A real shame.

This is the method that was used to publish the names of teachers in Los Angeles and call them failures or successes. They did not include evaluations and performance reviews even if they were excellent.

From an article by Michael Winerip in the NYT in March. This is the type of farce in evaluating teachers that is going on all over the country. NYC is leading the way.

...This may seem disconnected from reality, but it has real ramifications. Because of her 7th percentile, Ms. Isaacson was told in February that it was virtually certain that she would not be getting tenure this year. “My principal said that given the opportunity, she would advocate for me,” Ms. Isaacson said. “But she said don’t get your hopes up, with a 7th percentile, there wasn’t much she could do.”


The article points out that if the "mayor and governor have their way, and layoffs are no longer based on seniority but instead are based on the city’s formulas that scientifically identify good teachers, Ms. Isaacson is pretty sure she’d be cooked."

That's tragic.

Many are often criticized here for speaking out about the harm both parties are doing to education. A Democratic administration is leading the way in turning our public education tradition into one controlled by private companies who want to profit.

Problem is that it's going to be too late soon for many career teachers, for many special education students.

There is nothing wrong with expressing criticism of political leaders who are destroying a great American tradition for profit.

[madfloridian]

1. More on the complicated FL formula.

Florida's new formula for rating teachers makes Einstein's theory look simple

In Florida, by a formula that makes Einstein's Theory of Relativity look like child's play.

"It looks like hieroglyphics," said Debra Wilhelm, president of the Palm Beach County Classroom Teachers Association.

"Nobody really understands it," said Gary Itzkowitz, a Broward Teacher's Union field representative.

As Florida's new teacher evaluation system based on FCAT results has started to kick in, now might be a good time to let everyone know how crazy it is.

[Recursion]

2. A linear system makes relativity look like child's play?

Seriously? That equation should be clear to anyone who's taken linear algebra.

Why does a linear system of equations look like hieroglyphics to a veteran teacher?

[jsr]

23. A monkey can punch a button to run formulas

There is a hidden agenda behind the assumptions if you read the technical report. Mathematical quantification of social factors is subjective bullshit.

[Recursion]

3. The NY one there is very clear

And also addresses pretty much every complaint about outcome based evaluation (it takes into account student background, class size and makeup, school characteristics, previous achievement, and even district makeup).

Now, maybe the coefficients are bad (and I'm dead sure the district paid some think tank too much for them), but are you really claiming teachers can't be expected to understand linear algebra?

[madfloridian]

5. Which one refers to the child as an individual, and true learning as in-depth?

I am quite sure you were not attempting to make teachers sound ignorant....were you?

Or were you?

What is your explanation for the many teachers with outstanding evaluations who end up say...way down the scale on this?

I had a very good liberal arts education, at two highly rated colleges, well, one a university...just a small one.

I never took linear algebra. Is that a requirement now for a good education? When I looked at the faces of all the little individuals in my classroom...I thought of teaching them as a whole person. In fact linear algebra never entered my mind.

Duncan has done far more to encouraging condescending attitutes toward teachers than the Bush family's ed leaders before him ever did. He appears to be doing it with the tacit agreement of his boss.

That is shameful.

I don't usually respond to insulting posts like this, but I am finding that this administration has encouraged this atmosphere of looking down on teachers. So you are just responding in the tone that Arne Duncan has already set.

[Recursion]

8. Beta prime Chi sub i

Sorry, I can't do math symbols in a title. But, that's the element that treats each child as an individual.

Insulting? I'm not the one claiming teachers can't understand a very basic system of linear equations.

[Are_grits_groceries]

6. I didn't respond to this because

I wasn't sure if you forgot this .

[Recursion]

9. It's not too much to expect teachers to understand linear algebra

That's very, very basic math right there, and even teachers in non-math subjects have to use it when they make quantitative assessments of students.

[Are_grits_groceries]

14. So now teachers are supposed to have a quick and complete grasp of linear algebra or

whatever other subject their evaluations will be based on.

And if somebody doesn't understand it, you have prejudge them and are ready to shame them as being uneducated dolts.

Well, linear algebra doesn't spring readily to my mind. It's not second nature to me as you seem to think it should be and I don't give a rat's ass what you set up as some demarcation for teacher competence.

It's not too much to expect to not be jumped with some artificial formula that some education gawd has decided will quantify everything.

Everybody wants a magic formula that will decide it. There's no such thing. Those different equstions are a Republican's dream. Reduce it to numbers, coefficients, and get the people the hell out of it. If they complain, mock them for not knowing the math or questioning it.

[bobclark86]

16. Umm... yeah. We do expect someone who has a bachelor's or master's to know basic math.

All of this is spelled out in crayon in high school for New York students (For decades, it was part of the Sequence 1 and Sequence 2 classes taught to 8th/9th/10th grade students), and show what the hell to really do with it in dedicated Statistics 101 classes for first-year teaching students... or journalism students... or loads of other degree programs.

I suck at math. But I need it for work, so I keep a book on statistics at my desk at all times. It ain't hard and it's damn handy to figure out. Because guess what? As any good math or science teacher can tell you, one can break EVERYTHING in the universe down to math. From reading level of a book to building a nuclear bomb. All math. It's quite empowering to students to crack the mystical code controlling everything.

That's something the best math teacher I ever had taught me. I wish the ones after him were half as good, instead of just coasting to retirement.

And math is a hell of a lot better way of doing teacher evaluations than a principal saying "You're fine, because if I say you aren't, I'll get fired, too" under the system of qualitative assessments.

"I don't give a rat's ass what you set up as some demarcation for teacher competence."

That's nice. The benchmark is the worst of the existing teachers. If all teachers are awesome, then yeah, an awesome teacher is going to rank at the "you suck" level. Fortunately for good teachers, there's pretty much always a handful of crappy teachers who couldn't write a lesson plan to get themselves out of a paper bag.

BTW, a Republican's dream is shouting "It's God's will!" and firing gay teachers. They hate math because it's what atheists use to "prove" Earth is 4 billion years old and humans came from lower apes.

[dsc]

10. Unless we know where the coefficients come from then we don't understand the equation

I teach and tutor statistics so I do know something about the equations that are in these formulas. I also know that the coefficients, even when the input is good, are just estimates of what the true numbers are. I don't think it is a wise idea to use a formula like this to judge teachers. As some of the input, fine, but as the entire judgment, no. I also would like to know where the coefficients came from, what data did they use to get them. How often are they updated? When does a new test start counting for this purpose? None of those questions are answered in those articles and without those answers no one can truly understand those equations.

[Recursion]

11. The coefficients could be awful for all I know; I'm irritated by the claim that "math is hard"

The coefficients could be so awful that it would be better to flip a coin (and like I said I guarantee you the state paid too much for them). I'm just troubled that veteran teachers (including a math teacher, apparently) are being quoted as saying this is some kind of higher order math ordinary mortals can't understand, when anybody who's had a probability, stats, or linear algebra class (and I'd hope pretty much every teacher has) should be able to understand the form of the system from looking at it.

[LWolf]

62. No equation takes in student background.

It can take in one factor, or more than one factor. It can't take in what has not been quantified, though, and most of a child's background has not.

The writers of these formulas are quick to explain that when you talk to them face to face. They are also quick to defend the use of those formulas for high-stakes outcomes with the tired old "We only focus on what we CAN..." that has been used to such harm for many decades.

That was my experience when meeting with them, anyway. It was 2 or 3 years back; at the time, there were only a few in the entire nation writing such formulas for each state. They also metaphorically and paternalistically patted us on the head and said that the formulas were to complex for any but professional statisticians to understand.

I'm sure the formulas are evolving as the high-stakes policies continue to kick into higher gears.

[Recursion]

63. So it doesn't matter who teaches?

There's really no way to predict what teachers will have better outcomes with "difficult" students than others? Why bother, then?

[LWolf]

70. What we have known for decades BEFORE the high-stakes testing movement:

Teachers may be the biggest IN SCHOOL factor affecting student "achievement" in the form of standardized test scores, given reasonable working conditions and support system, which is NOT a given.

Outside factors, primarily parent SES, are a bigger factor than anything that happens in school, and any teacher.

[Recursion]

71. OK, so here are formulas that account for that.

They're right there in the OP.

[LWolf]

72. Please enlighten me.

Where in the formula does it account for whether or not a kid ate that day, or the day before, whether or not the kid was screamed at or hit, or molested by a neighbor, or too busy babysitting to do homework, or negotiating between vicious parental fights, or had a safe place to sleep, or got enough sleep, or any of the other things way too many of my students, of OUR students across the nation, deal with?

It doesn't. It looks at free and reduced lunch as THE factor for "poverty." It doesn't account for the difference between someone whose working poor family is stable or not, or has a greater infrastructure behind them in the form of extended family. It doesn't account for the reasons why some students are always behind; reasons that started in utero and have nothing to do with the school they attend or the people working at that school to serve them.

[liberal_at_heart]

4. K&R

[madfloridian]

7. .....

Thanks, speaking up for public schools and teachers now is almost useless. The propaganda has flourished to the point that teachers are considered lowest in capability and intelligence.

Congrats to Arne, he has been very convincing and is winning the war against public education.

[msanthrope]

12. I gotta agree with Recursion upthread...you were a veteran teacher, but you

don't recognize linear algebra?

[MattBaggins]

13. I'm willing to bet neither you nor recursion understand it either

Last edited Mon Feb 17, 2014, 01:38 PM - Edit history (1)

Just spouting out linear algebra.

[Recursion]

19. You honestly don't understand systems of linear equations?

Hell, this used to be high school stuff...

[AleksS]

33. Systems of linear equations are still high School stuff however

Systems of linear equations are still high School stuff. However, these are presented in such a way that to most folks with a liberal arts degree they'll look daunting. If someone puts some time and effort into thinking about them, the equations are not that complicated, especially if the terms are explained/defined. But most folks, (I'd wager almost all) when they see those equations, filled with greek characters, and subscripts, and mathematical operations they've rarely seen or used (if ever), are going to be pretty intimidated.

Now the article does a disservice using hyperbolic language like "makes relativity look like child's play" (paraphrased), and certainly hams up the opaquity of the equations, but I think you're going a bit far the other way on this.

Yes, the equations presented aren't difficult when you dig into them.
BUT
Yes, they are being presented in a way that most definitely will be intimidating to the vast majority of (even college educated) readers.

[msanthrope]

21. You think a poster who has named himself "Recursion" doesn't know linear algebra?

That's funny, and indicative of the fact that you don't actually know algebra.

As for me, I understand what I am looking at, and its purpose. Could I work it out? Sure...and I bet you can, too.

[roody]

39. I never took linear algebra. I have a M.A.

I took one year of calculus. I guess I'm not qualified to teach my first grade class.

[madfloridian]

44. .....

[Recursion]

53. You didn't take stats in the entire course of getting your MA?

OK, then yes, you have every reason not to understand the equation. I think teachers probably should know stats, personally, just like they should know English grammar pretty well even if they are teaching math. No, it's not particularly a requirement for the teaching itself (though I guess it limits the data analysis you can do on your students' performance -- it can be useful to know if the students' performance on something is normal, bimodal, whatever...), but it's very useful for dealing with the modern world. As this demonstrates.

[roody]

65. I can tell if they spelled a word correctly

or not. I didn't look at the equation. I love math. I loved teaching fifth grade math.

[Recursion]

66. So if you love math, learn linear algebra

It's really wonderful.

[roody]

68. I'm sure it is. Teaching first grade takes a lot of time.

Of course I and all the teachers at my school suck because our kids overall do not test high enough.

[jsr]

15. Looks like mathematical masturbation

[Donald Ian Rankin]

18. No, it's very simple, very dull, rather practical maths.

No mathematician would get excited over this sort of thing, I'm afraid, but it is useful.

Take a look at http://www.imo-official.org/ if you want to see some proper mathematical self-pleasuring.

[jsr]

24. Euclidean norm minimization was not meant for the fuzzy sciences

[Recursion]

20. It's the most basic ****ing mathematical tool in the whole world

First-order linear equation systems, for God's sake. We're not talking about differential equations or vector calculus or anything here. You don't really even need linear algebra or matrix manipulation to solve them, just ordinary algebra (though it takes longer).

The attached graphic is incredibly clear:

The post-test is the sum of:

The teacher's contribution

The pretest

The student's aptitude

The influence of the classroom

The influence of the school

The influence of the district

and a linearly-minimized error term (ie, the variance of a student's performance on the test is generally known, so the model can't get more accurate than the square root of that).

This AFAIK answers all the complaints people make about outcomes-based assessments, and takes into account all of the things that people say matter. The questionable bit is the actual value of the participation indicators, "I sub whatever", that weight the sums, but no doubt the state overpaid some think tank to come up with them based on some study in the journal of whatever.

I am very troubled that working educators claim this equation system is mystifying. What it is saying is completely clear. You may think it's a bad model (the actual values of the participation indicators are my big issue with it) but saying "this is too hard to understand" doesn't exactly make teachers look very good. This is something any college graduate who took a math class at some point (and that should be all of them) should be able to understand.

[Squinch]

31. The formula isn't hard. But the individual variables are not quantifiable characteristics.

[Recursion]

38. Anything is quantifiable, the question is how meaningfully

That said, there are a lot of skills and knowledge sets taught, particularly at the primary level, that really are amenable to quantifiable measurement.

Obviously, grade level and subject matter here: by high school, in many subjects, one would expect students to be able to start performing syntheses, which are notoriously difficult to measure quantitatively. But in the years before that, the student must master the elements of the synthesis she is to perform. So, taking history as an example, one might expect a high school upperclassman to perform a synthesis on, say, the European Age of Discovery: this will be basically impossible to reliably quantify. However, in the years leading up to that (which is where most students and most teachers are at any given time), the student will be needing to master the underlying historical facts: all the names, dates, countries, etc. that are quite easily testable. (And of course in math and science even fairly complex syntheses are pretty easily testable.)

This really does seem like the "right" way to do things. The teacher is judged per student based on the student's performance on the final test, minus:

Their performance on the pre-test

A model of their aptitude (ESL students, free lunch students, etc. are "worth more points" to the educator)

A system modeling how the class, school, and district interact and influence performance (already high-performing classes, schools, and districts are "worth fewer points&quot

Isn't this taking into account exactly the sorts of things educators keep complaining that outcome-based assessments don't take into account?

It's like the evaluators are damned if they do, damned if they don't. If they don't take these environmental factors into account, they're ignoring them. If they do, then they're criticized for producing evaluation equations that are "too complex".

The "system modeling how the class, school, and district interact and influence performance" isn't explicit in the formula (we don't know what the coefficients are, though there are very simple and well-known ways to construct them empirically given past datasets) --- and for that matter we don't really know that they interact in a linear fashion (but we don't really know that about most things even in the non-fuzzy sciences, but use linear systems anyways because even non-linear systems are generally linearizable over the short term).

This really just sounds suspiciously like teachers' adamantly refusing to be judged based on student outcomes in any way whatsoever. But, as I keep coming back to, if there's not a reliable way to say teachers impact student outcomes, you can't say it matters who teaches.

[madfloridian]

46. "adamantly refusing to be judged based on student outcomes in any way whatsoever"

Not true.

Teachers want students tested on what they have covered. The high stakes tests do not do that....only God and Pearson know what is going to be on it.

Teachers want to teach their students on a curriculum that is known. All this testing practice is concentrating on unknown stuff.

BTW you really need to learn how to disagree without trying to make everyone else sound stupid.

That formula has no relation whatever to anything that goes on in the classrooms of grades 1-6. I won't speak for high school, never taught there.

But I know how to present my side without being snide. Calling teachers ignorant is not necessary.

[Donald Ian Rankin]

54. I think unknown tests are a good idea, actually.

One of the big arguments against testing is that it incentivises teachers to "teach to the test", and neglect anything not in it.

If I were running an education system and wanted to monitor pupil and/or teacher performance, I'd administer a half-hour machine-markable multiple choice test once a week, which could contain anything from "which of these years did man first walk on the moon" to "which is the correctly-spelled word" to "what is the cosine of 45'" to test knowledge, and a two-hour essay on a choice of three closed questions, marked with an eye to "can you right coherently and convincingly" rather than to background knowledge once a month to test writing & thinking skills, and perhaps a monthly maths paper.

Teachers would get data on where their class was by centile in questions on each subject separately on the grounds that it's not much use to a teacher to know "you/they are great" or "you/they suck", but it probably *is* useful to be able to see that a class that are 60th centile in maths are only 40th centile in history, and hence that's where you need to be spending more time.

And, paradoxically, I think that more tests would probably actually make tests less of a burden, because the stakes in each are lower; unpreparable-for tests would hopefully likewise dominate less than things you can cram for.

[DebJ]

50. Quantify this: the impact of the student's home life on the student's

educational achievements.

Please show me the factors for:
- no food at home
- parent or guardian is an alcoholic or drug addict
- student allowed to stay up all night playing computer games or watching tv
-students 'regular' bedtime is midnight or past
-adults at home engage in regular fighting and even physical violence
-adults at home place no value on education and view it as a babysitting service
-adults think that education is just a matter of flipping up the top of a child's head and pouring it it....ie, no
effort is required on the part of the child
-adults think it is find for Suzy to text all during class


On Edit: I'm 58, and even way back when I was in elementary school, I repeatedly heard that
the single most important influence on a child's education is the parents. Still as true today
as ever. But too many parents refuse to accept this. They are in 'consumer' mode: they drop
the child off at school and expect to pick him up having gained 'what they paid for', with no
effort from themselves. This isn't ALL parents, but this is what inner city classrooms are chock full of.

[Recursion]

52. Well, again, that's in the term Beta prime Chi sub i

And the impact is also smoothed over the classroom, school, and district (a nutritionally insecure student in a nutritionally secure district does better than a similarly nutritionally insecure student in a nutritionally insecure district; Ta-Nehisi Coates just did an excellent piece on this which I now can't find).

How you find those is that you look at students in similar situations in the past and see how it actually has affected their performance. Did NY do a good job of actually performing that analysis? No idea -- that process should be more transparent than it is, and less expensive (hell, if they want to give me anonymized historical testing data with economic crosstabs I'll be happy to do it for free).

Over large populations (like, say, three years of five classes a day) the "noise" (I'm not dismissing it; that's a math term here -- there's nothing "mere" about noise) of different students' home lives will wash out to the overall composition of the district (with exceptions based on subjects; Latin teachers will have richer students than history teachers overall). We know from the past how students in various situations have done on tests, averaged over all of their teachers. This allows administrators to look at what a given teacher's contribution -- again, over a large population -- is to that average.

[Squinch]

61. Only that has nothing to do with how those variables are being defined.

You are assuming, because the formula is pretty, that all the variables are correctly and logically defined. This simply isn't the case. We can see this because teachers who routinely bring lower-ability students measurably farther than other teachers are often found to be unsatisfactory.

As I have said before, this is not about teachers refusing to be judged. This is about teachers wanting to be able to teach.

This is also about respecting AND TEACHING those children who have "student aptitude" that is not of a magnitude that can move the formula. That is not happening under this formula.

[knitter4democracy]

42. That's anything but clear.

How do we know what the teacher's contribution to the post-test is? Which test is being used, a final exam the teacher wrote or the NWEA MAP test, like in my district? For those of us who don't teach English or math, the reading test on the MAP is used--which could be 4 of the 6 teachers any student has in any given trimester, so which of the four had the contribution? All four? How much? How can we know?

How do we know the student's aptitude? Do we use an IQ test? Those are fallible, as are pretty much all other aptitude tests. Where do we get that measurement from, and how do we know that it's accurate?

How on earth can we quantify the influence of the classroom when it's a dynamic place that changes on a daily basis depending on who's there, what we're doing, the seating arrangements, how close the next big social event is, etc.? Is there an equation for this, and is someone gathering the data and figuring out that equation prior to working this out?

How can we quantify the influence of the school and the district when there are just too darn many variables?

It's not that the equation itself is mystifying. It's that what goes into it is.

[Donald Ian Rankin]

17. As has been pointed out upthread, it looks like quite a simple idea, presented obfuscatorily.

Practically any idea can be made to look incomprehensible if it's presented by someone who has a vested interest in it confusing the reader and turning them against it.

Because it's been deliberately obfuscated by just showing one step in the middle of a process, I'm having to guess at various details that aren't specified, but here's what I assume they're doing:

You have bunch of data points - test scores of various children in various schools and districts under various teachers at various times.

You hypothesise that every child's score at every time can be approximated by some function of a particular form in a bunch of unknowns - things like "how much difference does district X/school Y/teacher Z make".

You then use your choice of statistical techniques - probably linear regression - to pick the values of the variables that make your approximation best fit the data.

If you're ranking teachers according to pupil test scores, this looks like a good way of doing it.

By all means argue that things other than test scores should be taken into account. But this "look, maths, aargh!" bilge is just dumb: the maths is the right way of doing it, and if presented properly isn't even terribly confusing.

[madfloridian]

37. Still seems to be all test based. Is there an allowance for daily classroom performance...

based on teacher grades. You know how teachers keep grade books? Teachers give tests on work done and supposedly learned? Are those considered? Or it is all the one high-stake test?

Looks like teachers, students, schools are graded by formula now with little else considered.

[eppur_se_muova]

22. To express a preference, you need to assume the data are meaningful ...

otherwise, it's GIGO. Or at least the error bars are huge.

[reformist2]

25. Teachers are going to start treating at-risk students like hot potatoes.

[madfloridian]

26. Now that education is reduced to a formula, they will do that to survive.

That's a tragedy. In fact charter schools are getting away with excluding them right now. The test and the formula are now our education gods.

[reformist2]

29. Yep. Charter schools engage in the worst kind of cherry picking, and then don't even do that well!

[Squinch]

32. This is already happening. In every classroom, in a very complete way.

[reformist2]

35. Who's getting the at-risk kids?

[madfloridian]

36. The parents of these kids are wondering the same thing.

It's a very sad situation. Most end up back in public schools. I am not sure which districts mainstream some or all disabilities. There are some special schools but they are expensive.

[Squinch]

60. They are in the classrooms.

See them? Over there, on the computer, while the teacher concentrates on the kids who have a hope of passing the test.

And don't blame the teacher. If s/he doesn't get a certain percentage of her/his students to a certain score on the test, s/he gets an unsatisfactory. 3 "U's" and s/he is fired.

[Recursion]

43. They're "worth more" under this assessment

A student who did poorly last year, and meets certain "aptitude" requirements (IIRC ESL, free lunch, etc.) amplifies any improvement over the pre-test remarkably.

In fact, when a story on this was published last year, the complaint was the exact opposite: a teacher who was teaching a "good" class (rich native English speakers who don't get free school lunches) was rated badly by it despite being observationally identified as a good teacher.

[reformist2]

48. Yeah, *if* the at-risk students rebound, and that's a big if. Most teachers won't want to gamble.

[Recursion]

51. Well, I'd like to "know" that I won't be blamed for network outages, too...

How much of my uptime metrics are really my doing? I don't know; maybe half? That's one reason you should do this over a long time series (and NY at least looks at a few years of testing at a time).

But if I can't reliably keep servers up, then I don't expect I'll stay a sysadmin long. And if nobody can, then it's hard for me to say that it matters who is the sysadmin.

[LWolf]

27. Neither.

Evaluate me based on what I do. Evaluate my students based on what they do.

Assume that they are unique human individuals with a range of abilities and free will to apply themselves, and treat them like it. They aren't inert clay that I am shaping, carving, and firing. They aren't empty, passive vessels that I am filling.

[madfloridian]

28. .....

[WinkyDink]

30. This started when Supers wanted to be "CEO's". Then, of course, students became "products" of the

"corporation."

This analogy made teachers into interchangeable assemblers, all teaching the same approved lessons at the same pace, giving the same standardized tests (from for-profit companies).

The entire, and I do mean entire, ulterior motive is to ruin public education for the massive amounts of money to be given to for-profit "academies."

[madfloridian]

34. Yes...students are becoming "products" judged by formulas. Motive is profit.

You are so right.

[adirondacker]

40. Nothing but a scam and a joke that replicates our "complex" financial system. nt

[roody]

41. If you want to evaluate a teacher,

go into his or her room. Watch them teach. Look at kids' work. It is not hard to know if a teacher is good or not.

[Recursion]

45. What if you watch them teach, conclude they're a good teacher, and the kids don't actually learn?

And if you can't make a connection between better teaching and student improvement on these tests, it's hard to argue that it matters who's teaching.

Look, I realize DC public schools are screwed six ways to Sunday through no fault of the teachers, but it's the district I know. The teachers there had a model of how they were "supposed to" teach, taught to it admirably, were observed teaching to it admirably and rated "excellent" 97% of the time (what exactly is "excellent", in that case?) and absolutely failing to actually improve student performance. All told DC was spending more per student than Arlington County across the river and getting much worse results despite having fewer ESL students per capita (though much higher poverty -- also, the two districts accounted for spending per student differently, and DC had absurdly high transportation requirements for special needs students, so there's not really an apples to apples; DCPS really is swimming in money, though).

If your complaint is that a quantitative test score cannot measure educational progress, then I'm much more concerned about how those test scores impact students' lives than teachers', frankly. Over the past two decades, graduation rates, literacy rates, standardized test scores, college admissions, college graduations, and functional literacy assessments are all up, and are up for every ethnic and racial group except Native Americans/Aleutians (we need to do something about that, quickly, incidentally). African American students today perform better academically than white students did 25 years ago.

This is one of those inconvenient facts like the precipitous drop in violence over the same period (and for that matter probably has the same exogenous cause, whatever that may be -- lead abatement, Roe v. Wade, etc.) that removes the sense of crisis both "reformers" and antireformers seem to want to cultivate. Schools are not in crisis; they're doing well and getting better, and the idea that the past decade of reforms has damaged education simply isn't borne out by the actual performance of students. There are districts with problems, which we need to fix. Our industrial model of transporting children to a big central concrete box where they spend 8 hours a day sitting at a desk writing with pencils will probably seem hopelessly quaint in a generation or two, but it's what we have now and it's not broken. But finding the places where it isn't currently serving actual students' actual needs is a good thing, and ignoring the actual outputs here is just as silly as the "reform" caricature of ignoring all inputs.

[madfloridian]

47. Teachers are evaluated by just about everyone.

You can trust me. It's not just the principals. It's people from the district, from the county, it's parents...trust me parents really really evaluate.

What do you think about the fact that teacher grades, teacher-made tests on material covered, gradebooks on which teachers spend hours.......no longer count. Only the TEST with unknown material counts.

[Recursion]

49. Like I said above, that mismatch is stupid, wasteful, and unfair

It's like how the nation has both a minimum wage and a poverty line, and the two have nothing to do with one another.

But the problem there is not the concept of testing as part of teacher evaluation or with the model used to calculate its contribution; it's with the fact that we have national testing standards and no national curriculum, which is of course idiotic (and, again, an argument for a national curriculum, though try getting that past GOP state legislatures... a lot of them say they're for it now, but that rubber hasn't hit the road yet -- eg, no high school science assessment that does not include evolution through natural selection or human impact on climate could ever be taken seriously on a national level).

It's not just the principals. It's people from the district, from the county, it's parents

If you want to double, triple, quadruple, whatever teachers' pay and school facilities funding, sign me up. If you want to provide a buffer between teachers and pushy parents who are absolutely certain their precious Trevor or Caitlyn couldn't possibly be acting out in class, I'm completely with you (in fact, parent ombudsman positions are starting to become more common, from my anecdotal experience). If you want to say principals and districts should back off and let teachers do the damn jobs they're trained for, I'll shout that from the rooftops (that, incidentally, is exactly the reason most of my friends who teach at charters give for that decision). If you want to say that PEPs, while a good idea in principle, have become a monstrosity and a completely unfair burden to classroom teachers, I've been saying that for years too. And most importantly if you want to say that a lot of "education reform" is fly-by-night scam artists who take the money and run, I can point you to three "charter schools" in my old neighborhood in DC that literally took the money and never opened and say I completely agree. But I'll also say that the "five year winnowing" of early teacher attrition (I forget who called it that, but it's a good name) can't be looked at as the last word on who should and shouldn't be a teacher; that in the current "pendulum", pedagogy is overemphasized with respect to subject mastery (though that swing back has started in the past decade or so); that outcomes absolutely should have a place in teacher evaluation (with significant consideration to inputs); and that, just bluntly, districts need a way to remove teachers who aren't doing as good a job as they used to. I won't even couch that in terms of "help them improve", because I don't believe that's what "remediation" is really about any more than you do. Hell, maybe we need NYC's rubber rooms everywhere; if it costs a lot, fine (I'm for stimulus spending, after all).

Many factors lead to students' having academic problems. Only a few of those (school facilities, security, and teachers) are under district control and paid for with public money. So the simple fact is those are the places where districts can take action.

[Sancho]

56. If you really want to get into the details of value added formulas...

First, you are correct that the "public version" is a linear equation....but

-it's created using structural equation modeling (SEM) and the authors of the model usually sell the formula to the states and will not reveal the exact way they determine each component. The authors purposefully produce confusing and simplified models for the public. The VA authors sell a proprietary formula.

-many experimental studies show that the value-added "score" generated by the formulas are unstable at the teacher level. That means it's arbitrary or situational (not reliable over time). This is usually related to the fact that pre-post testing (gain scores) have reliability issues that are exponential (not additive) so that you can't depend on results. If the pre-post is over five year (five testings), the lack of reliability (error) exceeds any score value of interest (for a given teacher). That notion plays out in anecdotal evidence when the teacher of the year gets a low VA score, etc. Also, different VA formulas produce wildly different values for the same teacher over the same time period, so again, there is no "formula" that seems to work at this time.

-other experimental studies question the validity of the models for some components of the model. One of the most common validity issues is the high-stakes test itself. Yes, kids can be measured precisely, but it would take 3 days of testing with an individual psychologist to get accuracy useful for these VA formulas, which would be cost prohibitive.

-teachers who take a couple algebra courses do not see anything beyond a quadratic, so the VA formulas are not the linear algebra that they would have likely studied in college classes. If you were a math major or science major or engineer, you might have seen similar indexed equations, but even then (as you pointed out) the devil is in the details! SEM would be typical of graduate students in statistics. Just like relativity, the basic equation belies the complexity.

-believe me, everyone (even smart college students) don't find linear algebra "simple". Of course, there are engineers who can't write a coherent sentence, etc. My wife (5 degrees) teaches music and struggled with college math. She doesn't understand why everyone can't sing on pitch or simply remember every tune the first time they heard it!! In other words, be careful not to be condescending when you say everyone should understand algebra. I've taught middle school math, college algebra, and graduate statistics. No matter the level, students vary widely in their ability to catch on to basic math concepts.

[Recursion]

57. I'll grant those points; I'll also complain that I see no feedback diagram here

The whole point should be about feedback, right? But that's nonexistent here.

Again, I'm complaining mostly that a veteran math teacher complained to the press that this was "too complex", not that this was "wrong".

[Sancho]

59. A few states have feedback or remediation based on VA scores, but it's a nightmare...

the VA scores don't diagnose what happened well enough to be helpful, so the schools go back to the same observations and professional improvement plans that they've always used anyway. Even when the teach completes the plan, it doesn't always show up in the VA score.

I don't know if the math teacher's complaint is on the algebra formula alone. I've seen DOE officials unable to explain their own VA formula. In fact, I was at a conference with the author of one of the most widely used formulas, and he couldn't explain to the audience of Ph.D.'s how he determined some of the coefficients! For example, we know that one predictor of good teachers is the "value and belief" of the teacher (called dispositions in the INTASC principles). These VA formulas never assess or include those values - but virtually all teacher education programs and clinical observers rely on those constructs! The VA author could not explain why that component was missing. BTW, when that component in included in his formula it rises to the top of predictors!

I suspect that the math teacher is caught in a world of mis-information, confusing targets, and changing formulas, but I wasn't there.

I think we all wish that teachers were more numerate. If I was math competent, and 22 years old; why teach for half the salary? Even when I started teaching in the mid-70's, my first contract was $8900 a year. My first year I was offered $20,000 a year to work for Shell Oil. I didn't take the job, but it's still true today for STEM graduates!

[Donald Ian Rankin]

55. Teacher-made class-specific tests are useless for inter-class or interteacher comparison.

They're a valuable way of comparing two pupils in the same class.

But, obviously, they're 100% useless for comparing either pupils in different classes who've taken different tests, or different teachers.

[madfloridian]

64. I am saying they don't count at all.

So an honor student is not really an honor student unless they are compared with those outside the class and school?

Every school is different. My last school before I retired could in no way compete with the schools in the elite part of town. It was in a neighborhood of drug dealers and worse.

They were learning, we were good teachers. But there was little parental support, a principal who cared nothing about kids or teachers....only their own career.

There were few textbooks. Science texts were over 25 years old. The elite schools had all new books, a set for school and a set for home.

Until they equalize the experience of learning....the damn formulas are useless, actually harmful. And a teacher's career is decided on them.

[Donald Ian Rankin]

67. No, these formulas specifically account for that.

There *is* a strong argument against formulas like this: all they can measure is how good a teacher is at getting students to pass tests, and they'll encourage teachers to neglect anything except test scores.

But *if* you view the job of a teacher as being simply to get the best possible test scores for their pupils, then grading them with formula like these (I can't vouch for the details, because I haven't seen them) is a moderately good and fair way of doing it - there are (or at least easily can be, and I think are) terms in the formula to account for things like differences between schools and pupil intake, and to ensure that what you're grading is value added by an individual rather than just having bright students or being in a good school.

[Recursion]

69. That's exactly what that formula is for

It's a way of comparing how the student actually performed on the test to how you would "expect" a similar student in a similar school to perform.

So an honor student is not really an honor student unless they are compared with those outside the class and school?

College admissions departments certainly don't think so; they have ways of translating high school GPA + high school characteristics to how a kid can be expected to perform in college.

[Sancho]

73. ...and you have identified the problem!!

We know that the amount of variance in many "school-wide" test scores is larger for demographic changes than due to the teacher. In other words, every year the immigration policy changes, a highway is built, a factory closes, etc...and those things change school scores MORE than what the teachers do!!

For that reason, most school scores over five years are not stable. Meanwhile, you can't "follow the child" because many schools have 20 to 50% turnover every year.

In other words, Madfloridian is correct. The only stable scores where the SES and students are constant are in the high functioning suburban schools!!

Colleges don't just use HS GPA and SAT's. They have complex formulas and tracking based on their own GPA computation over specific courses, evaluation of writing submitted, sometimes interviews, and economic factors. Of course, the admissions processes vary widely, but you'd be surprised at how sophisticated some schools track the predicted success of students. And even if they don't admit it, many schools target a mix of race, gender, and origin so that they can have a diverse student body if that's part of the university's mission. Since a lot of students are homogeneous that apply to a given school, and even state universities often admit about a third of applicants - there's plenty of room for admissions offices to target the profile they want. In fact, many schools predict the likelihood the family will become donators to the campus too.

[Starry Messenger]

58. k&r