Study:Teach For America Teachers Outperform Their Peers

[YBR31]

A big new study (PDF) from the U.S. Department of Education's Institute of Education Sciences performed by Mathematica Policy Research confirms that Teach For America teachers outperform peer teachers in hard-to-staff schools and also indicates that teachers recruiting through The New Teacher Project's Teaching Fellows program are about the same as peer teachers recruited through traditional certification methods.
[link:http://www.slate.com/blogs/moneybox/2013/09/10/mathematica_study_of_tfa_and_teaching_fellows.html|
http://www.slate.com/blogs/moneybox/2013/09/10/mathematica_study_of_tfa_and_teaching_fellows.html
The study found:
"TFA teachers were more effective than comparison teachers from both TC and less selective AC routes. Students of TFA teachers outperformed students of teachers from TC routes by 0.06 standard deviations and outperformed students of teachers from less selective AC routes by 0.09 standard deviations.

Novice TFA teachers were more effective than both novice and experienced comparison teachers. Students of novice TFA teachers—those in their first three years of teaching—outscored students of novice comparison teachers by 0.08 standard deviations. Students of novice TFA teachers also outperformed students of more experienced comparison teachers (those with more than three years of experience) by 0.07 standard deviations. This latter finding is particularly important given the fact that TFA requires its teachers to make only a two-year commitment to teaching.

TFA teachers were more effective than comparison teachers in both middle and high schools. Students of TFA teachers outscored those of comparison teachers by 0.06 standard deviations in middle school and 0.13 standard deviations in high school."

[winterpark]

1. I find that very hard to believe.. n/t

[kelliekat44]

35. TFA teachers probably have more heart in their work. nt

[Are_grits_groceries]

53. You have got to be kidding.

Some teachers become jaded, but even the most worn out still care about the kids.
I know several who helped them personally with clothes, money, etc.

There are some teachers beyond helping anybody, but they are in the minority.

[winterpark]

76. I'm saying it's hard to believe a TFA teacher is more effective than a regular one n/t

[Are_grits_groceries]

79. I was answering the post that said TFA teachers have more heart.

Your post made a lot of sense to me.

[roody]

106. Yeah that is why they studied to

become teachers! For six weeks that is. Oops, five weeks.

[YvonneCa]

2. EFFECTIVENESS is not yet defined. n/t

[YBR31]

4. The study defined effectiveness for its purposes

You can look at the study methodology. They studied math teachers. Gave kids math tests in the beginning of the school year and end of school year. Seems pretty straight forward. Either kids knew the math or didn't.

[joeybee12]

6. So they tested the kids, right?

And we know how TFA doctors the results, so we know this is BS.

[YBR31]

8. The study was conducted by Mathematica Policy Research.

I looked them up. Seems like a good, solid firm, from what I can tell. Check them out.
http://www.mathematica-mpr.com/About_Us/company_history.asp

[YvonneCa]

13. You are pretty naive on this subject. Determining teacher effectiveness...

...by student test performance is not a fair way to measure teacher effectiveness. There is much debate on this issue, by educators and non-educators.

[YBR31]

16. Well either the kids learned the material or didn't

With math, you can measure it to some degree. Can't do the same with writing & reading or critical thinking. I'm sure that's why they studied math.

[dionysus]

78. there's too many outside factors influencing whether or not kids can or will learn the material.

[Recursion]

90. So why does it matter who teaches, in that case? (nt)

[Recursion]

89. Then why does it matter who teaches?

Seriously.

If testable student performance isn't reliably improved more by some teachers than others, who cares who teaches our children?

[ananda]

3. My bs meter is off the chart.

The teacher as cheap wageslave mentality means better reviews for
any system that puts those kinds of teachers in place.

[YBR31]

7. This seems empirical.

They looked at math. It looks like its an objective not subjective measure. I imagine that's why they chose to study math since math is objective. Tested kids in beginning of year and at end of year. Either the kids knew the material or they didn't.

[YvonneCa]

14. So...

...were the students from a wide range of schools, public and charter, hi-poverty areas and low...etc.?

[YBR31]

17. It's all in the study.

They matched on free/reduced lunch etc.

[HereSince1628]

73. But there is little about numbers that mean anything without interpretation

and that interpretation seems at question.

What is going on is a two-sided comparison, what's being sought is an understanding of difference in effectiveness of two approaches which are quite complex. From the presentation we don't know that comparisons were made between matched pairs.

What we are presented with are two numbers representing small fractions of standard deviations of difference. The statistical meaning of these differences are influenced by things which effect sensitivity and reliability.

It is suggested that differences of .05 std deviation are meaningful. I suggest that may be quite open to question. It's pretty common to look in old-timey statistics for a probability of .05 of the two measures being the same (95% likely to be different).

That probability is often related to a difference of 1.96 standard deviations in the underlying measure of the difference. A difference of .09 std deviations isn't much and unless there is terrific sensitivity it's likely meaningless in terms of educational meaning.


And THAT is the crux of the problem. Not that there is a number value placed on the difference, but that the measured difference actually means something valuable.

I suspect what is really being shown in these small fractions of difference in std deviation is that there is difference but it is not meaningful educational difference between the two groups.

[hobbit709]

60. Here you go.

[proud2BlibKansan]

63. A+

[Smarmie Doofus]

5. Funding? n/t

[U4ikLefty]

20. No Answer?

Hmmmm

[Recursion]

40. Mathmatica. The people who make MATLAB

Funding, incidentally, doesn't actually change statistical significance, but in case you were wondering.

[BadgerKid]

61. MathWorks makes MATLAB. Wolfram makes Mathematica.

There have been efforts to make STEM education more computational, so of course these companies have an interest.

[Recursion]

70. Thank you

Wrong computational software package

[caraher]

9. Even at face value this is a somewhat sterile result

For the sake of argument, let's just accept that TFA math teachers are more effective. The study does nothing to explain why this might be so. In particular, because TFA is a selective program, one cannot conclude that traditional teacher training and certification programs are ineffective because there are two variables, the TFA preparation vs. conventional or other alternative training, and the TFA selection process vs. admission to other paths to teaching. Might TFA teachers have been even more effective had they undergone conventional training? Would expanding TFA result in a "regression to the mean" in TFA math teacher performance?

I teach at a university that contributes a lot of students to TFA, and getting into TFA is treated as a huge honor. The program draws from a different population - it's the ambitious, competitive and academically-successful student who applies to TFA (to the point where a cynical student paper piece dubbed the program "Teach for your Resume&quot .

I'd venture that if you looked the teaching effectiveness of math teachers as a function of their college GPA you'd get a similar story.

[Nevernose]

28. A huge honor that most quit after two years

I really liked every TFA teacher I worked with, but all but one quit after two or occasionally three years. Even the ones who didn't have promising careers elsewhere said that they couldn't teach inner city kids as a long term career.

Literally every single one of them I met -- a couple of dozen, probably -- got into TFA because they thought that public school teachers were failing urban children. And literally every single one of them has apologized to me. Most people don't realize how many challenges there are, and how teaching at an urban school can be incredibly psychically draining.

[Pholus]

55. +100 on this. nt

[caraher]

74. Right

Thanks for what you do - that kind of teaching seems a truly Sisyphean task!

Long-term retention is another dimension this study can't touch. Retention is a problem across the board and especially in more demanding environments... I'm glad your experience has been that TFAers are committed (which speaks well of their selection process); I've heard more than a few students idly muse about applying who clearly didn't share that commitment.

It is hard, especially for college students from a privileged background, to properly judge the challenge. When my wife was student teaching (math) in an urban high school she collected some very basic data at the beginning - things like how many books are in your home. The results were pretty sobering!

[Nevernose]

87. The "books in the home" thing is from several studies

Done in the 80s. I'm drawing a blank on the dude who popularized them -- I want to say it was the guy who wrote "Star Teacher" -- but, basically and statistically, the biggest determinor of a child's success in life (again, statistically) is the number of books in the home (in the West), followed closely (worldwide) by the educational level of the mother.

In my district, the dropout rate for teachers is eighty percent after seven years. Of the many reasons cited, the biggest reason for moving districts is poor administration. My mom was saying the same damned thing in the seventies, forty years ago. I'm only still at it because I can't picture myself doing anything else; I'm a very strong atheist, but I understand the concept of "a calling" better than most religious people.

[denverbill]

10. I wonder why they only evaluated teachers at 'hard to staff' schools?

My guess would be these schools pay shit wages for the work and that any teacher worth his salt probably moves to greener pastures ASAP, if they even take a job there in the first place. That would leave the worst teachers and the TFAers, and apparently the TFAers are very slightly better than the remaining traditional teachers.

I'd like to see the results from 'easy to staff' and 'average' schools. If they aren't better there, this study is simply a matter of selecting the traditionally recruited teachers most likely to be bad.

[YvonneCa]

15. Or, some of those teachers worth his/her salt...

...who are committed to their school and students' success STAY at the low-performing school only to be targeted as ineffective after many years of committed service.

[hunter]

27. I was a teacher at a hard to staff school in the 'eighties.

Hell on earth. The only teachers who survived were saints or dead to humanity.

I was merely human. I fled.

[Nevernose]

30. TFA is only assigned at "hard to staff" schools

Because teachers, good AND bad, can only put up with the challenges for so long.

The old joke in our large district is that of we took the staff wholesale from the worst performing school in the worst neighborhood and switched them with the staff at the best performing school, you wouldn't see a difference.

There are decades worth of research on the effects of poverty, and teachers aren't part of that. Statisticians can tell with 95% accuracy how well a child will do in life, based on just a few factors: educational level of the mother, age of the mother, race, neighborhood, income. And they can tell this at the age of three.

[hunter]

34. Yep.

I couldn't survive the "challenges."

Teaching in a messed up community was the hardest job I ever had. I still have nightmares.

A couple years ago I was offered a job similar to what I'd experienced and I said I'd sleep on it. I relived nightmares and declined the next day. Even though I needed a good job with a good health plan.

My wife's sister still does it. She is a saint.

[DeltaLitProf]

36. Not correct

TFA teachers have been competing in all subject areas with new (many of them exemplary) teacher ed graduates from the university where I teach and beating them out for jobs in the hometown schools near here. It's a scandal. These TFAers come to our university for a 6-week training session, then fan out into schools our graduates are trying to get obtain jobs in.

[Nevernose]

85. That's the accusation in my district, too.

And it's probably an honest one. Why hire a union teacher when you can hire a well-meaning scab from an Ivy League school, right?

However, I think a part of the program is that they only go to "under performing" schools, which -- at least in my district -- usually means hard to staff.

[Recursion]

47. I'm inclined to agree, but that undermines the basic argument of teachers' unions

Because you're basically saying that who teaches doesn't matter.

If poverty, hunger, etc. are all-conquering, why does it matter who teaches in our schools?

[Nevernose]

86. Because those who are dedicated

Make the difference. And not just one who are dedicated for a couple of years, but one who are dedicated to a lifetime of service.

God knows that I know some burned out teachers that either need to find a new school or a new career, but most of us who work in "the hood" are as good as it gets for the long term.

We can't fix poverty or injustice in the schools, and we can't fix a crappy home life. But it takes the dedicated teachers to make a difference at school, and most of the TF's I know (most of whom are/were amazing at their jobs) don't have what it takes to make a difference in a school for the long run.

I'm probably not explaining myself as well as I'd like to, but the truth is: it's the third week of school and I'm already exhausted. I teach day school, I teach night school, I advise one club two days a week and the GSA club every other week (and I was a homophobe before DU!), I tutor after school, I volunteer tutor after school, I work with abused and neglected foster kids on the weekend, I'm trying to get a community garden off the ground, I have a wife, a kid, and three dogs, and I'm thinking about getting a Ph.D in Educational Psychology. Shit. I either need a therapist or a nap! Ah. The life of a dedicated liberal...

[Recursion]

88. Either some people are better at teaching or not

If some people are better at teaching, find a way of measuring that and having the people best at it do it. Specifically, either some teachers are capable of improving students' performance or they aren't. If students' performance isn't improving, all of the adherence to prior models doesn't matter. This is my big problem with teacher evaluations: the ones I've been involved with are all about "we have these preset ideas of what a good teacher does; how well does this teacher follow them?" and not at all about "are the children actually learning the things they are here to learn?" It's looking only at inputs and not at all at outputs.

You may say standardized tests are bad indicators at student performance. I largely agree. However, since they greatly influence students' futures, it doesn't seem any more unjust to also have them influence the teachers'; standardized tests are a huge part of which students get in to what colleges. And for that matter, my own opinion is that the educational theory pendulum has swung a bit too far, and the importance of actually testable rote learning is currently undervalued. History, language, etc. require synthesis, yes, but they also require an actual (and testable) rote mastery of facts first -- particularly at the primary level. The 4th grade history class across the room from the math class I tutored in had 9 year olds writing essays about how they feel about Columbus's journey. Any writing practice is good, of course, but what kind of informed synthesis can a 4th grader really do about that? Particularly without an understanding of the Reconquista, the Inquisition, the Portuguese-Spanish rivalry. For that matter the teacher ran with the tired "Europeans thought the world was flat" nonsense; that BS will now survive another generation, but that's a different rant.

A few years ago, 95% of DC public school teachers were assessed as "Excellent". This is absurd. The corporate money grab at public schools is real, but it didn't pop up in a vacuum. If teachers were more serious about actually policing their own, it would never have found the support it has among parents.

[Recursion]

46. For the same reason you compare police stations in high crime districts

It doesn't particularly matter what kind of teachers Trevor and Caitlyn in the rich suburbs have; they'll do fine.

[ancianita]

11. The Big TFA Fail is in COMMITMENT to communities; thus, communities should not commit to them.

Without comparisons of similar demographic samplings of students, this does not impress -- this silly sliver of 'win.'

[YBR31]

18. Read the study--they did match

[ancianita]

21. Fair enough. Then put them in private prep schools and see how they're received by that

socioeconomic class. The only, only reason that TFA might seem better is that university departments of education are clueless about proper teacher training.

[liberal_at_heart]

37. exactly. There's no way rich people woul allow TFA teachers to teach their kids in prep schools.

Teachers in prep schools have Masters in Education, professional teachers certification, and probably go through regular continuing education.

[Recursion]

45. Huh? No. Private schools are much more open to non-traditional teachers

That's one of the whole points.

[ancianita]

56. Then why is "non-traditional" only good enough for the public's kids. You get what you pay for.

The public must refuse to take this "cost-benefit" standard handed to them by the rich, who themselves spare no expense in educating their children.

[Recursion]

72. Private schools hire non traditional teachers all the time

I may be missing your point?

[ancianita]

80. I mean that TFA falls into the category of 'non-traditional,' which is really code for cheaping out.

I stand by what Nevernose has said about decades of studies that prove that socioeconomic status is the determining factor in student success.

My point is that privileging some kinds of teacher success -- in math, no less -- over other kinds of teacher success, or even teachers, traditional or not, in other subject areas or not, ignores the whole rigged game that systematically wounds this country's youth.

When the public wants to seriously pay for the kind of schooling the rich get for their kids, it will get successful schools.

[Smarmie Doofus]

12. Mathematica takes $$$ from Gates.

http://www.gatesfoundation.org/How-We-Work/Quick-Links/Grants-Database/Grants/2009/07/OPP53404

He's got a horse in this race. Or should I say "dog".

[YBR31]

19. Does that invalidate the study?

I read the methodology. It looked kosher to me. I admit I've been out of grad school more years than I want to admit but the study design looked well thought out and controlled. I couldn't find the flaws. If someone does, I'd like to hear what they are but to criticize the study just because one doesn't like the outcome is much like what the climate change deniers do. If the study is well designed and executed, I see no reason no to believe the conclusions.

[ancianita]

22. Their work should be replicable by non-TFA teachers who should succeed across a broad spectrum

schools, then, by the end of this school year.

[U4ikLefty]

23. Small sample size = bad stats.

Too much variation with less than 200 teachers in the sample.

BTW, how do you feel about Charter Schools?

[YBR31]

24. Sample size shouldn't be an issue

Confidence coefficient is 95.

As far as charter schools, I'm neutral. There are great ones and terrible ones. The bad ones should be fixed or lose their charters. I'm for good schools that educate kids, period.

[U4ikLefty]

25. Confidence coefficient does not indicate correctness of the conclusion. Just the repeatability

that a set of data will show the same results.

Shouldn't bad Charter Schools be replaced by good Public Schools?

Everybody is "for good schools that educate kids." That tells me nothing.

How about this...do you think teacher's unions have the best interests of the students in mind?

[YBR31]

29. I don't care whether a school is a public charter or a regular public school as long as it gets the

Job done.

I think the job of the teachers' unions first & foremost is to represent the best interests of the teachers. The teachers deserve vigorous representation of their interests. Sometimes that aligns with the best interests of the students, sometimes it doesn't.

[liberal_at_heart]

48. what helps the teachers does help the students. Graduation rates are usually much higher in

districts where teachers have Masters in Education, continuing education opportunities, and a good salary.

[YBR31]

109. Can you link to a study on this?

I searched graduation rates and the factors you cited but came up empty. There seems to be lots of info on what contributes to graduation rates but I couldn't find any study that supported what you stated. I think in more affluent areas, they would have more educated and higher paid teachers and also higher graduation rates but I am not sure you could prove any sort of causation.

[Pholus]

84. Oh noes, a "95% confidence coefficient!"

[DeltaLitProf]

38. No. But it should be noted in the description of the study. It's a conflict of interest.

Those of us who follow TFA also know of the many Gates and Walmart-supported studies rating their program's members as doing well. Most of the time we do catch these and expose them.

Studies can be manipulated for certain results in ways that would not be traceable in the reports.

When studies NOT funded by Gates and crew start to show TFAers as more effective, then we'll have good reason to change our minds about TFA. But in almost 20 years those studies have yet to appear in enough numbers.

[liberal_at_heart]

39. I saw the last few minutes of Teach the documentary.

They sure try to tug on your heart strings. My child is struggling in school, but I can tell you that documentary did not pull on my heart strings. I see through their bs.

[flyingfysh]

26. Baloney

My wife (now retired) was a highly-trained, nationally certified science teacher who over the years learned all the tricks of how to manage a classroom and teach effectively. She saw some of these TFA people, they just weren't very good at it.

[mythology]

32. I haven't read the study, but that's not evidence of anything

At the very best, your wife would have only seen a small number of TFA teachers in a limited area. At worst, she may have had a bias that prevented her from objectively rating the TFA teachers.

The study may well be crap, but I would hope that a science teacher would understand that a single observation point isn't enough to make a valid conclusion.

[LWolf]

62. Yes.

Of course, I've heard the same thing from every teacher I've talked to that works in a building with TFA teachers, in states across the nation. For whatever that's worth.

I talk to a lot of teachers from a lot of places.

Here's another piece of anecdotal evidence, that I find interesting:

I had a student a decade ago. Average intelligence, average ability, and a complete slacker. When he got to high school, he didn't last long. He ended up in an "alternative" program for failing students, because he couldn't follow through with anything. He was on track for graduation through the alternative program. He came back to visit on a regular basis to reassure me that he was actually going to graduate. Two months before graduation, he very excitedly told me that he was being recruited by TFA. I thought they recruited ivy-league types to do a couple of years slumming it across the tracks before moving on to their REAL careers. I was surprised to hear that they were also recruiting from students who couldn't make it through regular high school.

Here are some questions I always ask before accepting conclusions about any data presented to me:

Who collected and reported, and what is their bias?

How was the data collected, and how was it organized for reporting purposes?

What information was left out?

After all, the same data can be manipulated to suggest completely different things, depending on the goal of the presenter. When someone offers up some data that doesn't seem to fit with our own observations and experiences, we are understandably skeptical, at least until those questions are answered satisfactorily.

[Recursion]

44. Hm. I thought it was conservatives that just flat-out ignored data, not us.

Interesting.

[dsc]

31. I decided to look at the study itself and found something very, very interesting

which Slate somehow decided not to mention. 41% of the teachers that TFA were compared to went to "less selective alternative certification programs". This is from page xxii of the study (a pdf linked at slate). One wonders who the TFA did against the 59% who went to teaching programs. Reading Slate one wouldn't know. I have to go to bed so I will have to wait until tomorrow to see if the study answers that question. Yes, one would expect TFA to perform better than people who got into teaching the same way they did but were in less selective programs.

[Recursion]

42. That's the larger point of Yglesias's article

You probably won't like it, but the result there is that traditional teacher certification programs really don't do any good, and ex post selection seems to be better than ex ante. Which sucks for the two years of kids between the ante and the post, but there we are.

[dsc]

49. but that study surely doesn't prove that

41% of the teachers who were being compared to TFA DIDN'T GET AN EDUCATION DEGREE. That is what the words alternative certification mean. Nowhere in Yglesias' article does he state what the TFA did compared to those who did get an education degree (I am presuming because the study didn't provide that data). So Yglesias can come to his conclusion all he wants but that study doesn't back it up.

[Recursion]

50. No, that's the second study he quotes

I'm not terribly invested on either side of this debate, but I did notice he was quoting two studies here and acting like they were one study. People with more dog on this fight may look into that.

[dsc]

83. No that is not

I checked again just to be sure, but on page xxii of the study that the OP and slate is discussing it says, in no uncertain terms, that 41% of the comparision teachers that the TFA were compared to entered education via alternative certification. I have no idea what any other study says but I do know what that one says.

[YBR31]

96. I think you might need to re-read that

I am pretty sure that 41% was in reference to Teaching Fellows. The Study made multiple comparisons. It compared TFA teachers to Teaching Fellows. It also compared TFA Teacher to traditional teachers. The 41% refers to Teaching Fellows.

[dsc]

100. No I do not

I have read and reread the section in question. It clearly states, in utterly unambiguous language, in the final paragraph of page xxii, starting four lines from the bottom and two words from the right:

Finally, TFA and teaching fellow teachers were also compared with different types of teachers; 41 percent of the teachers with whom TFA teachers were compared entered teaching through less selective alternative routes to certificaiton . . .

certification is the 11th to last word on page xxii

I don't have any earthly idea how much clearer I can state it.

[madfloridian]

33. TFA alums are speaking out now about harm being done by TFA.

http://www.democraticunderground.com/10023639689

"Last month, TFA alumni and members critical of the organization joined students, parents, and community activists at Chicago's Free Minds/Free People education conference for a summit titled “Organizing Resistance to Teach for America and Its Role in Privatization.” (The Education for Liberation Network, which runs the conference, works with organizers but does not control the outcomes of summits.) It was the launch of the first national campaign against TFA and the first national-level convergence of dissident TFA rank and file.

While debate over TFA traditionally revolves around the effectiveness of its teaching model—recruits receive just five weeks of pre-service training and commit to only two years of teaching—organizers are focused on TFA’s broader political impact. With formidable corporate funding and partnership, TFA is part of a market-oriented reform movement that involves expanding charter schools to compete with district schools, pegging teachers' job security to students' standardized-test scores, and churning in fresh teachers while weeding out those who “underperform,” regardless of experience. These moves purport to enhance student outcomes; they also increase teacher turnover and destabilize school systems."

[ellenrr]

52. What's the Difference Between Teach For America and a Scab Temp Agency?

If you're a public school parent or student, there's none. If you're an educator or other school employee or a friend or relative of any school employee, there's none. If you live in a community where the local public school is one of the last hopeful possibilities that might bind a neighborhood together, there's none.

If you're a school CEO or administrator trying to hollow out your public schools to justify their closing and privatzation, or a mayor trying to justify those campaign contributions, there's no difference at all, either. If you're a hedge fund investor, like the charter school sugar daddies who contribute billions to Barack Obama and a host of black and white politicians in state and local government across the country; there's still no difference.

http://www.blackagendareport.com/tfa-scab

[Recursion]

41. The difference is statistically significant but not terribly useful

That is, we can say with 95% confidence that it's not a fluke, but then again a reliable 0.7% increase isn't terribly important, either.

[Starry Messenger]

43. Sounds like nonsense.

[Recursion]

54. Hm. I thought it was conservatives who avoided data

I guess that's changed?

[Starry Messenger]

64. ?

Why be nasty? That was unwarranted.

[Nye Bevan]

66. To be fair, that wasn't exactly much of a rebuttal you were able to come up with.

Obviously you don't like the conclusion, but calling a scientific study "nonsense" doesn't really refute it.

[Starry Messenger]

67. I'll be sure to run my replies past the committee next time.

[Nye Bevan]

68. You did. We are "the committee" (nt)

[Starry Messenger]

69. Pholus' reply in #59 reflects my views

Most of these studies are cherry-picked absurdities meant to "prove" a pre-set agenda.

After observing a steady stream of them for years, I'm standing by my first remark.

[Katashi_itto]

51. Sure they did...

[reformist2]

57. Yayayayay! Teachers with no experience and no long-term commitment to teaching do best!!! Absurd.

Even if one were to believe the results of this particular study, the implications are patently ridiculous. Why not have entire schools staffed by 24 year-olds right out of college?

[Nye Bevan]

58. Perhaps because then you would have to accept many more TFA applicants,

so you would have to make the program much less selective than it is now, reducing the overall quality of the TFA teachers, meaning that the results of the study would no longer apply?

[Recursion]

92. Well, why not?

Seriously, why is that so absurd? I agree this study doesn't show we should do that (the difference, while statistically significant, isn't large enough to justify big changes). But if the data showed that doing that got the best results, why on earth wouldn't you want that? The purpose of schools is not for teachers to have jobs, but for children to be educated.

[reformist2]

94. Don't be ridiculous. There is no profession in the world best done by novices.

[Pholus]

59. Wow, I see statistical proof that "white kids" are better teachers!

Table IV.2 compare to IV.5. Out of all the "data" of which none is actually presented in the report (just summaries, lacking in crucial diagnostic data or even scatterplots), that is the most striking apparent correlation.

Seems to me the actual key point of the study gets only a small mention on page xxiv and its actual significance is completely underplayed: while the TFA teachers demographically match teachers nationwide (80% white) the "Comparison Teachers" strikingly do not.... (30% white) .

Doesn't that beg the question why the "Comparison Teachers" are so demographically different in these underperforming schools? Why no text discussing that? What factors lead to this difference?

Oh yeah, that's cause it's economics and the answer isn't sending a bunch of preppie college kids in, it's actually financially supporting the districts in place. The "Comparison Teachers" have a whole host of duties I figure the TFA kids are exempted from as well. Not actually discussed, but you know the volunteers are always treated more lightly than the actual professionals.

This is supposed to be about trumpeting the full superiority of TFA teachers in a cherry picked study where they are dropped into what looks like the worst of the public school system and eke out an approximate 10% improvement in scores regarding a relatively specialized subject.

Agenda driven papers suck!

[Starry Messenger]

65. +1

[adirondacker]

71. +1000. This country is losing major ground in education prepping the preppers for a bullshit exam.nt

[Recursion]

93. Because majority-minority schools have majority-minority teachers, generally

And those are the same schools that are considered "hard to staff", often because the white teachers don't want to work in them.

[Pholus]

95. This study is flawed because those variables are not considered.

Last edited Thu Sep 12, 2013, 09:59 AM - Edit history (1)

I am stunned how the authors are claiming it is teaching quality from such sparse data and with such blatant signs of such a large uncontrolled effect that simply screams socioeconomics. Frankly I also see multiple unconsidered explanations not involving "effectiveness" that could explain this 10 percent effect. 1) different expectations in collateral duties giving one group more contact or preparation time than thank the others. 2) TFA classroom selection having hidden biases (for example if I were getting a new person helping me out I tend to give them a gentler schedule or nicer classes or even an assistant) 3) The "placebo effect " where they have some goodwill from the students because of a buildup where the TFA teachers are sold as special people who are here as a special opportunity. The resulting esprit-de-corps leads to the student acceptance of different expectations.

An alternate experiment which would probably yield even better results is giving each comparison teacher 1 to 2 volunteer aides so that education can be their sole focus.

The poor quality of the papers' analysis does cause further lessening of my opinion about the general quality of the Wolfram's products though... I already despise software licenses because it makes the application of what I have spent my limited time learning conditional on paying some rentiers.

Open source every chance I get and I will spend development time so that what I use is there for me whenever I want and it can be shared freely without economics getting involved.

[dionysus]

77. honest question.. how do you realistically gauge that sort of thing? There's so many factors.. you

can have the best teacher but if the students are unwilling or unable to learn (poverty, bad parents ect) how can you blame the teacher if the kids don't do well?

I know someone who teaches 8-10 years olds in the inner city and he says it's a nightmare. How can you really judge effectiveness?

[Smarmie Doofus]

81. This is still going on? Silly season. Listen: Find a study that isn't conducted by a ...

Gates front group and people will take it seriously. Nice of Mathematica to list the Gates connection on it's website, btw. (It doesn't. Does that sound ethical to you?)

Seems like you may be new to all of this so I don't necessarily blame you, but we've been hashing out this shit for years on DU and it's like going around in circles. Respectfully, learn about corporate ties to the TFA and the corporate ed "reform" movement in general and Gates in particular.


Here. Read this:

http://www.nytimes.com/2011/05/22/education/22gates.html?_r=3&pagewanted=all&amp&

[Recursion]

91. How does a study's funding magically change how well kids do math?

Come on. Ignoring data is what conservatives do, not us.

[LWolf]

107. Like this.

http://cseweb.ucsd.edu/~ricko/CSE3/Lie_with_Statistics.pdf

[YBR31]

97. Should teachers be laid off by seniority rather than performance?

I don't think the concept is wrong. The question is: "How do you measure performance?"

[dsc]

101. One year's performance shouldn't be the be all and end all

the fact is say a school has a choice between laying off two teachers. Teacher A is 10 years from retirement, has 20 years experience all at her current school, and shows no signs at all of wanting to leave. Teacher B is in his fourth year of teaching, is taking law classes at night which he will graduate in 2 years. His test scores are as much better as the TFA teachers were in this study. Now, is the school better off having teacher a for 10 more years, or teacher b for 2 years followed by his replacement who will have to be trained, etc? It is for this reason we have seniority based layoffs.

[Teamster Jeff]

82. Scabbing for cheap in USA Inc.

Scabs are beloved and praised by those interested in breaking unions

[CabalPowered]

98. With all due respect to Slate..

The conclusions are quite clear on page 98:

"The experimental analysis suggests that Teaching Fellows teaching secondary math were neither more nor less effective than comparison teachers."

Having worked on efficacy studies myself, the next sentence is how the researchers save face:

"However, this comparison of average effectiveness masked several differences in effectiveness between specific subgroups of teachers: .. "

Then they offer four sub points, three of which are not strong arguments. The only one they can hang their hat on is..

"Students of Teaching Fellows outperformed students of comparison teachers from less selective AC routes by 0.13 standard deviations."

Yet on page 96 that details this finding, there's this gem..

"Novice Teaching Fellows were less effective than experienced comparison teachers."

In short, they didn't find any results and they'll never be published by a peer-reviewed journal.

[Pholus]

99. Yup, statistical hit job.

That was obvious the moment they didn't dirty up their paper with nasty things like actual plots of the data.

[YBR31]

105. There is confusion because the study looked at 2 distinct groups: Teaching Fellows and TFA

The study looked at both. You are referring to Teaching Fellows. From the study:
"Two programs—Teach For America (TFA) and the TNTP Teaching Fellows programs— take a distinctive approach to addressing the need for high-quality teachers of hard-to-staff subjects in high-poverty schools."
From the Study
"EXECUTIVE SUMMARY
Teach For America (TFA) and the Teaching Fellows programs are an important and growing source of teachers of hard-to-staff subjects in high-poverty schools, but comprehensive evidence of their effectiveness has been limited. This report presents findings from the first large-scale random assignment study of secondary math teachers from these programs. The study separately examined the effectiveness of TFA and Teaching Fellows teachers, comparing secondary math teachers from each program with other secondary math teachers teaching the same math courses in the same schools. The study focused on secondary math because this is a subject in which schools face particular staffing difficulties.
The study had two main findings, one for each program studied:
1. TFA teachers were more effective than the teachers with whom they were compared. On average, students assigned to TFA teachers scored 0.07 standard deviations higher on end-of-year math assessments than students assigned to comparison teachers, a statistically significant difference. This impact is equivalent to an additional 2.6 months of school for the average student nationwide.
2. Teaching Fellows were neither more nor less effective than the teachers with whom they were compared. On average, students of Teaching Fellows and students of comparison teachers had similar scores on end-of-year math assessments."

[Pholus]

108. It's not confusion at all. It's statistical dishonesty.

It has nothing to do with two groups. The "Teaching Fellows" didn't accomplish anything, the TFA supposedly did.

But that second statement depends on having to trust the authors' interpretation since they don't deign to show their raw data. IN MY EXPERIENCE as a scientist, refusal to display raw data immediately sets off the bullshit detector. Certainly, some results are plotted, just not the ones relevant to the main conclusions. Why?

Fishy, fishy, fishy.

[YBR31]

102. Teaching Fellows are a separate category. The Conclusion about TFA are on page 61-61

Teaching Fellows are Teachers who come in through a different certification route than TFA or traditional teachers.

[YBR31]

103. P. 96-98 is about Teaching Fellows not TFA. Conclusions about TFA are on p.61-62

Teaching Fellows are not TFA. The study explains the difference.

[YBR31]

104. TFA & Teaching Fellows are 2 totally different programs.

The study looked at both. You are referring to Teaching Fellows. From the study:
"Two programs—Teach For America (TFA) and the TNTP Teaching Fellows programs— take a distinctive approach to addressing the need for high-quality teachers of hard-to-staff subjects in high-poverty schools."
From the Study
"EXECUTIVE SUMMARY
Teach For America (TFA) and the Teaching Fellows programs are an important and growing source of teachers of hard-to-staff subjects in high-poverty schools, but comprehensive evidence of their effectiveness has been limited. This report presents findings from the first large-scale random assignment study of secondary math teachers from these programs. The study separately examined the effectiveness of TFA and Teaching Fellows teachers, comparing secondary math teachers from each program with other secondary math teachers teaching the same math courses in the same schools. The study focused on secondary math because this is a subject in which schools face particular staffing difficulties.
The study had two main findings, one for each program studied:
1. TFA teachers were more effective than the teachers with whom they were compared. On average, students assigned to TFA teachers scored 0.07 standard deviations higher on end-of-year math assessments than students assigned to comparison teachers, a statistically significant difference. This impact is equivalent to an additional 2.6 months of school for the average student nationwide.
2. Teaching Fellows were neither more nor less effective than the teachers with whom they were compared. On average, students of Teaching Fellows and students of comparison teachers had similar scores on end-of-year math assessments."